From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4C057A054F for ; Fri, 5 Mar 2021 22:29:40 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2C48E40691; Fri, 5 Mar 2021 22:29:40 +0100 (CET) Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by mails.dpdk.org (Postfix) with ESMTP id D755E40147 for ; Fri, 5 Mar 2021 22:29:38 +0100 (CET) Received: by mail-lf1-f41.google.com with SMTP id v9so6065170lfa.1 for ; Fri, 05 Mar 2021 13:29:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HFSH0WTR0YUUSqKyDbslu63XTaEGAtuAiWSBqH0bCxc=; b=cWt7D9lUMCvbtqPIBYYjafuQMXFv0rgk/VMpbi1Ozqh/Ow4d1u6Xcx0tX9bF+fICoS yZx5U/8kLqCH47gH9v2mJPJlL3hvrHJmvE5BL8LDc68+yq5RowPGKWrJ14XJSrwdWx1+ h4aWL6Y0BQ/5SQbUpvXmATeTZW1sOjU/kcgy0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HFSH0WTR0YUUSqKyDbslu63XTaEGAtuAiWSBqH0bCxc=; b=Ft/jAV19vteaA3uZNIEodWhcFB2PJHyM63MawhCx71+Nezu+kXvwFJ/oIV16aceMl1 XC0qUEQCoS2wtGnMeJrdXiYc+vwWjk3N2Dvrbz6w+7sGjzghZRjGPj/Y2NWzUrQLlhoJ qqqmAuPExJJmo02NqkyWfP9mADJh1B8asW4Gm7Ar3IbaBSjDiJapS1taAHJncPwws1+J FADSwCpnzWXEBIf1/HZ75QPIaiMM4sG9erLMPyqTdxJDBo2LYcdMEhqXFHoFSGZOBRgH q6xP8nB6nXKs1TVKlRS9+bmIcabT2pS9dp2as+M36Gid9yclHwm+col91wP2Owxn/F2m 01uw== X-Gm-Message-State: AOAM532tUlM63llUszptPXJ/rdHTHhFPbx6I+llEyMokV7/BfVEX2Ill CA1HuLVghUNNG56zTtFZkmgZk7YxHYRKSUJKimuuVA== X-Google-Smtp-Source: ABdhPJzfJmC+QsFOo7TTEqzE5Rf7908RvfwfroKdI7/VBbgxNkNBOHFn5luASGwxo6nwoP9/fPBci33GxaQrjp7ioWI= X-Received: by 2002:a05:6512:1284:: with SMTP id u4mr6990475lfs.175.1614979778264; Fri, 05 Mar 2021 13:29:38 -0800 (PST) MIME-Version: 1.0 References: <6b0b25c338e840929f72560a3889f9f8@pantheon.tech> In-Reply-To: From: David Liu Date: Fri, 5 Mar 2021 16:29:02 -0500 Message-ID: To: =?UTF-8?Q?Juraj_Linke=C5=A1?= Cc: Ajit Khaparde , Owen Hilyard , "ci@dpdk.org" , dpdklab Content-Type: multipart/alternative; boundary="00000000000000d5f605bcd0c87e" Subject: Re: [dpdk-ci] [dpdklab] Re: [DPDKLAB] 25G Broadcom NIC in Arm hardware X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" --00000000000000d5f605bcd0c87e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Ajit, Juraj, For simplicity, we will keep the functional and performance testing disable during the weekend. In this way, you will have no testing interfere with you during troubleshooting. Then we will rerun all the jobs that are skipped. Please let me know if you have any questions. Best, David On Wed, Mar 3, 2021 at 4:05 AM Juraj Linke=C5=A1 wrote: > Hi David, Ajit, > > > > Friday 10am is 4pm for me a that's fine. > > > > Juraj > > > > *From:* David Liu > *Sent:* Tuesday, March 2, 2021 7:06 PM > *To:* Ajit Khaparde > *Cc:* Juraj Linke=C5=A1 ; Owen Hilyard < > ohilyard@iol.unh.edu>; ci@dpdk.org; dpdklab > *Subject:* Re: [dpdk-ci] [dpdklab] Re: [DPDKLAB] 25G Broadcom NIC in Arm > hardware > > > > Hi, Ajit > I changed it to Friday at 10 AM for now. > > We can change again if that doesn't work for Juraj > > Best, > > David > > > > > > On Tue, Mar 2, 2021 at 10:28 AM Ajit Khaparde > wrote: > > Hi David, > My Thursdays are crazy. Can we do Friday instead? > Juraj, can you make Friday 10AM (Eastern time) call? > > Thanks > Ajit > > On Tue, Mar 2, 2021 at 6:29 AM David Liu wrote: > > > > Hi Ajit and Juraj, > > > > I send out an invite for Thursday, Mar 4 at 10:00 AM. But feel free to > adjust the time or let me know if there is a better time that works for y= ou. > > > > For me, I can do: > > Tuesday 10:00 AM - 1:00 PM. > > Thursday 10:00 AM - 1:00 PM. > > Friday 10:00 AM - 12:00 PM and 2:00 PM - 5:00 PM > > > > Best, > > David > > > > On Mon, Mar 1, 2021 at 10:12 AM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > >> David, > >> I was busy with a customer issue and could not get to my emails. > >> I am seeing this now. Let me close out any issues needing urgent > attention. > >> Let's sync on a suitable time after that. > >> > >> Juraj, > >> Sure. We will add you. > >> > >> Thanks > >> Ajit > >> > >> On Mon, Mar 1, 2021 at 7:00 AM Juraj Linke=C5=A1 > wrote: > >> > > >> > Hi Ajit, David, > >> > > >> > > >> > > >> > If you're setting up a call, could you please send me an invite? I'd > like to get some insight into the issues on Arm's behalf. > >> > > >> > > >> > > >> > Thanks, > >> > > >> > Juraj > >> > > >> > > >> > > >> > From: ci On Behalf Of David Liu > >> > Sent: Thursday, February 25, 2021 7:47 PM > >> > To: Ajit Khaparde > >> > Cc: Owen Hilyard ; ci@dpdk.org; dpdklab < > dpdklab@iol.unh.edu> > >> > Subject: Re: [dpdk-ci] [dpdklab] Re: [DPDKLAB] 25G Broadcom NIC in > Arm hardware > >> > > >> > > >> > > >> > Hi Ajit, > >> > > >> > > >> > > >> > Sorry for this issue. > >> > > >> > If possible I would like to set up a time where we can do this in a > call to help get this solved faster. > >> > > >> > Please let me know what would be a good time for you. > >> > > >> > > >> > > >> > Best, > >> > > >> > David > >> > > >> > > >> > > >> > > >> > > >> > On Thu, Feb 25, 2021 at 12:55 PM David Liu wrote: > >> > > >> > Hi Ajit, > >> > > >> > > >> > > >> > I changed the permission to allow you to run sudo. > >> > > >> > This should work now. > >> > > >> > Let me know if you still have issues. > >> > > >> > > >> > > >> > Best, > >> > > >> > David > >> > > >> > > >> > > >> > On Mon, Feb 22, 2021 at 11:11 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > > >> > David, > >> > > >> > I tried the option to use sudo. And that didn't work. > >> > > >> > > >> > > >> > Thanks > >> > > >> > Ajit > >> > > >> > > >> > > >> > ajitkhaparde is not allowed to run sudo on arm-gigabyte. This > incident will be reported. > >> > > >> > > >> > > >> > ---- > >> > > >> > sudo su > >> > [sudo] password for ajitkhaparde: > >> > ajitkhaparde is not allowed to run sudo on arm-dell. This incident > will be reported. > >> > ajitkhaparde@arm-dell:/opt/dts$ sudo ./dts -s > >> > [sudo] password for ajitkhaparde: > >> > ajitkhaparde is not allowed to run sudo on arm-dell. This incident > will be reported. > >> > > >> > > >> > > >> > On Thu, Feb 18, 2021 at 10:03 AM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > > >> > > >> > > >> > > >> > > >> > On Thu, Feb 18, 2021 at 10:01 AM David Liu wrote: > >> > > >> > Hi Ajit, > >> > > >> > > >> > > >> > This is likely caused by when dts try to ssh into another machine as > a normal user which will prompt you for the password. > >> > > >> > > >> > > >> > One solution would be "sudo su" as root to run the command line sinc= e > you should have root access. > >> > > >> > David, > >> > > >> > Let me try this method first. > >> > > >> > If it does not work, we could try with the public ssh keys. > >> > > >> > I will let you know. > >> > > >> > > >> > > >> > Thanks > >> > > >> > Ajit > >> > > >> > > >> > > >> > Another solution would be to let me add your public ssh key into bot= h > machines. > >> > > >> > > >> > > >> > Let me know if this helps. > >> > > >> > > >> > > >> > Best, > >> > > >> > David > >> > > >> > > >> > > >> > On Fri, Feb 12, 2021 at 5:48 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > > >> > Thanks Owen. > >> > > >> > > >> > > >> > When I run the dts command line suggested, I get the following error= . > >> > > >> > > >> > > >> > Thanks > >> > > >> > Ajit > >> > > >> > > >> > > >> > Traceback (most recent call last): > >> > File "/opt/dts/framework/ssh_pexpect.py", line 60, in _connect_hos= t > >> > self.password, original_prompt=3D'[$#>]', > password_regex=3Dr'(?i)(?:password:)|(?:passphrase for key)|(?i)(password= for > .+:)') > >> > File "/usr/local/lib/python3.6/dist-packages/pexpect/pxssh.py", > line 440, in login > >> > raise ExceptionPxssh('password refused') > >> > pexpect.pxssh.ExceptionPxssh: password refused > >> > > >> > ^^^^^^^^^^^^^^ > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > On Fri, Feb 12, 2021 at 12:09 PM Owen Hilyard > wrote: > >> > > >> > Hello Ajit, > >> > > >> > > >> > > >> > There was a misconfiguration in our identity management system. The > issue should now be resolved. > >> > > >> > > >> > > >> > Owen Hilyard > >> > > >> > > >> > > >> > On Thu, Feb 11, 2021 at 9:38 PM 'Ajit Khaparde' via dpdklab < > dpdklab@iol.unh.edu> wrote: > >> > > >> > David, > >> > > >> > I can use my credentials to login to rhea. > >> > > >> > But the same credentials are not allowing me to login onto the > servers you provided. > >> > > >> > > >> > > >> > Thanks > >> > > >> > Ajit > >> > > >> > > >> > > >> > ssh -l ajitkhaparde arm-gigabyte.dpdklab.iol.unh.edu > >> > Connection closed by 172.18.0.41 port 22 > >> > > >> > > >> > > >> > ssh -l ajitkhaparde arm-dell.dpdklab.iol.unh.edu > >> > Connection closed by 172.18.0.40 port 22 > >> > > >> > > >> > > >> > On Tue, Feb 9, 2021 at 6:36 AM David Liu wrote: > >> > > >> > Hi Ajit, > >> > > >> > > >> > > >> > Thanks for the update. Let me know if you need anything from me. > >> > > >> > > >> > > >> > > >> > > >> > Best, > >> > > >> > David > >> > > >> > > >> > > >> > On Sun, Feb 7, 2021 at 11:04 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > > >> > David, > >> > > >> > I had some trouble with my system over the past week. > >> > > >> > So I could not login into the setup. I am hoping that my system will > be restored early this week > >> > > >> > I will login into the setup after that. > >> > > >> > > >> > > >> > Thanks > >> > > >> > Ajit > >> > > >> > > >> > > >> > On Tue, Feb 2, 2021 at 10:57 AM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > > >> > Hi David, > >> > I actually did miss it. I will take a look. > >> > Thanks for the refresh mail. > >> > > >> > Thanks > >> > Ajit > >> > > >> > On Tue, Feb 2, 2021 at 8:58 AM David Liu wrote: > >> > > > >> > > Hi Ajit, > >> > > > >> > > Ping again in case if you miss the last email. > >> > > > >> > > Did you have a chance to give it a try on login to the machines? > >> > > > >> > > Just some notes, we are currently running compile and some > performance/functional testing. > >> > > Which it may be hard for you to run dts, so please let me know if > you want me to stop these tests for you to debug. > >> > > > >> > > Step I run: > >> > > CONF=3D"conf_dpdk" ./switch.sh Broadcom #this is to switch the d= ts > configuration to Broadcom > >> > > ./dts -s > #To start the dts > >> > > > >> > > Informatio: > >> > > arm-dell.dpdklab.iol.unh.edu > >> > > - dts - located in /opt/dts > >> > > - Trex - located in /opt/trex/v2.87 > >> > > > >> > > arm-gigabyte.dpdklab.iol.unh.edu > >> > > - dpdk - located in /root/dpdk > >> > > > >> > > Best, > >> > > David > >> > > > >> > > On Tue, Jan 26, 2021 at 4:59 PM David Liu wrote= : > >> > >> > >> > >> Also, I had changed the configuration when I run Broadcom nic. > >> > >> In the dts folder, you can start it with ./dts -s > >> > >> > >> > >> On Tue, Jan 26, 2021 at 4:54 PM David Liu > wrote: > >> > >>> > >> > >>> Hi Ajit, > >> > >>> Sorry for the late reply. > >> > >>> > >> > >>> You can ssh into the machine using the same VPN Username and > Password. > >> > >>> Can you try and see if that works? > >> > >>> > >> > >>> Informatio: > >> > >>> arm-dell.dpdklab.iol.unh.edu > >> > >>> - dts - located in /opt/dts > >> > >>> - Trex - located in /opt/trex/v2.87 > >> > >>> > >> > >>> arm-gigabyte.dpdklab.iol.unh.edu > >> > >>> - dpdk - located in /root/dpdk > >> > >>> > >> > >>> > >> > >>> Thanks, > >> > >>> David > >> > >>> > >> > >>> On Tue, Jan 19, 2021 at 5:20 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > >>>> > >> > >>>> Hi David, > >> > >>>> Thanks for pinging me. Do you think I could login into the setu= p > and > >> > >>>> see what is happening? > >> > >>>> I do have the credentials for two systems in your lab - > >> > >>>> rhea.dpdklab.iol.unh.edu and io.dpdklab.iol.unh.edu > >> > >>>> If you can provide me details of the systems you are trying to > >> > >>>> configure, I could take a look. > >> > >>>> > >> > >>>> Thanks > >> > >>>> Ajit > >> > >>>> > >> > >>>> On Tue, Jan 19, 2021 at 1:41 PM David Liu > wrote: > >> > >>>> > > >> > >>>> > Hi Ajit, > >> > >>>> > > >> > >>>> > Thank you for helping out. > >> > >>>> > > >> > >>>> > How are you doing with this issue? > >> > >>>> > > >> > >>>> > Please let me know if you need anything information from us. > >> > >>>> > > >> > >>>> > Just some extra information that both Trex and dts are on the > same intel tester machine. > >> > >>>> > > >> > >>>> > Best, > >> > >>>> > David > >> > >>>> > > >> > >>>> > On Thu, Jan 14, 2021 at 2:12 PM David Liu > wrote: > >> > >>>> >> > >> > >>>> >> Yes, so this is the same NIC work on x86. > >> > >>>> >> By seeing the link, I am assuming you mean the ports that ar= e > using for Trex. If so they are UP before I start the Trex. > >> > >>>> >> > >> > >>>> >> Thank, > >> > >>>> >> David > >> > >>>> >> > >> > >>>> >> > >> > >>>> >> On Thu, Jan 14, 2021 at 1:11 AM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > >>>> >>> > >> > >>>> >>> Does the same NIC work on x86? > >> > >>>> >>> Also do you see link before you start the trex? I am trying > to see if the port failed to successfully come up > >> > >>>> >>> or if there is a problem with the link. Can you check and > tell? > >> > >>>> >>> > >> > >>>> >>> Thanks > >> > >>>> >>> Ajit > >> > >>>> >>> > >> > >>>> >>> On Wed, Jan 13, 2021 at 4:20 PM David Liu > wrote: > >> > >>>> >>>> > >> > >>>> >>>> Hi Ajit, > >> > >>>> >>>> > >> > >>>> >>>> Thank you for helping out. > >> > >>>> >>>> > >> > >>>> >>>> We have it working on the x86, but not the Arm. > >> > >>>> >>>> > >> > >>>> >>>> > >> > >>>> >>>> Thanks, > >> > >>>> >>>> David > >> > >>>> >>>> > >> > >>>> >>>> On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde < > ajit.khaparde@broadcom.com> wrote: > >> > >>>> >>>>> > >> > >>>> >>>>> Hi David, > >> > >>>> >>>>> I will take a look. > >> > >>>> >>>>> Do you see similar issues on x86? I am asking because > >> > >>>> >>>>> even I would start with that to create a baseline and the= n > >> > >>>> >>>>> attempt on ARM. > >> > >>>> >>>>> > >> > >>>> >>>>> Thanks > >> > >>>> >>>>> Ajit > >> > >>>> >>>>> > >> > >>>> >>>>> On Wed, Jan 13, 2021 at 12:08 PM David Liu < > dliu@iol.unh.edu> wrote: > >> > >>>> >>>>>> > >> > >>>> >>>>>> Hi Ajit, > >> > >>>> >>>>>> > >> > >>>> >>>>>> We have a 25G Broadcom NIC that is installed on an Arm > machine. > >> > >>>> >>>>>> And I am running into a problem when I try to run > nic_single_core_perf on the NIC. > >> > >>>> >>>>>> Currently, we are using: Trex v2.86 and dts from > http://git.dpdk.org/tools/dts/ > >> > >>>> >>>>>> All the NICs are up and running when testing. > >> > >>>> >>>>>> > >> > >>>> >>>>>> The problem is an error when I try to run the test case > inside the nic_single_core_perf: > >> > >>>> >>>>>> > >> > >>>> >>>>>>> TestNicSingleCorePerf: Test running at parameters: > framesize: 64, rxd/txd: 512 > >> > >>>> >>>>>>> dut.172.18.0.41: > x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4 > --file-prefix=3Ddpdk_15311_20210113190237 -a 0000:93:00.0 -a 0000:93:00= .1 > -- -i --portmask=3D0x3 --txd=3D512 --rxd=3D512 --nb-cores=3D1 > >> > >>>> >>>>>>> dut.172.18.0.41: start > >> > >>>> >>>>>>> TestNicSingleCorePerf: Test Case > test_perf_nic_single_core Result ERROR: Traceback (most recent call last)= : > >> > >>>> >>>>>>> File "/opt/dts/framework/test_case.py", line 319, in > _execute_test_case > >> > >>>> >>>>>>> case_obj() > >> > >>>> >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line > 196, in test_perf_nic_single_core > >> > >>>> >>>>>>> self.perf_test(self.nb_ports) > >> > >>>> >>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line > 270, in perf_test > >> > >>>> >>>>>>> _, packets_received =3D > self.tester.pktgen.measure_throughput(stream_ids=3Dstreams, > options=3Dtraffic_opt) > >> > >>>> >>>>>>> File "/opt/dts/framework/pktgen_base.py", line 245, i= n > measure_throughput > >> > >>>> >>>>>>> self._prepare_transmission(stream_ids=3Dstream_ids) > >> > >>>> >>>>>>> File "/opt/dts/framework/pktgen_trex.py", line 778, i= n > _prepare_transmission > >> > >>>> >>>>>>> self._conn.reset(ports=3Dself._ports) > >> > >>>> >>>>>>> File > "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/tr= ex_api_annotators.py", > line 51, in wrap2 > >> > >>>> >>>>>>> ret =3D f(*args, **kwargs) > >> > >>>> >>>>>>> File > "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_= stl_client.py", > line 339, in reset > >> > >>>> >>>>>>> self.clear_stats(ports) > >> > >>>> >>>>>>> File > "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/tr= ex_api_annotators.py", > line 51, in wrap2 > >> > >>>> >>>>>>> ret =3D f(*args, **kwargs) > >> > >>>> >>>>>>> File > "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_= stl_client.py", > line 1492, in clear_stats > >> > >>>> >>>>>>> self._clear_stats_common(ports, clear_global, > clear_xstats) > >> > >>>> >>>>>>> File > "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/tr= ex_client.py", > line 2876, in _clear_stats_common > >> > >>>> >>>>>>> raise TRexError(rc) > >> > >>>> >>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - > Failed to get server response from tcp://172.18.0.40:4501 > >> > >>>> >>>>>> > >> > >>>> >>>>>> > >> > >>>> >>>>>> I also try to run TRex in stateless mode with./t-rex-64 > -i --cfg /etc/trex_cfg.yaml and connect with ./trex-console > >> > >>>> >>>>>> > >> > >>>> >>>>>> Then start sending traffic with command > >> > >>>> >>>>>>> > >> > >>>> >>>>>>> trex>start -f stl/imix.py > >> > >>>> >>>>>> > >> > >>>> >>>>>> > >> > >>>> >>>>>> But this will cause an error to turn off Trex. > >> > >>>> >>>>>> > >> > >>>> >>>>>>> trex> > >> > >>>> >>>>>>> 2021-01-13 19:56:21 - [server][warning] - Server has > been shutdown - cause: 'WATCHDOG: task 'Trex DP core 1' has not responded > for more than 1.00135 seconds - timeout is 1 seconds > >> > >>>> >>>>>>> *** traceback follows *** > >> > >>>> >>>>>>> 1 0x561be173cf5a ./_t-rex-64(+0x19af5a) > [0x561be173cf5a] > >> > >>>> >>>>>>> 2 0x7feea3ac0980 > /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7feea3ac0980] > >> > >>>> >>>>>>> 3 0x561be1a05a2a rte_delay_us_block + 106 > >> > >>>> >>>>>>> 4 0x561be16ce874 > CCoreEthIF::send_burst(CCorePerPort*, unsigned short, > CVirtualIFPerSideStats*) + 3220 > >> > >>>> >>>>>>> 5 0x561be16abf57 CCoreEthIF::flush_tx_queue() + 3= 1 > >> > >>>> >>>>>>> 6 0x561be171e3d9 > CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265 > >> > >>>> >>>>>>> 7 0x561be171f7ec > CNodeGenerator::handle_flow_sync(CGenNode*, CFlowGenListPerThread*, bool&= ) > + 92 > >> > >>>> >>>>>>> 8 0x561be171fc98 > CNodeGenerator::handle_slow_messages(unsigned char, CGenNode*, > CFlowGenListPerThread*, bool) + 184 > >> > >>>> >>>>>>> 9 0x561be16cb5f1 int > CNodeGenerator::flush_file_realtime<23, false>(double, double, > CFlowGenListPerThread*, double&) + 881 > >> > >>>> >>>>>>> 10 0x561be1905212 > TrexStatelessDpCore::start_scheduler() + 226 > >> > >>>> >>>>>>> 11 0x561be1883ae9 TrexDpCore::start() + 89 > >> > >>>> >>>>>>> 12 0x561be1714113 > CFlowGenListPerThread::start(std::__cxx11::basic_string std::char_traits, std::allocator >&, CPreviewMode&) + 115 > >> > >>>> >>>>>>> 13 0x561be16af8dd CGlobalTRex::run_in_core(unsigne= d > char) + 487 > >> > >>>> >>>>>>> 14 0x561be16d11ad ./_t-rex-64(+0x12f1ad) > [0x561be16d11ad] > >> > >>>> >>>>>>> 15 0x561be1a1dfaa eal_thread_loop + 426 > >> > >>>> >>>>>>> 16 0x7feea3ab56db > /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7feea3ab56db] > >> > >>>> >>>>>>> 17 0x7feea2a8571f clone + 63 > >> > >>>> >>>>>>> > >> > >>>> >>>>>>> *** addr2line information follows *** > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ??:0 > >> > >>>> >>>>>>> ' > >> > >>>> >>>>>> > >> > >>>> >>>>>> > >> > >>>> >>>>>> I believe if everything is working, this should not turn > off trex but please correct me if I am wrong. > >> > >>>> >>>>>> I wonder if you have any suggestion of how I can approac= h > this issue? > >> > >>>> >>>>>> > >> > >>>> >>>>>> Best, > >> > >>>> >>>>>> David > >> > >>>> >>>>>> > >> > >>>> >>>>>> > > --00000000000000d5f605bcd0c87e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Ajit, Juraj,

For simplicity, we will= keep the functional and performance testing disable during the weekend.
In this way, you will have no testing interfere with you du= ring troubleshooting.

Then we will rerun all the jobs tha= t are skipped.

Please let me know if you have any = questions.

Best,
David

On Wed, Mar = 3, 2021 at 4:05 AM Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech> wro= te:

Hi David, Ajit,

=C2=A0

Friday 10am is 4pm for me a that's fine.=

=C2=A0

Juraj

=C2=A0

From: David Liu <dliu@iol.unh.edu>
Sent: Tuesday, March 2, 2021 7:06 PM
To: Ajit Khaparde <ajit.khaparde@broadcom.com>
Cc: Juraj Linke=C5=A1 <juraj.linkes@pantheon.tech>; Owen Hilya= rd <ohilyard@i= ol.unh.edu>; ci@dpd= k.org; dpdklab <dpdklab@iol.unh.edu>
Subject: Re: [dpdk-ci] [dpdklab] Re: [DPDKLAB] 25G Broadcom NIC in A= rm hardware

=C2=A0

Hi, Ajit
I changed it to Friday at 10 AM for now.

We can change again if that doesn't work for=C2= =A0Juraj

Best,

David

=C2=A0

=C2=A0

On Tue, Mar 2, 2021 at 10:28 AM Ajit Khaparde <ajit.khaparde= @broadcom.com> wrote:

Hi David,
My Thursdays are crazy. Can we do Friday instead?
Juraj, can you make Friday 10AM (Eastern time) call?

Thanks
Ajit

On Tue, Mar 2, 2021 at 6:29 AM David Liu <dliu@iol.unh.edu> wrote:
>
> Hi Ajit and Juraj,
>
> I send out an invite for Thursday, Mar 4 at 10:00 AM. But feel free to= adjust the time or let me know if there is a better time that works for yo= u.
>
> For me, I can do:
> Tuesday 10:00 AM - 1:00 PM.
> Thursday 10:00 AM - 1:00 PM.
> Friday 10:00 AM - 12:00 PM and 2:00 PM - 5:00 PM
>
> Best,
> David
>
> On Mon, Mar 1, 2021 at 10:12 AM Ajit Khaparde <ajit.khaparde@broadcom.com&= gt; wrote:
>>
>> David,
>> I was busy with a customer issue and could not get to my emails. >> I am seeing this now. Let me close out any issues needing urgent a= ttention.
>> Let's sync on a suitable time after that.
>>
>> Juraj,
>> Sure. We will add you.
>>
>> Thanks
>> Ajit
>>
>> On Mon, Mar 1, 2021 at 7:00 AM Juraj Linke=C5=A1 <juraj.linkes@pantheon.te= ch> wrote:
>> >
>> > Hi Ajit, David,
>> >
>> >
>> >
>> > If you're setting up a call, could you please send me an = invite? I'd like to get some insight into the issues on Arm's behal= f.
>> >
>> >
>> >
>> > Thanks,
>> >
>> > Juraj
>> >
>> >
>> >
>> > From: ci <ci-bounces@dpdk.org> On Behalf Of David Liu
>> > Sent: Thursday, February 25, 2021 7:47 PM
>> > To: Ajit Khaparde <ajit.khaparde@broadcom.com>
>> > Cc: Owen Hilyard <ohilyard@iol.unh.edu>; ci@dpdk.org; dpdklab &= lt;dpdklab@iol.unh= .edu>
>> > Subject: Re: [dpdk-ci] [dpdklab] Re: [DPDKLAB] 25G Broadcom N= IC in Arm hardware
>> >
>> >
>> >
>> > Hi Ajit,
>> >
>> >
>> >
>> > Sorry for this issue.
>> >
>> > If possible I would like to set up a time where we can do thi= s in a call to help get this solved faster.
>> >
>> > Please let me know what would be a good time for you.
>> >
>> >
>> >
>> > Best,
>> >
>> > David
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Feb 25, 2021 at 12:55 PM David Liu <dliu@iol.unh.edu> wrote:
>> >
>> > Hi Ajit,
>> >
>> >
>> >
>> > I changed the permission to allow you to run sudo.
>> >
>> > This should work now.
>> >
>> > Let me know if you still have issues.
>> >
>> >
>> >
>> > Best,
>> >
>> > David
>> >
>> >
>> >
>> > On Mon, Feb 22, 2021 at 11:11 PM Ajit Khaparde <ajit.khaparde@broadco= m.com> wrote:
>> >
>> > David,
>> >
>> > I tried the option to use sudo. And that didn't work.
>> >
>> >
>> >
>> > Thanks
>> >
>> > Ajit
>> >
>> >
>> >
>> > ajitkhaparde is not allowed to run sudo on arm-gigabyte.=C2= =A0 This incident will be reported.
>> >
>> >
>> >
>> > ----
>> >
>> > sudo su
>> > [sudo] password for ajitkhaparde:
>> > ajitkhaparde is not allowed to run sudo on arm-dell.=C2=A0 Th= is incident will be reported.
>> > ajitkhaparde@arm-dell:/opt/dts$ sudo ./dts -s
>> > [sudo] password for ajitkhaparde:
>> > ajitkhaparde is not allowed to run sudo on arm-dell.=C2=A0 Th= is incident will be reported.
>> >
>> >
>> >
>> > On Thu, Feb 18, 2021 at 10:03 AM Ajit Khaparde <ajit.khaparde@broadco= m.com> wrote:
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Feb 18, 2021 at 10:01 AM David Liu <dliu@iol.unh.edu> wrote:
>> >
>> > Hi Ajit,
>> >
>> >
>> >
>> > This is likely caused by when dts try to ssh into another mac= hine as a normal user which will prompt you for the password.
>> >
>> >
>> >
>> > One solution would be "sudo su" as root to run the = command line since you should have root access.
>> >
>> > David,
>> >
>> > Let me try this method first.
>> >
>> > If it does not work, we could try with the public ssh keys. >> >
>> > I will let you know.
>> >
>> >
>> >
>> > Thanks
>> >
>> > Ajit
>> >
>> >
>> >
>> > Another solution would be to let me add your public ssh key i= nto both machines.
>> >
>> >
>> >
>> > Let me know if this helps.
>> >
>> >
>> >
>> > Best,
>> >
>> > David
>> >
>> >
>> >
>> > On Fri, Feb 12, 2021 at 5:48 PM Ajit Khaparde <ajit.khaparde@broadcom= .com> wrote:
>> >
>> > Thanks Owen.
>> >
>> >
>> >
>> > When I run the dts command line suggested, I get the followin= g error.
>> >
>> >
>> >
>> > Thanks
>> >
>> > Ajit
>> >
>> >
>> >
>> > Traceback (most recent call last):
>> >=C2=A0 =C2=A0File "/opt/dts/framework/ssh_pexpect.py"= ;, line 60, in _connect_host
>> >=C2=A0 =C2=A0 =C2=A0self.password, original_prompt=3D'[$#&= gt;]', password_regex=3Dr'(?i)(?:password:)|(?:passphrase for key)|= (?i)(password for .+:)')
>> >=C2=A0 =C2=A0File "/usr/local/lib/python3.6/dist-packages= /pexpect/pxssh.py", line 440, in login
>> >=C2=A0 =C2=A0 =C2=A0raise ExceptionPxssh('password refused= ')
>> > pexpect.pxssh.ExceptionPxssh: password refused
>> >
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 ^^^^^^^^^^^^^^
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Fri, Feb 12, 2021 at 12:09 PM Owen Hilyard <ohilyard@iol.unh.edu>= wrote:
>> >
>> > Hello Ajit,
>> >
>> >
>> >
>> > There was a misconfiguration in our identity management syste= m. The issue should now be resolved.
>> >
>> >
>> >
>> > Owen Hilyard
>> >
>> >
>> >
>> > On Thu, Feb 11, 2021 at 9:38 PM 'Ajit Khaparde' via d= pdklab <dpdklab= @iol.unh.edu> wrote:
>> >
>> > David,
>> >
>> > I can use my credentials to login to rhea.
>> >
>> > But the same credentials are not allowing me to login onto th= e servers you provided.
>> >
>> >
>> >
>> > Thanks
>> >
>> > Ajit
>> >
>> >
>> >
>> > ssh -l ajitkhaparde arm-gigabyte.dpdklab.iol.unh.edu
>> > Connection closed by 172.18.0.41 port 22
>> >
>> >
>> >
>> > ssh -l ajitkhaparde arm-dell.dpdklab.iol.unh.edu
>> > Connection closed by 172.18.0.40 port 22
>> >
>> >
>> >
>> > On Tue, Feb 9, 2021 at 6:36 AM David Liu <dliu@iol.unh.edu> wrote:
>> >
>> > Hi Ajit,
>> >
>> >
>> >
>> > Thanks for the update. Let me know if you need anything from = me.
>> >
>> >
>> >
>> >
>> >
>> > Best,
>> >
>> > David
>> >
>> >
>> >
>> > On Sun, Feb 7, 2021 at 11:04 PM Ajit Khaparde <ajit.khaparde@broadcom= .com> wrote:
>> >
>> > David,
>> >
>> > I had some trouble with my system over the past week.
>> >
>> > So I could not login into the setup. I am hoping that my syst= em will be restored early this week
>> >
>> > I will login into the setup after that.
>> >
>> >
>> >
>> > Thanks
>> >
>> > Ajit
>> >
>> >
>> >
>> > On Tue, Feb 2, 2021 at 10:57 AM Ajit Khaparde <ajit.khaparde@broadcom= .com> wrote:
>> >
>> > Hi David,
>> > I actually did miss it. I will take a look.
>> > Thanks for the refresh mail.
>> >
>> > Thanks
>> > Ajit
>> >
>> > On Tue, Feb 2, 2021 at 8:58 AM David Liu <dliu@iol.unh.edu> wrote:
>> > >
>> > > Hi Ajit,
>> > >
>> > > Ping again in case if you miss the last email.
>> > >
>> > > Did you have a chance to give it a try on login to the m= achines?
>> > >
>> > > Just some notes, we are currently running compile and so= me performance/functional testing.
>> > > Which it may be hard for you to run dts, so please let m= e know if you want me to stop these tests for you to debug.
>> > >
>> > > Step I run:
>> > > CONF=3D"conf_dpdk" ./switch.sh Broadcom=C2=A0 = =C2=A0#this is to switch the dts configuration to Broadcom
>> > > ./dts -s=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0#To start the dts
>> > >
>> > > Informatio:
>> > > arm-dell.dpdklab.iol.unh.edu
>> > > - dts - located in /opt/dts
>> > > - Trex - located in /opt/trex/v2.87
>> > >
>> > > arm-gigabyte.dpdklab.iol.unh.edu
>> > > - dpdk - located in /root/dpdk
>> > >
>> > > Best,
>> > > David
>> > >
>> > > On Tue, Jan 26, 2021 at 4:59 PM David Liu <dliu@iol.unh.edu> wrote:=
>> > >>
>> > >> Also, I had changed the configuration when I run Bro= adcom nic.
>> > >> In the dts folder, you can start it with ./dts -s >> > >>
>> > >> On Tue, Jan 26, 2021 at 4:54 PM David Liu <dliu@iol.unh.edu> wr= ote:
>> > >>>
>> > >>> Hi Ajit,
>> > >>> Sorry for the late reply.
>> > >>>
>> > >>> You can ssh into the machine using the same VPN = Username and Password.
>> > >>> Can you try and see if that works?
>> > >>>
>> > >>> Informatio:
>> > >>> arm-dell.dpdklab.iol.unh.edu
>> > >>> - dts - located in /opt/dts
>> > >>> - Trex - located in /opt/trex/v2.87
>> > >>>
>> > >>> arm-gigabyte.dpdklab.iol.unh.edu
>> > >>> - dpdk - located in /root/dpdk
>> > >>>
>> > >>>
>> > >>> Thanks,
>> > >>> David
>> > >>>
>> > >>> On Tue, Jan 19, 2021 at 5:20 PM Ajit Khaparde &l= t;ajit.khap= arde@broadcom.com> wrote:
>> > >>>>
>> > >>>> Hi David,
>> > >>>> Thanks for pinging me. Do you think I could = login into the setup and
>> > >>>> see what is happening?
>> > >>>> I do have the credentials for two systems in= your lab -
>> > >>>> rhea.dpdklab.iol.unh.edu and io.dpdklab.iol.= unh.edu
>> > >>>> If you can provide me details of the systems= you are trying to
>> > >>>> configure, I could take a look.
>> > >>>>
>> > >>>> Thanks
>> > >>>> Ajit
>> > >>>>
>> > >>>> On Tue, Jan 19, 2021 at 1:41 PM David Liu &l= t;dliu@iol.unh.edu> wrote:
>> > >>>> >
>> > >>>> > Hi Ajit,
>> > >>>> >
>> > >>>> > Thank you for helping out.
>> > >>>> >
>> > >>>> > How are you doing with this issue?
>> > >>>> >
>> > >>>> > Please let me know if you need anything= information from us.
>> > >>>> >
>> > >>>> > Just some extra information that both T= rex and dts are on the same intel tester machine.
>> > >>>> >
>> > >>>> > Best,
>> > >>>> > David
>> > >>>> >
>> > >>>> > On Thu, Jan 14, 2021 at 2:12 PM David L= iu <dliu@iol.unh.e= du> wrote:
>> > >>>> >>
>> > >>>> >> Yes, so this is the same NIC work o= n x86.
>> > >>>> >> By seeing the link, I am assuming y= ou mean the ports that are using for Trex. If so they are UP before I start= the Trex.
>> > >>>> >>
>> > >>>> >> Thank,
>> > >>>> >> David
>> > >>>> >>
>> > >>>> >>
>> > >>>> >> On Thu, Jan 14, 2021 at 1:11 AM Aji= t Khaparde <ajit.khaparde@broadcom.com> wrote:
>> > >>>> >>>
>> > >>>> >>> Does the same NIC work on x86?<= br> >> > >>>> >>> Also do you see link before you= start the trex? I am trying to see if the port failed to successfully come= up
>> > >>>> >>> or if there is a problem with t= he link. Can you check and tell?
>> > >>>> >>>
>> > >>>> >>> Thanks
>> > >>>> >>> Ajit
>> > >>>> >>>
>> > >>>> >>> On Wed, Jan 13, 2021 at 4:20 PM= David Liu <dliu@i= ol.unh.edu> wrote:
>> > >>>> >>>>
>> > >>>> >>>> Hi Ajit,
>> > >>>> >>>>
>> > >>>> >>>> Thank you for helping out.<= br> >> > >>>> >>>>
>> > >>>> >>>> We have it working on the x= 86, but not the Arm.
>> > >>>> >>>>
>> > >>>> >>>>
>> > >>>> >>>> Thanks,
>> > >>>> >>>> David
>> > >>>> >>>>
>> > >>>> >>>> On Wed, Jan 13, 2021 at 5:3= 4 PM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:
>> > >>>> >>>>>
>> > >>>> >>>>> Hi David,
>> > >>>> >>>>> I will take a look.
>> > >>>> >>>>> Do you see similar issu= es on x86? I am asking because
>> > >>>> >>>>> even I would start with= that to create a baseline and then
>> > >>>> >>>>> attempt on ARM.
>> > >>>> >>>>>
>> > >>>> >>>>> Thanks
>> > >>>> >>>>> Ajit
>> > >>>> >>>>>
>> > >>>> >>>>> On Wed, Jan 13, 2021 at= 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
>> > >>>> >>>>>>
>> > >>>> >>>>>> Hi Ajit,
>> > >>>> >>>>>>
>> > >>>> >>>>>> We have a 25G Broad= com NIC that is installed on an Arm machine.
>> > >>>> >>>>>> And I am running in= to a problem when I try to run nic_single_core_perf on the NIC.
>> > >>>> >>>>>> Currently, we are u= sing: Trex v2.86 and dts from http://git.dpdk.org/tools/dts/
>> > >>>> >>>>>> All the NICs are up= and running when testing.
>> > >>>> >>>>>>
>> > >>>> >>>>>> The problem is an e= rror when I try to run the test case inside the nic_single_core_perf:
>> > >>>> >>>>>>
>> > >>>> >>>>>>> TestNicSingleCo= rePerf: Test running at parameters: framesize: 64, rxd/txd: 512
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dut.172.18.0.41: x86_64-native-li= nuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4=C2=A0 --file-prefix=3Ddpdk_15311_= 20210113190237=C2=A0 =C2=A0-a 0000:93:00.0 -a 0000:93:00.1 -- -i --portmask= =3D0x3 --txd=3D512 --rxd=3D512 --nb-cores=3D1
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dut.172.18.0.41: start
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 TestNicSingleCorePerf: Test Case test_perf_nic_single_= core Result ERROR: Traceback (most recent call last):
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/dts/framework/test_case.py", line 319, in _execute_test_c= ase
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0case_obj()
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "tests/TestSuite_nic_single_core_perf.py", line 196, in test_pe= rf_nic_single_core
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0self.perf_test(self.nb_ports)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "tests/TestSuite_nic_single_core_perf.py", line 270, in perf_te= st
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0_, packets_received =3D self.tester.pktgen.measure_throughput(stream_= ids=3Dstreams, options=3Dtraffic_opt)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/dts/framework/pktgen_base.py", line 245, in measure_throu= ghput
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0self._prepare_transmission(stream_ids=3Dstream_ids)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/dts/framework/pktgen_trex.py", line 778, in _prepare_tran= smission
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0self._conn.reset(ports=3Dself._ports)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/comm= on/trex_api_annotators.py", line 51, in wrap2
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0ret =3D f(*args, **kwargs)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/= trex_stl_client.py", line 339, in reset
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0self.clear_stats(ports)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/comm= on/trex_api_annotators.py", line 51, in wrap2
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0ret =3D f(*args, **kwargs)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/= trex_stl_client.py", line 1492, in clear_stats
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0self._clear_stats_common(ports, clear_global, clear_xstats)
>> > >>>> >>>>>>>=C2=A0 =C2=A0Fil= e "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/comm= on/trex_client.py", line 2876, in _clear_stats_common
>> > >>>> >>>>>>>=C2=A0 =C2=A0 = =C2=A0raise TRexError(rc)
>> > >>>> >>>>>>> trex.common.tre= x_exceptions.TRexError: *** [RPC] - Failed to get server response from tcp:= //172.18.0.40:4501
>> > >>>> >>>>>>
>> > >>>> >>>>>>
>> > >>>> >>>>>> I also try to run T= Rex in stateless mode with./t-rex-64 -i --cfg /etc/trex_cfg.yaml and=C2=A0 = connect with ./trex-console
>> > >>>> >>>>>>
>> > >>>> >>>>>> Then start sending = traffic with command
>> > >>>> >>>>>>>
>> > >>>> >>>>>>> trex>start -= f stl/imix.py
>> > >>>> >>>>>>
>> > >>>> >>>>>>
>> > >>>> >>>>>> But this will cause= an error to turn off Trex.
>> > >>>> >>>>>>
>> > >>>> >>>>>>> trex>
>> > >>>> >>>>>>> 2021-01-13 19:5= 6:21 - [server][warning]=C2=A0 - Server has been shutdown - cause: 'WAT= CHDOG: task 'Trex DP core 1' has not responded for more than 1.0013= 5 seconds - timeout is 1 seconds
>> > >>>> >>>>>>> *** traceback f= ollows ***
>> > >>>> >>>>>>> 1=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>> > >>>> >>>>>>> 2=C2=A0 =C2=A0 = =C2=A0 =C2=A00x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)= [0x7feea3ac0980]
>> > >>>> >>>>>>> 3=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be1a05a2a rte_delay_us_block + 106
>> > >>>> >>>>>>> 4=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned = short, CVirtualIFPerSideStats*) + 3220
>> > >>>> >>>>>>> 5=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>> > >>>> >>>>>>> 6=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be171e3d9 CNodeGenerator::handle_maintenance(CFlowGenList= PerThread*) + 265
>> > >>>> >>>>>>> 7=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*, CFl= owGenListPerThread*, bool&) + 92
>> > >>>> >>>>>>> 8=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned c= har, CGenNode*, CFlowGenListPerThread*, bool) + 184
>> > >>>> >>>>>>> 9=C2=A0 =C2=A0 = =C2=A0 =C2=A00x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23, = false>(double, double, CFlowGenListPerThread*, double&) + 881
>> > >>>> >>>>>>> 10=C2=A0 =C2=A0= =C2=A0 0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>> > >>>> >>>>>>> 11=C2=A0 =C2=A0= =C2=A0 0x561be1883ae9 TrexDpCore::start() + 89
>> > >>>> >>>>>>> 12=C2=A0 =C2=A0= =C2=A0 0x561be1714113 CFlowGenListPerThread::start(std::__cxx11::basic_str= ing<char, std::char_traits<char>, std::allocator<char> >&= amp;, CPreviewMode&) + 115
>> > >>>> >>>>>>> 13=C2=A0 =C2=A0= =C2=A0 0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>> > >>>> >>>>>>> 14=C2=A0 =C2=A0= =C2=A0 0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>> > >>>> >>>>>>> 15=C2=A0 =C2=A0= =C2=A0 0x561be1a1dfaa eal_thread_loop + 426
>> > >>>> >>>>>>> 16=C2=A0 =C2=A0= =C2=A0 0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f= eea3ab56db]
>> > >>>> >>>>>>> 17=C2=A0 =C2=A0= =C2=A0 0x7feea2a8571f clone + 63
>> > >>>> >>>>>>>
>> > >>>> >>>>>>> *** addr2line i= nformation follows ***
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> ??:0
>> > >>>> >>>>>>> '
>> > >>>> >>>>>>
>> > >>>> >>>>>>
>> > >>>> >>>>>> I believe if everyt= hing is working, this should not turn off trex but please correct me if I a= m wrong.
>> > >>>> >>>>>> I wonder if you hav= e any suggestion of how I can approach this issue?
>> > >>>> >>>>>>
>> > >>>> >>>>>> Best,
>> > >>>> >>>>>> David
>> > >>>> >>>>>>
>> > >>>> >>>>>>

--00000000000000d5f605bcd0c87e--