From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4CF0EA0A05 for ; Tue, 19 Jan 2021 22:41:04 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 25A04140F9F; Tue, 19 Jan 2021 22:41:04 +0100 (CET) Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by mails.dpdk.org (Postfix) with ESMTP id A5EBE140F9E for ; Tue, 19 Jan 2021 22:41:02 +0100 (CET) Received: by mail-lf1-f42.google.com with SMTP id s26so31185212lfc.8 for ; Tue, 19 Jan 2021 13:41:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cBXAI5yts+IPepfhMtiGCvC+9DjSLcWJoqgsJloWZV8=; b=YI426N13F/ccX7ICAEDrWnPpPMWdQ/cTXLZON2CFq74Rnljt8/yYDEaHFAXgBeotos SITuTaBL77BUpv6KqsZ75iYLuvcqu5hLCrqkXm0ILv0GodWm+NXKVZjab4mrBbuGxjXM w7uUicJUfOuOQvlpCil4+Z/bfUV+20BujuqFU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cBXAI5yts+IPepfhMtiGCvC+9DjSLcWJoqgsJloWZV8=; b=kriVQ8vcXr4gsKiJmnfFODwiXKDHkhIGDr1fQU8CdZW5TEaXnPy6zdPXYLs5Mc7MA4 4zomJNIeEqiq/wAbRVgwPk/LCUFtZ9BS6R8mlfyHqFOQgHhC3BS3tOlHoVq9xrvN+d4+ O/sdDFLfMubgdzqI3Doh4cU1nMGzhlBqJWZD62UbgDrcAn1GOFuekHxymGRv9nCXQOXt hH7xIWt0QHUbYNnGucbY0/5q0bQxbbgilGjBO/12FJLgBjDWnO1vl9spl8GX/PVvZDBF HgyBKR1X5MR6dECltrBHd3tr0M9Oz0fc6R74a4MD6QLymnjVcUaT6q1aVduyRNEiha06 Oo9w== X-Gm-Message-State: AOAM531ecAHW0AeW3CLfGOXPVS7zK+Afwnb5zWDr5JqWxMP7Oq6ySjmm AGcJSmBuI8IdbMdC5G9LQ7wLzEDm1NteD6rHLCRfGQ== X-Google-Smtp-Source: ABdhPJzT64Tp5IXE6lzeCeOepcvEq2yConf695eg0jGh5APxykAgRsHV/DCvlL7MRs3j6rOeexP9AaWqe7GSjLWic/k= X-Received: by 2002:ac2:5309:: with SMTP id c9mr2795049lfh.447.1611092462144; Tue, 19 Jan 2021 13:41:02 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: David Liu Date: Tue, 19 Jan 2021 16:40:26 -0500 Message-ID: To: Ajit Khaparde Cc: ci@dpdk.org, dpdklab Content-Type: multipart/alternative; boundary="000000000000e823aa05b947b1bf" Subject: Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Sender: "ci" --000000000000e823aa05b947b1bf Content-Type: text/plain; charset="UTF-8" Hi Ajit, Thank you for helping out. How are you doing with this issue? Please let me know if you need anything information from us. Just some extra information that both Trex and dts are on the same intel tester machine. Best, David On Thu, Jan 14, 2021 at 2:12 PM David Liu wrote: > Yes, so this is the same NIC work on x86. > By seeing the link, I am assuming you mean the ports that are using for > Trex. If so they are UP before I start the Trex. > > Thank, > David > > > On Thu, Jan 14, 2021 at 1:11 AM Ajit Khaparde > wrote: > >> Does the same NIC work on x86? >> Also do you see link before you start the trex? I am trying to see if the >> port failed to successfully come up >> or if there is a problem with the link. Can you check and tell? >> >> Thanks >> Ajit >> >> On Wed, Jan 13, 2021 at 4:20 PM David Liu wrote: >> >>> Hi Ajit, >>> >>> Thank you for helping out. >>> >>> We have it working on the x86, but not the Arm. >>> >>> >>> Thanks, >>> David >>> >>> On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde < >>> ajit.khaparde@broadcom.com> wrote: >>> >>>> Hi David, >>>> I will take a look. >>>> Do you see similar issues on x86? I am asking because >>>> even I would start with that to create a baseline and then >>>> attempt on ARM. >>>> >>>> Thanks >>>> Ajit >>>> >>>> On Wed, Jan 13, 2021 at 12:08 PM David Liu wrote: >>>> >>>>> Hi Ajit, >>>>> >>>>> We have a 25G Broadcom NIC that is installed on an Arm machine. >>>>> And I am running into a problem when I try to run nic_single_core_perf >>>>> on the NIC. >>>>> Currently, we are using: Trex v2.86 and dts from >>>>> http://git.dpdk.org/tools/dts/ >>>>> All the NICs are up and running when testing. >>>>> >>>>> The problem is an error when I try to run the test case inside >>>>> the nic_single_core_perf: >>>>> >>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64, >>>>>> rxd/txd: 512 >>>>>> dut.172.18.0.41: >>>>>> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4 >>>>>> --file-prefix=dpdk_15311_20210113190237 -a 0000:93:00.0 -a 0000:93:00.1 >>>>>> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1 >>>>>> dut.172.18.0.41: start >>>>>> TestNicSingleCorePerf: Test Case test_perf_nic_single_core >>>>>> Result ERROR: Traceback (most recent call last): >>>>>> File "/opt/dts/framework/test_case.py", line 319, in >>>>>> _execute_test_case >>>>>> case_obj() >>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 196, in >>>>>> test_perf_nic_single_core >>>>>> self.perf_test(self.nb_ports) >>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 270, in >>>>>> perf_test >>>>>> _, packets_received = >>>>>> self.tester.pktgen.measure_throughput(stream_ids=streams, >>>>>> options=traffic_opt) >>>>>> File "/opt/dts/framework/pktgen_base.py", line 245, in >>>>>> measure_throughput >>>>>> self._prepare_transmission(stream_ids=stream_ids) >>>>>> File "/opt/dts/framework/pktgen_trex.py", line 778, in >>>>>> _prepare_transmission >>>>>> self._conn.reset(ports=self._ports) >>>>>> File >>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", >>>>>> line 51, in wrap2 >>>>>> ret = f(*args, **kwargs) >>>>>> File >>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", >>>>>> line 339, in reset >>>>>> self.clear_stats(ports) >>>>>> File >>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", >>>>>> line 51, in wrap2 >>>>>> ret = f(*args, **kwargs) >>>>>> File >>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", >>>>>> line 1492, in clear_stats >>>>>> self._clear_stats_common(ports, clear_global, clear_xstats) >>>>>> File >>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py", >>>>>> line 2876, in _clear_stats_common >>>>>> raise TRexError(rc) >>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get >>>>>> server response from tcp://172.18.0.40:4501 >>>>> >>>>> >>>>> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg >>>>> /etc/trex_cfg.yaml and connect with ./trex-console >>>>> >>>>> Then start sending traffic with command >>>>> >>>>>> trex>start -f stl/imix.py >>>>> >>>>> >>>>> But this will cause an error to turn off Trex. >>>>> >>>>> trex> >>>>>> 2021-01-13 19:56:21 - [server][warning] - Server has been shutdown - >>>>>> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than >>>>>> 1.00135 seconds - timeout is 1 seconds >>>>>> *** traceback follows *** >>>>>> 1 0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a] >>>>>> 2 0x7feea3ac0980 >>>>>> /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7feea3ac0980] >>>>>> 3 0x561be1a05a2a rte_delay_us_block + 106 >>>>>> 4 0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned >>>>>> short, CVirtualIFPerSideStats*) + 3220 >>>>>> 5 0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31 >>>>>> 6 0x561be171e3d9 >>>>>> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265 >>>>>> 7 0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*, >>>>>> CFlowGenListPerThread*, bool&) + 92 >>>>>> 8 0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned >>>>>> char, CGenNode*, CFlowGenListPerThread*, bool) + 184 >>>>>> 9 0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23, >>>>>> false>(double, double, CFlowGenListPerThread*, double&) + 881 >>>>>> 10 0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226 >>>>>> 11 0x561be1883ae9 TrexDpCore::start() + 89 >>>>>> 12 0x561be1714113 >>>>>> CFlowGenListPerThread::start(std::__cxx11::basic_string>>>>> std::char_traits, std::allocator >&, CPreviewMode&) + 115 >>>>>> 13 0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487 >>>>>> 14 0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad] >>>>>> 15 0x561be1a1dfaa eal_thread_loop + 426 >>>>>> 16 0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) >>>>>> [0x7feea3ab56db] >>>>>> 17 0x7feea2a8571f clone + 63 >>>>>> >>>>>> *** addr2line information follows *** >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ??:0 >>>>>> ' >>>>> >>>>> >>>>> I believe if everything is working, this should not turn off trex but >>>>> please correct me if I am wrong. >>>>> I wonder if you have any suggestion of how I can approach this issue? >>>>> >>>>> Best, >>>>> David >>>>> >>>>> >>>>> --000000000000e823aa05b947b1bf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Ajit,

Thank you for helping out.

How are you doing with this issue?

Please let me know if you need anything information from us.

Just some extra information=C2=A0that both Trex and dts a= re on the same intel=C2=A0tester machine.

Best,
David

On Thu, Jan 14, 2021 at 2:12 PM David Liu <dliu@iol.unh.edu> wrote:=
Yes, so this is the same NIC work on x86.
By seeing the link, I am as= suming=C2=A0you=C2=A0mean the ports that are using for Trex. If so they are= UP before I start the Trex.

Thank,
Davi= d


On Thu, Jan 14, 2021 at 1:11 AM Ajit Khaparde <ajit.khaparde@= broadcom.com> wrote:
Does the same NIC work on x86?
Also do you see link b= efore you start the trex? I am trying to see if the port failed to successf= ully come up
or if there is a problem with the link. Can you check and tell?=
=
Thanks
Ajit

On Wed, Jan 13, 2021 at 4:20 PM David Liu <dliu@iol.unh.edu> wro= te:
Hi Ajit,

Thank you for helping out.

<= /div>
We have it working on the x86, but not the Arm.


Thanks,
David

On Wed, Jan 13, 2021 at= 5:34 PM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:
Hi=C2=A0David,
I will t= ake a look.
Do you see similar issues on x86? I am asking because
even I woul= d start with that to create a baseline and then
attempt on ARM.

Thank= s
Ajit

On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
Hi Ajit,

We have a 25G = Broadcom NIC that is installed on an Arm machine.
And=C2=A0I am running into a problem when I try to= run nic_single_core_perf on the NIC.
Currently, we are using: Trex v2.86 and dts from=C2=A0http://git.dpdk.org/too= ls/dts/
All the NICs = are up and running when testing.

The problem is an error when I try to run the test c= ase inside the=C2=A0nic_single_core_perf:

TestNicSingleCorePerf: Test r= unning at parameters: framesize: 64, rxd/txd: 512
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0dut.172.18.0.41: x86_64-native-linuxapp-g= cc/app/dpdk-testpmd -l 32,33 -n 4 =C2=A0--file-prefix=3Ddpdk_15311_20210113= 190237 =C2=A0 -a 0000:93:00.0 -a 0000:93:00.1 -- -i --portmask=3D0x3 --txd= =3D512 --rxd=3D512 --nb-cores=3D1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0dut.172.18.0.41: start
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR:= Traceback (most recent call last):
=C2=A0 File "/opt/dts/framework= /test_case.py", line 319, in _execute_test_case
=C2=A0 =C2=A0 case_= obj()
=C2=A0 File "tests/TestSuite_nic_single_core_perf.py", l= ine 196, in test_perf_nic_single_core
=C2=A0 =C2=A0 self.perf_test(self.= nb_ports)
=C2=A0 File "tests/TestSuite_nic_single_core_perf.py"= ;, line 270, in perf_test
=C2=A0 =C2=A0 _, packets_received =3D self.tes= ter.pktgen.measure_throughput(stream_ids=3Dstreams, options=3Dtraffic_opt)<= br>=C2=A0 File "/opt/dts/framework/pktgen_base.py", line 245, in = measure_throughput
=C2=A0 =C2=A0 self._prepare_transmission(stream_ids= =3Dstream_ids)
=C2=A0 File "/opt/dts/framework/pktgen_trex.py"= , line 778, in _prepare_transmission
=C2=A0 =C2=A0 self._conn.reset(port= s=3Dself._ports)
=C2=A0 File "/opt/trex/v2.86/automation/trex_contr= ol_plane/interactive/trex/common/trex_api_annotators.py", line 51, in = wrap2
=C2=A0 =C2=A0 ret =3D f(*args, **kwargs)
=C2=A0 File "/opt= /trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_cli= ent.py", line 339, in reset
=C2=A0 =C2=A0 self.clear_stats(ports)=C2=A0 File "/opt/trex/v2.86/automation/trex_control_plane/interacti= ve/trex/common/trex_api_annotators.py", line 51, in wrap2
=C2=A0 = =C2=A0 ret =3D f(*args, **kwargs)
=C2=A0 File "/opt/trex/v2.86/auto= mation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", li= ne 1492, in clear_stats
=C2=A0 =C2=A0 self._clear_stats_common(ports, cl= ear_global, clear_xstats)
=C2=A0 File "/opt/trex/v2.86/automation/t= rex_control_plane/interactive/trex/common/trex_client.py", line 2876, = in _clear_stats_common
=C2=A0 =C2=A0 raise TRexError(rc)
trex.common.= trex_exceptions.TRexError: *** [RPC] - Failed to get server response from t= cp://172.18.0.40:4501=
<= /blockquote>
=
=
<= /blockquote>
=

I also try to run=C2=A0TRex in stateless mode with./t-rex-64 -i --cfg /etc/trex_cfg.yaml=C2=A0and=C2=A0 connect with=C2=A0./trex-console

<= div>Th= en start sending traffic with command
trex>start -f stl/imix.py=C2=A0=
=C2=A0
= But this will cause an error to turn off T= rex.

trex>
2021-01-13 19:56:21 - [server][warning] =C2=A0- Serv= er has been shutdown - cause: 'WATCHDOG: task 'Trex DP core 1' = has not responded for more than 1.00135 seconds - timeout is 1 seconds
*= ** traceback follows ***
1 =C2=A0 =C2=A0 =C2=A0 0x561be173cf5a ./_t-rex-= 64(+0x19af5a) [0x561be173cf5a]
2 =C2=A0 =C2=A0 =C2=A0 0x7feea3ac0980 /li= b/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7feea3ac0980]
3 =C2=A0 = =C2=A0 =C2=A0 0x561be1a05a2a rte_delay_us_block + 106
4 =C2=A0 =C2=A0 = =C2=A0 0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned short,= CVirtualIFPerSideStats*) + 3220
5 =C2=A0 =C2=A0 =C2=A0 0x561be16abf57 C= CoreEthIF::flush_tx_queue() + 31
6 =C2=A0 =C2=A0 =C2=A0 0x561be171e3d9 C= NodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
7 =C2=A0= =C2=A0 =C2=A0 0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*, C= FlowGenListPerThread*, bool&) + 92
8 =C2=A0 =C2=A0 =C2=A0 0x561be171= fc98 CNodeGenerator::handle_slow_messages(unsigned char, CGenNode*, CFlowGe= nListPerThread*, bool) + 184
9 =C2=A0 =C2=A0 =C2=A0 0x561be16cb5f1 int C= NodeGenerator::flush_file_realtime<23, false>(double, double, CFlowGe= nListPerThread*, double&) + 881
10 =C2=A0 =C2=A0 =C2=A00x561be190521= 2 TrexStatelessDpCore::start_scheduler() + 226
11 =C2=A0 =C2=A0 =C2=A00x= 561be1883ae9 TrexDpCore::start() + 89
12 =C2=A0 =C2=A0 =C2=A00x561be1714= 113 CFlowGenListPerThread::start(std::__cxx11::basic_string<char, std::c= har_traits<char>, std::allocator<char> >&, CPreviewMode&= amp;) + 115
13 =C2=A0 =C2=A0 =C2=A00x561be16af8dd CGlobalTRex::run_in_co= re(unsigned char) + 487
14 =C2=A0 =C2=A0 =C2=A00x561be16d11ad ./_t-rex-6= 4(+0x12f1ad) [0x561be16d11ad]
15 =C2=A0 =C2=A0 =C2=A00x561be1a1dfaa eal_= thread_loop + 426
16 =C2=A0 =C2=A0 =C2=A00x7feea3ab56db /lib/x86_64-linu= x-gnu/libpthread.so.0(+0x76db) [0x7feea3ab56db]
17 =C2=A0 =C2=A0 =C2=A00= x7feea2a8571f clone + 63

*** addr2line information follows ***
??= :0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0<= br>??:0
??:0
??:0
??:0
??:0
??:0
??:0
'
=C2=A0
I believe if e= verything is working, this should not turn off trex but please correct me i= f I am wrong.
I wonder if you have any suggestion of how I ca= n approach=C2=A0this issue?

Best,
David


<= /code>
--000000000000e823aa05b947b1bf--