DPDK CI discussions
 help / color / mirror / Atom feed
* [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
@ 2021-01-13 20:07 David Liu
  2021-01-13 22:34 ` Ajit Khaparde
  0 siblings, 1 reply; 7+ messages in thread
From: David Liu @ 2021-01-13 20:07 UTC (permalink / raw)
  To: Ajit Khaparde; +Cc: ci, dpdklab

[-- Attachment #1: Type: text/plain, Size: 4686 bytes --]

Hi Ajit,

We have a 25G Broadcom NIC that is installed on an Arm machine.
And I am running into a problem when I try to run nic_single_core_perf on
the NIC.
Currently, we are using: Trex v2.86 and dts from
http://git.dpdk.org/tools/dts/
All the NICs are up and running when testing.

The problem is an error when I try to run the test case inside
the nic_single_core_perf:

TestNicSingleCorePerf: Test running at parameters: framesize: 64, rxd/txd:
> 512
>                dut.172.18.0.41:
> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4
>  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1
> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>                dut.172.18.0.41: start
>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result
> ERROR: Traceback (most recent call last):
>   File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
>     case_obj()
>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in
> test_perf_nic_single_core
>     self.perf_test(self.nb_ports)
>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in perf_test
>     _, packets_received =
> self.tester.pktgen.measure_throughput(stream_ids=streams,
> options=traffic_opt)
>   File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
>     self._prepare_transmission(stream_ids=stream_ids)
>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
> _prepare_transmission
>     self._conn.reset(ports=self._ports)
>   File
> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
> line 51, in wrap2
>     ret = f(*args, **kwargs)
>   File
> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
> line 339, in reset
>     self.clear_stats(ports)
>   File
> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
> line 51, in wrap2
>     ret = f(*args, **kwargs)
>   File
> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
> line 1492, in clear_stats
>     self._clear_stats_common(ports, clear_global, clear_xstats)
>   File
> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py",
> line 2876, in _clear_stats_common
>     raise TRexError(rc)
> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server
> response from tcp://172.18.0.40:4501


I also try to run TRex in stateless mode with./t-rex-64 -i --cfg
/etc/trex_cfg.yaml and  connect with ./trex-console

Then start sending traffic with command

> trex>start -f stl/imix.py


But this will cause an error to turn off Trex.

trex>
> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown -
> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than
> 1.00135 seconds - timeout is 1 seconds
> *** traceback follows ***
> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
> 2       0x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)
> [0x7feea3ac0980]
> 3       0x561be1a05a2a rte_delay_us_block + 106
> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned
> short, CVirtualIFPerSideStats*) + 3220
> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
> 6       0x561be171e3d9
> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*,
> CFlowGenListPerThread*, bool&) + 92
> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned char,
> CGenNode*, CFlowGenListPerThread*, bool) + 184
> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23,
> false>(double, double, CFlowGenListPerThread*, double&) + 881
> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
> 11      0x561be1883ae9 TrexDpCore::start() + 89
> 12      0x561be1714113
> CFlowGenListPerThread::start(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
> 15      0x561be1a1dfaa eal_thread_loop + 426
> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)
> [0x7feea3ab56db]
> 17      0x7feea2a8571f clone + 63
>
> *** addr2line information follows ***
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> ??:0
> '


I believe if everything is working, this should not turn off trex but
please correct me if I am wrong.
I wonder if you have any suggestion of how I can approach this issue?

Best,
David

[-- Attachment #2: Type: text/html, Size: 14428 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
  2021-01-13 20:07 [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware David Liu
@ 2021-01-13 22:34 ` Ajit Khaparde
  2021-01-14  0:19   ` David Liu
  0 siblings, 1 reply; 7+ messages in thread
From: Ajit Khaparde @ 2021-01-13 22:34 UTC (permalink / raw)
  To: David Liu; +Cc: ci, dpdklab


[-- Attachment #1.1: Type: text/plain, Size: 5086 bytes --]

Hi David,
I will take a look.
Do you see similar issues on x86? I am asking because
even I would start with that to create a baseline and then
attempt on ARM.

Thanks
Ajit

On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:

> Hi Ajit,
>
> We have a 25G Broadcom NIC that is installed on an Arm machine.
> And I am running into a problem when I try to run nic_single_core_perf on
> the NIC.
> Currently, we are using: Trex v2.86 and dts from
> http://git.dpdk.org/tools/dts/
> All the NICs are up and running when testing.
>
> The problem is an error when I try to run the test case inside
> the nic_single_core_perf:
>
> TestNicSingleCorePerf: Test running at parameters: framesize: 64, rxd/txd:
>> 512
>>                dut.172.18.0.41:
>> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4
>>  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1
>> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>>                dut.172.18.0.41: start
>>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core
>> Result ERROR: Traceback (most recent call last):
>>   File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
>>     case_obj()
>>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in
>> test_perf_nic_single_core
>>     self.perf_test(self.nb_ports)
>>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in perf_test
>>     _, packets_received =
>> self.tester.pktgen.measure_throughput(stream_ids=streams,
>> options=traffic_opt)
>>   File "/opt/dts/framework/pktgen_base.py", line 245, in
>> measure_throughput
>>     self._prepare_transmission(stream_ids=stream_ids)
>>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
>> _prepare_transmission
>>     self._conn.reset(ports=self._ports)
>>   File
>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>> line 51, in wrap2
>>     ret = f(*args, **kwargs)
>>   File
>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>> line 339, in reset
>>     self.clear_stats(ports)
>>   File
>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>> line 51, in wrap2
>>     ret = f(*args, **kwargs)
>>   File
>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>> line 1492, in clear_stats
>>     self._clear_stats_common(ports, clear_global, clear_xstats)
>>   File
>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py",
>> line 2876, in _clear_stats_common
>>     raise TRexError(rc)
>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server
>> response from tcp://172.18.0.40:4501
>
>
> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg
> /etc/trex_cfg.yaml and  connect with ./trex-console
>
> Then start sending traffic with command
>
>> trex>start -f stl/imix.py
>
>
> But this will cause an error to turn off Trex.
>
> trex>
>> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown -
>> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than
>> 1.00135 seconds - timeout is 1 seconds
>> *** traceback follows ***
>> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>> 2       0x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)
>> [0x7feea3ac0980]
>> 3       0x561be1a05a2a rte_delay_us_block + 106
>> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned
>> short, CVirtualIFPerSideStats*) + 3220
>> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>> 6       0x561be171e3d9
>> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
>> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*,
>> CFlowGenListPerThread*, bool&) + 92
>> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned
>> char, CGenNode*, CFlowGenListPerThread*, bool) + 184
>> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23,
>> false>(double, double, CFlowGenListPerThread*, double&) + 881
>> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>> 11      0x561be1883ae9 TrexDpCore::start() + 89
>> 12      0x561be1714113
>> CFlowGenListPerThread::start(std::__cxx11::basic_string<char,
>> std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
>> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>> 15      0x561be1a1dfaa eal_thread_loop + 426
>> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)
>> [0x7feea3ab56db]
>> 17      0x7feea2a8571f clone + 63
>>
>> *** addr2line information follows ***
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> ??:0
>> '
>
>
> I believe if everything is working, this should not turn off trex but
> please correct me if I am wrong.
> I wonder if you have any suggestion of how I can approach this issue?
>
> Best,
> David
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 15677 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4175 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
  2021-01-13 22:34 ` Ajit Khaparde
@ 2021-01-14  0:19   ` David Liu
  2021-01-14  6:11     ` Ajit Khaparde
  0 siblings, 1 reply; 7+ messages in thread
From: David Liu @ 2021-01-14  0:19 UTC (permalink / raw)
  To: Ajit Khaparde; +Cc: ci, dpdklab

[-- Attachment #1: Type: text/plain, Size: 5423 bytes --]

Hi Ajit,

Thank you for helping out.

We have it working on the x86, but not the Arm.


Thanks,
David

On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde <ajit.khaparde@broadcom.com>
wrote:

> Hi David,
> I will take a look.
> Do you see similar issues on x86? I am asking because
> even I would start with that to create a baseline and then
> attempt on ARM.
>
> Thanks
> Ajit
>
> On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
>
>> Hi Ajit,
>>
>> We have a 25G Broadcom NIC that is installed on an Arm machine.
>> And I am running into a problem when I try to run nic_single_core_perf on
>> the NIC.
>> Currently, we are using: Trex v2.86 and dts from
>> http://git.dpdk.org/tools/dts/
>> All the NICs are up and running when testing.
>>
>> The problem is an error when I try to run the test case inside
>> the nic_single_core_perf:
>>
>> TestNicSingleCorePerf: Test running at parameters: framesize: 64,
>>> rxd/txd: 512
>>>                dut.172.18.0.41:
>>> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4
>>>  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1
>>> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>>>                dut.172.18.0.41: start
>>>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core
>>> Result ERROR: Traceback (most recent call last):
>>>   File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
>>>     case_obj()
>>>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in
>>> test_perf_nic_single_core
>>>     self.perf_test(self.nb_ports)
>>>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in perf_test
>>>     _, packets_received =
>>> self.tester.pktgen.measure_throughput(stream_ids=streams,
>>> options=traffic_opt)
>>>   File "/opt/dts/framework/pktgen_base.py", line 245, in
>>> measure_throughput
>>>     self._prepare_transmission(stream_ids=stream_ids)
>>>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
>>> _prepare_transmission
>>>     self._conn.reset(ports=self._ports)
>>>   File
>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>> line 51, in wrap2
>>>     ret = f(*args, **kwargs)
>>>   File
>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>> line 339, in reset
>>>     self.clear_stats(ports)
>>>   File
>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>> line 51, in wrap2
>>>     ret = f(*args, **kwargs)
>>>   File
>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>> line 1492, in clear_stats
>>>     self._clear_stats_common(ports, clear_global, clear_xstats)
>>>   File
>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py",
>>> line 2876, in _clear_stats_common
>>>     raise TRexError(rc)
>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server
>>> response from tcp://172.18.0.40:4501
>>
>>
>> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg
>> /etc/trex_cfg.yaml and  connect with ./trex-console
>>
>> Then start sending traffic with command
>>
>>> trex>start -f stl/imix.py
>>
>>
>> But this will cause an error to turn off Trex.
>>
>> trex>
>>> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown -
>>> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than
>>> 1.00135 seconds - timeout is 1 seconds
>>> *** traceback follows ***
>>> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>>> 2       0x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)
>>> [0x7feea3ac0980]
>>> 3       0x561be1a05a2a rte_delay_us_block + 106
>>> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned
>>> short, CVirtualIFPerSideStats*) + 3220
>>> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>>> 6       0x561be171e3d9
>>> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
>>> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*,
>>> CFlowGenListPerThread*, bool&) + 92
>>> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned
>>> char, CGenNode*, CFlowGenListPerThread*, bool) + 184
>>> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23,
>>> false>(double, double, CFlowGenListPerThread*, double&) + 881
>>> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>>> 11      0x561be1883ae9 TrexDpCore::start() + 89
>>> 12      0x561be1714113
>>> CFlowGenListPerThread::start(std::__cxx11::basic_string<char,
>>> std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
>>> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>>> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>>> 15      0x561be1a1dfaa eal_thread_loop + 426
>>> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)
>>> [0x7feea3ab56db]
>>> 17      0x7feea2a8571f clone + 63
>>>
>>> *** addr2line information follows ***
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> ??:0
>>> '
>>
>>
>> I believe if everything is working, this should not turn off trex but
>> please correct me if I am wrong.
>> I wonder if you have any suggestion of how I can approach this issue?
>>
>> Best,
>> David
>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 16261 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
  2021-01-14  0:19   ` David Liu
@ 2021-01-14  6:11     ` Ajit Khaparde
  2021-01-14 19:12       ` David Liu
  0 siblings, 1 reply; 7+ messages in thread
From: Ajit Khaparde @ 2021-01-14  6:11 UTC (permalink / raw)
  To: David Liu; +Cc: ci, dpdklab


[-- Attachment #1.1: Type: text/plain, Size: 5879 bytes --]

Does the same NIC work on x86?
Also do you see link before you start the trex? I am trying to see if the
port failed to successfully come up
or if there is a problem with the link. Can you check and tell?

Thanks
Ajit

On Wed, Jan 13, 2021 at 4:20 PM David Liu <dliu@iol.unh.edu> wrote:

> Hi Ajit,
>
> Thank you for helping out.
>
> We have it working on the x86, but not the Arm.
>
>
> Thanks,
> David
>
> On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde <ajit.khaparde@broadcom.com>
> wrote:
>
>> Hi David,
>> I will take a look.
>> Do you see similar issues on x86? I am asking because
>> even I would start with that to create a baseline and then
>> attempt on ARM.
>>
>> Thanks
>> Ajit
>>
>> On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
>>
>>> Hi Ajit,
>>>
>>> We have a 25G Broadcom NIC that is installed on an Arm machine.
>>> And I am running into a problem when I try to run nic_single_core_perf
>>> on the NIC.
>>> Currently, we are using: Trex v2.86 and dts from
>>> http://git.dpdk.org/tools/dts/
>>> All the NICs are up and running when testing.
>>>
>>> The problem is an error when I try to run the test case inside
>>> the nic_single_core_perf:
>>>
>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64,
>>>> rxd/txd: 512
>>>>                dut.172.18.0.41:
>>>> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4
>>>>  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1
>>>> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>>>>                dut.172.18.0.41: start
>>>>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core
>>>> Result ERROR: Traceback (most recent call last):
>>>>   File "/opt/dts/framework/test_case.py", line 319, in
>>>> _execute_test_case
>>>>     case_obj()
>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in
>>>> test_perf_nic_single_core
>>>>     self.perf_test(self.nb_ports)
>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in perf_test
>>>>     _, packets_received =
>>>> self.tester.pktgen.measure_throughput(stream_ids=streams,
>>>> options=traffic_opt)
>>>>   File "/opt/dts/framework/pktgen_base.py", line 245, in
>>>> measure_throughput
>>>>     self._prepare_transmission(stream_ids=stream_ids)
>>>>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
>>>> _prepare_transmission
>>>>     self._conn.reset(ports=self._ports)
>>>>   File
>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>>> line 51, in wrap2
>>>>     ret = f(*args, **kwargs)
>>>>   File
>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>>> line 339, in reset
>>>>     self.clear_stats(ports)
>>>>   File
>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>>> line 51, in wrap2
>>>>     ret = f(*args, **kwargs)
>>>>   File
>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>>> line 1492, in clear_stats
>>>>     self._clear_stats_common(ports, clear_global, clear_xstats)
>>>>   File
>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py",
>>>> line 2876, in _clear_stats_common
>>>>     raise TRexError(rc)
>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server
>>>> response from tcp://172.18.0.40:4501
>>>
>>>
>>> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg
>>> /etc/trex_cfg.yaml and  connect with ./trex-console
>>>
>>> Then start sending traffic with command
>>>
>>>> trex>start -f stl/imix.py
>>>
>>>
>>> But this will cause an error to turn off Trex.
>>>
>>> trex>
>>>> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown -
>>>> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than
>>>> 1.00135 seconds - timeout is 1 seconds
>>>> *** traceback follows ***
>>>> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>>>> 2       0x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)
>>>> [0x7feea3ac0980]
>>>> 3       0x561be1a05a2a rte_delay_us_block + 106
>>>> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned
>>>> short, CVirtualIFPerSideStats*) + 3220
>>>> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>>>> 6       0x561be171e3d9
>>>> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
>>>> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*,
>>>> CFlowGenListPerThread*, bool&) + 92
>>>> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned
>>>> char, CGenNode*, CFlowGenListPerThread*, bool) + 184
>>>> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23,
>>>> false>(double, double, CFlowGenListPerThread*, double&) + 881
>>>> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>>>> 11      0x561be1883ae9 TrexDpCore::start() + 89
>>>> 12      0x561be1714113
>>>> CFlowGenListPerThread::start(std::__cxx11::basic_string<char,
>>>> std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
>>>> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>>>> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>>>> 15      0x561be1a1dfaa eal_thread_loop + 426
>>>> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)
>>>> [0x7feea3ab56db]
>>>> 17      0x7feea2a8571f clone + 63
>>>>
>>>> *** addr2line information follows ***
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> ??:0
>>>> '
>>>
>>>
>>> I believe if everything is working, this should not turn off trex but
>>> please correct me if I am wrong.
>>> I wonder if you have any suggestion of how I can approach this issue?
>>>
>>> Best,
>>> David
>>>
>>>
>>>

[-- Attachment #1.2: Type: text/html, Size: 17266 bytes --]

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4175 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
  2021-01-14  6:11     ` Ajit Khaparde
@ 2021-01-14 19:12       ` David Liu
  2021-01-19 21:40         ` David Liu
  0 siblings, 1 reply; 7+ messages in thread
From: David Liu @ 2021-01-14 19:12 UTC (permalink / raw)
  To: Ajit Khaparde; +Cc: ci, dpdklab

[-- Attachment #1: Type: text/plain, Size: 6321 bytes --]

Yes, so this is the same NIC work on x86.
By seeing the link, I am assuming you mean the ports that are using for
Trex. If so they are UP before I start the Trex.

Thank,
David


On Thu, Jan 14, 2021 at 1:11 AM Ajit Khaparde <ajit.khaparde@broadcom.com>
wrote:

> Does the same NIC work on x86?
> Also do you see link before you start the trex? I am trying to see if the
> port failed to successfully come up
> or if there is a problem with the link. Can you check and tell?
>
> Thanks
> Ajit
>
> On Wed, Jan 13, 2021 at 4:20 PM David Liu <dliu@iol.unh.edu> wrote:
>
>> Hi Ajit,
>>
>> Thank you for helping out.
>>
>> We have it working on the x86, but not the Arm.
>>
>>
>> Thanks,
>> David
>>
>> On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde <ajit.khaparde@broadcom.com>
>> wrote:
>>
>>> Hi David,
>>> I will take a look.
>>> Do you see similar issues on x86? I am asking because
>>> even I would start with that to create a baseline and then
>>> attempt on ARM.
>>>
>>> Thanks
>>> Ajit
>>>
>>> On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
>>>
>>>> Hi Ajit,
>>>>
>>>> We have a 25G Broadcom NIC that is installed on an Arm machine.
>>>> And I am running into a problem when I try to run nic_single_core_perf
>>>> on the NIC.
>>>> Currently, we are using: Trex v2.86 and dts from
>>>> http://git.dpdk.org/tools/dts/
>>>> All the NICs are up and running when testing.
>>>>
>>>> The problem is an error when I try to run the test case inside
>>>> the nic_single_core_perf:
>>>>
>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64,
>>>>> rxd/txd: 512
>>>>>                dut.172.18.0.41:
>>>>> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4
>>>>>  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1
>>>>> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>>>>>                dut.172.18.0.41: start
>>>>>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core
>>>>> Result ERROR: Traceback (most recent call last):
>>>>>   File "/opt/dts/framework/test_case.py", line 319, in
>>>>> _execute_test_case
>>>>>     case_obj()
>>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in
>>>>> test_perf_nic_single_core
>>>>>     self.perf_test(self.nb_ports)
>>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in
>>>>> perf_test
>>>>>     _, packets_received =
>>>>> self.tester.pktgen.measure_throughput(stream_ids=streams,
>>>>> options=traffic_opt)
>>>>>   File "/opt/dts/framework/pktgen_base.py", line 245, in
>>>>> measure_throughput
>>>>>     self._prepare_transmission(stream_ids=stream_ids)
>>>>>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
>>>>> _prepare_transmission
>>>>>     self._conn.reset(ports=self._ports)
>>>>>   File
>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>>>> line 51, in wrap2
>>>>>     ret = f(*args, **kwargs)
>>>>>   File
>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>>>> line 339, in reset
>>>>>     self.clear_stats(ports)
>>>>>   File
>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>>>> line 51, in wrap2
>>>>>     ret = f(*args, **kwargs)
>>>>>   File
>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>>>> line 1492, in clear_stats
>>>>>     self._clear_stats_common(ports, clear_global, clear_xstats)
>>>>>   File
>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py",
>>>>> line 2876, in _clear_stats_common
>>>>>     raise TRexError(rc)
>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get
>>>>> server response from tcp://172.18.0.40:4501
>>>>
>>>>
>>>> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg
>>>> /etc/trex_cfg.yaml and  connect with ./trex-console
>>>>
>>>> Then start sending traffic with command
>>>>
>>>>> trex>start -f stl/imix.py
>>>>
>>>>
>>>> But this will cause an error to turn off Trex.
>>>>
>>>> trex>
>>>>> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown -
>>>>> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than
>>>>> 1.00135 seconds - timeout is 1 seconds
>>>>> *** traceback follows ***
>>>>> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>>>>> 2       0x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)
>>>>> [0x7feea3ac0980]
>>>>> 3       0x561be1a05a2a rte_delay_us_block + 106
>>>>> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned
>>>>> short, CVirtualIFPerSideStats*) + 3220
>>>>> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>>>>> 6       0x561be171e3d9
>>>>> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
>>>>> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*,
>>>>> CFlowGenListPerThread*, bool&) + 92
>>>>> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned
>>>>> char, CGenNode*, CFlowGenListPerThread*, bool) + 184
>>>>> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23,
>>>>> false>(double, double, CFlowGenListPerThread*, double&) + 881
>>>>> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>>>>> 11      0x561be1883ae9 TrexDpCore::start() + 89
>>>>> 12      0x561be1714113
>>>>> CFlowGenListPerThread::start(std::__cxx11::basic_string<char,
>>>>> std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
>>>>> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>>>>> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>>>>> 15      0x561be1a1dfaa eal_thread_loop + 426
>>>>> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)
>>>>> [0x7feea3ab56db]
>>>>> 17      0x7feea2a8571f clone + 63
>>>>>
>>>>> *** addr2line information follows ***
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> ??:0
>>>>> '
>>>>
>>>>
>>>> I believe if everything is working, this should not turn off trex but
>>>> please correct me if I am wrong.
>>>> I wonder if you have any suggestion of how I can approach this issue?
>>>>
>>>> Best,
>>>> David
>>>>
>>>>
>>>>

[-- Attachment #2: Type: text/html, Size: 17906 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
  2021-01-14 19:12       ` David Liu
@ 2021-01-19 21:40         ` David Liu
  2021-01-19 22:19           ` Ajit Khaparde
  0 siblings, 1 reply; 7+ messages in thread
From: David Liu @ 2021-01-19 21:40 UTC (permalink / raw)
  To: Ajit Khaparde; +Cc: ci, dpdklab

[-- Attachment #1: Type: text/plain, Size: 6816 bytes --]

Hi Ajit,

Thank you for helping out.

How are you doing with this issue?

Please let me know if you need anything information from us.

Just some extra information that both Trex and dts are on the same
intel tester machine.

Best,
David

On Thu, Jan 14, 2021 at 2:12 PM David Liu <dliu@iol.unh.edu> wrote:

> Yes, so this is the same NIC work on x86.
> By seeing the link, I am assuming you mean the ports that are using for
> Trex. If so they are UP before I start the Trex.
>
> Thank,
> David
>
>
> On Thu, Jan 14, 2021 at 1:11 AM Ajit Khaparde <ajit.khaparde@broadcom.com>
> wrote:
>
>> Does the same NIC work on x86?
>> Also do you see link before you start the trex? I am trying to see if the
>> port failed to successfully come up
>> or if there is a problem with the link. Can you check and tell?
>>
>> Thanks
>> Ajit
>>
>> On Wed, Jan 13, 2021 at 4:20 PM David Liu <dliu@iol.unh.edu> wrote:
>>
>>> Hi Ajit,
>>>
>>> Thank you for helping out.
>>>
>>> We have it working on the x86, but not the Arm.
>>>
>>>
>>> Thanks,
>>> David
>>>
>>> On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde <
>>> ajit.khaparde@broadcom.com> wrote:
>>>
>>>> Hi David,
>>>> I will take a look.
>>>> Do you see similar issues on x86? I am asking because
>>>> even I would start with that to create a baseline and then
>>>> attempt on ARM.
>>>>
>>>> Thanks
>>>> Ajit
>>>>
>>>> On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
>>>>
>>>>> Hi Ajit,
>>>>>
>>>>> We have a 25G Broadcom NIC that is installed on an Arm machine.
>>>>> And I am running into a problem when I try to run nic_single_core_perf
>>>>> on the NIC.
>>>>> Currently, we are using: Trex v2.86 and dts from
>>>>> http://git.dpdk.org/tools/dts/
>>>>> All the NICs are up and running when testing.
>>>>>
>>>>> The problem is an error when I try to run the test case inside
>>>>> the nic_single_core_perf:
>>>>>
>>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64,
>>>>>> rxd/txd: 512
>>>>>>                dut.172.18.0.41:
>>>>>> x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4
>>>>>>  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1
>>>>>> -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>>>>>>                dut.172.18.0.41: start
>>>>>>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core
>>>>>> Result ERROR: Traceback (most recent call last):
>>>>>>   File "/opt/dts/framework/test_case.py", line 319, in
>>>>>> _execute_test_case
>>>>>>     case_obj()
>>>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in
>>>>>> test_perf_nic_single_core
>>>>>>     self.perf_test(self.nb_ports)
>>>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in
>>>>>> perf_test
>>>>>>     _, packets_received =
>>>>>> self.tester.pktgen.measure_throughput(stream_ids=streams,
>>>>>> options=traffic_opt)
>>>>>>   File "/opt/dts/framework/pktgen_base.py", line 245, in
>>>>>> measure_throughput
>>>>>>     self._prepare_transmission(stream_ids=stream_ids)
>>>>>>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
>>>>>> _prepare_transmission
>>>>>>     self._conn.reset(ports=self._ports)
>>>>>>   File
>>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>>>>> line 51, in wrap2
>>>>>>     ret = f(*args, **kwargs)
>>>>>>   File
>>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>>>>> line 339, in reset
>>>>>>     self.clear_stats(ports)
>>>>>>   File
>>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
>>>>>> line 51, in wrap2
>>>>>>     ret = f(*args, **kwargs)
>>>>>>   File
>>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
>>>>>> line 1492, in clear_stats
>>>>>>     self._clear_stats_common(ports, clear_global, clear_xstats)
>>>>>>   File
>>>>>> "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py",
>>>>>> line 2876, in _clear_stats_common
>>>>>>     raise TRexError(rc)
>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get
>>>>>> server response from tcp://172.18.0.40:4501
>>>>>
>>>>>
>>>>> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg
>>>>> /etc/trex_cfg.yaml and  connect with ./trex-console
>>>>>
>>>>> Then start sending traffic with command
>>>>>
>>>>>> trex>start -f stl/imix.py
>>>>>
>>>>>
>>>>> But this will cause an error to turn off Trex.
>>>>>
>>>>> trex>
>>>>>> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown -
>>>>>> cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than
>>>>>> 1.00135 seconds - timeout is 1 seconds
>>>>>> *** traceback follows ***
>>>>>> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>>>>>> 2       0x7feea3ac0980
>>>>>> /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7feea3ac0980]
>>>>>> 3       0x561be1a05a2a rte_delay_us_block + 106
>>>>>> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned
>>>>>> short, CVirtualIFPerSideStats*) + 3220
>>>>>> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>>>>>> 6       0x561be171e3d9
>>>>>> CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
>>>>>> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*,
>>>>>> CFlowGenListPerThread*, bool&) + 92
>>>>>> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned
>>>>>> char, CGenNode*, CFlowGenListPerThread*, bool) + 184
>>>>>> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23,
>>>>>> false>(double, double, CFlowGenListPerThread*, double&) + 881
>>>>>> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>>>>>> 11      0x561be1883ae9 TrexDpCore::start() + 89
>>>>>> 12      0x561be1714113
>>>>>> CFlowGenListPerThread::start(std::__cxx11::basic_string<char,
>>>>>> std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
>>>>>> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>>>>>> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>>>>>> 15      0x561be1a1dfaa eal_thread_loop + 426
>>>>>> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db)
>>>>>> [0x7feea3ab56db]
>>>>>> 17      0x7feea2a8571f clone + 63
>>>>>>
>>>>>> *** addr2line information follows ***
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> ??:0
>>>>>> '
>>>>>
>>>>>
>>>>> I believe if everything is working, this should not turn off trex but
>>>>> please correct me if I am wrong.
>>>>> I wonder if you have any suggestion of how I can approach this issue?
>>>>>
>>>>> Best,
>>>>> David
>>>>>
>>>>>
>>>>>

[-- Attachment #2: Type: text/html, Size: 18638 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware
  2021-01-19 21:40         ` David Liu
@ 2021-01-19 22:19           ` Ajit Khaparde
  0 siblings, 0 replies; 7+ messages in thread
From: Ajit Khaparde @ 2021-01-19 22:19 UTC (permalink / raw)
  To: David Liu; +Cc: ci, dpdklab

[-- Attachment #1: Type: text/plain, Size: 7085 bytes --]

Hi David,
Thanks for pinging me. Do you think I could login into the setup and
see what is happening?
I do have the credentials for two systems in your lab -
rhea.dpdklab.iol.unh.edu and io.dpdklab.iol.unh.edu
If you can provide me details of the systems you are trying to
configure, I could take a look.

Thanks
Ajit

On Tue, Jan 19, 2021 at 1:41 PM David Liu <dliu@iol.unh.edu> wrote:
>
> Hi Ajit,
>
> Thank you for helping out.
>
> How are you doing with this issue?
>
> Please let me know if you need anything information from us.
>
> Just some extra information that both Trex and dts are on the same intel tester machine.
>
> Best,
> David
>
> On Thu, Jan 14, 2021 at 2:12 PM David Liu <dliu@iol.unh.edu> wrote:
>>
>> Yes, so this is the same NIC work on x86.
>> By seeing the link, I am assuming you mean the ports that are using for Trex. If so they are UP before I start the Trex.
>>
>> Thank,
>> David
>>
>>
>> On Thu, Jan 14, 2021 at 1:11 AM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:
>>>
>>> Does the same NIC work on x86?
>>> Also do you see link before you start the trex? I am trying to see if the port failed to successfully come up
>>> or if there is a problem with the link. Can you check and tell?
>>>
>>> Thanks
>>> Ajit
>>>
>>> On Wed, Jan 13, 2021 at 4:20 PM David Liu <dliu@iol.unh.edu> wrote:
>>>>
>>>> Hi Ajit,
>>>>
>>>> Thank you for helping out.
>>>>
>>>> We have it working on the x86, but not the Arm.
>>>>
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>> On Wed, Jan 13, 2021 at 5:34 PM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:
>>>>>
>>>>> Hi David,
>>>>> I will take a look.
>>>>> Do you see similar issues on x86? I am asking because
>>>>> even I would start with that to create a baseline and then
>>>>> attempt on ARM.
>>>>>
>>>>> Thanks
>>>>> Ajit
>>>>>
>>>>> On Wed, Jan 13, 2021 at 12:08 PM David Liu <dliu@iol.unh.edu> wrote:
>>>>>>
>>>>>> Hi Ajit,
>>>>>>
>>>>>> We have a 25G Broadcom NIC that is installed on an Arm machine.
>>>>>> And I am running into a problem when I try to run nic_single_core_perf on the NIC.
>>>>>> Currently, we are using: Trex v2.86 and dts from http://git.dpdk.org/tools/dts/
>>>>>> All the NICs are up and running when testing.
>>>>>>
>>>>>> The problem is an error when I try to run the test case inside the nic_single_core_perf:
>>>>>>
>>>>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64, rxd/txd: 512
>>>>>>>                dut.172.18.0.41: x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 32,33 -n 4  --file-prefix=dpdk_15311_20210113190237   -a 0000:93:00.0 -a 0000:93:00.1 -- -i --portmask=0x3 --txd=512 --rxd=512 --nb-cores=1
>>>>>>>                dut.172.18.0.41: start
>>>>>>>          TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR: Traceback (most recent call last):
>>>>>>>   File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
>>>>>>>     case_obj()
>>>>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 196, in test_perf_nic_single_core
>>>>>>>     self.perf_test(self.nb_ports)
>>>>>>>   File "tests/TestSuite_nic_single_core_perf.py", line 270, in perf_test
>>>>>>>     _, packets_received = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
>>>>>>>   File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
>>>>>>>     self._prepare_transmission(stream_ids=stream_ids)
>>>>>>>   File "/opt/dts/framework/pktgen_trex.py", line 778, in _prepare_transmission
>>>>>>>     self._conn.reset(ports=self._ports)
>>>>>>>   File "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
>>>>>>>     ret = f(*args, **kwargs)
>>>>>>>   File "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 339, in reset
>>>>>>>     self.clear_stats(ports)
>>>>>>>   File "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
>>>>>>>     ret = f(*args, **kwargs)
>>>>>>>   File "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 1492, in clear_stats
>>>>>>>     self._clear_stats_common(ports, clear_global, clear_xstats)
>>>>>>>   File "/opt/trex/v2.86/automation/trex_control_plane/interactive/trex/common/trex_client.py", line 2876, in _clear_stats_common
>>>>>>>     raise TRexError(rc)
>>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server response from tcp://172.18.0.40:4501
>>>>>>
>>>>>>
>>>>>> I also try to run TRex in stateless mode with./t-rex-64 -i --cfg /etc/trex_cfg.yaml and  connect with ./trex-console
>>>>>>
>>>>>> Then start sending traffic with command
>>>>>>>
>>>>>>> trex>start -f stl/imix.py
>>>>>>
>>>>>>
>>>>>> But this will cause an error to turn off Trex.
>>>>>>
>>>>>>> trex>
>>>>>>> 2021-01-13 19:56:21 - [server][warning]  - Server has been shutdown - cause: 'WATCHDOG: task 'Trex DP core 1' has not responded for more than 1.00135 seconds - timeout is 1 seconds
>>>>>>> *** traceback follows ***
>>>>>>> 1       0x561be173cf5a ./_t-rex-64(+0x19af5a) [0x561be173cf5a]
>>>>>>> 2       0x7feea3ac0980 /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7feea3ac0980]
>>>>>>> 3       0x561be1a05a2a rte_delay_us_block + 106
>>>>>>> 4       0x561be16ce874 CCoreEthIF::send_burst(CCorePerPort*, unsigned short, CVirtualIFPerSideStats*) + 3220
>>>>>>> 5       0x561be16abf57 CCoreEthIF::flush_tx_queue() + 31
>>>>>>> 6       0x561be171e3d9 CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 265
>>>>>>> 7       0x561be171f7ec CNodeGenerator::handle_flow_sync(CGenNode*, CFlowGenListPerThread*, bool&) + 92
>>>>>>> 8       0x561be171fc98 CNodeGenerator::handle_slow_messages(unsigned char, CGenNode*, CFlowGenListPerThread*, bool) + 184
>>>>>>> 9       0x561be16cb5f1 int CNodeGenerator::flush_file_realtime<23, false>(double, double, CFlowGenListPerThread*, double&) + 881
>>>>>>> 10      0x561be1905212 TrexStatelessDpCore::start_scheduler() + 226
>>>>>>> 11      0x561be1883ae9 TrexDpCore::start() + 89
>>>>>>> 12      0x561be1714113 CFlowGenListPerThread::start(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115
>>>>>>> 13      0x561be16af8dd CGlobalTRex::run_in_core(unsigned char) + 487
>>>>>>> 14      0x561be16d11ad ./_t-rex-64(+0x12f1ad) [0x561be16d11ad]
>>>>>>> 15      0x561be1a1dfaa eal_thread_loop + 426
>>>>>>> 16      0x7feea3ab56db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7feea3ab56db]
>>>>>>> 17      0x7feea2a8571f clone + 63
>>>>>>>
>>>>>>> *** addr2line information follows ***
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> ??:0
>>>>>>> '
>>>>>>
>>>>>>
>>>>>> I believe if everything is working, this should not turn off trex but please correct me if I am wrong.
>>>>>> I wonder if you have any suggestion of how I can approach this issue?
>>>>>>
>>>>>> Best,
>>>>>> David
>>>>>>
>>>>>>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4175 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-01-19 22:20 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-13 20:07 [dpdk-ci] [DPDKLAB] 25G Broadcom NIC in Arm hardware David Liu
2021-01-13 22:34 ` Ajit Khaparde
2021-01-14  0:19   ` David Liu
2021-01-14  6:11     ` Ajit Khaparde
2021-01-14 19:12       ` David Liu
2021-01-19 21:40         ` David Liu
2021-01-19 22:19           ` Ajit Khaparde

DPDK CI discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/ci/0 ci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ci ci/ http://inbox.dpdk.org/ci \
		ci@dpdk.org
	public-inbox-index ci

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.ci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git