Hi Ajit,
I believe DTS/TREX is run on the io machine as a tester. Rhea runs
testpmd (through an ssh session from io --> rhea) to catch information
as a DUT.
The only commands that I run (on io) are:
cd /opt/dts
export DTS_CFG_FOLDER='conf_100g'
./dts -s
The rest is managed by DTS itself.
The issue occurs when DTS is trying to send packets to rhea from io on
the new 100G NIC.
It seems to happen after it tries to measure throughput using TREX's API.
Thanks,
Brandon
On Thu, Aug 20, 2020 at 12:29 AM Ajit Khaparde
<ajit.khaparde@broadcom.com> wrote:
>
> Hi Brandon,
> I was trying to see what exactly is happening on the setup and what is causing the problem.
> But I will need your help to use the proper code and commands.
>
> I believe you run the testpmd command on io. And the trex is executed on rhea?
> Can you point me to the location of the code and the steps you are following while running the test?
>
> Thanks
> Ajit
>
> On Tue, Aug 11, 2020 at 10:47 AM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:
>>
>> Hi Brandon,
>> I haven't. I tried to login as well. But I had some issues doing it from the office.
>> I just have to remind myself to try it again once I get home before I connect to the company VPN.
>> Thanks for checking in. I will try to update you as soon as I can.
>>
>> Thanks
>> Ajit
>>
>> On Tue, Aug 11, 2020 at 10:46 AM Brandon Lo <blo@iol.unh.edu> wrote:
>>>
>>> Hi Ajit,
>>>
>>> I'm just checking in; have you heard of any updates on this issue?
>>>
>>> Thanks,
>>> Brandon
>>>
>>> On Tue, Aug 4, 2020 at 1:42 PM Brandon Lo <blo@iol.unh.edu> wrote:
>>>>
>>>> Hi Ajit,
>>>>
>>>> Yes, I believe the issue is coming from the trex/tester system with the 100G NIC.
>>>> I'm not sure what causes this issue; if I run trex using the command "cd /opt/v2.82;./t-rex-64 -i --cfg /etc/trex_cfg_100g.yaml -c 7", which is the same command used in DTS, it seems to launch without failing.
>>>>
>>>> If you want to replicate it, here are the steps that I ran:
>>>>
>>>> (on io) cd /opt/dts
>>>> export DTS_CFG_FOLDER='conf_100g'
>>>>
>>>> conf_100g has the new configuration files to use the new PCI id and pktgen config file
>>>>
>>>> ./dts -s
>>>>
>>>>
>>>> Thanks for your help,
>>>> Brandon
>>>>
>>>> On Tue, Aug 4, 2020 at 12:38 PM Ajit Khaparde <ajit.khaparde@broadcom.com> wrote:
>>>>>
>>>>> Hi Brandon,
>>>>> No, I haven't seen or heard this before.
>>>>> But I will try to have someone run it again.
>>>>>
>>>>> Just to make sure -
>>>>> You are running trex on the 100G NIC and the problem is encountered on that setup?
>>>>> Or is it the system that is running testpmd where you are running into the issue?
>>>>>
>>>>> Thanks
>>>>> Ajit
>>>>>
>>>>> On Tue, Aug 4, 2020 at 9:16 AM Brandon Lo <blo@iol.unh.edu> wrote:
>>>>>>
>>>>>> Hi Ajit,
>>>>>>
>>>>>> I'm running into a problem with trying to run nic_single_core_perf on the new NIC.
>>>>>> The current configuration uses trex version v2.82.
>>>>>> However, I'm running into an error when it tries to actually do a test case in the nic_single_core_perf.
>>>>>>
>>>>>> The output looks like this when it reaches a test case:
>>>>>>
>>>>>>> TestNicSingleCorePerf: Test running at parameters: framesize: 64, rxd/txd: 512
>>>>>>> dut.rhea: ./x86_64-native-linuxapp-gcc/app/testpmd -l 16,17 -n 4 -w 0000:81:00.0 -w 0000:81:00.1 --file-prefix=dpdk_11307_20200804160513 -- -i --portmask=0x3 --txd=512 --rxd=512
>>>>>>> dut.rhea: start
>>>>>>> TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR: Traceback (most recent call last):
>>>>>>> File "/opt/dts/framework/test_case.py", line 316, in _execute_test_case
>>>>>>> case_obj()
>>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 198, in test_perf_nic_single_core
>>>>>>> self.perf_test(self.nb_ports)
>>>>>>> File "tests/TestSuite_nic_single_core_perf.py", line 259, in perf_test
>>>>>>> _, packets_received = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
>>>>>>> File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
>>>>>>> self._prepare_transmission(stream_ids=stream_ids)
>>>>>>> File "/opt/dts/framework/pktgen_trex.py", line 779, in _prepare_transmission
>>>>>>> self._conn.reset(ports=self._ports)
>>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
>>>>>>> ret = f(*args, **kwargs)
>>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 339, in reset
>>>>>>> self.clear_stats(ports)
>>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
>>>>>>> ret = f(*args, **kwargs)
>>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 1467, in clear_stats
>>>>>>> self._clear_stats_common(ports, clear_global, clear_xstats)
>>>>>>> File "/opt/v2.82/automation/trex_control_plane/interactive/trex/common/trex_client.py", line 2840, in _clear_stats_common
>>>>>>> raise TRexError(rc)
>>>>>>> trex.common.trex_exceptions.TRexError: *** [RPC] - Failed to get server response from tcp://127.0.0.1:4501
>>>>>>
>>>>>>
>>>>>> I have found one similar case on the github repository for trex, but the solution was vendor-specific: https://github.com/cisco-system-traffic-generator/trex-core/issues/147.
>>>>>> Have you ran into this issue before?
>>>>>>
>>>>>> Thanks,
>>>>>> Brandon
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Brandon Lo
>>>>
>>>> UNH InterOperability Laboratory
>>>>
>>>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>>>>
>>>> blo@iol.unh.edu
>>>>
>>>> www.iol.unh.edu
>>>
>>>
>>>
>>> --
>>>
>>> Brandon Lo
>>>
>>> UNH InterOperability Laboratory
>>>
>>> 21 Madbury Rd, Suite 100, Durham, NH 03824
>>>
>>> blo@iol.unh.edu
>>>
>>> www.iol.unh.edu
--
Brandon Lo
UNH InterOperability Laboratory
21 Madbury Rd, Suite 100, Durham, NH 03824
blo@iol.unh.edu
www.iol.unh.edu