[dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[David Liu]

Hi Lijuan,

I currently have issues with running the Nic Single-Core Perf Test.
I believe this is the scapy naming conflict issue, so I follow the solution
mentioned on the website to replace system scapy with trex scapy.
Then I got into another error that I wasn't able to get pass:

root@arm-dell:/opt/dts# ./dts
Traceback (most recent call last):
  File "./main.py", line 50, in <module>
    import dts
  File "/opt/dts/framework/dts.py", line 47, in <module>
    from tester import Tester
  File "/opt/dts/framework/tester.py", line 52, in <module>
    from packet import SCAPY_IMP_CMD
  File "/opt/dts/framework/packet.py", line 72, in <module>
    locals().update({clazz: getattr(module, clazz)})
AttributeError: module 'scapy.contrib.gtp' has no attribute
'GTPPDUSessionContainer'

Do you have any clue how to fix this?

Thanks.
David

Re: [dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[Tu, Lijuan]

Hi David,



Snice some protocols are only supported by scapy 2.4.4, it's recemented that scapy 2.4.4 as major version for DTS.

However perf users might not care these advanced protocols, to be compatible with trex, dts captured these exceptions and let it go through.

Fix was contained in the latest DTS. Could you update it.



commit 362ae41844e1992163b4eed53a721729985644be

Author: Chen, BoX C <box.c.chen@intel.com>

Date:   Wed Nov 4 10:52:09 2020 +0800



    framework: fix Trex mode not working

    DTS requires Trex compatible scapy, modify package loading mode and scapy version verification rules



    Signed-off-by: Chen, BoX C <box.c.chen@intel.com>


From: David Liu <dliu@iol.unh.edu>
Sent: 2020年11月18日 4:24
To: Tu, Lijuan <lijuan.tu@intel.com>
Cc: dpdk-dev <dev@dpdk.org>
Subject: DTS Test Nic Single Core Perf Scapy Naming Conflict

Hi Lijuan,

I currently have issues with running the Nic Single-Core Perf Test.
I believe this is the scapy naming conflict issue, so I follow the solution
mentioned on the website to replace system scapy with trex scapy.
Then I got into another error that I wasn't able to get pass:

root@arm-dell:/opt/dts# ./dts
Traceback (most recent call last):
  File "./main.py", line 50, in <module>
    import dts
  File "/opt/dts/framework/dts.py", line 47, in <module>
    from tester import Tester
  File "/opt/dts/framework/tester.py", line 52, in <module>
    from packet import SCAPY_IMP_CMD
  File "/opt/dts/framework/packet.py", line 72, in <module>
    locals().update({clazz: getattr(module, clazz)})
AttributeError: module 'scapy.contrib.gtp' has no attribute 'GTPPDUSessionContainer'

Do you have any clue how to fix this?

Thanks.
David

Re: [dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[David Liu]

Hi Lijuan,

Thank you for the help.
I can run DTS without that error now,

Thanks,
David

On Tue, Nov 17, 2020 at 9:12 PM Tu, Lijuan <lijuan.tu@intel.com> wrote:

> Hi David,
>
>
>
> Snice some protocols are only supported by scapy 2.4.4, it's recemented
> that scapy 2.4.4 as major version for DTS.
>
> However perf users might not care these advanced protocols, to be
> compatible with trex, dts captured these exceptions and let it go through.
>
> Fix was contained in the latest DTS. Could you update it.
>
>
>
> commit 362ae41844e1992163b4eed53a721729985644be
>
> Author: Chen, BoX C <box.c.chen@intel.com>
>
> Date:   Wed Nov 4 10:52:09 2020 +0800
>
>
>
>     framework: fix Trex mode not working
>
>     DTS requires Trex compatible scapy, modify package loading mode and
> scapy version verification rules
>
>
>
>     Signed-off-by: Chen, BoX C <box.c.chen@intel.com>
>
>
>
>
>
> *From:* David Liu <dliu@iol.unh.edu>
> *Sent:* 2020年11月18日 4:24
> *To:* Tu, Lijuan <lijuan.tu@intel.com>
> *Cc:* dpdk-dev <dev@dpdk.org>
> *Subject:* DTS Test Nic Single Core Perf Scapy Naming Conflict
>
>
>
> Hi Lijuan,
>
>
>
> I currently have issues with running the Nic Single-Core Perf Test.
>
> I believe this is the scapy naming conflict issue, so I follow the
> solution
>
> mentioned on the website to replace system scapy with trex scapy.
>
> Then I got into another error that I wasn't able to get pass:
>
>
>
> root@arm-dell:/opt/dts# ./dts
> Traceback (most recent call last):
>   File "./main.py", line 50, in <module>
>     import dts
>   File "/opt/dts/framework/dts.py", line 47, in <module>
>     from tester import Tester
>   File "/opt/dts/framework/tester.py", line 52, in <module>
>     from packet import SCAPY_IMP_CMD
>   File "/opt/dts/framework/packet.py", line 72, in <module>
>     locals().update({clazz: getattr(module, clazz)})
> AttributeError: module 'scapy.contrib.gtp' has no attribute
> 'GTPPDUSessionContainer'
>
>
>
> Do you have any clue how to fix this?
>
>
>
> Thanks.
>
> David
>

Re: [dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[David Liu]

Hi Lijuan,

The DTS started without error but the nic single core test failed on a
"connection timeout - no response from TRex server" when I tried to run it
on an intel nic.
This happens when DTS uses arm-dell (traffic generator) to send packets
to arm-gigabyte (DUT) using the Intel nic {Ethernet Controller XL710 for
40GbE QSFP+}
I wonder if you have any clue on this issue?

19/11/2020 19:05:55          TestNicSingleCorePerf: Test Case
test_perf_nic_single_core Result ERROR: Traceback (most recent call last):
  File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
    case_obj()
  File "tests/TestSuite_nic_single_core_perf.py", line 200, in
test_perf_nic_single_core
    self.perf_test(self.nb_ports)
  File "tests/TestSuite_nic_single_core_perf.py", line 274, in perf_test
    _, packets_received =
self.tester.pktgen.measure_throughput(stream_ids=streams,
options=traffic_opt)
  File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
    self._prepare_transmission(stream_ids=stream_ids)
  File "/opt/dts/framework/pktgen_trex.py", line 778, in
_prepare_transmission
    self._conn.reset(ports=self._ports)
  File
"/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
line 51, in wrap2
    ret = f(*args, **kwargs)
  File
"/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
line 339, in reset
    self.clear_stats(ports)
  File
"/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
line 51, in wrap2
    ret = f(*args, **kwargs)
  File
"/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
line 1492, in clear_stats
    self._clear_stats_common(ports, clear_global, clear_xstats)
  File
"/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_client.py",
line 2876, in _clear_stats_common
    raise TRexError(rc)
trex.common.trex_exceptions.TRexError: ^[[1m*** [RPC] - Failed to get
server response from tcp://172.18.0.40:4501^[[22m

19/11/2020 19:05:55                            dts:
TEST SUITE ENDED: TestNicSingleCorePerf
19/11/2020 19:06:11          TestNicSingleCorePerf: tear_down_all failed:
Traceback (most recent call last):
  File "/opt/dts/framework/test_case.py", line 430, in execute_tear_downall
    self.tear_down_all()
  File "tests/TestSuite_nic_single_core_perf.py", line 405, in tear_down_all
    self.dut.build_install_dpdk(self.target)
  File "/opt/dts/framework/project_dpdk.py", line 283, in build_install_dpdk
    build_install_dpdk(target, extra_options)
  File "/opt/dts/framework/project_dpdk.py", line 316, in
build_install_dpdk_linux_meson
    self.send_expect("rm -rf " + target, "#")
  File "/opt/dts/framework/crb.py", line 93, in send_expect
    return self.session.send_expect(cmds, expected, timeout, verify)
  File "/opt/dts/framework/ssh_connection.py", line 64, in send_expect
    out = self.session.send_expect(cmds, expected, timeout, verify)
  File "/opt/dts/framework/ssh_pexpect.py", line 103, in send_expect
    raise(e)
  File "/opt/dts/framework/ssh_pexpect.py", line 90, in send_expect
    ret = self.send_expect_base(command, expected, timeout)
  File "/opt/dts/framework/ssh_pexpect.py", line 81, in send_expect_base
    self.__prompt(command, timeout)
  File "/opt/dts/framework/ssh_pexpect.py", line 149, in __prompt
    raise TimeoutException(command, self.get_output_all()) from None
exception.TimeoutException: TIMEOUT on rm -rf x86_64-native-linuxapp-gcc

Thanks,
David

On Thu, Nov 19, 2020 at 12:56 PM David Liu <dliu@iol.unh.edu> wrote:

> Hi Lijuan,
>
> Thank you for the help.
> I can run DTS without that error now,
>
> Thanks,
> David
>
> On Tue, Nov 17, 2020 at 9:12 PM Tu, Lijuan <lijuan.tu@intel.com> wrote:
>
>> Hi David,
>>
>>
>>
>> Snice some protocols are only supported by scapy 2.4.4, it's recemented
>> that scapy 2.4.4 as major version for DTS.
>>
>> However perf users might not care these advanced protocols, to be
>> compatible with trex, dts captured these exceptions and let it go through.
>>
>> Fix was contained in the latest DTS. Could you update it.
>>
>>
>>
>> commit 362ae41844e1992163b4eed53a721729985644be
>>
>> Author: Chen, BoX C <box.c.chen@intel.com>
>>
>> Date:   Wed Nov 4 10:52:09 2020 +0800
>>
>>
>>
>>     framework: fix Trex mode not working
>>
>>     DTS requires Trex compatible scapy, modify package loading mode and
>> scapy version verification rules
>>
>>
>>
>>     Signed-off-by: Chen, BoX C <box.c.chen@intel.com>
>>
>>
>>
>>
>>
>> *From:* David Liu <dliu@iol.unh.edu>
>> *Sent:* 2020年11月18日 4:24
>> *To:* Tu, Lijuan <lijuan.tu@intel.com>
>> *Cc:* dpdk-dev <dev@dpdk.org>
>> *Subject:* DTS Test Nic Single Core Perf Scapy Naming Conflict
>>
>>
>>
>> Hi Lijuan,
>>
>>
>>
>> I currently have issues with running the Nic Single-Core Perf Test.
>>
>> I believe this is the scapy naming conflict issue, so I follow the
>> solution
>>
>> mentioned on the website to replace system scapy with trex scapy.
>>
>> Then I got into another error that I wasn't able to get pass:
>>
>>
>>
>> root@arm-dell:/opt/dts# ./dts
>> Traceback (most recent call last):
>>   File "./main.py", line 50, in <module>
>>     import dts
>>   File "/opt/dts/framework/dts.py", line 47, in <module>
>>     from tester import Tester
>>   File "/opt/dts/framework/tester.py", line 52, in <module>
>>     from packet import SCAPY_IMP_CMD
>>   File "/opt/dts/framework/packet.py", line 72, in <module>
>>     locals().update({clazz: getattr(module, clazz)})
>> AttributeError: module 'scapy.contrib.gtp' has no attribute
>> 'GTPPDUSessionContainer'
>>
>>
>>
>> Do you have any clue how to fix this?
>>
>>
>>
>> Thanks.
>>
>> David
>>
>

Re: [dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[Tu, Lijuan]

It seems Trex server is not launched successfully. So DTS as a client can’t connect it.

Here are some tips to make sure Trex can work before integrating into DTS.

·        Simple manual test for trex
o   Go to $TREX/scripts: cd ./scripts
o   Launch dpdk application, e.g. testpmd on dut.  Testpmd can be launch at first, also can be launch after trex server launched, but no matter when you run testpmd, dut must be up before staring trex server.
o   Launch trex server: ./t-rex-64 -i -c 20 # -c means cores number, here we use 20 cores.
./t-rex-64 -c 20 -f /etc/trex_cfg.yaml # if you don't use default name
o   Launch trex client and start traffic:
§ ./trex-console, we give commands to trex through trex-console.
§ start -f stl/bench.py -t size=64,vm=cached --port 0 -m 100% --force  --pin (64 frame size + 100% LR + P0)
§ start -f stl/bench.py -t size=64,vm=cached --port 1 -m 100% --force  --pin (64 frame size + 100% LR + P1)
§ tui (you will get a more friendship view of statistics)
§ Note: you can get other traffic scripts from  ./stl,  stl/bench.py is just the one we used.
o   Notes: to get  the best performance, here are some suggestions:
§ Give trex enough cores.
§ Cores and NIC  are in the same socket.
§ Simply what you want trex to do, complicated computer will cause performance drop.


From: David Liu <dliu@iol.unh.edu>
Sent: 2020年11月20日 3:34
To: Tu, Lijuan <lijuan.tu@intel.com>
Cc: dpdk-dev <dev@dpdk.org>
Subject: Re: DTS Test Nic Single Core Perf Scapy Naming Conflict

Hi Lijuan,

The DTS started without error but the nic single core test failed on a "connection timeout - no response from TRex server" when I tried to run it on an intel nic.
This happens when DTS uses arm-dell (traffic generator) to send packets to arm-gigabyte (DUT) using the Intel nic {Ethernet Controller XL710 for 40GbE QSFP+}
I wonder if you have any clue on this issue?

19/11/2020 19:05:55          TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR: Traceback (most recent call last):
  File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
    case_obj()
  File "tests/TestSuite_nic_single_core_perf.py", line 200, in test_perf_nic_single_core
    self.perf_test(self.nb_ports)
  File "tests/TestSuite_nic_single_core_perf.py", line 274, in perf_test
    _, packets_received = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
  File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
    self._prepare_transmission(stream_ids=stream_ids)
  File "/opt/dts/framework/pktgen_trex.py", line 778, in _prepare_transmission
    self._conn.reset(ports=self._ports)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
    ret = f(*args, **kwargs)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 339, in reset
    self.clear_stats(ports)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
    ret = f(*args, **kwargs)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 1492, in clear_stats
    self._clear_stats_common(ports, clear_global, clear_xstats)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_client.py", line 2876, in _clear_stats_common
    raise TRexError(rc)
trex.common.trex_exceptions.TRexError: ^[[1m*** [RPC] - Failed to get server response from tcp://172.18.0.40:4501<http://172.18.0.40:4501>^[[22m

19/11/2020 19:05:55                            dts:
TEST SUITE ENDED: TestNicSingleCorePerf
19/11/2020 19:06:11          TestNicSingleCorePerf: tear_down_all failed:
Traceback (most recent call last):
  File "/opt/dts/framework/test_case.py", line 430, in execute_tear_downall
    self.tear_down_all()
  File "tests/TestSuite_nic_single_core_perf.py", line 405, in tear_down_all
    self.dut.build_install_dpdk(self.target)
  File "/opt/dts/framework/project_dpdk.py", line 283, in build_install_dpdk
    build_install_dpdk(target, extra_options)
  File "/opt/dts/framework/project_dpdk.py", line 316, in build_install_dpdk_linux_meson
    self.send_expect("rm -rf " + target, "#")
  File "/opt/dts/framework/crb.py", line 93, in send_expect
    return self.session.send_expect(cmds, expected, timeout, verify)
  File "/opt/dts/framework/ssh_connection.py", line 64, in send_expect
    out = self.session.send_expect(cmds, expected, timeout, verify)
  File "/opt/dts/framework/ssh_pexpect.py", line 103, in send_expect
    raise(e)
  File "/opt/dts/framework/ssh_pexpect.py", line 90, in send_expect
    ret = self.send_expect_base(command, expected, timeout)
  File "/opt/dts/framework/ssh_pexpect.py", line 81, in send_expect_base
    self.__prompt(command, timeout)
  File "/opt/dts/framework/ssh_pexpect.py", line 149, in __prompt
    raise TimeoutException(command, self.get_output_all()) from None
exception.TimeoutException: TIMEOUT on rm -rf x86_64-native-linuxapp-gcc

Thanks,
David

On Thu, Nov 19, 2020 at 12:56 PM David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>> wrote:
Hi Lijuan,

Thank you for the help.
I can run DTS without that error now,

Thanks,
David

On Tue, Nov 17, 2020 at 9:12 PM Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>> wrote:

Hi David,



Snice some protocols are only supported by scapy 2.4.4, it's recemented that scapy 2.4.4 as major version for DTS.

However perf users might not care these advanced protocols, to be compatible with trex, dts captured these exceptions and let it go through.

Fix was contained in the latest DTS. Could you update it.



commit 362ae41844e1992163b4eed53a721729985644be

Author: Chen, BoX C <box.c.chen@intel.com<mailto:box.c.chen@intel.com>>

Date:   Wed Nov 4 10:52:09 2020 +0800



    framework: fix Trex mode not working

    DTS requires Trex compatible scapy, modify package loading mode and scapy version verification rules



    Signed-off-by: Chen, BoX C <box.c.chen@intel.com<mailto:box.c.chen@intel.com>>


From: David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>>
Sent: 2020年11月18日 4:24
To: Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
Cc: dpdk-dev <dev@dpdk.org<mailto:dev@dpdk.org>>
Subject: DTS Test Nic Single Core Perf Scapy Naming Conflict

Hi Lijuan,

I currently have issues with running the Nic Single-Core Perf Test.
I believe this is the scapy naming conflict issue, so I follow the solution
mentioned on the website to replace system scapy with trex scapy.
Then I got into another error that I wasn't able to get pass:

root@arm-dell:/opt/dts# ./dts
Traceback (most recent call last):
  File "./main.py", line 50, in <module>
    import dts
  File "/opt/dts/framework/dts.py", line 47, in <module>
    from tester import Tester
  File "/opt/dts/framework/tester.py", line 52, in <module>
    from packet import SCAPY_IMP_CMD
  File "/opt/dts/framework/packet.py", line 72, in <module>
    locals().update({clazz: getattr(module, clazz)})
AttributeError: module 'scapy.contrib.gtp' has no attribute 'GTPPDUSessionContainer'

Do you have any clue how to fix this?

Thanks.
David

Re: [dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[David Liu]

Hi Lijuan,

Thank you for the suggestions.
After that is resolved we encounter another issue.

Currently, the kernel version we have is 5.4.0-53 and has a vfio-pci built
in. But we can not use the module from the nic_devbind.py.
Because we have vfio-pci drivers on the other machine's nic and trex
require both machines to have the same driver on the nic be used.
Do you know what we can approach to solving this issue?

Thank,
David

On Thu, Nov 19, 2020 at 9:10 PM Tu, Lijuan <lijuan.tu@intel.com> wrote:

> It seems Trex server is not launched successfully. So DTS as a client
> can’t connect it.
>
>
>
> Here are some tips to make sure Trex can work before integrating into DTS.
>
>
>
> ·        Simple manual test for trex
>
> o   Go to $TREX/scripts: cd ./scripts
>
> o   Launch dpdk application, e.g. testpmd on dut.  Testpmd can be launch
> at first, also can be launch after trex server launched, but no matter when
> you run testpmd, *dut must be up before staring trex server*.
>
> o   Launch trex server: ./t-rex-64 -i -c 20 # -c means cores number, here
> we use 20 cores.
>
> ./t-rex-64 -c 20 -f /etc/trex_cfg.yaml # if you don't use default name
>
> o   Launch trex client and start traffic:
>
> § ./trex-console, we give commands to trex through trex-console.
>
> § start -f stl/bench.py -t size=64,vm=cached --port 0 -m 100% --force
> --pin (64 frame size + 100% LR + P0)
>
> § start -f stl/bench.py -t size=64,vm=cached --port 1 -m 100% --force
> --pin (64 frame size + 100% LR + P1)
>
> § tui (you will get a more friendship view of statistics)
>
> § Note: you can get other traffic scripts from  ./stl,  stl/bench.py is
> just the one we used.
>
> o   Notes: to get  the best performance, here are some suggestions:
>
> § Give trex enough cores.
>
> § Cores and NIC  are in the same socket.
>
> § Simply what you want trex to do, complicated computer will cause
> performance drop.
>
>
>
>
>
> *From:* David Liu <dliu@iol.unh.edu>
> *Sent:* 2020年11月20日 3:34
> *To:* Tu, Lijuan <lijuan.tu@intel.com>
> *Cc:* dpdk-dev <dev@dpdk.org>
> *Subject:* Re: DTS Test Nic Single Core Perf Scapy Naming Conflict
>
>
>
> Hi Lijuan,
>
>
>
> The DTS started without error but the nic single core test failed on a
> "connection timeout - no response from TRex server" when I tried to run it
> on an intel nic.
>
> This happens when DTS uses arm-dell (traffic generator) to send packets
> to arm-gigabyte (DUT) using the Intel nic {Ethernet Controller XL710 for
> 40GbE QSFP+}
>
> I wonder if you have any clue on this issue?
>
>
>
> 19/11/2020 19:05:55          TestNicSingleCorePerf: Test Case
> test_perf_nic_single_core Result ERROR: Traceback (most recent call last):
>   File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
>     case_obj()
>   File "tests/TestSuite_nic_single_core_perf.py", line 200, in
> test_perf_nic_single_core
>     self.perf_test(self.nb_ports)
>   File "tests/TestSuite_nic_single_core_perf.py", line 274, in perf_test
>     _, packets_received =
> self.tester.pktgen.measure_throughput(stream_ids=streams,
> options=traffic_opt)
>   File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
>     self._prepare_transmission(stream_ids=stream_ids)
>   File "/opt/dts/framework/pktgen_trex.py", line 778, in
> _prepare_transmission
>     self._conn.reset(ports=self._ports)
>   File
> "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
> line 51, in wrap2
>     ret = f(*args, **kwargs)
>   File
> "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
> line 339, in reset
>     self.clear_stats(ports)
>   File
> "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py",
> line 51, in wrap2
>     ret = f(*args, **kwargs)
>   File
> "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py",
> line 1492, in clear_stats
>     self._clear_stats_common(ports, clear_global, clear_xstats)
>   File
> "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_client.py",
> line 2876, in _clear_stats_common
>     raise TRexError(rc)
> trex.common.trex_exceptions.TRexError: ^[[1m*** [RPC] - Failed to get
> server response from tcp://172.18.0.40:4501^[[22m
>
> 19/11/2020 19:05:55                            dts:
> TEST SUITE ENDED: TestNicSingleCorePerf
> 19/11/2020 19:06:11          TestNicSingleCorePerf: tear_down_all failed:
> Traceback (most recent call last):
>   File "/opt/dts/framework/test_case.py", line 430, in execute_tear_downall
>     self.tear_down_all()
>   File "tests/TestSuite_nic_single_core_perf.py", line 405, in
> tear_down_all
>     self.dut.build_install_dpdk(self.target)
>   File "/opt/dts/framework/project_dpdk.py", line 283, in
> build_install_dpdk
>     build_install_dpdk(target, extra_options)
>   File "/opt/dts/framework/project_dpdk.py", line 316, in
> build_install_dpdk_linux_meson
>     self.send_expect("rm -rf " + target, "#")
>   File "/opt/dts/framework/crb.py", line 93, in send_expect
>     return self.session.send_expect(cmds, expected, timeout, verify)
>   File "/opt/dts/framework/ssh_connection.py", line 64, in send_expect
>     out = self.session.send_expect(cmds, expected, timeout, verify)
>   File "/opt/dts/framework/ssh_pexpect.py", line 103, in send_expect
>     raise(e)
>   File "/opt/dts/framework/ssh_pexpect.py", line 90, in send_expect
>     ret = self.send_expect_base(command, expected, timeout)
>   File "/opt/dts/framework/ssh_pexpect.py", line 81, in send_expect_base
>     self.__prompt(command, timeout)
>   File "/opt/dts/framework/ssh_pexpect.py", line 149, in __prompt
>     raise TimeoutException(command, self.get_output_all()) from None
> exception.TimeoutException: TIMEOUT on rm -rf x86_64-native-linuxapp-gcc
>
>
>
> Thanks,
>
> David
>
>
>
> On Thu, Nov 19, 2020 at 12:56 PM David Liu <dliu@iol.unh.edu> wrote:
>
> Hi Lijuan,
>
>
>
> Thank you for the help.
> I can run DTS without that error now,
>
>
>
> Thanks,
>
> David
>
>
>
> On Tue, Nov 17, 2020 at 9:12 PM Tu, Lijuan <lijuan.tu@intel.com> wrote:
>
> Hi David,
>
>
>
> Snice some protocols are only supported by scapy 2.4.4, it's recemented
> that scapy 2.4.4 as major version for DTS.
>
> However perf users might not care these advanced protocols, to be
> compatible with trex, dts captured these exceptions and let it go through.
>
> Fix was contained in the latest DTS. Could you update it.
>
>
>
> commit 362ae41844e1992163b4eed53a721729985644be
>
> Author: Chen, BoX C <box.c.chen@intel.com>
>
> Date:   Wed Nov 4 10:52:09 2020 +0800
>
>
>
>     framework: fix Trex mode not working
>
>     DTS requires Trex compatible scapy, modify package loading mode and
> scapy version verification rules
>
>
>
>     Signed-off-by: Chen, BoX C <box.c.chen@intel.com>
>
>
>
>
>
> *From:* David Liu <dliu@iol.unh.edu>
> *Sent:* 2020年11月18日 4:24
> *To:* Tu, Lijuan <lijuan.tu@intel.com>
> *Cc:* dpdk-dev <dev@dpdk.org>
> *Subject:* DTS Test Nic Single Core Perf Scapy Naming Conflict
>
>
>
> Hi Lijuan,
>
>
>
> I currently have issues with running the Nic Single-Core Perf Test.
>
> I believe this is the scapy naming conflict issue, so I follow the
> solution
>
> mentioned on the website to replace system scapy with trex scapy.
>
> Then I got into another error that I wasn't able to get pass:
>
>
>
> root@arm-dell:/opt/dts# ./dts
> Traceback (most recent call last):
>   File "./main.py", line 50, in <module>
>     import dts
>   File "/opt/dts/framework/dts.py", line 47, in <module>
>     from tester import Tester
>   File "/opt/dts/framework/tester.py", line 52, in <module>
>     from packet import SCAPY_IMP_CMD
>   File "/opt/dts/framework/packet.py", line 72, in <module>
>     locals().update({clazz: getattr(module, clazz)})
> AttributeError: module 'scapy.contrib.gtp' has no attribute
> 'GTPPDUSessionContainer'
>
>
>
> Do you have any clue how to fix this?
>
>
>
> Thanks.
>
> David
>
>

Re: [dpdk-dev] DTS Test Nic Single Core Perf Scapy Naming Conflict

[Tu, Lijuan]

Sorry,  I am not quite understand your problem.

Do you mean that trex require same driver, but you use your build-in vfio-pci for one nic , and own-built on the other nic? If yes, I think you can make your own-built driver  built-in, install it in kernel package.

From: David Liu <dliu@iol.unh.edu>
Sent: 2020年11月25日 4:08
To: Tu, Lijuan <lijuan.tu@intel.com>
Cc: dpdk-dev <dev@dpdk.org>
Subject: Re: DTS Test Nic Single Core Perf Scapy Naming Conflict

Hi Lijuan,

Thank you for the suggestions.
After that is resolved we encounter another issue.

Currently, the kernel version we have is 5.4.0-53 and has a vfio-pci built in. But we can not use the module from the nic_devbind.py.
Because we have vfio-pci drivers on the other machine's nic and trex require both machines to have the same driver on the nic be used.
Do you know what we can approach to solving this issue?

Thank,
David

On Thu, Nov 19, 2020 at 9:10 PM Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>> wrote:
It seems Trex server is not launched successfully. So DTS as a client can’t connect it.

Here are some tips to make sure Trex can work before integrating into DTS.

•        Simple manual test for trex
o   Go to $TREX/scripts: cd ./scripts
o   Launch dpdk application, e.g. testpmd on dut.  Testpmd can be launch at first, also can be launch after trex server launched, but no matter when you run testpmd, dut must be up before staring trex server.
o   Launch trex server: ./t-rex-64 -i -c 20 # -c means cores number, here we use 20 cores.
./t-rex-64 -c 20 -f /etc/trex_cfg.yaml # if you don't use default name
o   Launch trex client and start traffic:
• ./trex-console, we give commands to trex through trex-console.
• start -f stl/bench.py -t size=64,vm=cached --port 0 -m 100% --force  --pin (64 frame size + 100% LR + P0)
• start -f stl/bench.py -t size=64,vm=cached --port 1 -m 100% --force  --pin (64 frame size + 100% LR + P1)
• tui (you will get a more friendship view of statistics)
• Note: you can get other traffic scripts from  ./stl,  stl/bench.py is just the one we used.
o   Notes: to get  the best performance, here are some suggestions:
• Give trex enough cores.
• Cores and NIC  are in the same socket.
• Simply what you want trex to do, complicated computer will cause performance drop.


From: David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>>
Sent: 2020年11月20日 3:34
To: Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
Cc: dpdk-dev <dev@dpdk.org<mailto:dev@dpdk.org>>
Subject: Re: DTS Test Nic Single Core Perf Scapy Naming Conflict

Hi Lijuan,

The DTS started without error but the nic single core test failed on a "connection timeout - no response from TRex server" when I tried to run it on an intel nic.
This happens when DTS uses arm-dell (traffic generator) to send packets to arm-gigabyte (DUT) using the Intel nic {Ethernet Controller XL710 for 40GbE QSFP+}
I wonder if you have any clue on this issue?

19/11/2020 19:05:55          TestNicSingleCorePerf: Test Case test_perf_nic_single_core Result ERROR: Traceback (most recent call last):
  File "/opt/dts/framework/test_case.py", line 319, in _execute_test_case
    case_obj()
  File "tests/TestSuite_nic_single_core_perf.py", line 200, in test_perf_nic_single_core
    self.perf_test(self.nb_ports)
  File "tests/TestSuite_nic_single_core_perf.py", line 274, in perf_test
    _, packets_received = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
  File "/opt/dts/framework/pktgen_base.py", line 245, in measure_throughput
    self._prepare_transmission(stream_ids=stream_ids)
  File "/opt/dts/framework/pktgen_trex.py", line 778, in _prepare_transmission
    self._conn.reset(ports=self._ports)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
    ret = f(*args, **kwargs)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 339, in reset
    self.clear_stats(ports)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_api_annotators.py", line 51, in wrap2
    ret = f(*args, **kwargs)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/stl/trex_stl_client.py", line 1492, in clear_stats
    self._clear_stats_common(ports, clear_global, clear_xstats)
  File "/opt/trex-v2.85/automation/trex_control_plane/interactive/trex/common/trex_client.py", line 2876, in _clear_stats_common
    raise TRexError(rc)
trex.common.trex_exceptions.TRexError: ^[[1m*** [RPC] - Failed to get server response from tcp://172.18.0.40:4501<http://172.18.0.40:4501>^[[22m

19/11/2020 19:05:55                            dts:
TEST SUITE ENDED: TestNicSingleCorePerf
19/11/2020 19:06:11          TestNicSingleCorePerf: tear_down_all failed:
Traceback (most recent call last):
  File "/opt/dts/framework/test_case.py", line 430, in execute_tear_downall
    self.tear_down_all()
  File "tests/TestSuite_nic_single_core_perf.py", line 405, in tear_down_all
    self.dut.build_install_dpdk(self.target)
  File "/opt/dts/framework/project_dpdk.py", line 283, in build_install_dpdk
    build_install_dpdk(target, extra_options)
  File "/opt/dts/framework/project_dpdk.py", line 316, in build_install_dpdk_linux_meson
    self.send_expect("rm -rf " + target, "#")
  File "/opt/dts/framework/crb.py", line 93, in send_expect
    return self.session.send_expect(cmds, expected, timeout, verify)
  File "/opt/dts/framework/ssh_connection.py", line 64, in send_expect
    out = self.session.send_expect(cmds, expected, timeout, verify)
  File "/opt/dts/framework/ssh_pexpect.py", line 103, in send_expect
    raise(e)
  File "/opt/dts/framework/ssh_pexpect.py", line 90, in send_expect
    ret = self.send_expect_base(command, expected, timeout)
  File "/opt/dts/framework/ssh_pexpect.py", line 81, in send_expect_base
    self.__prompt(command, timeout)
  File "/opt/dts/framework/ssh_pexpect.py", line 149, in __prompt
    raise TimeoutException(command, self.get_output_all()) from None
exception.TimeoutException: TIMEOUT on rm -rf x86_64-native-linuxapp-gcc

Thanks,
David

On Thu, Nov 19, 2020 at 12:56 PM David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>> wrote:
Hi Lijuan,

Thank you for the help.
I can run DTS without that error now,

Thanks,
David

On Tue, Nov 17, 2020 at 9:12 PM Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>> wrote:

Hi David,



Snice some protocols are only supported by scapy 2.4.4, it's recemented that scapy 2.4.4 as major version for DTS.

However perf users might not care these advanced protocols, to be compatible with trex, dts captured these exceptions and let it go through.

Fix was contained in the latest DTS. Could you update it.



commit 362ae41844e1992163b4eed53a721729985644be

Author: Chen, BoX C <box.c.chen@intel.com<mailto:box.c.chen@intel.com>>

Date:   Wed Nov 4 10:52:09 2020 +0800



    framework: fix Trex mode not working

    DTS requires Trex compatible scapy, modify package loading mode and scapy version verification rules



    Signed-off-by: Chen, BoX C <box.c.chen@intel.com<mailto:box.c.chen@intel.com>>


From: David Liu <dliu@iol.unh.edu<mailto:dliu@iol.unh.edu>>
Sent: 2020年11月18日 4:24
To: Tu, Lijuan <lijuan.tu@intel.com<mailto:lijuan.tu@intel.com>>
Cc: dpdk-dev <dev@dpdk.org<mailto:dev@dpdk.org>>
Subject: DTS Test Nic Single Core Perf Scapy Naming Conflict

Hi Lijuan,

I currently have issues with running the Nic Single-Core Perf Test.
I believe this is the scapy naming conflict issue, so I follow the solution
mentioned on the website to replace system scapy with trex scapy.
Then I got into another error that I wasn't able to get pass:

root@arm-dell:/opt/dts# ./dts
Traceback (most recent call last):
  File "./main.py", line 50, in <module>
    import dts
  File "/opt/dts/framework/dts.py", line 47, in <module>
    from tester import Tester
  File "/opt/dts/framework/tester.py", line 52, in <module>
    from packet import SCAPY_IMP_CMD
  File "/opt/dts/framework/packet.py", line 72, in <module>
    locals().update({clazz: getattr(module, clazz)})
AttributeError: module 'scapy.contrib.gtp' has no attribute 'GTPPDUSessionContainer'

Do you have any clue how to fix this?

Thanks.
David