From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E30FA0508; Wed, 6 Apr 2022 17:14:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD10C42929; Wed, 6 Apr 2022 17:11:43 +0200 (CEST) Received: from lb.pantheon.sk (lb.pantheon.sk [46.229.239.20]) by mails.dpdk.org (Postfix) with ESMTP id 8B2BC428FD for ; Wed, 6 Apr 2022 17:11:36 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by lb.pantheon.sk (Postfix) with ESMTP id EA89F129C28; Wed, 6 Apr 2022 17:11:35 +0200 (CEST) X-Virus-Scanned: amavisd-new at siecit.sk Received: from lb.pantheon.sk ([127.0.0.1]) by localhost (lb.pantheon.sk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mhe7S5udW6PL; Wed, 6 Apr 2022 17:11:34 +0200 (CEST) Received: from entguard.lab.pantheon.local (unknown [46.229.239.141]) by lb.pantheon.sk (Postfix) with ESMTP id CE74119E0DF; Wed, 6 Apr 2022 17:11:17 +0200 (CEST) From: =?UTF-8?q?Juraj=20Linke=C5=A1?= To: thomas@monjalon.net, david.marchand@redhat.com, Honnappa.Nagarahalli@arm.com, ohilyard@iol.unh.edu, lijuan.tu@intel.com Cc: dev@dpdk.org, =?UTF-8?q?Juraj=20Linke=C5=A1?= Subject: [RFC PATCH v1 23/24] dts: merge DTS doc/dts_gsg/usr_guide/trex.rst to DPDK Date: Wed, 6 Apr 2022 15:11:05 +0000 Message-Id: <20220406151106.2915304-24-juraj.linkes@pantheon.tech> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220406151106.2915304-1-juraj.linkes@pantheon.tech> References: <20220406151106.2915304-1-juraj.linkes@pantheon.tech> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --- dts/doc/dts_gsg/usr_guide/trex.rst | 377 +++++++++++++++++++++++++++++ 1 file changed, 377 insertions(+) create mode 100644 dts/doc/dts_gsg/usr_guide/trex.rst diff --git a/dts/doc/dts_gsg/usr_guide/trex.rst b/dts/doc/dts_gsg/usr_guide/trex.rst new file mode 100644 index 0000000000..8dfa66bdbb --- /dev/null +++ b/dts/doc/dts_gsg/usr_guide/trex.rst @@ -0,0 +1,377 @@ +Practice with TRex +================== + +Download TREX +------------- +TREX should be installed in Tester, it could be downloaded from http://trex-tgn.cisco.com/trex/release/. +We recommend to use the latest version v2.88. + +.. code-block:: console + + wget http://trex-tgn.cisco.com/trex/release/v2.88.tar.gz + +To read more about T-Rex stateless mode, read the +`T-Rex stateless support guide `__. + +Configure TREX +-------------- + +Extract the downloaded TREX tarball +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Extract the downloaded TREX file to /opt/trex/:: + + root@tester:/opt# mkdir trex + root@tester:/opt# cd trex + root@tester:/opt/trex# tar -zxvf v2.88.tar.gz + v2.88/ + v2.88/_t-rex-64-debug + v2.88/t-rex-64-debug + ... + v2.88/trex_client_v2.88.tar.gz + +Generate Configure File +~~~~~~~~~~~~~~~~~~~~~~~~ + +Identify the performance test ports in tester +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +execute the command like below to list the ports:: + + root@tester:/opt/trex/v2.88# ./dpdk_setup_ports.py -s + + Network devices using DPDK-compatible driver + ============================================ + + + Network devices using kernel driver + =================================== + 0000:03:00.0 'VMXNET3 Ethernet Controller' if=ens160 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic *Active* + 0000:0b:00.0 'Device 1592' if=ens192f0 drv=ice unused=igb_uio,vfio-pci,uio_pci_generic + 0000:0b:00.1 'Device 1592' if=ens192f1 drv=ice unused=igb_uio,vfio-pci,uio_pci_generic + Other network devices + ===================== + + +Generate Configure File +^^^^^^^^^^^^^^^^^^^^^^^ +Assume that we use 0000:0b:00.0, 0000:0b:00.1 to connect to DUT port. +And the ports topology as below:: + + trex.0 (0b:00.0/40:a6:b7:0b:55:78) <-------> DUT port 0 (05:00.0/b4:96:91:9f:63:68) + trex.1 (0b:00.1/40:a6:b7:0b:55:79) <-------> DUT port 1 (05:00.1/b4:96:91:9f:63:69) + +We need to generate the trex config file as below:: + + root@tester:/opt/v2.88# ./dpdk_setup_ports.py -i + By default, IP based configuration file will be created. Do you want to use MAC based config? (y/N)y + +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+ + | ID | NUMA | PCI | MAC | Name | Driver | Linux IF | Active | + +====+======+=========+===================+============================================+=========+==========+==========+ + | 0 | -1 | 03:00.0 | 00:0c:29:29:30:80 | VMXNET3 Ethernet Controller | vmxnet3 | ens160 | *Active* | + +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+ + | 1 | -1 | 0b:00.0 | 40:a6:b7:0b:55:78 | Device 1592 | ice | ens192f0 | | + +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+ + | 2 | -1 | 0b:00.1 | 40:a6:b7:0b:55:79 | Device 1592 | ice | ens192f1 | | + +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+ + Please choose an even number of interfaces from the list above, either by ID, PCI or Linux IF + Stateful will use order of interfaces: Client1 Server1 Client2 Server2 etc. for flows. + Stateless can be in any order. + Enter list of interfaces separated by space (for example: 1 3) : 1 2 + For interface 1, assuming loopback to its dual interface 2. + Destination MAC is 40:a6:b7:0b:55:79. Change it to MAC of DUT? (y/N).y + Please enter a new destination MAC of interface 1: b4:96:91:9f:63:68 + For interface 2, assuming loopback to its dual interface 1. + Destination MAC is 40:a6:b7:0b:55:78. Change it to MAC of DUT? (y/N).y + Please enter a new destination MAC of interface 2: b4:96:91:9f:63:69 + Print preview of generated config? (Y/n)Y + ### Config file generated by dpdk_setup_ports.py ### + + - version: 2 + interfaces: ['0b:00.0', '0b:00.1'] + port_info: + - dest_mac: b4:96:91:9f:63:68 + src_mac: 40:a6:b7:0b:55:78 + - dest_mac: b4:96:91:9f:63:69 + src_mac: 40:a6:b7:0b:55:79 + + platform: + master_thread_id: 0 + latency_thread_id: 1 + dual_if: + - socket: 0 + threads: [2,3,4,5,6,7] + + Save the config to file? (Y/n)Y + Default filename is /etc/trex_cfg.yaml + Press ENTER to confirm or enter new file: + File /etc/trex_cfg.yaml already exist, overwrite? (y/N)Y + Saved to /etc/trex_cfg.yaml. + root@tester:/opt/trex/v2.88# + +We could not modify the DUT mac during the configuration and modify it in the generated file /etc/trex_cfg.yaml. + +Modify Configure File +^^^^^^^^^^^^^^^^^^^^^ +Make sure the DUT mac of the generated TREX file is correct, and add prefix and limit_memory is better:: + + root@tester:/opt/trex/v2.88# cat /etc/trex_cfg.yaml + ### Config file generated by dpdk_setup_ports.py ### + + - version: 2 + interfaces: ['0b:00.0', '0b:00.1'] + prefix: TREX + limit_memory: 4096 + port_info: + - dest_mac: b4:96:91:9f:63:68 + src_mac: 40:a6:b7:0b:55:78 + - dest_mac: b4:96:91:9f:63:69 + src_mac: 40:a6:b7:0b:55:79 + + platform: + master_thread_id: 0 + latency_thread_id: 1 + dual_if: + - socket: 0 + threads: [2,3,4,5,6,7] + + root@tester:/opt/trex/v2.88# + +Replace system scapy with TREX scapy +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +As the know trex issue -- trex scapy lib will be conflict with scapy installed in OS, we need the replace the scapy +in system with trex scapy. + +#. backup your scapy:: + + cp -rf /usr/local/lib/python3.6/dist-packages/scapy /usr/local/lib/python3.6/dist-packages/scapy_backup + +#. unify scapy version with trex:: + + cp -rf /opt/trex/v2.88/external_libs/scapy-2.4.3/scapy/ /usr/local/lib/python3.6/dist-packages/scapy + +Configure DTS +-------------- + +Configure execution.cfg +~~~~~~~~~~~~~~~~~~~~~~~ +Add the performance test suite and set perf=true:: + + root@tester:dts# cat execution.cfg + [Execution1] + crbs=192.168.1.1 + drivername=vfio-pci + build_type=meson + test_suites= + tso, + targets= + x86_64-native-linuxapp-gcc + parameters=nic_type=cfg:perf=true + root@tester:dts# + +Configure $DTS_CFG_FOLDER/crbs.cfg +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Set the pktgen_group=trex or pktgen_group=TREX, this item is case insensitive:: + + root@tester:dts# cat $DTS_CFG_FOLDER/crbs.cfg + [192.168.1.1] + dut_ip=192.168.1.1 + dut_user=root + dut_passwd=dutpwd + os=linux + dut_arch= + tester_ip=192.168.1.2 + tester_passwd=testerpwd + ixia_group= + pktgen_group=trex + channels=4 + bypass_core0=True + +Configure $DTS_CFG_FOLDER/ports.cfg +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This configuration is just same with PF function test, so if you have completed some functional test, +you have no need to modify the $DTS_CFG_FOLDER/ports.cfg, just like below:: + + root@tester:dts# cat $DTS_CFG_FOLDER/ports.cfg + [192.168.1.1] + ports = + pci=0000:05:00.0,peer=0000:0b:00.0; + pci=0000:05:00.1,peer=0000:0b:00.1; + +In addition, it could be configured as below:: + + root@tester:dts# cat $DTS_CFG_FOLDER/ports.cfg + [192.168.1.1] + ports = + pci=0000:05:00.0,peer=TREX:0; + pci=0000:05:00.1,peer=TREX:1; + +We recommend to use the first format configuration, as it has no need to do modification when we do functional test. + +Configure $DTS_CFG_FOLDER/pktgen.cfg +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Fill in the $DTS_CFG_FOLDER/pktgen.cfg with your TREX setting, for the first time, you should set start_trex=yes. +Set the IP address of the device which you installed TREX into item ‘server=’, it should be the tester IP. +The configuration should as below:: + + root@tester:dts# cat $DTS_CFG_FOLDER/pktgen.cfg + [TREX] + trex_root_path=/opt/trex/v2.88 + trex_lib_path=/opt/trex/v2.88/automation/trex_control_plane/interactive + config_file=/etc/trex_cfg.yaml + server=192.168.1.1 + pcap_file=/opt/trex/v2.88/stl/sample.pcap + core_num=4 + #core_mask=0x3 + ip_src=16.0.0.1 + ip_dst=10.0.0.1 + warmup=15 + duration=-1 + start_trex=yes + +As the trex_lib_path may be different in different versions, you could find the correct path as blow command:: + + root@tester:/opt/trex/v2.88# find . -name trex_stl_lib + ./automation/trex_control_plane/interactive/trex_stl_lib + root@tester:/opt/trex/v2.88# + + +Run DTS performance test with TREX +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Now you can start DTS performance test with TREX:: + + root@tester:/home/zhaohy/dts# ./dts + dts: + DUT 192.168.1.1 + tester: ssh root@192.168.1.1 + ... + pktgen: ssh root@192.168.1.1 + pktgen: cd /opt/trex/v2.88;./t-rex-64 -i --cfg /etc/trex_cfg.yaml -c 4 + pktgen: Starting Scapy server..... Scapy server is started + Trying to bind to igb_uio ... + /usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:0b:00.0 0000:0b:00.1 + ... + TestTSO: Test Case test_perf_TSO_2ports Begin + dut.10.240.183.72: + tester: + TestTSO: Executing PMD using 1S/1C/2T + dut.10.240.183.72: x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1,45 -n 4 -a 0000:05:00.0 -a 0000:05:00.1 --file-prefix=dpdk_31529_20210324143008 -- -i --rxd=512 --txd=512 --burst=32 --rxfreet=64 --mbcache=128 --portmask=0x3 --max-pkt-len=9000 --txpt=36 --txht=0 --txwt=0 --txfreet=32 --txrst=32 + dut.10.240.183.72: EAL: Detected 88 lcore(s) + ... + pktgen: Rx Port 0 stats: + rx_port: 0, rx_bps: 25354096640.000000, rx_pps: 1239130.250000 + pktgen: throughput: pps_rx 5463897.750000, bps_rx 50961129472.000000 + pktgen: traffic completed. + ... + TestTSO: + +------------+---------------+------------+ + | Frame Size | 1S/1C/2T Mpps | % linerate | + +============+===============+============+ + | 128 | 5.371 | 4 | + +------------+---------------+------------+ + | 2500 | 5.464 | 56 | + +------------+---------------+------------+ + TestTSO: Test Case test_perf_TSO_2ports Result PASSED + +FAQ +--- + +dpdk hugepage management conflict issue +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +trex use older dpdk version than we release cycle source code. When dpdk change +the memory management merchanism, trex will meet the following issue. + +Trex should run on an independent platform. DUT/Trex should run on two platforms: + +* one is used as TESTER and trex server, another one is used as DUT.(dts/pktgen) +* one is used as trex server, another one is used as DUT/TESTER.(recommended scheme) + This scheme can make sure that trex run on its full status capability. + +When trex run with dts on the same platform, trex server sometimes boot up +failed for hugepage error. + +.. code-block:: console + + ./t-rex-64 -i --stl -k 4 + + Starting Scapy server..... Scapy server is started + Trying to bind to igb_uio ... + /usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:85:00.0 0000:8a:00.1 + The ports are bound/configured. + Starting TRex v2.41 please wait ... + EAL: Can only reserve 1766 pages from 4096 requested + Current CONFIG_RTE_MAX_MEMSEG=256 is not enough + Please either increase it or request less amount of memory. + EAL: FATAL: Cannot init memory + + EAL: Cannot init memory + + You might need to run ./trex-cfg once + EAL: Error - exiting with code: 1 + Cause: Invalid EAL arguments + +trex quit when using Niantic +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +when bind dut NNT port to igb_uio, peer port will get a link down status, then +trex server using NNT nic will quit. + +.. code-block:: console + + WATCHDOG: task 'master' has not responded for more than 2.00044 seconds - timeout is 2 seconds + + *** traceback follows *** + + 1 0x55a7c779561a ./_t-rex-64(+0x12761a) [0x55a7c779561a] + 2 0x7f23da4be1b0 /lib64/libpthread.so.0(+0x121b0) [0x7f23da4be1b0] + 3 0x55a7c7942d40 rte_delay_us_block + 128 + 4 0x55a7c798d731 ixgbe_setup_mac_link_multispeed_fiber + 337 + 5 0x55a7c79a8f14 ./_t-rex-64(+0x33af14) [0x55a7c79a8f14] + 6 0x55a7c7954c72 rte_eth_link_get_nowait + 114 + 7 0x55a7c776a988 DpdkTRexPortAttr::update_link_status_nowait() + 24 + 8 0x55a7c77856a6 CGlobalTRex::handle_slow_path() + 118 + 9 0x55a7c7785ad7 CGlobalTRex::run_in_master() + 759 + 10 0x55a7c7785e3c ./_t-rex-64(+0x117e3c) [0x55a7c7785e3c] + 11 0x55a7c793efba rte_eal_mp_remote_launch + 346 + 12 0x55a7c7789e1e main_test(int, char**) + 1038 + 13 0x7f23d9417f2a __libc_start_main + 234 + 14 0x55a7c7719b9d ./_t-rex-64(+0xabb9d) [0x55a7c7719b9d] + + + *** addr2line information follows *** + + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + ??:0 + + + ./t-rex-64: line 80: 25870 Aborted (core dumped) ./_$( + +other issues +~~~~~~~~~~~~ + +#. linux kernel version should not be too low. + +#. Trex only works with even number link peers. + +#. Trex only works with nics, which are using the same driver. + +#. Before boot up trex, please make sure the peer ports are on up status. + +#. If you have ran dpdk on the platform which you want to deploy trex-server, + reboot the platform to make sure that trex-server can work fine. + +#. If using i40e driver, Trex v2.41 version need i40e nic firmware version newer than 5.02. + +#. trex will drop the received packet, which dst mac is the port mac address. -- 2.20.1