[RFC PATCH v1 23/24] dts: merge DTS doc/dts_gsg/usr_guide/trex.rst to DPDK

["Juraj Linkeš" <juraj.linkes@pantheon.tech>]

---
 dts/doc/dts_gsg/usr_guide/trex.rst | 377 +++++++++++++++++++++++++++++
 1 file changed, 377 insertions(+)
 create mode 100644 dts/doc/dts_gsg/usr_guide/trex.rst

diff --git a/dts/doc/dts_gsg/usr_guide/trex.rst b/dts/doc/dts_gsg/usr_guide/trex.rst
new file mode 100644
index 0000000000..8dfa66bdbb
--- /dev/null
+++ b/dts/doc/dts_gsg/usr_guide/trex.rst
@@ -0,0 +1,377 @@
+Practice with TRex
+==================
+
+Download TREX
+-------------
+TREX should be installed in Tester, it could be downloaded from http://trex-tgn.cisco.com/trex/release/.
+We recommend to use the latest version v2.88.
+
+.. code-block:: console
+
+    wget http://trex-tgn.cisco.com/trex/release/v2.88.tar.gz
+
+To read more about T-Rex stateless mode, read the
+`T-Rex stateless support guide <https://trex-tgn.cisco.com/trex/doc/trex_stateless.html>`__.
+
+Configure TREX
+--------------
+
+Extract the downloaded TREX tarball
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Extract the downloaded TREX file to /opt/trex/::
+
+    root@tester:/opt# mkdir trex
+    root@tester:/opt# cd trex
+    root@tester:/opt/trex# tar -zxvf v2.88.tar.gz
+    v2.88/
+    v2.88/_t-rex-64-debug
+    v2.88/t-rex-64-debug
+    ...
+    v2.88/trex_client_v2.88.tar.gz
+
+Generate Configure File
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Identify the performance test ports in tester
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+execute the command like below to list the ports::
+
+    root@tester:/opt/trex/v2.88# ./dpdk_setup_ports.py -s
+
+    Network devices using DPDK-compatible driver
+    ============================================
+    <none>
+
+    Network devices using kernel driver
+    ===================================
+    0000:03:00.0 'VMXNET3 Ethernet Controller' if=ens160 drv=vmxnet3 unused=igb_uio,vfio-pci,uio_pci_generic *Active*
+    0000:0b:00.0 'Device 1592' if=ens192f0 drv=ice unused=igb_uio,vfio-pci,uio_pci_generic
+    0000:0b:00.1 'Device 1592' if=ens192f1 drv=ice unused=igb_uio,vfio-pci,uio_pci_generic
+    Other network devices
+    =====================
+    <none>
+
+Generate Configure File
+^^^^^^^^^^^^^^^^^^^^^^^
+Assume that we use 0000:0b:00.0, 0000:0b:00.1 to connect to DUT port.
+And the ports topology as below::
+
+    trex.0 (0b:00.0/40:a6:b7:0b:55:78) <-------> DUT port 0 (05:00.0/b4:96:91:9f:63:68)
+    trex.1 (0b:00.1/40:a6:b7:0b:55:79) <-------> DUT port 1 (05:00.1/b4:96:91:9f:63:69)
+
+We need to generate the trex config file as below::
+
+    root@tester:/opt/v2.88# ./dpdk_setup_ports.py -i
+    By default, IP based configuration file will be created. Do you want to use MAC based config? (y/N)y
+    +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+
+    | ID | NUMA |   PCI   |        MAC        |                    Name                    | Driver  | Linux IF |  Active  |
+    +====+======+=========+===================+============================================+=========+==========+==========+
+    | 0  | -1   | 03:00.0 | 00:0c:29:29:30:80 | VMXNET3 Ethernet Controller                | vmxnet3 | ens160   | *Active* |
+    +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+
+    | 1  | -1   | 0b:00.0 | 40:a6:b7:0b:55:78 | Device 1592                                | ice     | ens192f0 |          |
+    +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+
+    | 2  | -1   | 0b:00.1 | 40:a6:b7:0b:55:79 | Device 1592                                | ice     | ens192f1 |          |
+    +----+------+---------+-------------------+--------------------------------------------+---------+----------+----------+
+    Please choose an even number of interfaces from the list above, either by ID, PCI or Linux IF
+    Stateful will use order of interfaces: Client1 Server1 Client2 Server2 etc. for flows.
+    Stateless can be in any order.
+    Enter list of interfaces separated by space (for example: 1 3) : 1 2
+    For interface 1, assuming loopback to its dual interface 2.
+    Destination MAC is 40:a6:b7:0b:55:79. Change it to MAC of DUT? (y/N).y
+    Please enter a new destination MAC of interface 1: b4:96:91:9f:63:68
+    For interface 2, assuming loopback to its dual interface 1.
+    Destination MAC is 40:a6:b7:0b:55:78. Change it to MAC of DUT? (y/N).y
+    Please enter a new destination MAC of interface 2: b4:96:91:9f:63:69
+    Print preview of generated config? (Y/n)Y
+    ### Config file generated by dpdk_setup_ports.py ###
+
+    - version: 2
+      interfaces: ['0b:00.0', '0b:00.1']
+      port_info:
+          - dest_mac: b4:96:91:9f:63:68
+            src_mac:  40:a6:b7:0b:55:78
+          - dest_mac: b4:96:91:9f:63:69
+            src_mac:  40:a6:b7:0b:55:79
+
+      platform:
+          master_thread_id: 0
+          latency_thread_id: 1
+          dual_if:
+            - socket: 0
+              threads: [2,3,4,5,6,7]
+
+    Save the config to file? (Y/n)Y
+    Default filename is /etc/trex_cfg.yaml
+    Press ENTER to confirm or enter new file:
+    File /etc/trex_cfg.yaml already exist, overwrite? (y/N)Y
+    Saved to /etc/trex_cfg.yaml.
+    root@tester:/opt/trex/v2.88#
+
+We could not modify the DUT mac during the configuration and modify it in the generated file /etc/trex_cfg.yaml.
+
+Modify Configure File
+^^^^^^^^^^^^^^^^^^^^^
+Make sure the DUT mac of the generated TREX file is correct, and add prefix and limit_memory is better::
+
+    root@tester:/opt/trex/v2.88# cat /etc/trex_cfg.yaml
+    ### Config file generated by dpdk_setup_ports.py ###
+
+    - version: 2
+      interfaces: ['0b:00.0', '0b:00.1']
+      prefix: TREX
+      limit_memory: 4096
+      port_info:
+          - dest_mac: b4:96:91:9f:63:68
+            src_mac:  40:a6:b7:0b:55:78
+          - dest_mac: b4:96:91:9f:63:69
+            src_mac:  40:a6:b7:0b:55:79
+
+      platform:
+          master_thread_id: 0
+          latency_thread_id: 1
+          dual_if:
+            - socket: 0
+              threads: [2,3,4,5,6,7]
+
+    root@tester:/opt/trex/v2.88#
+
+Replace system scapy with TREX scapy
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+As the know trex issue -- trex scapy lib will be conflict with scapy installed in OS, we need the replace the scapy
+in system with trex scapy.
+
+#. backup your scapy::
+
+    cp -rf /usr/local/lib/python3.6/dist-packages/scapy /usr/local/lib/python3.6/dist-packages/scapy_backup
+
+#. unify scapy version with trex::
+
+    cp  -rf /opt/trex/v2.88/external_libs/scapy-2.4.3/scapy/ /usr/local/lib/python3.6/dist-packages/scapy
+
+Configure DTS
+--------------
+
+Configure execution.cfg
+~~~~~~~~~~~~~~~~~~~~~~~
+Add the performance test suite and set perf=true::
+
+    root@tester:dts# cat execution.cfg
+    [Execution1]
+    crbs=192.168.1.1
+    drivername=vfio-pci
+    build_type=meson
+    test_suites=
+            tso,
+    targets=
+        x86_64-native-linuxapp-gcc
+    parameters=nic_type=cfg:perf=true
+    root@tester:dts#
+
+Configure $DTS_CFG_FOLDER/crbs.cfg
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Set the pktgen_group=trex or pktgen_group=TREX, this item is case insensitive::
+
+    root@tester:dts# cat $DTS_CFG_FOLDER/crbs.cfg
+    [192.168.1.1]
+    dut_ip=192.168.1.1
+    dut_user=root
+    dut_passwd=dutpwd
+    os=linux
+    dut_arch=
+    tester_ip=192.168.1.2
+    tester_passwd=testerpwd
+    ixia_group=
+    pktgen_group=trex
+    channels=4
+    bypass_core0=True
+
+Configure $DTS_CFG_FOLDER/ports.cfg
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+This configuration is just same with PF function test, so if you have completed some functional test,
+you have no need to modify the $DTS_CFG_FOLDER/ports.cfg, just like below::
+
+    root@tester:dts# cat $DTS_CFG_FOLDER/ports.cfg
+    [192.168.1.1]
+    ports =
+        pci=0000:05:00.0,peer=0000:0b:00.0;
+        pci=0000:05:00.1,peer=0000:0b:00.1;
+
+In addition, it could be configured as below::
+
+    root@tester:dts# cat $DTS_CFG_FOLDER/ports.cfg
+    [192.168.1.1]
+    ports =
+        pci=0000:05:00.0,peer=TREX:0;
+        pci=0000:05:00.1,peer=TREX:1;
+
+We recommend to use the first format configuration, as it has no need to do modification when we do functional test.
+
+Configure $DTS_CFG_FOLDER/pktgen.cfg
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Fill in the $DTS_CFG_FOLDER/pktgen.cfg with your TREX setting, for the first time, you should set start_trex=yes.
+Set the IP address of the device which you installed TREX into item ‘server=’, it should be the tester IP.
+The configuration should as below::
+
+    root@tester:dts# cat $DTS_CFG_FOLDER/pktgen.cfg
+    [TREX]
+    trex_root_path=/opt/trex/v2.88
+    trex_lib_path=/opt/trex/v2.88/automation/trex_control_plane/interactive
+    config_file=/etc/trex_cfg.yaml
+    server=192.168.1.1
+    pcap_file=/opt/trex/v2.88/stl/sample.pcap
+    core_num=4
+    #core_mask=0x3
+    ip_src=16.0.0.1
+    ip_dst=10.0.0.1
+    warmup=15
+    duration=-1
+    start_trex=yes
+
+As the trex_lib_path may be different in different versions, you could find the correct path as blow command::
+
+    root@tester:/opt/trex/v2.88# find . -name trex_stl_lib
+    ./automation/trex_control_plane/interactive/trex_stl_lib
+    root@tester:/opt/trex/v2.88#
+
+
+Run DTS performance test with TREX
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Now you can start DTS performance test with TREX::
+
+    root@tester:/home/zhaohy/dts# ./dts
+                           dts:
+    DUT 192.168.1.1
+                        tester: ssh root@192.168.1.1
+    ...
+    pktgen: ssh root@192.168.1.1
+    pktgen: cd /opt/trex/v2.88;./t-rex-64 -i --cfg /etc/trex_cfg.yaml -c 4
+                        pktgen: Starting Scapy server..... Scapy server is started
+    Trying to bind to igb_uio ...
+    /usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:0b:00.0 0000:0b:00.1
+    ...
+                     TestTSO: Test Case test_perf_TSO_2ports Begin
+             dut.10.240.183.72:
+                        tester:
+                       TestTSO: Executing PMD using 1S/1C/2T
+             dut.10.240.183.72: x86_64-native-linuxapp-gcc/app/dpdk-testpmd  -l 1,45 -n 4 -a 0000:05:00.0 -a 0000:05:00.1  --file-prefix=dpdk_31529_20210324143008   -- -i --rxd=512 --txd=512 --burst=32 --rxfreet=64 --mbcache=128 --portmask=0x3 --max-pkt-len=9000 --txpt=36 --txht=0 --txwt=0 --txfreet=32 --txrst=32
+             dut.10.240.183.72: EAL: Detected 88 lcore(s)
+             ...
+    pktgen: Rx Port 0 stats:
+    rx_port: 0,  rx_bps: 25354096640.000000, rx_pps: 1239130.250000
+                        pktgen: throughput: pps_rx 5463897.750000, bps_rx 50961129472.000000
+                        pktgen: traffic completed.
+    ...
+                       TestTSO:
+    +------------+---------------+------------+
+    | Frame Size | 1S/1C/2T Mpps | % linerate |
+    +============+===============+============+
+    | 128        | 5.371         | 4          |
+    +------------+---------------+------------+
+    | 2500       | 5.464         | 56         |
+    +------------+---------------+------------+
+                           TestTSO: Test Case test_perf_TSO_2ports Result PASSED
+
+FAQ
+---
+
+dpdk hugepage management conflict issue
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+trex use older dpdk version than we release cycle source code. When dpdk change
+the memory management merchanism, trex will meet the following issue.
+
+Trex should run on an independent platform. DUT/Trex should run on two platforms:
+
+* one is used as TESTER and trex server, another one is used as DUT.(dts/pktgen)
+* one is used as trex server, another one is used as DUT/TESTER.(recommended scheme)
+  This scheme can make sure that trex run on its full status capability.
+
+When trex run with dts on the same platform, trex server sometimes boot up
+failed for hugepage error.
+
+.. code-block:: console
+
+      ./t-rex-64  -i --stl -k 4
+
+         Starting Scapy server..... Scapy server is started
+         Trying to bind to igb_uio ...
+         /usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:85:00.0 0000:8a:00.1
+         The ports are bound/configured.
+         Starting  TRex v2.41 please wait  ...
+         EAL: Can only reserve 1766 pages from 4096 requested
+         Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
+         Please either increase it or request less amount of memory.
+         EAL: FATAL: Cannot init memory
+
+         EAL: Cannot init memory
+
+          You might need to run ./trex-cfg  once
+         EAL: Error - exiting with code: 1
+           Cause: Invalid EAL arguments
+
+trex quit when using Niantic
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+when bind dut NNT port to igb_uio, peer port will get a link down status, then
+trex server using NNT nic will quit.
+
+.. code-block:: console
+
+   WATCHDOG: task 'master' has not responded for more than 2.00044 seconds - timeout is 2 seconds
+
+   *** traceback follows ***
+
+   1       0x55a7c779561a ./_t-rex-64(+0x12761a) [0x55a7c779561a]
+   2       0x7f23da4be1b0 /lib64/libpthread.so.0(+0x121b0) [0x7f23da4be1b0]
+   3       0x55a7c7942d40 rte_delay_us_block + 128
+   4       0x55a7c798d731 ixgbe_setup_mac_link_multispeed_fiber + 337
+   5       0x55a7c79a8f14 ./_t-rex-64(+0x33af14) [0x55a7c79a8f14]
+   6       0x55a7c7954c72 rte_eth_link_get_nowait + 114
+   7       0x55a7c776a988 DpdkTRexPortAttr::update_link_status_nowait() + 24
+   8       0x55a7c77856a6 CGlobalTRex::handle_slow_path() + 118
+   9       0x55a7c7785ad7 CGlobalTRex::run_in_master() + 759
+   10      0x55a7c7785e3c ./_t-rex-64(+0x117e3c) [0x55a7c7785e3c]
+   11      0x55a7c793efba rte_eal_mp_remote_launch + 346
+   12      0x55a7c7789e1e main_test(int, char**) + 1038
+   13      0x7f23d9417f2a __libc_start_main + 234
+   14      0x55a7c7719b9d ./_t-rex-64(+0xabb9d) [0x55a7c7719b9d]
+
+
+   *** addr2line information follows ***
+
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+   ??:0
+
+
+   ./t-rex-64: line 80: 25870 Aborted                 (core dumped) ./_$(
+
+other issues
+~~~~~~~~~~~~
+
+#. linux kernel version should not be too low.
+
+#. Trex only works with even number link peers.
+
+#. Trex only works with nics, which are using the same driver.
+
+#. Before boot up trex, please make sure the peer ports are on up status.
+
+#. If you have ran dpdk on the platform which you want to deploy trex-server,
+   reboot the platform to make sure that trex-server can work fine.
+
+#. If using i40e driver, Trex v2.41 version need i40e nic firmware version newer than 5.02.
+
+#. trex will drop the received packet, which dst mac is the port mac address.
-- 
2.20.1