* [PATCH V2 0/3] add vswitch_sample_dsa
@ 2023-03-27 7:30 Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 1/3] test_plans/index: " Wei Ling
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Wei Ling @ 2023-03-27 7:30 UTC (permalink / raw)
To: dts; +Cc: Wei Ling
Add vswitch_sample_dsa testplan and testsuite to test dpdk-vhost APP
with host DMA operation callbacks for DSA PMD and vhost async data-path.
Wei Ling (3):
test_plans/index: add vswitch_sample_dsa
test_plans/vswitch_sample_dsa: add new testplan
tests/vswitch_sample_dsa: add new testsuite
test_plans/index.rst | 1 +
test_plans/vswitch_sample_dsa_test_plan.rst | 968 +++++++++++++++++
tests/TestSuite_vswitch_sample_dsa.py | 1033 +++++++++++++++++++
3 files changed, 2002 insertions(+)
create mode 100644 test_plans/vswitch_sample_dsa_test_plan.rst
create mode 100644 tests/TestSuite_vswitch_sample_dsa.py
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V2 1/3] test_plans/index: add vswitch_sample_dsa
2023-03-27 7:30 [PATCH V2 0/3] add vswitch_sample_dsa Wei Ling
@ 2023-03-27 7:30 ` Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 2/3] test_plans/vswitch_sample_dsa: add new testplan Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 3/3] tests/vswitch_sample_dsa: add new testsuite Wei Ling
2 siblings, 0 replies; 5+ messages in thread
From: Wei Ling @ 2023-03-27 7:30 UTC (permalink / raw)
To: dts; +Cc: Wei Ling
Add vswitch_sample_dsa_test_plan in test_plans/index.rst.
Signed-off-by: Wei Ling <weix.ling@intel.com>
---
test_plans/index.rst | 1 +
1 file changed, 1 insertion(+)
diff --git a/test_plans/index.rst b/test_plans/index.rst
index 0770a935..69a81697 100644
--- a/test_plans/index.rst
+++ b/test_plans/index.rst
@@ -243,6 +243,7 @@ The following are the test plans for the DPDK DTS automated test system.
dpdk_gro_lib_cbdma_test_plan
dpdk_gso_lib_test_plan
vswitch_sample_cbdma_test_plan
+ vswitch_sample_dsa_test_plan
vswitch_pvp_multi_paths_performance_with_cbdma_test_plan
vxlan_gpe_support_in_i40e_test_plan
pvp_diff_qemu_version_test_plan
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V2 2/3] test_plans/vswitch_sample_dsa: add new testplan
2023-03-27 7:30 [PATCH V2 0/3] add vswitch_sample_dsa Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 1/3] test_plans/index: " Wei Ling
@ 2023-03-27 7:30 ` Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 3/3] tests/vswitch_sample_dsa: add new testsuite Wei Ling
2 siblings, 0 replies; 5+ messages in thread
From: Wei Ling @ 2023-03-27 7:30 UTC (permalink / raw)
To: dts; +Cc: Wei Ling
Add vswitch_sample_dsa testplan to test dpdk-vhost APP with host DMA operation
callbacks for DSA PMD and vhost async data-path.
Signed-off-by: Wei Ling <weix.ling@intel.com>
---
test_plans/vswitch_sample_dsa_test_plan.rst | 968 ++++++++++++++++++++
1 file changed, 968 insertions(+)
create mode 100644 test_plans/vswitch_sample_dsa_test_plan.rst
diff --git a/test_plans/vswitch_sample_dsa_test_plan.rst b/test_plans/vswitch_sample_dsa_test_plan.rst
new file mode 100644
index 00000000..f3e71fb0
--- /dev/null
+++ b/test_plans/vswitch_sample_dsa_test_plan.rst
@@ -0,0 +1,968 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2023 Intel Corporation
+
+========================================================
+Vswitch sample test with vhost async data path test plan
+========================================================
+
+Description
+===========
+
+Vswitch sample can leverage DMA to accelerate vhost async data-path from dpdk 20.11. This plan test
+vhost DMA operation callbacks for DSA PMD and vhost async data-path in vhost sample.
+
+--dmas This parameter is used to specify the assigned DMA device of a vhost device. Async vhost-user
+net driver will be used if --dmas is set. For example –dmas [txd0@wq0.0,txd1@wq0.1,rxd0@wq0.2,rxd1@wq0.3]
+means use DMA channel wq0.0/wq0.2 for vhost device 0 enqueue/dequeue operation and use DMA channel
+wq0.1/wq0.3 for vhost device 1 enqueue/dequeue operation. The index of the device corresponds to
+the socket file in order, that means vhost device 0 is created through the first socket file,
+vhost device 1 is created through the second socket file, and so on.
+
+For more about vswitch example, please refer to the DPDK docment:
+http://doc.dpdk.org/guides/sample_app_ug/vhost.html
+
+Prerequisites
+=============
+
+Hardware
+--------
+Supportted NICs: NIC that supports VMDQ
+
+Software
+--------
+Trex:http://trex-tgn.cisco.com/trex/release/v2.26.tar.gz
+
+General set up
+--------------
+1. Compile DPDK and vhost example::
+
+ # CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib -Dexamples=all --default-library=static x86_64-native-linuxapp-gcc
+ # meson configure -Dexamples=vhost x86_64-native-linuxapp-gcc
+ # ninja -C x86_64-native-linuxapp-gcc -j 110
+
+2. Get the PCI device of DUT, for example, 0000:29:00.0 is NIC port, 0000:6a:01.0 - 0000:f6:01.0 are DSA devices::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -s
+
+ Network devices using kernel driver
+ ===================================
+ 0000:29:00.0 'Ethernet Controller E810-C for QSFP 1592' drv=ice unused=vfio-pci
+
+ DMA devices using kernel driver
+ ===============================
+ 0000:6a:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:6f:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:74:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:79:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:e7:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:ec:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:f1:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+ 0000:f6:01.0 'Device 0b25' drv=idxd unused=vfio-pci
+
+Test case
+=========
+
+Common steps
+------------
+1. Bind 1 NIC port to vfio-pci::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <nic_pci>
+
+ For example:
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:29:00.0
+
+2. Bind DSA devices to DPDK vfio-pci driver::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci <dsa_pci>
+
+ For example, bind 2 DSA devices to vfio-pci driver:
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:6a:01.0 0000:6f:01.0
+
+.. note::
+
+ One DPDK DSA device can create 8 WQ at most. Below is an example, where DPDK DSA device will create one and
+ eight WQ for DSA deivce 0000:6a:01.0 and 0000:6f:01.0. The value of “max_queues” is 1~8:
+ <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 4 -n 4 -a 0000:6a:01.0,max_queues=1 -- -i
+ <dpdk dir># ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 4 -n 4 -a 0000:6f:01.0,max_queues=8 -- -i
+
+3. Bind DSA devices to kernel idxd driver, and configure Work Queue (WQ)::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd <dsa_pci>
+ <dpdk dir># ./drivers/dma/idxd/dpdk_idxd_cfg.py -q <wq_num> <dsa_idx>
+
+.. note::
+
+ Better to reset WQ when need operate DSA devices that bound to idxd drvier:
+ <dpdk dir># ./drivers/dma/idxd/dpdk_idxd_cfg.py --reset <dsa_idx>
+ You can check it by 'ls /dev/dsa'
+ dsa_idx: Index of DSA devices, where 0<=dsa_idx<=7, corresponding to 0000:6a:01.0 - 0000:f6:01.0
+ wq_num: Number of workqueues per DSA endpoint, where 1<=wq_num<=8
+
+ For example, bind 2 DSA devices to idxd driver and configure WQ:
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0 6f:01.0
+ <dpdk dir># ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 1 0
+ <dpdk dir># ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 4 1
+ Check WQ by 'ls /dev/dsa' and can find "wq0.0 wq1.0 wq1.1 wq1.2 wq1.3"
+
+Test Case 1: VM2VM virtio-user forwarding test when vhost async operation using DSA dpdk driver
+-----------------------------------------------------------------------------------------------
+This case tests that the imix packets can forward normally in VM2VM topology(virtio-user as front-end) when vhost async operation using DSA dpdk drvier, both split ring and packed ring have been covered.
+
+1.Bind 1 NIC port and 1 DSA device to vfio-pci.
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=4 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd0@0000:6a:01.0-q1,txd1@0000:6a:01.0-q2,rxd1@0000:6a:01.0-q3] \
+ --client --total-num-mbufs 600000
+
+3. Launch virtio-user::
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \
+ --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/root/dpdk/vhost-net0,queues=1,server=1,mrg_rxbuf=1,in_order=0,packed_vq=1 \
+ -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 31-32 -n 4 --no-pci --file-prefix=testpmd1 \
+ --vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=/root/dpdk/vhost-net1,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1 \
+ -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1
+
+4. Loop packets between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected::
+
+ testpmd0>set fwd mac
+ testpmd0>start tx_first
+ testpmd0>stop
+ testpmd0>set eth-peer 0 00:11:22:33:44:11
+ testpmd0>start
+ testpmd1>set fwd mac
+ testpmd1>set eth-peer 0 00:11:22:33:44:10
+ testpmd1>set txpkts 64
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000,2000,2000,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 64,256,2000,64,256,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+
+5. Stop dpdk-vhost side and relaunch it with below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=4 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd1@0000:6a:01.0-q1] \
+ --client --total-num-mbufs 600000
+
+6. Rerun step 4.
+
+Test Case 2: VM2VM virtio-pmd split ring test with DSA dpdk driver register/unregister stable check
+---------------------------------------------------------------------------------------------------
+This case checks that the split ring with DSA dpdk driver can work stably when the virtio-pmd port is registed and unregisted for many times.
+
+1. Bind 1 NIC port and 1 DSA device to vfio-pci.
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=4 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd0@0000:6a:01.0-q1,txd1@0000:6a:01.0-q2,rxd1@0000:6a:01.0-q3] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. Bind virtio port to vfio-pci in both two VMs::
+
+ modprobe vfio enable_unsafe_noiommu_mode=1
+ modprobe vfio-pci
+ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+6. Start testpmd in VMs seperately::
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-2 -n 4 -- -i --rxq=1 --txq=1 --nb-cores=1 --txd=1024 --rxd=1024
+
+7. Loop pkts between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected::
+
+ testpmd0>set fwd mac
+ testpmd0>start tx_first
+ testpmd0>stop
+ testpmd0>set eth-peer 0 52:54:00:00:00:02
+ testpmd0>start
+ testpmd1>set fwd mac
+ testpmd1>set eth-peer 0 52:54:00:00:00:01
+ testpmd1>set txpkts 64
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000,2000,2000,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 64,256,2000,64,256,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+
+8. Quit two testpmd in two VMs, bind virtio-pmd port to virtio-pci,then bind port back to vfio-pci, rerun below cmd 50 times::
+
+ ./usertools/dpdk-devbind.py -u 00:05.0
+ ./usertools/dpdk-devbind.py --bind=virtio-pci 00:05.0
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+9. Rerun step 6-7, check vhost can stable work and get expected throughput.
+
+10. Quit and relaunch dpdk-vhost with below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=4 -- -p 0x1 --mergeable 1 --vm2vm 1 \
+ --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd1@0000:6a:01.0-q3] \
+ --client --total-num-mbufs 600000
+
+11. Rerun step 6-9, check vhost can stable work and get expected throughput.
+
+Test Case 3: VM2VM virtio-pmd packed ring test with DSA dpdk driver register/unregister stable check
+----------------------------------------------------------------------------------------------------
+This case checks that the packed ring with DSA dpdk driver can work stably when the virtio-pmd port is registed and unregisted for many times.
+
+1. Bind 1 NIC port and 1 DSA devices to vfio-pci.
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=4 -a 0000:6f:01.0,max_queues=4 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd0@0000:6a:01.0-q1,txd1@0000:6f:01.0-q2,rxd1@0000:6f:01.0-q3] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. Bind virtio port to vfio-pci in both two VMs::
+
+ modprobe vfio enable_unsafe_noiommu_mode=1
+ modprobe vfio-pci
+ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+6. Start testpmd in VMs seperately::
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-2 -n 4 -- -i --rxq=1 --txq=1 --nb-cores=1 --txd=1024 --rxd=1024
+
+7. Loop packets between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected::
+
+ testpmd0>set fwd mac
+ testpmd0>start tx_first
+ testpmd0>stop
+ testpmd0>set eth-peer 0 52:54:00:00:00:02
+ testpmd0>start
+ testpmd1>set fwd mac
+ testpmd1>set eth-peer 0 52:54:00:00:00:01
+ testpmd1>set txpkts 64
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000,2000,2000,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 64,256,2000,64,256,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+
+8. Quit two testpmd in two VMs, bind virtio-pmd port to virtio-pci,then bind port back to vfio-pci, rerun below cmd 50 times::
+
+ ./usertools/dpdk-devbind.py -u 00:05.0
+ ./usertools/dpdk-devbind.py --bind=virtio-pci 00:05.0
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+9. Rerun step 6-7，check vhost can stable work and get expected throughput.
+
+10. Quit VMs and rerun step 3-9.
+
+Test Case 4: VM2VM virtio-net split ring test with DSA dpdk driver and iperf stable check
+-----------------------------------------------------------------------------------------
+This case tests with split ring with DSA dpdk driver in two VMs, check that iperf/scp and reconnection can work stably between two virito-net.
+
+1. Bind 1 NIC port and 2 DSA devices to vfio-pci.
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 2-3 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=2 -a 0000:6f:01.0,max_queues=2 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd0@0000:6a:01.0-q1,txd1@0000:6f:01.0-q0,rxd1@0000:6f:01.0-q1] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. On VM1, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.2
+ arp -s 1.1.1.8 52:54:00:00:00:02
+
+6. On VM2, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.8
+ arp -s 1.1.1.2 52:54:00:00:00:01
+
+7. Check the iperf performance between two VMs by below commands::
+
+ Under VM1, run: `iperf -s -i 1`
+ Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60`
+
+8. Check iperf throughput can get x Gbits/sec.
+
+9. Scp 1MB file form VM0 to VM1, check packets can be forwarding success by scp::
+
+ Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name
+
+10. Relaunch dpdk-vhost, then rerun step 7-9 five times.
+
+11. Relaunch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 2-3 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=2 -- -p 0x1 --mergeable 1 --vm2vm 1 \
+ --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd1@0000:6a:01.0-q1] --client
+
+12. rerun step 7-9.
+
+Test Case 5: VM2VM virtio-net packed ring test with DSA dpdk driver and iperf stable check
+------------------------------------------------------------------------------------------
+This case tests with packed ring with DSA dpdk driver in two VMs, check that iperf/scp and reconnection can work stably between two virito-net.
+
+1. Bind 1 NIC port to vfio-pci and 2 DSA devices to vfio-pci.
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=2 -a 0000:6f:01.0,max_queues=2 -- -p 0x1 --mergeable 1 --vm2vm 1 \
+ --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd0@0000:6a:01.0-q1,txd1@0000:6f:01.0-q0,rxd1@0000:6f:01.0-q1] \
+ --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. On VM1, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.2
+ arp -s 1.1.1.8 52:54:00:00:00:02
+
+6. On VM2, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.8
+ arp -s 1.1.1.2 52:54:00:00:00:01
+
+7. Check the iperf performance between two VMs by below commands::
+
+ Under VM1, run: `iperf -s -i 1`
+ Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60`
+
+8. Check iperf throughput can get x Gbits/sec.
+
+9. Scp 1MB file form VM1 to VM2M, check packets can be forwarding success by scp::
+
+ Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name
+
+10. Rerun step 7-9 five times.
+
+11. Relaunch VMs with mrg_rxbuf=on.
+
+12. Rerun step 5-10.
+
+Test Case 6: VM2VM virtio-net packed ring test with 2 DSA WQ with dpdk driver and iperf stable check
+----------------------------------------------------------------------------------------------------
+This case tests with packed ring with 2 DSA WQ with dpdk driver in two VMs, check that iperf/scp and reconnection can work stably between two virito-net.
+
+1. Bind 1 NIC port to vfio-pci and 2 DSA devices to vfio-pci.
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:3a:00.0 -a 0000:6a:01.0,max_queues=2 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@0000:6a:01.0-q0,rxd1@0000:6a:01.0-q1] --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. On VM1, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.2
+ arp -s 1.1.1.8 52:54:00:00:00:02
+
+6. On VM2, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.8
+ arp -s 1.1.1.2 52:54:00:00:00:01
+
+7. Check the iperf performance between two VMs by below commands::
+
+ Under VM1, run: `iperf -s -i 1`
+ Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60`
+
+8. Check iperf throughput can get x Gbits/sec.
+
+9. Scp 1MB file form VM1 to VM2M, check packets can be forwarding success by scp::
+
+ Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name
+
+10. Rerun step 7-9 five times.
+
+11. Relaunch VMs with mrg_rxbuf=on.
+
+12. Rerun step 5-10.
+
+Test Case 7: VM2VM virtio-user forwarding test when vhost async operation using DSA kernel driver
+-------------------------------------------------------------------------------------------------
+This case tests that the imix packets can forward normally in VM2VM topology(virtio-user as front-end) when vhost async operation using DSA kernel drvier, both split ring and packed ring have been covered.
+
+1.Bind 1 NIC port and 1 DSA device to idxd::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 6a:00.0
+
+ ls /mnt/dsa #check wq configure, reset if exist
+ <dpdk dir># ./usertools/dpdk-devbind.py -u 6a:01.0
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 4 0
+ ls /mnt/dsa #check wq configure success
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd0@wq0.1,txd1@wq0.2,rxd1@wq0.3] \
+ --client --total-num-mbufs 600000
+
+3. Launch virtio-user::
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 29-30 -n 4 --no-pci --file-prefix=testpmd0 \
+ --vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=/root/dpdk/vhost-net0,queues=1,server=1,mrg_rxbuf=1,in_order=0,packed_vq=1 \
+ -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 31-32 -n 4 --no-pci --file-prefix=testpmd1 \
+ --vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=/root/dpdk/vhost-net1,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1 \
+ -- -i --rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1
+
+4. Loop packets between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected::
+
+ testpmd0>set fwd mac
+ testpmd0>start tx_first
+ testpmd0>stop
+ testpmd0>set eth-peer 0 00:11:22:33:44:11
+ testpmd0>start
+ testpmd1>set fwd mac
+ testpmd1>set eth-peer 0 00:11:22:33:44:10
+ testpmd1>set txpkts 64
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000,2000,2000,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 64,256,2000,64,256,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+
+5. Stop dpdk-vhost side and relaunch it with below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd1@wq0.1] \
+ --client --total-num-mbufs 600000
+
+6. Rerun step 4.
+
+Test Case 8: VM2VM virtio-pmd split ring test with DSA kernel driver register/unregister stable check
+------------------------------------------------------------------------------------------------------
+This case checks that the split ring with DSA kernel driver can work stably when the virtio-pmd port is registed and unregisted for many times.
+
+1. Bind 1 NIC port and 1 DSA device to idxd::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 6a:00.0
+
+ ls /mnt/dsa #check wq configure, reset if exist
+ <dpdk dir># ./usertools/dpdk-devbind.py -u 6a:01.0
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 4 0
+ ls /mnt/dsa #check wq configure success
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd0@wq0.1,txd1@wq0.2,rxd1@wq0.3] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=true,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. Bind virtio port to vfio-pci in both two VMs::
+
+ modprobe vfio enable_unsafe_noiommu_mode=1
+ modprobe vfio-pci
+ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+6. Start testpmd in VMs seperately::
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-2 -n 4 -- -i --rxq=1 --txq=1 --nb-cores=1 --txd=1024 --rxd=1024
+
+7. Loop pkts between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected::
+
+ testpmd0>set fwd mac
+ testpmd0>start tx_first
+ testpmd0>stop
+ testpmd0>set eth-peer 0 52:54:00:00:00:02
+ testpmd0>start
+ testpmd1>set fwd mac
+ testpmd1>set eth-peer 0 52:54:00:00:00:01
+ testpmd1>set txpkts 64
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000,2000,2000,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 64,256,2000,64,256,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+
+8. Quit two testpmd in two VMs, bind virtio-pmd port to virtio-pci,then bind port back to vfio-pci, rerun below cmd 50 times::
+
+ ./usertools/dpdk-devbind.py -u 00:05.0
+ ./usertools/dpdk-devbind.py --bind=virtio-pci 00:05.0
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+9. Rerun step 6-7, check vhost can stable work and get expected throughput.
+
+10. Quit and relaunch dpdk-vhost with below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd1@wq0.3] \
+ --client --total-num-mbufs 600000
+
+11. Rerun step 6-9, check vhost can stable work and get expected throughput.
+
+Test Case 9: VM2VM virtio-pmd packed ring test with DSA kernel driver register/unregister stable check
+-------------------------------------------------------------------------------------------------------
+This case checks that the packed ring with DSA kernel driver can work stably when the virtio-pmd port is registed and unregisted for many times.
+
+1. Bind 1 NIC port and 2 DSA devices to idxd::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 6a:00.0
+
+ ls /mnt/dsa #check wq configure, reset if exist
+ <dpdk dir># ./usertools/dpdk-devbind.py -u 6a:01.0 6f:01.0
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0 6f:01.0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 4 0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 4 1
+ ls /mnt/dsa #check wq configure success
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd0@wq0.1,txd1@wq1.0,rxd1@wq1.1] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=on,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. Bind virtio port to vfio-pci in both two VMs::
+
+ modprobe vfio enable_unsafe_noiommu_mode=1
+ modprobe vfio-pci
+ echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+6. Start testpmd in VMs seperately::
+
+ ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd -l 1-2 -n 4 -- -i --rxq=1 --txq=1 --nb-cores=1 --txd=1024 --rxd=1024
+
+7. Loop packets between two virtio-user sides, record performance number with 64b/2000b/8000b/IMIX pkts can get expected::
+
+ testpmd0>set fwd mac
+ testpmd0>start tx_first
+ testpmd0>stop
+ testpmd0>set eth-peer 0 52:54:00:00:00:02
+ testpmd0>start
+ testpmd1>set fwd mac
+ testpmd1>set eth-peer 0 52:54:00:00:00:01
+ testpmd1>set txpkts 64
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 2000,2000,2000,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+ testpmd1>stop
+ testpmd1>set txpkts 64,256,2000,64,256,2000
+ testpmd1>start tx_first
+ testpmd1>show port stats all
+
+8. Quit two testpmd in two VMs, bind virtio-pmd port to virtio-pci,then bind port back to vfio-pci, rerun below cmd 50 times::
+
+ ./usertools/dpdk-devbind.py -u 00:05.0
+ ./usertools/dpdk-devbind.py --bind=virtio-pci 00:05.0
+ ./usertools/dpdk-devbind.py --bind=vfio-pci 00:05.0
+
+9. Rerun step 6-7，check vhost can stable work and get expected throughput.
+
+10. Quit VMs and rerun step 3-9.
+
+Test Case 10: VM2VM virtio-net split ring test with DSA kernel driver and iperf stable check
+--------------------------------------------------------------------------------------------
+This case tests with split ring with DSA kenrel driver in two VMs, check that iperf/scp and reconnection can work stably between two virito-net.
+
+1. Bind 1 NIC port and 2 DSA devices to idxd::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 6a:00.0
+
+ ls /mnt/dsa #check wq configure, reset if exist
+ <dpdk dir># ./usertools/dpdk-devbind.py -u 6a:01.0 6f:01.0
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0 6f:01.0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 2 0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 2 1
+ ls /mnt/dsa #check wq configure success
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 2-3 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --tso 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd0@wq0.1,txd1@wq1.0,rxd1@wq1.1] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=true,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. On VM1, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.2
+ arp -s 1.1.1.8 52:54:00:00:00:02
+
+6. On VM2, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.8
+ arp -s 1.1.1.2 52:54:00:00:00:01
+
+7. Check the iperf performance between two VMs by below commands::
+
+ Under VM1, run: `iperf -s -i 1`
+ Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60`
+
+8. Check iperf throughput can get x Gbits/sec.
+
+9. Scp 1MB file form VM0 to VM1, check packets can be forwarding success by scp::
+
+ Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name
+
+10. Relaunch dpdk-vhost, then rerun step 7-9 five times.
+
+11. Relaunch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 2-3 -n 4 -a 0000:6a:01.0 -- -p 0x1 --tso 1 --vm2vm 1 \
+ --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd1@wq0.1] --client --total-num-mbufs 600000
+
+12. rerun step 7-9.
+
+Test Case 11: VM2VM virtio-net packed ring test with DSA kenrel driver and iperf stable check
+---------------------------------------------------------------------------------------------
+This case tests with packed ring with DSA kernel driver in two VMs, check that iperf/scp and reconnection can work stably between two virito-net.
+
+1. Bind 1 NIC port to vfio-pci and 2 DSA devices to idxd::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 6a:00.0
+
+ ls /mnt/dsa #check wq configure, reset if exist
+ <dpdk dir># ./usertools/dpdk-devbind.py -u 6a:01.0 6f:01.0
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0 6f:01.0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 2 0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 2 1
+ ls /mnt/dsa #check wq configure success
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --tso 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd0@wq0.1,txd1@wq1.0,rxd1@wq1.1] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. On VM1, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.2
+ arp -s 1.1.1.8 52:54:00:00:00:02
+
+6. On VM2, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.8
+ arp -s 1.1.1.2 52:54:00:00:00:01
+
+7. Check the iperf performance between two VMs by below commands::
+
+ Under VM1, run: `iperf -s -i 1`
+ Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60`
+
+8. Check iperf throughput can get x Gbits/sec.
+
+9. Scp 1MB file form VM1 to VM2M, check packets can be forwarding success by scp::
+
+ Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name
+
+10. Rerun step 7-9 five times.
+
+11. Relaunch VMs with mrg_rxbuf=on.
+
+12. Rerun step 5-10.
+
+Test Case 12: VM2VM virtio-net packed ring test with 2 DSA WQ with kernel driver and iperf stable check
+-------------------------------------------------------------------------------------------------------
+This case tests with packed ring with 2 DSA WQ with dpdk driver in two VMs, check that iperf/scp and reconnection can work stably between two virito-net.
+
+1. Bind 1 NIC port to vfio-pci and 1 DSA device to idxd::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 6a:00.0
+
+ ls /mnt/dsa #check wq configure, reset if exist
+ <dpdk dir># ./usertools/dpdk-devbind.py -u 6a:01.0
+ <dpdk dir># ./usertools/dpdk-devbind.py -b idxd 6a:01.0
+ ./drivers/dma/idxd/dpdk_idxd_cfg.py -q 2 0
+ ls /mnt/dsa #check wq configure success
+
+2. On host, launch dpdk-vhost by below command::
+
+ ./x86_64-native-linuxapp-gcc/examples/dpdk-vhost -l 26-28 -n 4 -a 0000:6a:01.0 \
+ -- -p 0x1 --mergeable 1 --vm2vm 1 --socket-file /root/dpdk/vhost-net0 --socket-file /root/dpdk/vhost-net1 \
+ --dmas [txd0@wq0.0,rxd1@wq0.1] \
+ --client --total-num-mbufs 600000
+
+3. Start VM0 with qemu::
+
+ qemu-system-x86_64 -name vm0 -enable-kvm -pidfile /tmp/.vm0.pid \
+ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6000-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net0,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:01,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm0_qga0.sock,server,nowait,id=vm0_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm0_qga0,name=org.qemu.guest_agent.0 -vnc :4 -drive file=/home/image/ubuntu2004.img
+
+4. Start VM1 with qemu::
+
+ qemu-system-x86_64 -name vm1 -enable-kvm -pidfile /tmp/.vm1.pid \
+ -daemonize -monitor unix:/tmp/vm1_monitor.sock,server,nowait \
+ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6001-:22 -device e1000,netdev=nttsip1 \
+ -chardev socket,id=char0,path=./vhost-net1,server -netdev type=vhost-user,id=netdev0,chardev=char0,vhostforce \
+ -device virtio-net-pci,netdev=netdev0,mac=52:54:00:00:00:02,disable-modern=false,mrg_rxbuf=off,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on,packed=on -cpu host -smp 8 -m 4096 \
+ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on -numa node,memdev=mem -mem-prealloc -chardev socket,path=/tmp/vm1_qga0.sock,server,nowait,id=vm1_qga0 \
+ -device virtio-serial -device virtserialport,chardev=vm1_qga0,name=org.qemu.guest_agent.0 -vnc :5 -drive file=/home/image/ubuntu2004_2.img
+
+5. On VM1, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.2
+ arp -s 1.1.1.8 52:54:00:00:00:02
+
+6. On VM2, set virtio device IP and run arp protocal::
+
+ ifconfig ens5 1.1.1.8
+ arp -s 1.1.1.2 52:54:00:00:00:01
+
+7. Check the iperf performance between two VMs by below commands::
+
+ Under VM1, run: `iperf -s -i 1`
+ Under VM2, run: `iperf -c 1.1.1.2 -i 1 -t 60`
+
+8. Check iperf throughput can get x Gbits/sec.
+
+9. Scp 1MB file form VM1 to VM2M, check packets can be forwarding success by scp::
+
+ Under VM1, run: `scp [xxx] root@1.1.1.8:/` [xxx] is the file name
+
+10. Rerun step 7-9 five times.
+
+11. Relaunch VMs with mrg_rxbuf=on.
+
+12. Rerun step 5-10.
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V2 3/3] tests/vswitch_sample_dsa: add new testsuite
2023-03-27 7:30 [PATCH V2 0/3] add vswitch_sample_dsa Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 1/3] test_plans/index: " Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 2/3] test_plans/vswitch_sample_dsa: add new testplan Wei Ling
@ 2023-03-27 7:30 ` Wei Ling
2023-03-28 1:13 ` lijuan.tu
2 siblings, 1 reply; 5+ messages in thread
From: Wei Ling @ 2023-03-27 7:30 UTC (permalink / raw)
To: dts; +Cc: Wei Ling
Add vswitch_sample_dsa testsuite to test dpdk-vhost APP with host DMA
operation callbacks for DSA PMD and vhost async data-path.
Signed-off-by: Wei Ling <weix.ling@intel.com>
---
tests/TestSuite_vswitch_sample_dsa.py | 1033 +++++++++++++++++++++++++
1 file changed, 1033 insertions(+)
create mode 100644 tests/TestSuite_vswitch_sample_dsa.py
diff --git a/tests/TestSuite_vswitch_sample_dsa.py b/tests/TestSuite_vswitch_sample_dsa.py
new file mode 100644
index 00000000..bf1e0f25
--- /dev/null
+++ b/tests/TestSuite_vswitch_sample_dsa.py
@@ -0,0 +1,1033 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023 Intel Corporation
+#
+
+"""
+DPDK Test suite.
+"""
+
+import random
+import re
+import string
+import time
+
+import framework.utils as utils
+from framework.pmd_output import PmdOutput
+from framework.test_case import TestCase
+from framework.virt_common import VM
+
+from .virtio_common import dsa_common as DC
+
+
+class TestVswitchSampleDSA(TestCase):
+ def set_up_all(self):
+ """
+ Run at the start of each test suite.
+ """
+ out = self.dut.build_dpdk_apps("./examples/vhost")
+ self.verify("Error" not in out, "compilation vhost error")
+ self.vhost_path = self.dut.apps_name["vhost"]
+ self.tester_tx_port_num = 1
+ self.dut_ports = self.dut.get_ports()
+ self.ports_socket = self.dut.get_numa_id(self.dut_ports[0])
+ self.cores = self.dut.get_core_list("all", socket=self.ports_socket)
+ self.vhost_core_list = self.cores[0:2]
+ self.vhost_core_range = "%s-%s" % (
+ self.vhost_core_list[0],
+ self.vhost_core_list[-1],
+ )
+ self.vuser0_core_list = self.cores[2:4]
+ self.vuser1_core_list = self.cores[4:6]
+ self.mem_channels = self.dut.get_memory_channels()
+ self.out_path = "/tmp"
+ out = self.tester.send_expect("ls -d %s" % self.out_path, "# ")
+ if "No such file or directory" in out:
+ self.tester.send_expect("mkdir -p %s" % self.out_path, "# ")
+ self.base_dir = self.dut.base_dir.replace("~", "/root")
+ self.virtio_dst_mac0 = "00:11:22:33:44:10"
+ self.virtio_dst_mac1 = "00:11:22:33:44:11"
+ self.vm_dst_mac0 = "52:54:00:00:00:01"
+ self.vm_dst_mac1 = "52:54:00:00:00:02"
+ self.vm_num = 2
+ # create an instance to set stream field setting
+ self.vhost_user = self.dut.new_session(suite="vhost-user")
+ self.virtio_user0 = self.dut.new_session(suite="virtio-user0")
+ self.virtio_user1 = self.dut.new_session(suite="virtio-user1")
+ self.virtio_user0_pmd = PmdOutput(self.dut, self.virtio_user0)
+ self.virtio_user1_pmd = PmdOutput(self.dut, self.virtio_user1)
+ self.random_string = string.ascii_letters + string.digits
+ self.virtio_ip0 = "1.1.1.2"
+ self.virtio_ip1 = "1.1.1.3"
+ self.rerun_times = 5
+ self.DC = DC(self)
+
+ def set_up(self):
+ """
+ Run before each test case.
+ """
+ self.table_header = ["Frame Size(Byte)", "Mode", "Throughput(Mpps)"]
+ self.result_table_create(self.table_header)
+ self.dut.send_expect("rm -rf %s/vhost-net*" % self.base_dir, "#")
+ self.dut.send_expect("killall -I dpdk-vhost", "#", 20)
+ self.dut.send_expect("killall -I qemu-system-x86_64", "#", 20)
+ self.vm_dut = []
+ self.vm = []
+
+ @property
+ def check_2M_env(self):
+ out = self.dut.send_expect(
+ "cat /proc/meminfo |grep Hugepagesize|awk '{print($2)}'", "# "
+ )
+ return True if out == "2048" else False
+
+ def launch_vhost_app(
+ self, eal_params, vdev_num, dmas, mergeable=True, tso=False, client_mode=True
+ ):
+ """
+ launch the vhost app on vhost side
+ """
+ vdev_param = ""
+ for i in range(vdev_num):
+ vdev_param += "--socket-file ./vhost-net{} ".format(i)
+ mergeable_param = "--mergeable 1" if mergeable else ""
+ tso_param = "--tso 1" if tso else ""
+ client_param = "--client" if client_mode else ""
+ cmd = (
+ "%s %s -- -p 0x1 %s %s --vm2vm 1 --stats 1 \
+ %s --dmas [%s] \
+ %s --total-num-mbufs 600000"
+ % (
+ self.vhost_path,
+ eal_params,
+ tso_param,
+ mergeable_param,
+ vdev_param,
+ dmas,
+ client_param,
+ )
+ )
+ self.vhost_user.send_command(cmd)
+ time.sleep(3)
+
+ def start_virtio_testpmd_with_vhost_net0(self, eal_param="", param=""):
+ """
+ launch the testpmd as virtio with vhost_net0
+ """
+ if self.check_2M_env:
+ eal_param += " --single-file-segments"
+ self.virtio_user0_pmd.start_testpmd(
+ cores=self.vuser0_core_list,
+ eal_param=eal_param,
+ param=param,
+ no_pci=True,
+ prefix="virtio-user0",
+ fixed_prefix=True,
+ )
+
+ def start_virtio_testpmd_with_vhost_net1(self, eal_param="", param=""):
+ """
+ launch the testpmd as virtio with vhost_net1
+ """
+ if self.check_2M_env:
+ eal_param += " --single-file-segments"
+ self.virtio_user1_pmd.start_testpmd(
+ cores=self.vuser1_core_list,
+ eal_param=eal_param,
+ param=param,
+ no_pci=True,
+ prefix="virtio-user1",
+ fixed_prefix=True,
+ )
+
+ def start_vms(
+ self,
+ moderns=["true", "true"],
+ mrg_rxbuf=True,
+ packed=False,
+ server_mode=False,
+ set_target=True,
+ bind_dev=True,
+ ):
+ """
+ start two VM, each VM has one virtio device
+ """
+ mrg_rxbuf = "on" if mrg_rxbuf else "off"
+ packed = ",packed=on" if packed else ""
+ for i in range(self.vm_num):
+ vm_dut = None
+ setting_format = "disable-modern=%s,mrg_rxbuf=%s,csum=on,guest_csum=on,host_tso4=on,guest_tso4=on,guest_ecn=on%s"
+ setting_args = setting_format % (moderns[i], mrg_rxbuf, packed)
+ vm_info = VM(self.dut, "vm%d" % i, "vhost_sample")
+ vm_params = {}
+ vm_params["driver"] = "vhost-user"
+ if server_mode:
+ vm_params["opt_path"] = "./vhost-net%d" % i + ",server"
+ else:
+ vm_params["opt_path"] = "./vhost-net%d" % i
+ vm_params["opt_mac"] = "52:54:00:00:00:0%d" % (i + 1)
+ vm_params["opt_settings"] = setting_args
+ vm_info.set_vm_device(**vm_params)
+ time.sleep(3)
+ try:
+ vm_dut = vm_info.start(set_target=set_target, bind_dev=bind_dev)
+ if vm_dut is None:
+ raise Exception("Set up VM ENV failed")
+ except Exception as e:
+ print((utils.RED("Failure for %s" % str(e))))
+ raise e
+ self.vm_dut.append(vm_dut)
+ self.vm.append(vm_info)
+
+ def start_vm_testpmd(self, pmd_session):
+ """
+ launch the testpmd in vm
+ """
+ self.vm_cores = [1, 2]
+ param = "--rxq=1 --txq=1 --nb-cores=1 --txd=1024 --rxd=1024"
+ pmd_session.start_testpmd(cores=self.vm_cores, param=param)
+
+ def repeat_bind_driver(self, dut, repeat_times=50):
+ i = 0
+ while i < repeat_times:
+ dut.unbind_interfaces_linux()
+ dut.bind_interfaces_linux(driver="virtio-pci")
+ dut.bind_interfaces_linux(driver="vfio-pci")
+ i += 1
+
+ def let_vswitch_know_mac(self, virtio_pmd, relaunch=False):
+ if not relaunch:
+ virtio_pmd.execute_cmd("set fwd mac")
+ virtio_pmd.execute_cmd("start tx_first")
+ else:
+ virtio_pmd.execute_cmd("stop")
+ virtio_pmd.execute_cmd("start tx_first")
+
+ def get_receive_throughput(self, pmd_session, count=10):
+ i = 0
+ while i < count:
+ pmd_session.execute_cmd("show port stats all")
+ i += 1
+ else:
+ out = pmd_session.execute_cmd("show port stats all")
+ pmd_session.execute_cmd("stop")
+ rx_throughput = re.compile("Rx-pps: \s+(.*?)\s+?").findall(out, re.S)
+ return float(rx_throughput[0]) / 1000000.0
+
+ def set_testpmd0_param(self, pmd_session, eth_peer_mac):
+ pmd_session.execute_cmd("set fwd mac")
+ pmd_session.execute_cmd("start tx_first")
+ pmd_session.execute_cmd("stop")
+ pmd_session.execute_cmd("set eth-peer 0 %s" % eth_peer_mac)
+ pmd_session.execute_cmd("start")
+
+ def set_testpmd1_param(self, pmd_session, eth_peer_mac):
+ pmd_session.execute_cmd("set fwd mac")
+ pmd_session.execute_cmd("set eth-peer 0 %s" % eth_peer_mac)
+
+ def send_pkts_from_testpmd1(self, pmd_session, pkt_len):
+ if pkt_len in [64, 2000]:
+ pmd_session.execute_cmd("set txpkts %s" % pkt_len)
+ elif pkt_len == 8000:
+ pmd_session.execute_cmd("set txpkts 2000,2000,2000,2000")
+ elif pkt_len == "imix":
+ pmd_session.execute_cmd("set txpkts 64,256,2000,64,256,2000")
+ pmd_session.execute_cmd("start tx_first")
+
+ def vm2vm_check_with_two_dsa(self):
+ frame_sizes = [64, 2000, 8000, "imix"]
+ self.set_testpmd0_param(self.virtio_user0_pmd, self.virtio_dst_mac1)
+ self.set_testpmd1_param(self.virtio_user1_pmd, self.virtio_dst_mac0)
+
+ rx_throughput = {}
+ for frame_size in frame_sizes:
+ self.send_pkts_from_testpmd1(
+ pmd_session=self.virtio_user1_pmd, pkt_len=frame_size
+ )
+ rx_pps = self.get_receive_throughput(pmd_session=self.virtio_user1_pmd)
+ rx_throughput[frame_size] = rx_pps
+ return rx_throughput
+
+ def test_vm2vm_virtio_user_forwarding_test_using_dsa_dpdk_driver(self):
+ """
+ Test Case 1: VM2VM virtio-user forwarding test when vhost async operation using DSA dpdk driver
+ """
+ perf_result = []
+ dsas = self.DC.bind_dsa_to_dpdk_driver(
+ dsa_num=1, driver_name="vfio-pci", socket=self.ports_socket
+ )
+ ports = dsas
+ ports.append(self.dut.ports_info[0]["pci"])
+ port_options = {dsas[0]: "max_queues=4"}
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports, port_options=port_options
+ )
+ dmas = "txd0@%s-q0,rxd0@%s-q1,txd1@%s-q2,rxd1@%s-q3" % (
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ )
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+ virtio0_eal_param = "--vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=./vhost-net0,queues=1,server=1,mrg_rxbuf=1,in_order=0,packed_vq=1"
+ virtio0_param = "--rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1"
+ self.start_virtio_testpmd_with_vhost_net0(
+ eal_param=virtio0_eal_param, param=virtio0_param
+ )
+
+ virtio1_eal_param = "--vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=./vhost-net1,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1"
+ virtio1_param = "--rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1"
+ self.start_virtio_testpmd_with_vhost_net1(
+ eal_param=virtio1_eal_param, param=virtio1_param
+ )
+ before_relunch_result = self.vm2vm_check_with_two_dsa()
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ dmas = "txd0@%s-q0,rxd1@%s-q1" % (dsas[0], dsas[0])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+ self.virtio_user0_pmd.execute_cmd("stop")
+ after_relunch_result = self.vm2vm_check_with_two_dsa()
+
+ for key in before_relunch_result.keys():
+ perf_result.append(
+ [key, "Before Re-launch vhost", before_relunch_result[key]]
+ )
+ for key in after_relunch_result.keys():
+ perf_result.append(
+ [key, "After Re-launch vhost ", after_relunch_result[key]]
+ )
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def vm2vm_check_with_two_vhost_device(self):
+ rx_throughput = {}
+ self.frame_sizes = [64, 2000, 8000, "imix"]
+ for frame_size in self.frame_sizes:
+ self.send_pkts_from_testpmd1(pmd_session=self.vm1_pmd, pkt_len=frame_size)
+ rx_pps = self.get_receive_throughput(pmd_session=self.vm1_pmd)
+ rx_throughput[frame_size] = rx_pps
+ return rx_throughput
+
+ def start_vms_testpmd_and_test(
+ self, moderns=["true", "true"], mrg_rxbuf=True, need_start_vm=True, packed=False
+ ):
+ if need_start_vm:
+ self.start_vms(
+ moderns=moderns,
+ mrg_rxbuf=mrg_rxbuf,
+ packed=packed,
+ server_mode=True,
+ set_target=True,
+ bind_dev=True,
+ )
+ self.vm0_pmd = PmdOutput(self.vm_dut[0])
+ self.vm1_pmd = PmdOutput(self.vm_dut[1])
+ self.start_vm_testpmd(self.vm0_pmd)
+ self.start_vm_testpmd(self.vm1_pmd)
+ self.set_testpmd0_param(self.vm0_pmd, self.vm_dst_mac1)
+ self.set_testpmd1_param(self.vm1_pmd, self.vm_dst_mac0)
+ perf_result = self.vm2vm_check_with_two_vhost_device()
+ self.vm0_pmd.quit()
+ self.vm1_pmd.quit()
+ return perf_result
+
+ def test_vm2vm_virtio_pmd_split_ring_test_with_dsa_dpdk_driver_register_and_unregister_stable_check(
+ self,
+ ):
+ """
+ Test Case 2: VM2VM virtio-pmd split ring test with DSA dpdk driver register/unregister stable check
+ """
+ perf_result = []
+ dsas = self.DC.bind_dsa_to_dpdk_driver(
+ dsa_num=1, driver_name="vfio-pci", socket=self.ports_socket
+ )
+ ports = dsas
+ ports.append(self.dut.ports_info[0]["pci"])
+ port_options = {dsas[0]: "max_queues=4"}
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports, port_options=port_options
+ )
+ dmas = "txd0@%s-q0,rxd0@%s-q1,txd1@%s-q2,rxd1@%s-q3" % (
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ )
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_rebind = self.start_vms_testpmd_and_test(
+ moderns=["true", "false"], mrg_rxbuf=True, need_start_vm=True, packed=False
+ )
+
+ # repeat bind 50 time from virtio-pci to vfio-pci
+ self.repeat_bind_driver(dut=self.vm_dut[0], repeat_times=50)
+ self.repeat_bind_driver(dut=self.vm_dut[1], repeat_times=50)
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ dmas = "txd0@%s-q0,rxd1@%s-q3" % (dsas[0], dsas[0])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ after_rebind = self.start_vms_testpmd_and_test(need_start_vm=False)
+
+ for key in before_rebind.keys():
+ perf_result.append([key, "Before rebind driver", before_rebind[key]])
+
+ for key in after_rebind.keys():
+ perf_result.append([key, "After rebind driver", after_rebind[key]])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ for i in perf_result:
+ self.verify(i[2] > 0, "%s Frame Size(Byte) is less than 0 Mpps" % i[0])
+
+ def test_vm2vm_virtio_pmd_packed_ring_test_with_dsa_dpdk_driver_register_and_unregister_stable_check(
+ self,
+ ):
+ """
+ Test Case 3: VM2VM virtio-pmd packed ring test with DSA dpdk driver register/unregister stable check
+ """
+ perf_result = []
+ dsas = self.DC.bind_dsa_to_dpdk_driver(
+ dsa_num=1, driver_name="vfio-pci", socket=self.ports_socket
+ )
+ ports = dsas
+ ports.append(self.dut.ports_info[0]["pci"])
+ port_options = {dsas[0]: "max_queues=4"}
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports, port_options=port_options
+ )
+ dmas = "txd0@%s-q0,rxd0@%s-q1,txd1@%s-q2,rxd1@%s-q3" % (
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ )
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_rebind = self.start_vms_testpmd_and_test(
+ moderns=["false", "false"], mrg_rxbuf=True, need_start_vm=True, packed=True
+ )
+
+ # repeat bind 50 time from virtio-pci to vfio-pci
+ self.repeat_bind_driver(dut=self.vm_dut[0], repeat_times=50)
+ self.repeat_bind_driver(dut=self.vm_dut[1], repeat_times=50)
+
+ after_rebind = self.start_vms_testpmd_and_test(need_start_vm=False)
+
+ for key in before_rebind.keys():
+ perf_result.append([key, "Before rebind driver", before_rebind[key]])
+
+ for key in after_rebind.keys():
+ perf_result.append([key, "After rebind driver", after_rebind[key]])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ for i in perf_result:
+ self.verify(i[2] > 0, "%s Frame Size(Byte) is less than 0 Mpps" % i[0])
+
+ def config_vm_env(self):
+ """
+ set virtio device IP and run arp protocal
+ """
+ vm0_intf = self.vm_dut[0].ports_info[0]["intf"]
+ vm1_intf = self.vm_dut[1].ports_info[0]["intf"]
+ self.vm_dut[0].send_expect(
+ "ifconfig %s %s" % (vm0_intf, self.virtio_ip0), "#", 10
+ )
+ self.vm_dut[1].send_expect(
+ "ifconfig %s %s" % (vm1_intf, self.virtio_ip1), "#", 10
+ )
+ self.vm_dut[0].send_expect(
+ "arp -s %s %s" % (self.virtio_ip1, self.vm_dst_mac1), "#", 10
+ )
+ self.vm_dut[1].send_expect(
+ "arp -s %s %s" % (self.virtio_ip0, self.vm_dst_mac0), "#", 10
+ )
+
+ def start_iperf_test(self):
+ """
+ run perf command between to vms
+ """
+ iperf_server = "iperf -f g -s -i 1"
+ iperf_client = "iperf -f g -c 1.1.1.2 -i 1 -t 60"
+ self.vm_dut[0].send_expect("%s > iperf_server.log &" % iperf_server, "", 10)
+ self.vm_dut[1].send_expect("%s > iperf_client.log &" % iperf_client, "", 60)
+ time.sleep(90)
+
+ def get_iperf_result(self):
+ """
+ get the iperf test result
+ """
+ self.vm_dut[0].send_expect("pkill iperf", "# ")
+ self.vm_dut[1].session.copy_file_from("%s/iperf_client.log" % self.dut.base_dir)
+ fp = open("./iperf_client.log")
+ fmsg = fp.read()
+ fp.close()
+ # remove the server report info from msg
+ index = fmsg.find("Server Report")
+ if index != -1:
+ fmsg = fmsg[:index]
+ iperfdata = re.compile("\S*\s*[M|G]bits/sec").findall(fmsg)
+ # the last data of iperf is the ave data from 0-30 sec
+ self.verify(len(iperfdata) != 0, "The iperf data between to vms is 0")
+ self.logger.info("The iperf data between vms is %s" % iperfdata[-1])
+ self.verify(
+ (iperfdata[-1].split()[1]) == "Gbits/sec"
+ and float(iperfdata[-1].split()[0]) >= 1,
+ "the throughput must be above 1Gbits/sec",
+ )
+ # rm the iperf log file in vm
+ self.vm_dut[0].send_expect("rm iperf_server.log", "#", 10)
+ self.vm_dut[1].send_expect("rm iperf_client.log", "#", 10)
+ return float(iperfdata[-1].split()[0])
+
+ def check_scp_file_valid_between_vms(self, file_size=1024):
+ """
+ scp file form VM1 to VM2, check the data is valid
+ """
+ # default file_size=1024K
+ data = ""
+ for _ in range(file_size * 1024):
+ data += random.choice(self.random_string)
+ self.vm_dut[0].send_expect('echo "%s" > /tmp/payload' % data, "# ")
+ # scp this file to vm1
+ out = self.vm_dut[1].send_command(
+ "scp root@%s:/tmp/payload /root" % self.virtio_ip0, timeout=5
+ )
+ if "Are you sure you want to continue connecting" in out:
+ self.vm_dut[1].send_command("yes", timeout=3)
+ self.vm_dut[1].send_command(self.vm[0].password, timeout=3)
+ # get the file info in vm1, and check it valid
+ md5_send = self.vm_dut[0].send_expect("md5sum /tmp/payload", "# ")
+ md5_revd = self.vm_dut[1].send_expect("md5sum /root/payload", "# ")
+ md5_send = md5_send[: md5_send.find(" ")]
+ md5_revd = md5_revd[: md5_revd.find(" ")]
+ self.verify(
+ md5_send == md5_revd, "the received file is different with send file"
+ )
+
+ def start_iperf_and_scp_test_in_vms(
+ self,
+ moderns=["true", "true"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=False,
+ server_mode=False,
+ ):
+ if need_start_vm:
+ self.start_vms(
+ moderns=moderns,
+ mrg_rxbuf=mrg_rxbuf,
+ server_mode=server_mode,
+ set_target=True,
+ bind_dev=False,
+ )
+ self.vm0_pmd = PmdOutput(self.vm_dut[0])
+ self.vm1_pmd = PmdOutput(self.vm_dut[1])
+ self.config_vm_env()
+ self.check_scp_file_valid_between_vms()
+ self.start_iperf_test()
+ iperfdata = self.get_iperf_result()
+ return iperfdata
+
+ def test_vm2vm_virtio_net_split_ring_test_with_dsa_dpdk_driver_and_iperf_stable_check(
+ self,
+ ):
+ """
+ Test Case 4: VM2VM virtio-net split ring test with DSA dpdk driver and iperf stable check
+ """
+ perf_result = []
+ dsas = self.DC.bind_dsa_to_dpdk_driver(
+ dsa_num=1, driver_name="vfio-pci", socket=self.ports_socket
+ )
+ ports = dsas
+ ports.append(self.dut.ports_info[0]["pci"])
+ port_options = {dsas[0]: "max_queues=2"}
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports, port_options=port_options
+ )
+ dmas = "txd0@%s-q0,rxd0@%s-q1,txd1@%s-q0,rxd1@%s-q1" % (
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ )
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["true", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=False,
+ server_mode=True,
+ )
+ perf_result.append(["split ring", "Before relaunch test", before_relaunch])
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["split ring", "Before relaunch test", rerun_result])
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ dmas = "txd0@%s-q0,rxd1@%s-q1" % (dsas[0], dsas[0])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ after_relaunch = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+
+ perf_result.append(["split ring", "After relaunch test", after_relaunch])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def test_vm2vm_virtio_net_packed_ring_test_with_dsa_dpdk_driver_and_iperf_stable_check(
+ self,
+ ):
+ """
+ Test Case 5: VM2VM virtio-net packed ring test with DSA dpdk driver and iperf stable check
+ """
+ perf_result = []
+ dsas = self.DC.bind_dsa_to_dpdk_driver(
+ dsa_num=1, driver_name="vfio-pci", socket=self.ports_socket
+ )
+ ports = dsas
+ ports.append(self.dut.ports_info[0]["pci"])
+ port_options = {dsas[0]: "max_queues=2"}
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports, port_options=port_options
+ )
+ dmas = "txd0@%s-q0,rxd0@%s-q1,txd1@%s-q0,rxd1@%s-q1" % (
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ dsas[0],
+ )
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+ perf_result.append(["packed ring", "Before relaunch test", before_relaunch])
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["packed ring", "Before relaunch test", rerun_result])
+
+ self.stop_vms()
+ after_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=True,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+
+ perf_result.append(["packed ring", "After relaunch test", after_relaunch])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def test_vm2vm_virtio_net_packed_ring_test_with_2_dsa_wq_with_dpdk_driver_and_iperf_stable_check(
+ self,
+ ):
+ """
+ Test Case 6: VM2VM virtio-net packed ring test with 2 DSA WQ with dpdk driver and iperf stable check
+ """
+ perf_result = []
+ dsas = self.DC.bind_dsa_to_dpdk_driver(
+ dsa_num=1, driver_name="vfio-pci", socket=self.ports_socket
+ )
+ ports = dsas
+ ports.append(self.dut.ports_info[0]["pci"])
+ port_options = {dsas[0]: "max_queues=2"}
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports, port_options=port_options
+ )
+ dmas = "txd0@%s-q0,rxd1@%s-q1" % (dsas[0], dsas[0])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+ perf_result.append(["packed ring", "Before relaunch test", before_relaunch])
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["packed ring", "Before relaunch test", rerun_result])
+
+ self.stop_vms()
+ after_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=True,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+ perf_result.append(["packed ring", "After relaunch test", after_relaunch])
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["packed ring", "After relaunch test", rerun_result])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def test_vm2vm_virtio_user_forwarding_test_using_dsa_kernel_driver(self):
+ """
+ Test Case 7: VM2VM virtio-user forwarding test when vhost async operation using DSA kernel driver
+ """
+ perf_result = []
+ wqs = self.DC.create_wq(wq_num=4, dsa_idxs=[0])
+ ports = [self.dut.ports_info[0]["pci"]]
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports
+ )
+ dmas = "txd0@%s,rxd0@%s,txd1@%s,rxd1@%s" % (wqs[0], wqs[1], wqs[2], wqs[3])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+ virtio0_eal_param = "--vdev=net_virtio_user0,mac=00:11:22:33:44:10,path=./vhost-net0,queues=1,server=1,mrg_rxbuf=1,in_order=0,packed_vq=1"
+ virtio0_param = "--rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1"
+ self.start_virtio_testpmd_with_vhost_net0(
+ eal_param=virtio0_eal_param, param=virtio0_param
+ )
+
+ virtio1_eal_param = "--vdev=net_virtio_user0,mac=00:11:22:33:44:11,path=./vhost-net1,queues=1,server=1,mrg_rxbuf=1,in_order=1,vectorized=1"
+ virtio1_param = "--rxq=1 --txq=1 --txd=1024 --rxd=1024 --nb-cores=1"
+ self.start_virtio_testpmd_with_vhost_net1(
+ eal_param=virtio1_eal_param, param=virtio1_param
+ )
+ before_relunch_result = self.vm2vm_check_with_two_dsa()
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ dmas = "txd0@wq0.0,rxd1@wq0.1"
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+ self.virtio_user0_pmd.execute_cmd("stop")
+ after_relunch_result = self.vm2vm_check_with_two_dsa()
+
+ for key in before_relunch_result.keys():
+ perf_result.append(
+ [key, "Before Re-launch vhost", before_relunch_result[key]]
+ )
+ for key in after_relunch_result.keys():
+ perf_result.append(
+ [key, "After Re-launch vhost ", after_relunch_result[key]]
+ )
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def test_vm2vm_virtio_pmd_split_ring_test_with_dsa_kernle_driver_register_and_unregister_stable_check(
+ self,
+ ):
+ """
+ Test Case 8: VM2VM virtio-pmd split ring test with DSA kernel driver register/unregister stable check
+ """
+ perf_result = []
+ wqs = self.DC.create_wq(wq_num=4, dsa_idxs=[0])
+ ports = [self.dut.ports_info[0]["pci"]]
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports
+ )
+ dmas = "txd0@%s,rxd0@%s,txd1@%s,rxd1@%s" % (wqs[0], wqs[1], wqs[2], wqs[3])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_rebind = self.start_vms_testpmd_and_test(
+ moderns=["false", "true"], mrg_rxbuf=True, need_start_vm=True, packed=False
+ )
+
+ # repeat bind 50 time from virtio-pci to vfio-pci
+ self.repeat_bind_driver(dut=self.vm_dut[0], repeat_times=50)
+ self.repeat_bind_driver(dut=self.vm_dut[1], repeat_times=50)
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ dmas = "txd0@%s,rxd1@%s" % (wqs[0], wqs[3])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ after_rebind = self.start_vms_testpmd_and_test(need_start_vm=False)
+
+ for key in before_rebind.keys():
+ perf_result.append([key, "Before rebind driver", before_rebind[key]])
+
+ for key in after_rebind.keys():
+ perf_result.append([key, "After rebind driver", after_rebind[key]])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ for i in perf_result:
+ self.verify(i[2] > 0, "%s Frame Size(Byte) is less than 0 Mpps" % i[0])
+
+ def test_vm2vm_virtio_pmd_packed_ring_test_with_dsa_kernel_driver_register_and_unregister_stable_check(
+ self,
+ ):
+ """
+ Test Case 9: VM2VM virtio-pmd packed ring test with DSA kernel driver register/unregister stable check
+ """
+ perf_result = []
+ wqs = self.DC.create_wq(wq_num=2, dsa_idxs=[0, 1])
+ ports = [self.dut.ports_info[0]["pci"]]
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports
+ )
+ dmas = "txd0@%s,rxd0@%s,txd1@%s,rxd1@%s" % (wqs[0], wqs[1], wqs[2], wqs[3])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_rebind = self.start_vms_testpmd_and_test(
+ moderns=["false", "false"], mrg_rxbuf=True, need_start_vm=True, packed=True
+ )
+
+ # repeat bind 50 time from virtio-pci to vfio-pci
+ self.repeat_bind_driver(dut=self.vm_dut[0], repeat_times=50)
+ self.repeat_bind_driver(dut=self.vm_dut[1], repeat_times=50)
+
+ after_rebind = self.start_vms_testpmd_and_test(need_start_vm=False)
+
+ for key in before_rebind.keys():
+ perf_result.append([key, "Before rebind driver", before_rebind[key]])
+
+ for key in after_rebind.keys():
+ perf_result.append([key, "After rebind driver", after_rebind[key]])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ for i in perf_result:
+ self.verify(i[2] > 0, "%s Frame Size(Byte) is less than 0 Mpps" % i[0])
+
+ def test_vm2vm_virtio_net_split_ring_test_with_dsa_kernel_driver_and_iperf_stable_check(
+ self,
+ ):
+ """
+ Test Case 10: VM2VM virtio-net split ring test with DSA kernel driver and iperf stable check
+ """
+ perf_result = []
+ wqs = self.DC.create_wq(wq_num=2, dsa_idxs=[0, 1])
+ ports = [self.dut.ports_info[0]["pci"]]
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports
+ )
+ dmas = "txd0@%s,rxd0@%s,txd1@%s,rxd1@%s" % (wqs[0], wqs[1], wqs[2], wqs[3])
+ self.launch_vhost_app(
+ eal_params=eal_params,
+ vdev_num=2,
+ dmas=dmas,
+ mergeable=False,
+ tso=True,
+ client_mode=True,
+ )
+
+ before_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["true", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=False,
+ server_mode=True,
+ )
+ perf_result.append(["split ring", "Before relaunch test", before_relaunch])
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ self.launch_vhost_app(
+ eal_params=eal_params,
+ vdev_num=2,
+ dmas=dmas,
+ mergeable=False,
+ tso=True,
+ client_mode=True,
+ )
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["split ring", "Before relaunch test", rerun_result])
+
+ self.vhost_user.send_expect("^C", "# ", 20)
+ dmas = "txd0@%s,rxd1@%s" % (wqs[0], wqs[1])
+ self.launch_vhost_app(
+ eal_params=eal_params,
+ vdev_num=2,
+ dmas=dmas,
+ mergeable=False,
+ tso=True,
+ client_mode=True,
+ )
+
+ after_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["true", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=False,
+ packed=False,
+ server_mode=True,
+ )
+
+ perf_result.append(["split ring", "After relaunch test", after_relaunch])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def test_vm2vm_virtio_net_packed_ring_test_with_dsa_kernel_driver_and_iperf_stable_check(
+ self,
+ ):
+ """
+ Test Case 11: VM2VM virtio-net packed ring test with DSA kenrel driver and iperf stable check
+ """
+ perf_result = []
+ wqs = self.DC.create_wq(wq_num=2, dsa_index=[0, 1])
+ ports = [self.dut.ports_info[0]["pci"]]
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports
+ )
+ dmas = "txd0@%s,rxd0@%s,txd1@%s,rxd1@%s" % (wqs[0], wqs[1], wqs[2], wqs[3])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, tso=True, client_mode=True
+ )
+
+ before_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+ perf_result.append(["packed ring", "Before relaunch test", before_relaunch])
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["packed ring", "Before relaunch test", rerun_result])
+
+ self.stop_vms()
+ after_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=True,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+
+ perf_result.append(["packed ring", "After relaunch test", after_relaunch])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def test_vm2vm_virtio_net_packed_ring_test_with_2_dsa_wq_with_kernel_driver_and_iperf_stable_check(
+ self,
+ ):
+ """
+ Test Case 12: VM2VM virtio-net packed ring test with 2 DSA WQ with kernel driver and iperf stable check
+ """
+ perf_result = []
+ wqs = self.DC.create_wq(wq_num=2, dsa_idxs=[0])
+ ports = [self.dut.ports_info[0]["pci"]]
+ eal_params = self.dut.create_eal_parameters(
+ cores=self.vhost_core_list, ports=ports
+ )
+ dmas = "txd0@%s,rxd1@%s" % (wqs[0], wqs[1])
+ self.launch_vhost_app(
+ eal_params=eal_params, vdev_num=2, dmas=dmas, client_mode=True
+ )
+
+ before_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=False,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+ perf_result.append(["packed ring", "Before relaunch test", before_relaunch])
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["packed ring", "Before relaunch test", rerun_result])
+
+ self.stop_vms()
+ after_relaunch = self.start_iperf_and_scp_test_in_vms(
+ moderns=["false", "false"],
+ mrg_rxbuf=True,
+ need_start_vm=True,
+ packed=True,
+ server_mode=True,
+ )
+ perf_result.append(["packed ring", "After relaunch test", after_relaunch])
+
+ for _ in range(self.rerun_times):
+ rerun_result = self.start_iperf_and_scp_test_in_vms(need_start_vm=False)
+ perf_result.append(["packed ring", "After relaunch test", rerun_result])
+
+ for table_row in perf_result:
+ self.result_table_add(table_row)
+ self.result_table_print()
+
+ def stop_vms(self):
+ for i in range(len(self.vm)):
+ self.vm[i].stop()
+ self.vm_dut = []
+ self.vm = []
+
+ def close_all_session(self):
+ if getattr(self, "vhost_user", None):
+ self.dut.close_session(self.vhost_user)
+ if getattr(self, "virtio-user0", None):
+ self.dut.close_session(self.virtio_user0)
+ if getattr(self, "virtio-user1", None):
+ self.dut.close_session(self.virtio_user1)
+
+ def tear_down(self):
+ """
+ Run after each test case.
+ """
+ self.stop_vms()
+ self.vhost_user.send_expect("^C", "# ", 20)
+ self.dut.kill_all()
+
+ def tear_down_all(self):
+ """
+ Run after each test suite.
+ """
+ self.close_all_session()
--
2.25.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V2 3/3] tests/vswitch_sample_dsa: add new testsuite
2023-03-27 7:30 ` [dts][PATCH V2 3/3] tests/vswitch_sample_dsa: add new testsuite Wei Ling
@ 2023-03-28 1:13 ` lijuan.tu
0 siblings, 0 replies; 5+ messages in thread
From: lijuan.tu @ 2023-03-28 1:13 UTC (permalink / raw)
To: dts, Wei Ling; +Cc: Wei Ling
On Mon, 27 Mar 2023 15:30:16 +0800, Wei Ling <weix.ling@intel.com> wrote:
> Add vswitch_sample_dsa testsuite to test dpdk-vhost APP with host DMA
> operation callbacks for DSA PMD and vhost async data-path.
>
> Signed-off-by: Wei Ling <weix.ling@intel.com>
Series applied, thanks
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-03-28 1:13 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-27 7:30 [PATCH V2 0/3] add vswitch_sample_dsa Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 1/3] test_plans/index: " Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 2/3] test_plans/vswitch_sample_dsa: add new testplan Wei Ling
2023-03-27 7:30 ` [dts][PATCH V2 3/3] tests/vswitch_sample_dsa: add new testsuite Wei Ling
2023-03-28 1:13 ` lijuan.tu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).