From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E3AFAA00C3; Tue, 19 Apr 2022 08:34:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D7BA74068E; Tue, 19 Apr 2022 08:34:50 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id 1921C40150 for ; Tue, 19 Apr 2022 08:34:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650350089; x=1681886089; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=wdlM9x7fG/ip3CdC02LsXHVrdHwIfXn3YAfTlmhabJM=; b=PUvA1Za7ig2ovBwGds3A2v1g1cD2RdSYzhz5c0FYm/xY/ySXgZW+KWkg K0uo3jJKetbYdwg1VPkW+0V6Pk0PP64v/TxrjkZz1GGYMtYJqyG4iGlVj KjojR5d+UNHM6TekhHkz9mF9lA3O8HtxvdYen6nDdpGHiUmp+qGiHla49 5/CrouoWKO49rTyzVfHvmK/XvWjm9JejGKT0yBYQatk9SSycFsHR6Rsrb 7M45ZSsHmiMscBu0wM1m1W/RGtKOtJhL2iuqWl8XVz798ZVYeAxk7s0kv 5e7XzmIBgG649yP7yeXFww/okfSC9j8PX9IcO/RhkaaMNgyqJRcv2gWY7 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10321"; a="263449080" X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="263449080" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Apr 2022 23:34:48 -0700 X-IronPort-AV: E=Sophos;i="5.90,272,1643702400"; d="scan'208";a="575945610" Received: from unknown (HELO localhost.localdomain) ([10.239.251.222]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Apr 2022 23:34:46 -0700 From: Wei Ling To: dts@dpdk.org Cc: Wei Ling Subject: [dts][PATCH V1 1/2] test_plans/vdev_primary_secondary_test_plan: add hotplug testcase2 Date: Tue, 19 Apr 2022 14:34:40 +0800 Message-Id: <20220419063440.987430-1-weix.ling@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Add hotplug testcase2 and modify testplan with new format. Signed-off-by: Wei Ling --- .../vdev_primary_secondary_test_plan.rst | 188 ++++++++---------- 1 file changed, 82 insertions(+), 106 deletions(-) diff --git a/test_plans/vdev_primary_secondary_test_plan.rst b/test_plans/vdev_primary_secondary_test_plan.rst index b98d2d04..9d3acbad 100644 --- a/test_plans/vdev_primary_secondary_test_plan.rst +++ b/test_plans/vdev_primary_secondary_test_plan.rst @@ -34,172 +34,148 @@ Virtio-pmd primary/secondary process test plan ============================================== -This test plan will test vdev primary/secondary with symmetric_mp, which demonstrates how a set of processes can run in parallel, -with each process performing the same set of packet processing operations. Also test vdev primary/secondary with hotplug_mp example. - -Symmetric MP Application Description ------------------------------------- - -This test is a multi-process test which demonstrates how multiple processes can -work together to perform packet I/O and packet processing in parallel, much as -other example application work by using multiple threads. In this example, each -process reads packets from all network ports being used - though from a different -RX queue in each case. Those packets are then forwarded by each process which -sends them out by writing them directly to a suitable TX queue. +Description +=========== + +This test plan will test vdev primary/secondary with symmetric_mp, +which demonstrates how a set of processes can run in parallel, +with each process performing the same set of packet processing operations, +each process reads packets from each of the network ports in use. +Each process reads a different RX queue and writes outgoing packets to a different TX queue on each port. +Also test vdev primary/secondary with hotplug_mp example. +The primary process work as a resource management process, it will create/destroy virtual device at runtime, +while the secondary process can deal with the network stuff with these devices, +that means attach or detach a device on any process will broadcast to all other processes through mp channel +then device information will be synchronized on all processes. +For more about the Symmetric MP Application example, +please refer to the dpdk docs: http://doc.dpdk.org/guides/sample_app_ug/multi_process.html Prerequisites -------------- - -Assuming that DPDK build has been set up and the multi-process sample -applications have been built. It is also assumed that a traffic generator has -been configured and plugged in to the NIC ports 0 and 1. - -Test Methodology ----------------- - -As with the simple_mp example, the first instance of the symmetric_mp process -must be run as the primary instance, though with a number of other application -specific parameters also provided after the EAL arguments. These additional -parameters are: +============= -* -p , where portmask is a hexadecimal bitmask of what ports on the - system are to be used. For example: -p 3 to use ports 0 and 1 only. -* --num-procs , where N is the total number of symmetric_mp instances that - will be run side-by-side to perform packet processing. This parameter is used to - configure the appropriate number of receive queues on each network port. -* --proc-id , where n is a numeric value in the range 0 <= n < N (number of - processes, specified above). This identifies which symmetric_mp instance is being - run, so that each process can read a unique receive queue on each network port. +Topology +-------- +Vhost-->Virtio-tmp -The secondary symmetric_mp instances must also have these parameters specified, -and the first two must be the same as those passed to the primary instance, or errors -result. +Hardware +-------- +Supportted NICs: all -For example, -to build symmetric_mp:: +General set up +-------------- +1. Compile DPDK in host and VM:: - meson configure -Dexamples=multi_process/symmetric_mp - ninja -C + # meson + # ninja -C -j 110 -to run a set of four symmetric_mp instances, running on lcores 1-4, all -performing level-2 forwarding of packets between ports 0 and 1, the following -commands can be used (assuming run as root):: +2. Compile symmetric_mp example application in VM:: - .//examples/dpdk-symmetric_mp -c 2 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=0 - .//examples/dpdk-symmetric_mp -c 4 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=1 - .//examples/dpdk-symmetric_mp -c 8 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=2 - .//examples/dpdk-symmetric_mp -c 10 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=3 + # meson configure -Dexamples=multi_process/symmetric_mp + # ninja -C -j 110 -To run only 1 or 2 instances, the above parameters to the 1 or 2 instances being -run should remain the same, except for the ``num-procs`` value, which should be -adjusted appropriately. +3. Compile hotplug_mp example application in VM:: -Hotplug MP Application Description ----------------------------------- + # meson configure -Dexamples=multi_process/hotplug_mp + # ninja -C -j 110 -Currently secondary process will only sync ethdev from primary process at -init stage, but it will not be aware if device is attached/detached on -primary process at runtime. +Test case +========= -While there is the requirement from application that take -primary-secondary process model. The primary process work as a resource -management process, it will create/destroy virtual device at runtime, -while the secondary process deal with the network stuff with these devices. +Common steps +------------ +1. Bind the virtio port to vfio-pci in VM:: -So the orignial intention is to fix this gap, but beyond that the patch -set provide a more comprehesive solution to handle different hotplug -cases in multi-process situation, it cover below scenario: + # modprobe vfio-pci + # echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode -* Attach a device from the primary -* Detach a device from the primary -* Attach a device from a secondary -* Detach a device from a secondary - -In primary-secondary process model, we assume ethernet devices are shared -by default, that means attach or detach a device on any process will -broadcast to all other processes through mp channel then device -information will be synchronized on all processes. - -Any failure during attaching process will cause inconsistent status -between processes, so proper rollback action should be considered. + # ./usertools/dpdk-devbind.py --bind=vfio-pci 0000:00:05.0 0000:00:06.0 Test Case 1: Virtio-pmd primary and secondary process symmetric test -==================================================================== +-------------------------------------------------------------------- +This case test vdev primary/secondary with two symmetric_mp process in VM, each process can reads a different RX queue on each port. -1. Bind one port to vfio-pci, launch testpmd by below command:: +1. Launch testpmd with 2 vdev by below command:: - .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 - testpmd>set fwd txonly - testpmd>start + # .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost \ + --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' \ + -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 2. Launch VM with two virtio ports, must set queues=2 as app receive packets from special queue which index same with proc-id:: - qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + # qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ - -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu16.img \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ - -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait \ + -device e1000,netdev=nttsip1 \ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ -chardev socket,id=char,path=./vhost-net,server -netdev type=vhost-user,id=mynet1,chardev=char,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet1,mrg_rxbuf=on,csum=on,mq=on,vectors=15 \ -chardev socket,id=char1,path=./vhost-net1,server -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:03,netdev=mynet2,mrg_rxbuf=on,csum=on,mq=on,vectors=15 -vnc :10 -daemonize -3. Bind virtio port to vfio-pci:: +3. In VM, get the virtio network device and bind bind it to vfio-pci as common steps 1. + +4. In Vm, Compile symmetric_mp and hotplug_mp example as General set up step 2-3. - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x +4. Launch two process by symmetric_mp example in VM, each process will using ports 0 and 1, primary process using queue 0 of each port, secondary process using queue 1 of each port:: -4. Launch two process by example:: + # .//examples/dpdk-symmetric_mp -l 1 -n 4 --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 + # .//examples/dpdk-symmetric_mp -l 2 -n 4 --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 - .//examples/dpdk-symmetric_mp -l 1 -n 4 --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 - .//examples/dpdk-symmetric_mp -l 2 -n 4 --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 +5. Send packets from vhost testpmd:: -5. Quit all process, check the packets number in rx/tx statistic like below for both primary process and secondary process:: + testpmd> set fwd mac + testpmd> start tx_first - Port 0: RX - 27511680, TX - 256, Drop - 27499168 - Port 1: RX - 27499424, TX - 256, Drop - 27511424 +6. Quit all process, check the packets number in rx/tx statistic like below for both primary process and secondary process:: + + Port 0: RX - 22322016, TX - 22321184, Drop - 0 + Port 1: RX - 22321184, TX - 22322016, Drop - 0 Test Case 2: Virtio-pmd primary and secondary process hotplug test -================================================================== +------------------------------------------------------------------ +This case use hotplug_mp example to test the MP Application hotplug, attach or detach a device on any process will broadcast to all other processes through mp channel, +then device information will be synchronized on all processes. it cover below scenario: attach a device from the primary, detach a device from the primary, +attach a device from a secondary,detach a device from a secondary. 1. Launch testpmd by below command:: - .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 - testpmd>set fwd txonly - testpmd>start + .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost \ + --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' \ + -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 + testpmd> set fwd txonly + testpmd> start 2. Launch VM with two virtio ports, must set queues=2 as app receive packets from special queue which index same with proc-id:: - qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + # qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ - -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu16.img \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ - -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait \ + -device e1000,netdev=nttsip1 \ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ -chardev socket,id=char,path=./vhost-net,server -netdev type=vhost-user,id=mynet1,chardev=char,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet1,mrg_rxbuf=on,csum=on,mq=on,vectors=15 \ -chardev socket,id=char1,path=./vhost-net1,server -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:03,netdev=mynet2,mrg_rxbuf=on,csum=on,mq=on,vectors=15 -vnc :10 -daemonize -3. Bind virtio port to vfio-pci:: - - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x +3. In VM, get the virtio network device and bind bind it to vfio-pci as common steps 1-2. -4. Start sample code as primary process:: +4. Start sample code as primary process and list all etherdev:: - .//examples/dpdk-symmetric_mp --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 + .//examples/dpdk-hotplug_mp --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 example> list list all etherdev 0 0000:00:05.0 1 0000:00:06.0 -5. Start sample code as secondary process:: +5. Start sample code as secondary process and list all etherdev:: - .//examples/dpdk-symmetric_mp --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 + .//examples/dpdk-hotplug_mp --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 example> list list all etherdev 0 0000:00:05.0 -- 2.25.1