From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3AD50A00C3; Tue, 19 Apr 2022 07:56:57 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 32550406A2; Tue, 19 Apr 2022 07:56:57 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 98FB940150 for ; Tue, 19 Apr 2022 07:56:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650347815; x=1681883815; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=O5w12qf645sjoecoV/r7ij0lVkqKjR9Azm+BZhzzDFA=; b=lcvUC4nAttRJahcwCaf3m9pHcaMKlJjsa5f7arhWo2D6epbFrXZDP2e6 mZSych0D8uRlbsbCipzGt7kStDeGjDPqjPaqmVaOSHaCLJJaYizpPKRg7 +NGXJdlHkU9Nn72TKsLoIb3Cgh+EXek4P4JEJpzboPVzzK72i1aM0c7zQ J4M6WroSRk5YKw2buFKhw0TQn1XIEtYtDI1isjUNMqddleJvxojDypVPA 0Sj3CdUTF/SgE8rjqKnVSfJDbn2SBlSdPzEkTbEObuLkVWA3scwnGFQ4z DNgNx7eGsAK/D9Ej7AewHxeAFfc9oynE9bvZcZNYVPxCbgNK6wWy8Lfqb g==; X-IronPort-AV: E=McAfee;i="6400,9594,10321"; a="326583632" X-IronPort-AV: E=Sophos;i="5.90,271,1643702400"; d="scan'208";a="326583632" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Apr 2022 22:56:53 -0700 X-IronPort-AV: E=Sophos;i="5.90,271,1643702400"; d="scan'208";a="529175296" Received: from unknown (HELO localhost.localdomain) ([10.239.251.222]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Apr 2022 22:56:52 -0700 From: Wei Ling To: dts@dpdk.org Cc: Wei Ling Subject: [dts][PATCH V1 1/2] test_plans/vdev_primary_secondary_test_plan: add hutplug testcase2 Date: Tue, 19 Apr 2022 13:56:45 +0800 Message-Id: <20220419055645.943384-1-weix.ling@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Add hutplug testcase2 and modify testplan with new format. Signed-off-by: Wei Ling --- .../vdev_primary_secondary_test_plan.rst | 184 ++++++++---------- 1 file changed, 80 insertions(+), 104 deletions(-) diff --git a/test_plans/vdev_primary_secondary_test_plan.rst b/test_plans/vdev_primary_secondary_test_plan.rst index b98d2d04..70e897b6 100644 --- a/test_plans/vdev_primary_secondary_test_plan.rst +++ b/test_plans/vdev_primary_secondary_test_plan.rst @@ -34,172 +34,148 @@ Virtio-pmd primary/secondary process test plan ============================================== -This test plan will test vdev primary/secondary with symmetric_mp, which demonstrates how a set of processes can run in parallel, -with each process performing the same set of packet processing operations. Also test vdev primary/secondary with hotplug_mp example. - -Symmetric MP Application Description ------------------------------------- - -This test is a multi-process test which demonstrates how multiple processes can -work together to perform packet I/O and packet processing in parallel, much as -other example application work by using multiple threads. In this example, each -process reads packets from all network ports being used - though from a different -RX queue in each case. Those packets are then forwarded by each process which -sends them out by writing them directly to a suitable TX queue. +Description +=========== + +This test plan will test vdev primary/secondary with symmetric_mp, +which demonstrates how a set of processes can run in parallel, +with each process performing the same set of packet processing operations, +each process reads packets from each of the network ports in use. +Each process reads a different RX queue and writes outgoing packets to a different TX queue on each port. +Also test vdev primary/secondary with hotplug_mp example. +The primary process work as a resource management process, it will create/destroy virtual device at runtime, +while the secondary process can deal with the network stuff with these devices, +that means attach or detach a device on any process will broadcast to all other processes through mp channel +then device information will be synchronized on all processes. +For more about the Symmetric MP Application example, +please refer to the dpdk docs: http://doc.dpdk.org/guides/sample_app_ug/multi_process.html Prerequisites -------------- - -Assuming that DPDK build has been set up and the multi-process sample -applications have been built. It is also assumed that a traffic generator has -been configured and plugged in to the NIC ports 0 and 1. - -Test Methodology ----------------- - -As with the simple_mp example, the first instance of the symmetric_mp process -must be run as the primary instance, though with a number of other application -specific parameters also provided after the EAL arguments. These additional -parameters are: - -* -p , where portmask is a hexadecimal bitmask of what ports on the - system are to be used. For example: -p 3 to use ports 0 and 1 only. -* --num-procs , where N is the total number of symmetric_mp instances that - will be run side-by-side to perform packet processing. This parameter is used to - configure the appropriate number of receive queues on each network port. -* --proc-id , where n is a numeric value in the range 0 <= n < N (number of - processes, specified above). This identifies which symmetric_mp instance is being - run, so that each process can read a unique receive queue on each network port. +============= -The secondary symmetric_mp instances must also have these parameters specified, -and the first two must be the same as those passed to the primary instance, or errors -result. +Topology +-------- +Vhost-->Virtio-tmp -For example, -to build symmetric_mp:: +Hardware +-------- +Supportted NICs: all - meson configure -Dexamples=multi_process/symmetric_mp - ninja -C +General set up +-------------- +1. Compile DPDK in host and VM:: -to run a set of four symmetric_mp instances, running on lcores 1-4, all -performing level-2 forwarding of packets between ports 0 and 1, the following -commands can be used (assuming run as root):: + # meson + # ninja -C -j 110 - .//examples/dpdk-symmetric_mp -c 2 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=0 - .//examples/dpdk-symmetric_mp -c 4 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=1 - .//examples/dpdk-symmetric_mp -c 8 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=2 - .//examples/dpdk-symmetric_mp -c 10 --proc-type=auto -- -p 3 --num-procs=4 --proc-id=3 +2. Compile symmetric_mp example application in VM:: -To run only 1 or 2 instances, the above parameters to the 1 or 2 instances being -run should remain the same, except for the ``num-procs`` value, which should be -adjusted appropriately. + # meson configure -Dexamples=multi_process/symmetric_mp + # ninja -C -j 110 -Hotplug MP Application Description ----------------------------------- +3. Compile hotplug_mp example application in VM:: -Currently secondary process will only sync ethdev from primary process at -init stage, but it will not be aware if device is attached/detached on -primary process at runtime. + # meson configure -Dexamples=multi_process/hotplug_mp + # ninja -C -j 110 -While there is the requirement from application that take -primary-secondary process model. The primary process work as a resource -management process, it will create/destroy virtual device at runtime, -while the secondary process deal with the network stuff with these devices. +Test case +========= -So the orignial intention is to fix this gap, but beyond that the patch -set provide a more comprehesive solution to handle different hotplug -cases in multi-process situation, it cover below scenario: +Common steps +------------ +1. Bind the virtio port to vfio-pci in VM:: -* Attach a device from the primary -* Detach a device from the primary -* Attach a device from a secondary -* Detach a device from a secondary + # modprobe vfio-pci + # echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode -In primary-secondary process model, we assume ethernet devices are shared -by default, that means attach or detach a device on any process will -broadcast to all other processes through mp channel then device -information will be synchronized on all processes. - -Any failure during attaching process will cause inconsistent status -between processes, so proper rollback action should be considered. + # ./usertools/dpdk-devbind.py --bind=vfio-pci 0000:00:05.0 0000:00:06.0 Test Case 1: Virtio-pmd primary and secondary process symmetric test -==================================================================== +-------------------------------------------------------------------- +This case test vdev primary/secondary with two symmetric_mp process in VM, each process can reads a different RX queue on each port. -1. Bind one port to vfio-pci, launch testpmd by below command:: +1. Launch testpmd with 2 vdev by below command:: - .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 - testpmd>set fwd txonly - testpmd>start + # .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost \ + --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' \ + -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 2. Launch VM with two virtio ports, must set queues=2 as app receive packets from special queue which index same with proc-id:: - qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + # qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ - -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu16.img \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ - -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait \ + -device e1000,netdev=nttsip1 \ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ -chardev socket,id=char,path=./vhost-net,server -netdev type=vhost-user,id=mynet1,chardev=char,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet1,mrg_rxbuf=on,csum=on,mq=on,vectors=15 \ -chardev socket,id=char1,path=./vhost-net1,server -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:03,netdev=mynet2,mrg_rxbuf=on,csum=on,mq=on,vectors=15 -vnc :10 -daemonize -3. Bind virtio port to vfio-pci:: +3. In VM, get the virtio network device and bind bind it to vfio-pci as common steps 1. - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x +4. In Vm, Compile symmetric_mp and hotplug_mp example as General set up step 2-3. -4. Launch two process by example:: +4. Launch two process by symmetric_mp example in VM, each process will using ports 0 and 1, primary process using queue 0 of each port, secondary process using queue 1 of each port:: - .//examples/dpdk-symmetric_mp -l 1 -n 4 --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 - .//examples/dpdk-symmetric_mp -l 2 -n 4 --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 + # .//examples/dpdk-symmetric_mp -l 1 -n 4 --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 + # .//examples/dpdk-symmetric_mp -l 2 -n 4 --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 -5. Quit all process, check the packets number in rx/tx statistic like below for both primary process and secondary process:: +5. Send packets from vhost testpmd:: - Port 0: RX - 27511680, TX - 256, Drop - 27499168 - Port 1: RX - 27499424, TX - 256, Drop - 27511424 + testpmd> set fwd mac + testpmd> start tx_first + +6. Quit all process, check the packets number in rx/tx statistic like below for both primary process and secondary process:: + + Port 0: RX - 22322016, TX - 22321184, Drop - 0 + Port 1: RX - 22321184, TX - 22322016, Drop - 0 Test Case 2: Virtio-pmd primary and secondary process hotplug test -================================================================== +------------------------------------------------------------------ +This case use hotplug_mp example to test the MP Application hotplug, attach or detach a device on any process will broadcast to all other processes through mp channel, +then device information will be synchronized on all processes. it cover below scenario: attach a device from the primary, detach a device from the primary, +attach a device from a secondary,detach a device from a secondary. 1. Launch testpmd by below command:: - .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 + .//app/dpdk-testpmd -l 1-6 -n 4 --file-prefix=vhost \ + --vdev 'net_vhost,iface=vhost-net,queues=2,client=1' --vdev 'net_vhost1,iface=vhost-net1,queues=2,client=1' \ + -- -i --nb-cores=4 --rxq=2 --txq=2 --txd=1024 --rxd=1024 testpmd>set fwd txonly testpmd>start 2. Launch VM with two virtio ports, must set queues=2 as app receive packets from special queue which index same with proc-id:: - qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ + # qemu-system-x86_64 -name vm1 -enable-kvm -cpu host -smp 4 -m 4096 \ -object memory-backend-file,id=mem,size=4096M,mem-path=/mnt/huge,share=on \ - -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu16.img \ + -numa node,memdev=mem -mem-prealloc -drive file=/home/osimg/ubuntu20-04.img \ -chardev socket,path=/tmp/vm2_qga0.sock,server,nowait,id=vm2_qga0 -device virtio-serial \ -device virtserialport,chardev=vm2_qga0,name=org.qemu.guest_agent.2 -daemonize \ - -monitor unix:/tmp/vm2_monitor.sock,server,nowait -device e1000,netdev=nttsip1 \ + -monitor unix:/tmp/vm2_monitor.sock,server,nowait \ + -device e1000,netdev=nttsip1 \ -netdev user,id=nttsip1,hostfwd=tcp:127.0.0.1:6002-:22 \ -chardev socket,id=char,path=./vhost-net,server -netdev type=vhost-user,id=mynet1,chardev=char,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:02,netdev=mynet1,mrg_rxbuf=on,csum=on,mq=on,vectors=15 \ -chardev socket,id=char1,path=./vhost-net1,server -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce,queues=2 \ -device virtio-net-pci,mac=52:54:00:00:00:03,netdev=mynet2,mrg_rxbuf=on,csum=on,mq=on,vectors=15 -vnc :10 -daemonize -3. Bind virtio port to vfio-pci:: - - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x - ./usertools/dpdk-devbind.py --bind=vfio-pci xx:xx.x +3. In VM, get the virtio network device and bind bind it to vfio-pci as common steps 1-2. -4. Start sample code as primary process:: +4. Start sample code as primary process and list all etherdev:: - .//examples/dpdk-symmetric_mp --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 + .//examples/dpdk-hotplug_mp --proc-type=auto -- -p 3 --num-procs=2 --proc-id=0 example> list list all etherdev 0 0000:00:05.0 1 0000:00:06.0 -5. Start sample code as secondary process:: +5. Start sample code as secondary process and list all etherdev:: - .//examples/dpdk-symmetric_mp --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 + .//examples/dpdk-hotplug_mp --proc-type=secondary -- -p 3 --num-procs=2 --proc-id=1 example> list list all etherdev 0 0000:00:05.0 -- 2.25.1