Re: [dts] [PATCH] Add basic packet rxtx test case for sriov VF

["Liu, Yong" <yong.liu@intel.com>]

One more comment, thanks.

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Liu, Yong
> Sent: Thursday, December 17, 2015 8:38 PM
> To: Pei, Yulong; dts@dpdk.org
> Subject: Re: [dts] [PATCH] Add basic packet rxtx test case for sriov VF
> 
> Yulong, thanks for your patch. There're some comments in test suite.
> 
> > -----Original Message-----
> > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Yulong Pei
> > Sent: Wednesday, December 16, 2015 4:52 PM
> > To: dts@dpdk.org
> > Subject: [dts] [PATCH] Add basic packet rxtx test case for sriov VF
> >
> > An basic test case for sriov VFs, based on kernel PF drivver and dpdk VF
> > driver.
> >
> > Signed-off-by: Yulong Pei <yulong.pei@intel.com>
> > ---
> >  conf/vf_packet_rxtx.cfg                 | 105 ++++++++++++++++
> >  test_plans/vf_packet_rxtx_test_plan.rst | 100 ++++++++++++++++
> >  tests/TestSuite_vf_packet_rxtx.py       | 204
> > ++++++++++++++++++++++++++++++++
> >  3 files changed, 409 insertions(+)
> >  create mode 100644 conf/vf_packet_rxtx.cfg
> >  create mode 100644 test_plans/vf_packet_rxtx_test_plan.rst
> >  create mode 100644 tests/TestSuite_vf_packet_rxtx.py
> >
> > diff --git a/conf/vf_packet_rxtx.cfg b/conf/vf_packet_rxtx.cfg
> > new file mode 100644
> > index 0000000..986d289
> > --- /dev/null
> > +++ b/conf/vf_packet_rxtx.cfg
> > @@ -0,0 +1,105 @@
> > +# QEMU options
> > +# name
> > +#       name: vm0
> > +#
> > +# enable_kvm
> > +#       enable: [yes | no]
> > +#
> > +# cpu
> > +#       model: [host | core2duo | ...]
> > +#           usage:
> > +#               choose model value from the command
> > +#                   qemu-system-x86_64 -cpu help
> > +#       number: '4' #number of vcpus
> > +#       cpupin: '3 4 5 6' # host cpu list
> > +#
> > +# mem
> > +#       size: 1024
> > +#
> > +# disk
> > +#       file: /path/to/image/test.img
> > +#
> > +# net
> > +#        type: [nic | user | tap | bridge | ...]
> > +#           nic
> > +#               opt_vlan: 0
> > +#                   note: Default is 0.
> > +#               opt_macaddr: 00:00:00:00:01:01
> > +#                   note: if creating a nic, it`s better to specify a
> MAC,
> > +#                         else it will get a random number.
> > +#               opt_model:["e1000" | "virtio" | "i82551" | ...]
> > +#                   note: Default is e1000.
> > +#               opt_name: 'nic1'
> > +#               opt_addr: ''
> > +#                   note: PCI cards only.
> > +#               opt_vectors:
> > +#                   note: This option currently only affects virtio
> cards.
> > +#           user
> > +#               opt_vlan: 0
> > +#                   note: default is 0.
> > +#               opt_hostfwd: [tcp|udp]:[hostaddr]:hostport-
> > [guestaddr]:guestport
> > +#                   note: If not specified, it will be setted
> > automatically.
> > +#           tap
> > +#               opt_vlan: 0
> > +#                   note: default is 0.
> > +#               opt_br: br0
> > +#                   note: if choosing tap, need to specify bridge name,
> > +#                         else it will be br0.
> > +#               opt_script: QEMU_IFUP_PATH
> > +#                   note: if not specified, default is
> > self.QEMU_IFUP_PATH.
> > +#               opt_downscript: QEMU_IFDOWN_PATH
> > +#                   note: if not specified, default is
> > self.QEMU_IFDOWN_PATH.
> > +#
> > +# device
> > +#       driver: [pci-assign | virtio-net-pci | ...]
> > +#           pci-assign
> > +#               prop_host: 08:00.0
> > +#               prop_addr: 00:00:00:00:01:02
> > +#           virtio-net-pci
> > +#               prop_netdev: mynet1
> > +#               prop_id: net1
> > +#               prop_mac: 00:00:00:00:01:03
> > +#               prop_bus: pci.0
> > +#               prop_addr: 0x3
> > +#
> > +# monitor
> > +#       port: 6061
> > +#           note: if adding monitor to vm, need to specicy
> > +#                 this port, else it will get a free port
> > +#                 on the host machine.
> > +#
> > +# qga
> > +#       enable: [yes | no]
> > +#
> > +# serial_port
> > +#       enable: [yes | no]
> > +#
> > +# vnc
> > +#       displayNum: 1
> > +#           note: you can choose a number not used on the host.
> > +#
> > +# daemon
> > +#       enable: 'yes'
> > +#           note:
> > +#               By default VM will start with the daemonize status.
> > +#               Not support starting it on the stdin now.
> > +
> > +# vm configuration for pmd sriov case
> > +[vm0]
> > +cpu =
> > +    model=host,number=4,cpupin=5 6 7 8;
> > +disk =
> > +    file=/home/image/sriov-fc20-1.img;
> > +login =
> > +    user=root,password=tester;
> > +net =
> > +   type=nic,opt_vlan=0;
> > +   type=user,opt_vlan=0;
> > +monitor =
> > +    port=;
> > +qga =
> > +    enable=yes;
> > +vnc =
> > +    displayNum=1;
> > +daemon =
> > +    enable=yes;
> > diff --git a/test_plans/vf_packet_rxtx_test_plan.rst
> > b/test_plans/vf_packet_rxtx_test_plan.rst
> > new file mode 100644
> > index 0000000..df14ccf
> > --- /dev/null
> > +++ b/test_plans/vf_packet_rxtx_test_plan.rst
> > @@ -0,0 +1,100 @@
> > +.. Copyright (c) <2015>, Intel Corporation
> > +      All rights reserved.
> > +
> > +   Redistribution and use in source and binary forms, with or without
> > +   modification, are permitted provided that the following conditions
> > +   are met:
> > +
> > +   - Redistributions of source code must retain the above copyright
> > +     notice, this list of conditions and the following disclaimer.
> > +
> > +   - Redistributions in binary form must reproduce the above copyright
> > +     notice, this list of conditions and the following disclaimer in
> > +     the documentation and/or other materials provided with the
> > +     distribution.
> > +
> > +   - Neither the name of Intel Corporation nor the names of its
> > +     contributors may be used to endorse or promote products derived
> > +     from this software without specific prior written permission.
> > +
> > +   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> > +   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> > +   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> > +   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> > +   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> > +   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> > +   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> > +   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
> > +   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
> > +   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> > +   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
> > +   OF THE POSSIBILITY OF SUCH DAMAGE.
> > +
> > +
> > +Test Case 1: create 2VFs from 2 kernel PFs, passthrough 2VFs to 1VM,
> > using dpdk vf in VM, do packet rx tx test.
> >
> +=========================================================================
> > ====================================
> > +
> > +1. got the pci device id of DUT, for example,
> > +
> > +./dpdk_nic_bind.py --st
> > +
> > +0000:81:00.0 'Ethernet Controller X710 for 10GbE SFP+' if=ens259f0
> > drv=i40e unused=
> > +0000:81:00.1 'Ethernet Controller X710 for 10GbE SFP+' if=ens259f1
> > drv=i40e unused=
> > +
> > +2. create 2 VFs from 2 PFs,
> > +
> > +echo 1 > /sys/bus/pci/devices/0000\:81\:00.0/sriov_numvfs
> > +echo 1 > /sys/bus/pci/devices/0000\:81\:00.1/sriov_numvfs
> > +./dpdk_nic_bind.py --st
> > +
> > +0000:81:00.0 'Ethernet Controller X710 for 10GbE SFP+' if=ens259f0
> > drv=i40e unused=
> > +0000:81:00.1 'Ethernet Controller X710 for 10GbE SFP+' if=ens259f1
> > drv=i40e unused=
> > +0000:81:02.0 'XL710/X710 Virtual Function' unused=
> > +0000:81:0a.0 'XL710/X710 Virtual Function' unused=
> > +
> > +3. detach VFs from the host, bind them to pci-stub driver,
> > +
> > +/sbin/modprobe pci-stub
> > +
> > +echo "8086 154c" > /sys/bus/pci/drivers/pci-stub/new_id
> > +echo 0000:81:02.0 > /sys/bus/pci/devices/0000:08:02.0/driver/unbind
> > +echo 0000:81:02.0 > /sys/bus/pci/drivers/pci-stub/bind
> > +
> > +echo "8086 154c" > /sys/bus/pci/drivers/pci-stub/new_id
> > +echo 0000:81:0a.0 > /sys/bus/pci/devices/0000:08:0a.0/driver/unbind
> > +echo 0000:81:0a.0 > /sys/bus/pci/drivers/pci-stub/bind
> > +
> > +or using the following more easy way,
> > +
> > +virsh nodedev-detach pci_0000_81_02_0;
> > +virsh nodedev-detach pci_0000_81_0a_0;
> > +
> > +./dpdk_nic_bind.py --st
> > +
> > +0000:81:00.0 'Ethernet Controller X710 for 10GbE SFP+' if=ens259f0
> > drv=i40e unused=
> > +0000:81:00.1 'Ethernet Controller X710 for 10GbE SFP+' if=ens259f1
> > drv=i40e unused=
> > +0000:81:02.0 'XL710/X710 Virtual Function' if= drv=pci-stub unused=
> > +0000:81:0a.0 'XL710/X710 Virtual Function' if= drv=pci-stub unused=
> > +
> > +it can be seen that VFs 81:02.0 & 81:0a.0 's drv is pci-stub.
> > +
> > +4. passthrough VFs 81:02.0 & 81:0a.0 to vm0, and start vm0,
> > +
> > +/usr/bin/qemu-system-x86_64  -name vm0 -enable-kvm \
> > +-cpu host -smp 4 -m 2048 -drive file=/home/image/sriov-fc20-1.img -
> vnc :1
> > \
> > +-device pci-assign,host=81:02.0,id=pt_0 \
> > +-device pci-assign,host=81:0a.0,id=pt_1
> > +
> > +5. login vm0, got VFs pci device id in vm0, assume they are 00:06.0 &
> > 00:07.0, bind them to igb_uio driver,
> > +and then start testpmd, set it in mac forward mode,
> > +
> > +./tools/dpdk_nic_bind.py --bind=igb_uio 00:06.0 00:07.0
> > +./x86_64-native-linuxapp-gcc/app/testpmd -c 0x0f -n 4 -w 00:06.0 -w
> > 00:07.0 -- -i --portmask=0x3 --txqflags=0
> > +
> > +testpmd> set fwd mac
> > +testpmd> start
> > +
> > +6. get mac address of one VF and use it as dest mac, using scapy to
> send
> > 2000 ethernet frames from tester,
> > +verify the frames can be received by one VF and can be forward to
> another
> > VF correctly.
> > +
> > +sendp([Ether(dst="%s")/IP()/TCP()/("\x50"*1450)], iface="%s", count=%d)
> > diff --git a/tests/TestSuite_vf_packet_rxtx.py
> > b/tests/TestSuite_vf_packet_rxtx.py
> > new file mode 100644
> > index 0000000..93a8432
> > --- /dev/null
> > +++ b/tests/TestSuite_vf_packet_rxtx.py
> > @@ -0,0 +1,204 @@
> > +# <COPYRIGHT_TAG>
> > +
> > +import re
> > +import time
> > +
> > +import dts
> > +from qemu_kvm import QEMUKvm
> > +from test_case import TestCase
> > +from pmd_output import PmdOutput
> > +
> > +FRAME_SIZE_64 = 64
> > +VM_CORES_MASK = 'all'
> > +
> > +
> > +class TestVfPacketRxtx(TestCase):
> > +
> > +    def set_up_all(self):
> > +
> > +        self.dut_ports = self.dut.get_ports(self.nic)
> > +        self.verify(len(self.dut_ports) >= 1, "Insufficient ports")
> I think this suite need at least two ports, am I right?
> > +        self.vm0 = None
> > +
> > +    def set_up(self):
> > +
> > +        self.setup_2pf_2vf_1vm_env_flag = 0
> > +
> > +    def get_stats(self, dut, portid, rx_tx):
> > +
> > +        stats = dut.testpmd.get_pmd_stats(portid)
> > +
> > +        if rx_tx == "rx":
> > +            stats_result = [
> > +                stats['RX-packets'], stats['RX-missed'], stats['RX-
> > bytes']]
> > +        elif rx_tx == "tx":
> > +            stats_result = [
> > +                stats['TX-packets'], stats['TX-errors'], stats['TX-
> > bytes']]
> > +        else:
> > +            return None
> > +
> > +        return stats_result
> > +
> > +    def send_packet(self,
> > +                    dut,
> > +                    dut_ports,
> > +                    dest_port,
> > +                    tester_intf=False,
> > +                    frame_size=FRAME_SIZE_64,
> > +                    count=1):
> > +        """
> > +        dut: which you want to send packet to
> > +        dest_port: the port num must be the index of dut.get_ports()
> > +        count: number of packet
> > +        """
> > +
> > +        gp0rx_pkts, gp0rx_err, gp0rx_bytes = [int(_)
> > +                                              for _ in
> self.get_stats(dut,
> > dest_port, "rx")]
> > +        try:
> > +            dut_dest_port = dut_ports[dest_port]
> > +        except Exception as e:
> > +            print e
> > +
> > +        tester_port = self.tester.get_local_port(self.dut_ports[0])
> > +
> > +        if not tester_intf:
> > +            itf = self.tester.get_interface(tester_port)
> > +        else:
> > +            itf = tester_intf
> > +
> > +        dest_mac = dut.get_mac_address(dut_dest_port)
> > +        src_mac = dut.tester.get_mac(tester_port)
> > +
> > +        self.tester.scapy_foreground()
> > +        send_cmd = 'sendp([Ether(dst="%s",
> > src="%s")/IP()/TCP()/("\x50"*1450)], iface="%s", count=%d)' % (dest_mac,
> > src_mac, itf, count)
> > +        self.tester.scapy_append(send_cmd)
> > +        self.tester.scapy_execute()
> > +
> > +        time.sleep(.5)
> > +
> > +        p0rx_pkts, p0rx_err, p0rx_bytes = [int(_)
> > +                                           for _ in self.get_stats(dut,
> > dest_port, "rx")]
> > +
> Could you try Packet module to transmit and receive packets, we will use
> that module to handler packets in the future.
> 

Can you try "check_random_pkts" in tester module? By this function we can compare packet content and make sure I/O correct.

> > +        p0rx_pkts -= gp0rx_pkts
> > +        p0rx_bytes -= gp0rx_bytes
> > +        print("p0rx_pkts=%d" % p0rx_pkts)
> > +        print("p0rx_bytes=%d" % p0rx_bytes)
> > +
> This is debug message, please remove it. If this is import, please use
> command "self.logger.info" to log those informations.
> 
> > +        self.verify(p0rx_pkts == count, "Data not received by port")
> > +
> > +
> > +    def setup_2pf_2vf_1vm_env(self, driver='default'):
> > +
> > +        self.used_dut_port_0 = self.dut_ports[0]
> > +        self.dut.generate_sriov_vfs_by_port(self.used_dut_port_0, 1,
> > driver=driver)
> > +        self.sriov_vfs_port_0 =
> > self.dut.ports_info[self.used_dut_port_0]['vfs_port']
> > +
> > +        self.used_dut_port_1 = self.dut_ports[1]
> > +        self.dut.generate_sriov_vfs_by_port(self.used_dut_port_1, 1,
> > driver=driver)
> > +        self.sriov_vfs_port_1 =
> > self.dut.ports_info[self.used_dut_port_1]['vfs_port']
> > +
> > +        try:
> > +
> > +            for port in self.sriov_vfs_port_0:
> > +                port.bind_driver('pci-stub')
> > +
> > +            for port in self.sriov_vfs_port_1:
> > +                port.bind_driver('pci-stub')
> > +
> > +            time.sleep(1)
> > +            vf0_prop = {'opt_host': self.sriov_vfs_port_0[0].pci}
> > +            vf1_prop = {'opt_host': self.sriov_vfs_port_1[0].pci}
> > +
> > +            if driver == 'igb_uio':
> > +                # start testpmd without the two VFs on the host
> > +                self.host_testpmd = PmdOutput(self.dut)
> > +                eal_param = '-b %(vf0)s -b %(vf1)s' % {'vf0':
> > self.sriov_vfs_port_0[0].pci,
> > +                                                       'vf1':
> > self.sriov_vfs_port_1[0].pci}
> > +                self.host_testpmd.start_testpmd("1S/2C/2T",
> > eal_param=eal_param)
> > +
> > +            # set up VM0 ENV
> > +            self.vm0 = QEMUKvm(self.dut, 'vm0', 'vf_packet_rxtx')
> > +            self.vm0.set_vm_device(driver='pci-assign', **vf0_prop)
> > +            self.vm0.set_vm_device(driver='pci-assign', **vf1_prop)
> > +            self.vm_dut_0 = self.vm0.start()
> > +            if self.vm_dut_0 is None:
> > +                raise Exception("Set up VM0 ENV failed!")
> > +
> > +            self.setup_2pf_2vf_1vm_env_flag = 1
> > +        except Exception as e:
> > +            self.destroy_2pf_2vf_1vm_env()
> > +            raise Exception(e)
> > +
> > +    def destroy_2pf_2vf_1vm_env(self):
> > +        if getattr(self, 'vm0', None):
> > +            #destroy testpmd in vm0
> > +            self.vm0_testpmd.execute_cmd('stop')
> > +            self.vm0_testpmd.execute_cmd('quit', '# ')
> > +            self.vm0_testpmd = None
> > +            self.vm0_dut_ports = None
> > +            #destroy vm0
> > +            self.vm0.stop()
> > +            self.vm0 = None
> > +
> > +        if getattr(self, 'host_testpmd', None):
> > +            self.host_testpmd.execute_cmd('quit', '# ')
> > +            self.host_testpmd = None
> > +
> > +        if getattr(self, 'used_dut_port_0', None):
> > +            self.dut.destroy_sriov_vfs_by_port(self.used_dut_port_0)
> > +            port = self.dut.ports_info[self.used_dut_port_0]['port']
> > +            port.bind_driver()
> > +            self.used_dut_port_0 = None
> > +
> > +        if getattr(self, 'used_dut_port_1', None):
> > +            self.dut.destroy_sriov_vfs_by_port(self.used_dut_port_1)
> > +            port = self.dut.ports_info[self.used_dut_port_1]['port']
> > +            port.bind_driver()
> > +            self.used_dut_port_1 = None
> > +
> > +        for port_id in self.dut_ports:
> > +            port = self.dut.ports_info[port_id]['port']
> > +            port.bind_driver()
> > +
> > +        self.setup_2pf_2vf_1vm_env_flag = 0
> > +
> > +######1. test case for kernel pf and dpdk vf 2pf_2vf_1vm scenario
> > +
> > +    def test_packet_rx_tx_kernel_2pf_2vf_1vm(self):
> > +
> The case is too long, since this is in vf_rx_tx suite, we can use
> "test_kernel_1vf_rxtx" for short name.
> 
> > +        self.setup_2pf_2vf_1vm_env(driver='')
> > +
> > +        self.vm0_dut_ports = self.vm_dut_0.get_ports('any')
> > +
> > +        port_id_0 = 0
> > +        port_id_1 = 1
> > +        packet_num = 2000
> > +
> > +        self.vm0_testpmd = PmdOutput(self.vm_dut_0)
> > +        self.vm0_testpmd.start_testpmd(VM_CORES_MASK)
> > +        self.vm0_testpmd.execute_cmd('set fwd mac')
> > +        self.vm0_testpmd.execute_cmd('start')
> > +
> > +        time.sleep(2)
> > +
> > +        port_id_1_start_stats =
> self.vm0_testpmd.get_pmd_stats(port_id_1)
> > +        self.send_packet(
> > +            self.vm_dut_0, self.vm0_dut_ports, port_id_0,
> > count=packet_num)
> > +        port_id_1_end_stats = self.vm0_testpmd.get_pmd_stats(port_id_1)
> > +        self.verify(
> > +            port_id_1_end_stats['TX-packets'] -
> > port_id_1_start_stats['TX-packets'] == packet_num,
> > +            "VF0 failed to forward packets to VF1")
> > +
> > +    def tear_down(self):
> > +
> > +        if self.setup_2pf_2vf_1vm_env_flag == 1:
> > +            self.destroy_2pf_2vf_1vm_env()
> > +
> > +    def tear_down_all(self):
> > +
> > +        if getattr(self, 'vm0', None):
> > +            self.vm0.stop()
> > +
> > +        for port_id in self.dut_ports:
> > +            self.dut.destroy_sriov_vfs_by_port(port_id)
> > +
> > --
> > 2.1.0