[dts] [PATCH V4 0/4] l3fwd: support l3fwd pf&vf test scenario

[dts] [PATCH V4 0/4] l3fwd: support l3fwd pf&vf test scenario

[yufengmx]

 support l3fwd pf&vf test scenario. 

v4: 
 -  rebase. 
 -  

v3: 
 - close dpdk CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC compile flag when use vf. 

v2: 
 -  change pf_driver to be kernel driver. 
 -  change vf_driver to be configured in execution.cfg. 
 -  remove cases test_perf_vf_rfc2544_ipv4_em/test_perf_vf_throughput_ipv4_em. 
 -  add case test_perf_vf_throughput_ipv4_lpm. 

yufengmx (4):
  conf/vf_l3fwd_base: stream flow configuration
  conf/vf_l3fwd_kernelpf: suite testing configuration
  tests/vf_l3fwd_kernelpf: upload automation script
  vf_l3fwd: support l3fwd pf&vf test scenario

 conf/vf_l3fwd_base.cfg               |  14 +
 conf/vf_l3fwd_kernelpf.cfg           | 104 +++++++
 tests/TestSuite_vf_l3fwd_kernelpf.py |  90 ++++++
 tests/l3fwd_base.py                  | 405 +++++++++++++++++++++++----
 4 files changed, 554 insertions(+), 59 deletions(-)
 create mode 100644 conf/vf_l3fwd_base.cfg
 create mode 100644 conf/vf_l3fwd_kernelpf.cfg
 create mode 100644 tests/TestSuite_vf_l3fwd_kernelpf.py

-- 
2.21.0

[dts] [PATCH V4 1/4] conf/vf_l3fwd_base: stream flow configuration

[yufengmx]

add vf_l3fwd_kernelpf suite traffic stream flow configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/vf_l3fwd_base.cfg | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 conf/vf_l3fwd_base.cfg

diff --git a/conf/vf_l3fwd_base.cfg b/conf/vf_l3fwd_base.cfg
new file mode 100644
index 0000000..9d9ede3
--- /dev/null
+++ b/conf/vf_l3fwd_base.cfg
@@ -0,0 +1,14 @@
+[suite]
+l3fwd_flows = {
+    "ipv4": {
+        "lpm": [
+            "198.18.0.0/24",
+            "198.18.1.0/24",
+            "198.18.2.0/24",
+            "198.18.3.0/24",
+            "198.18.4.0/24",
+            "198.18.5.0/24",
+            "198.18.6.0/24",
+            "198.18.7.0/24"
+        ]
+    }, }
\ No newline at end of file
-- 
2.21.0

[dts] [PATCH V4 2/4] conf/vf_l3fwd_kernelpf: suite testing configuration

[yufengmx]

add vf_l3fwd_kernelpf suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/vf_l3fwd_kernelpf.cfg | 104 +++++++++++++++++++++++++++++++++++++
 1 file changed, 104 insertions(+)
 create mode 100644 conf/vf_l3fwd_kernelpf.cfg

diff --git a/conf/vf_l3fwd_kernelpf.cfg b/conf/vf_l3fwd_kernelpf.cfg
new file mode 100644
index 0000000..b0d4ad0
--- /dev/null
+++ b/conf/vf_l3fwd_kernelpf.cfg
@@ -0,0 +1,104 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g':
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == True, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the beginning, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as usual.
+
+[suite]
+update_expected = True
+test_duration = 12
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_throughput = {
+    'test_perf_vf_throughput_ipv4_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},}
\ No newline at end of file
-- 
2.21.0

[dts] [PATCH V4 3/4] tests/vf_l3fwd_kernelpf: upload automation script

[yufengmx]

vf_l3fwd_kernelpf suite automation script.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_vf_l3fwd_kernelpf.py | 90 ++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)
 create mode 100644 tests/TestSuite_vf_l3fwd_kernelpf.py

diff --git a/tests/TestSuite_vf_l3fwd_kernelpf.py b/tests/TestSuite_vf_l3fwd_kernelpf.py
new file mode 100644
index 0000000..f279fb6
--- /dev/null
+++ b/tests/TestSuite_vf_l3fwd_kernelpf.py
@@ -0,0 +1,90 @@
+# BSD LICENSE
+#
+# Copyright(c) 2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import os
+import time
+from test_case import TestCase
+from l3fwd_base import L3fwdBase, IP_TYPE, MATCH_MODE, SUITE_TYPE
+
+
+class TestVfL3fwdKernelPf(TestCase, L3fwdBase):
+    #
+    # Test cases.
+    #
+
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        """
+        self.verify(self.nic in [
+            "niantic",
+            "fortville_spirit",
+            "fortville_25g",
+            "fortville_eagle",
+            "columbiaville_100g",
+            "columbiaville_25g"], "NIC Unsupported: " + str(self.nic))
+        self.dut_ports = self.dut.get_ports(self.nic)
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
+        # get socket and cores
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        cores = self.dut.get_core_list("1S/6C/1T", socket=socket)
+        self.verify(cores, "Requested 6 cores failed")
+        # init l3fwd common base class parameters
+        self.l3fwd_init(valports, socket, mode=SUITE_TYPE.VF)
+        # preset testing environment
+        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        self.l3fwd_destroy_resource()
+        self.l3fwd_save_results()
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        self.l3fwd_reset_cur_case()
+
+    def test_perf_vf_throughput_ipv4_lpm(self):
+        self.l3fwd_set_cur_case('test_perf_vf_throughput_ipv4_lpm')
+        self.ms_throughput(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
-- 
2.21.0

[dts] [PATCH V4 4/4] vf_l3fwd: support l3fwd pf&vf test scenario

[yufengmx]

support l3fwd pf&vf test scenario.
pf driver use kernel driver(ice, ixgbe,i40e).
vf driver use driver configured in execution.cfg.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/l3fwd_base.py | 405 +++++++++++++++++++++++++++++++++++++-------
 1 file changed, 346 insertions(+), 59 deletions(-)

diff --git a/tests/l3fwd_base.py b/tests/l3fwd_base.py
index ca5a9f8..2b4934c 100644
--- a/tests/l3fwd_base.py
+++ b/tests/l3fwd_base.py
@@ -37,7 +37,7 @@ import time
 import traceback
 import texttable
 import json
-from enum import Enum
+from enum import Enum, unique
 from pprint import pformat
 from itertools import product
 from copy import deepcopy
@@ -50,12 +50,32 @@ from exception import VerifyFailure
 import utils
 
 
+@unique
+class SUITE_TYPE(Enum):
+    VF = 'vf_l3fwd'
+    PF = 'pf_l3fwd'
+
+
+@unique
+class SUITE_NAME(Enum):
+    VF_KERNELPF = 'vf_l3fwd_kernelpf'
+
+
+@unique
+class NIC_DRV(Enum):
+    PCI_STUB = 'pci-stub'  # linux system default driver, pci-stub
+    IGB_UIO = 'igb_uio'  # dpdk nic driver
+    VFIO_PCI = 'vfio-pci'
+
+
+@unique
 class MATCH_MODE(Enum):
     # LPM(longest prefix match) mode
     LPM = 'lpm'
     # EM(Exact-Match) mode
     EM = 'em'
 
+
 # LPM(longest prefix match) mode
 LPM = MATCH_MODE.LPM
 # EM(Exact-Match) mode
@@ -63,14 +83,17 @@ EM = MATCH_MODE.EM
 
 
 # stream internet protocol layer types
+@unique
 class IP_TYPE(Enum):
     V6 = 'ipv6'
     V4 = 'ipv4'
 
+
 L3_IPV6 = IP_TYPE.V6
 L3_IPV4 = IP_TYPE.V4
 
 
+@unique
 class STREAM_TYPE(Enum):
     UDP = 'UDP'
     RAW = 'RAW'
@@ -96,23 +119,30 @@ def get_enum_name(value, enum_cls):
 
 class L3fwdBase(object):
 
-    def l3fwd_init(self, valports, socket):
+    def l3fwd_init(self, valports, socket, mode=None):
+        self.__mode = mode or SUITE_TYPE.PF
+        self.__suite = None
         self.__valports = valports
-        self.__white_list = None
         self.__socket = socket
+        self.__core_offset = 2
         self.__nic_name = self.nic
         self.__pkt_typ = STREAM_TYPE.RAW
-        # for result
-        self.__cur_case = None
-        self.__json_results = {}
-        # binary file process
+        self.__vf_driver = None
+        self.__pf_driver = None
+        self.__vf_ports_info = None
+        self.__is_l3fwd_on = None
+        self.__l3fwd_white_list = None
         self.__l3fwd_restart = True
         self.__pre_l3fwd_cmd = None
         self.__l3fwd_wait_up = 0
         self.__traffic_stop_wait_time = 0
+        self.__is_pmd_on = None
+        self.__pmd_session = None
+        self.__cur_case = None
+        self.__json_results = {}
 
     @property
-    def output_path(self):
+    def __output_path(self):
         suiteName = self.suite_name
         if self.logger.log_path.startswith(os.sep):
             output_path = os.path.join(self.logger.log_path, suiteName)
@@ -126,6 +156,29 @@ class L3fwdBase(object):
 
         return output_path
 
+    @property
+    def __target_dir(self):
+        target_dir = '/root' + self.dut.base_dir[1:] \
+                     if self.dut.base_dir.startswith('~') else \
+                     self.dut.base_dir
+        return target_dir
+
+    @property
+    def __sockets(self):
+        sockets = [
+            cpu.get('socket')
+            for cpu in self.dut.get_all_cores() if cpu.get('socket')]
+        total = len(set(sockets))
+        self.verify(total > 0, 'cpu socket should not be zero')
+        return total
+
+    def __pmd_con(self, cmd):
+        if not self.__pmd_session:
+            return
+        _cmd = [cmd, '# ', 10] if isinstance(cmd, str) else cmd
+        output = self.__pmd_session.session.send_expect(*_cmd)
+        return output
+
     def d_con(self, cmd):
         _cmd = [cmd, '# ', 10] if isinstance(cmd, (str)) else cmd
         return self.dut.send_expect(*_cmd)
@@ -237,6 +290,16 @@ class L3fwdBase(object):
 
         return pkt.pktgen.pkt
 
+    def __get_mac_layer(self, port_id):
+        if self.__mode is SUITE_TYPE.VF:
+            smac = self.__vf_ports_info[port_id]['src_mac']
+            dmac = self.__vf_ports_info[port_id]['vf_mac']
+            layer = {'ether': {'src': smac, 'dst': dmac, }, }
+        else:
+            dmac = self.dut.get_mac_address(port_id)
+            layer = {'ether': {'dst': dmac, }, }
+        return layer
+
     def __preset_flows_configs(self):
         flows = self.__test_content.get('flows')
         if not flows:
@@ -253,8 +316,7 @@ class L3fwdBase(object):
                         if index >= len(self.__valports):
                             break
                         port_id = self.__valports[index]
-                        dmac = self.dut.get_mac_address(port_id)
-                        _layer = {'ether': {'dst': dmac, }, }
+                        _layer = self.__get_mac_layer(port_id)
                         _layer2, fields_config = \
                             self.__get_ipv4_lpm_vm_config(config) \
                             if _name is IP_TYPE.V4 else \
@@ -265,8 +327,7 @@ class L3fwdBase(object):
                             break
                         port_id = \
                             self.__valports[int(index / 2) % len(self.__valports)]
-                        dmac = self.dut.get_mac_address(port_id)
-                        _layer = {'ether': {'dst': dmac, }, }
+                        _layer = self.__get_mac_layer(port_id)
                         _layer.update(config)
                         fields_config = None
                     flows_configs.setdefault((_name, _mode), []).append(
@@ -410,43 +471,171 @@ class L3fwdBase(object):
 
         return result
 
+    def __vf_init(self):
+        self.__vf_ports_info = {}
+        drvs = []
+        if self.__pf_driver is not NIC_DRV.PCI_STUB and \
+           self.__pf_driver.value != self.drivername:
+            drvs.append(self.__pf_driver.value)
+        if self.__vf_driver.value != self.drivername:
+            drvs.append(self.__vf_driver.value)
+        for driver in drvs:
+            self.dut.setup_modules(self.target, driver, '')
+
+    def __vf_create(self):
+        for index, port_id in enumerate(self.__valports):
+            port_obj = self.dut.ports_info[port_id]['port']
+            pf_driver = port_obj.default_driver \
+                if self.__pf_driver is NIC_DRV.PCI_STUB else \
+                self.__pf_driver.value
+            self.dut.generate_sriov_vfs_by_port(port_id, 1, driver=pf_driver)
+            pf_pci = port_obj.pci
+            sriov_vfs_port = self.dut.ports_info[port_id].get('vfs_port')
+            if not sriov_vfs_port:
+                msg = f"failed to create vf on dut port {pf_pci}"
+                self.logger.error(msg)
+                continue
+            for port in sriov_vfs_port:
+                port.bind_driver(driver=self.__vf_driver.value)
+            vf_mac = "00:12:34:56:78:0%d" % (index + 1)
+            self.__vf_ports_info[port_id] = {
+                'pf_pci': pf_pci,
+                'vfs_pci': port_obj.get_sriov_vfs_pci(),
+                'vf_mac': vf_mac,
+                'src_mac': "02:00:00:00:00:0%d" % index, }
+            # ignore non pci stub
+            if self.__pf_driver is not NIC_DRV.PCI_STUB:
+                continue
+            time.sleep(1)
+            # set vf mac address.
+            port_obj.set_vf_mac_addr(mac=vf_mac)
+            pf_intf = port_obj.get_interface_name()
+            cmd = (
+                f"ifconfig {pf_intf} up;"
+                f"ethtool {pf_intf} | grep Speed")
+            self.d_con(cmd)
+        self.logger.debug(self.__vf_ports_info)
+
+    def __vf_destroy(self):
+        if not self.__vf_ports_info:
+            return
+        for port_id, _ in self.__vf_ports_info.items():
+            self.dut.destroy_sriov_vfs_by_port(port_id)
+            port_obj = self.dut.ports_info[port_id]['port']
+            port_obj.bind_driver(self.drivername)
+        self.__vf_ports_info = None
+
+    def __preset_dpdk_compilation(self):
+        # Update config file and rebuild to get best perf on FVL
+        if self.__mode is SUITE_TYPE.PF:
+            if self.nic in ["fortville_spirit", "fortville_eagle", "fortville_25g"]:
+                self.d_con(
+                    ("sed -i -e 's/"
+                     "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n/"
+                     "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=y/' "
+                     "./config/common_base"))
+                self.dut.build_install_dpdk(self.target)
+
+    def __restore_compilation(self):
+        if self.__mode is SUITE_TYPE.PF:
+            if self.nic in ["fortville_spirit", "fortville_eagle", "fortville_25g"]:
+                self.d_con(
+                    ("sed -i -e 's/"
+                     "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=y/"
+                     "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n/' "
+                     "./config/common_base"))
+                self.dut.build_install_dpdk(self.target)
+
     def __preset_compilation(self):
         # Update config file and rebuild to get best perf on FVL
-        if self.nic in ["fortville_spirit", "fortville_eagle", "fortville_25g"]:
-            self.d_con(
-                ("sed -i -e 's/"
-                 "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n/"
-                 "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=y/' "
-                 "./config/common_base"))
-            self.dut.build_install_dpdk(self.target)
+        self.__preset_dpdk_compilation()
         # init l3fwd binary file
-        self.logger.info(
-            "Configure RX/TX descriptor to 2048, re-build ./examples/l3fwd")
-        self.d_con((
-            "sed -i -e 's/"
-            "define RTE_TEST_RX_DESC_DEFAULT.*$/"
-            "define RTE_TEST_RX_DESC_DEFAULT 2048/' "
-            "./examples/l3fwd/l3fwd.h"))
-        self.d_con((
-            "sed -i -e 's/"
-            "define RTE_TEST_TX_DESC_DEFAULT.*$/"
-            "define RTE_TEST_TX_DESC_DEFAULT 2048/' "
-            "./examples/l3fwd/l3fwd.h"))
-        self.__l3fwd_em = self.__init_l3fwd(MATCH_MODE.EM)
-        self.__l3fwd_lpm = self.__init_l3fwd(MATCH_MODE.LPM)
-
-    def __init_l3fwd(self, mode):
+        if self.nic not in ["columbiaville_100g", "columbiaville_25g"]:
+            self.logger.info(
+                "Configure RX/TX descriptor to 2048, re-build ./examples/l3fwd")
+            self.d_con((
+                "sed -i -e 's/"
+                "define RTE_TEST_RX_DESC_DEFAULT.*$/"
+                "define RTE_TEST_RX_DESC_DEFAULT 2048/' "
+                "./examples/l3fwd/l3fwd.h"))
+            self.d_con((
+                "sed -i -e 's/"
+                "define RTE_TEST_TX_DESC_DEFAULT.*$/"
+                "define RTE_TEST_TX_DESC_DEFAULT 2048/' "
+                "./examples/l3fwd/l3fwd.h"))
+        if self.__mode is SUITE_TYPE.VF:
+            self.__l3fwd_lpm = self.__l3fwd_em = \
+                self.__init_l3fwd(MATCH_MODE.EM, rename=False)
+            # init testpmd
+            if self.__pf_driver is not NIC_DRV.PCI_STUB:
+                self.__init_testpmd()
+        else:
+            self.__l3fwd_em = self.__init_l3fwd(MATCH_MODE.EM)
+            self.__l3fwd_lpm = self.__init_l3fwd(MATCH_MODE.LPM)
+
+    def __init_testpmd(self):
+        self.__pmd_session_name = 'testpmd'
+        self.__pmd_session = self.dut.create_session(self.__pmd_session_name)
+        self.__testpmd = "{}/{}/app/testpmd".format(
+            self.__target_dir, self.dut.target)
+
+    def __start_testpmd(self):
+        """
+        require enough PF ports,using kernel or dpdk driver, create 1 VF from each PF.
+        """
+        corelist = self.dut.get_core_list(
+            "1S/{}C/1T".format(self.__core_offset), socket=self.__socket)
+        core_mask = utils.create_mask(corelist[2:])
+        # set memory size
+        mem_size = ','.join(['1024'] * self.__sockets)
+        cmd = (
+            "{bin} "
+            " -v "
+            "-c {core_mask} "
+            "-n {mem_channel} "
+            "--socket-mem={memsize} "
+            "--file-prefix={prefix} "
+            "{whitelist} "
+            "-- -i ").format(**{
+                'bin': self.__testpmd,
+                'core_mask': core_mask,
+                'mem_channel': self.dut.get_memory_channels(),
+                'memsize': mem_size,
+                'whitelist': self.__get_testpmd_whitelist(),
+                'prefix': 'pf', })
+        self.__pmd_con([cmd, "testpmd> ", 120])
+        self.__is_pmd_on = True
+        index = 0
+        for _, info in self.__vf_ports_info.items():
+            cmd = 'set vf mac addr %d 0 %s' % (index, info.get('vf_mac'))
+            self.__pmd_con([cmd, "testpmd> ", 15])
+            index += 1
+        self.__pmd_con(['start', "testpmd> ", 15])
+        time.sleep(1)
+
+    def __close_testpmd(self):
+        """
+        destroy the setup VFs
+        """
+        if not self.__is_pmd_on:
+            return
+        self.__pmd_con(['quit', '# ', 15])
+        self.__is_pmd_on = False
+
+    def __init_l3fwd(self, mode, rename=True):
         """
         Prepare long prefix match table, __replace P(x) port pattern
         """
         l3fwd_method = '_'.join(['l3fwd', mode.value])
         self.d_con("make clean -C examples/l3fwd")
         flg = 1 if mode is MATCH_MODE.LPM else 0
-        out = self.dut.build_dpdk_apps(
-            "./examples/l3fwd",
-            "USER_FLAGS=-DAPP_LOOKUP_METHOD={}".format(flg))
+        _opt = "USER_FLAGS=-DAPP_LOOKUP_METHOD={}".format(flg) \
+            if self.__mode is SUITE_TYPE.PF else ''
+        out = self.dut.build_dpdk_apps("./examples/l3fwd", _opt)
         self.verify("Error" not in out, "compilation error 1")
         self.verify("No such file" not in out, "compilation error 2")
+        if not rename:
+            return "./examples/l3fwd/build/l3fwd"
         # rename binary file
         self.d_con(
             ("mv -f examples/l3fwd/build/l3fwd "
@@ -459,6 +648,7 @@ class L3fwdBase(object):
         # Start L3fwd application
         command_line = (
             "{bin} "
+            "-v "
             "-c {cores} "
             "-n {channel} "
             "{whitelist}"
@@ -469,7 +659,7 @@ class L3fwdBase(object):
                 'bin': bin,
                 'cores': core_mask,
                 'channel': self.dut.get_memory_channels(),
-                'whitelist': self.__white_list if self.__white_list else '',
+                'whitelist': self.__l3fwd_white_list if self.__l3fwd_white_list else '',
                 'port_mask': utils.create_mask(self.__valports),
                 'config': config, })
         if self.nic == "niantic":
@@ -509,7 +699,7 @@ class L3fwdBase(object):
             return
         if not self.__l3fwd_restart and self.__pre_l3fwd_cmd:
             return
-        self.d_con("^C")
+        self.d_con(["^C", '# ', 25])
         self.__is_l3fwd_on = False
 
     def __json_rfc2544(self, value):
@@ -569,7 +759,7 @@ class L3fwdBase(object):
             self.logger.error(msg)
             return
         _js_file = os.path.join(
-            self.output_path,
+            self.__output_path,
             json_file if json_file else 'l3fwd_result.json')
         with open(_js_file, 'w') as fp:
             json.dump(self.__json_results, fp, indent=4,
@@ -762,12 +952,13 @@ class L3fwdBase(object):
 
     def __parse_port_config(self, config):
         cores, total_threads, queue = config.split('/')
-        _thread = str(int(int(total_threads[:-1]) / int(cores[:-1]))) + 'T'
-        _cores = str(int(cores[:-1]) * len(self.__valports)) + 'C'
+        _thread_num = int(int(total_threads[:-1]) // int(cores[:-1]))
+        _thread = str(_thread_num) + 'T'
+        _cores = str(self.__core_offset + int(cores[:-1]) * len(self.__valports)) + 'C'
         # only use one socket
         cores_config = '/'.join(['1S', _cores, _thread])
         queues_per_port = int(queue[:-1])
-        return cores_config, queues_per_port
+        return cores_config, _thread_num, queues_per_port
 
     def __get_test_configs(self, options, ports, socket):
         if not options:
@@ -778,12 +969,12 @@ class L3fwdBase(object):
         for test_item, frame_sizes in sorted(options.items()):
             _frame_sizes = [int(frame_size) for frame_size in frame_sizes]
             frame_sizes_grp.extend([int(item) for item in _frame_sizes])
-            cores, queues_per_port = self.__parse_port_config(test_item)
+            cores, thread_num, queues_per_port = self.__parse_port_config(test_item)
             grp = [list(item)
                    for item in product(range(queues_per_port), range(ports))]
             corelist = self.dut.get_core_list(
                 cores, socket if cores.startswith('1S') else -1)
-            corelist = [str(int(core) + 2) for core in corelist]
+            corelist = corelist[self.__core_offset*thread_num:]
             cores_mask = utils.create_mask(corelist)
             total = len(grp)
             _corelist = (corelist * (total // len(corelist) + 1))[:total]
@@ -799,10 +990,70 @@ class L3fwdBase(object):
                 frame_size, ]) for frame_size in _frame_sizes]
         return configs, sorted(set(frame_sizes_grp))
 
+    def __get_suite_vf_pf_driver(self, test_content):
+        if self.__suite is SUITE_NAME.VF_KERNELPF:
+            pf_driver = NIC_DRV.PCI_STUB.value
+            vf_driver = None  # used configuration in execution.cfg
+        else:  # use config in <suite name>.cfg
+            pf_driver = test_content.get('pf_driver')
+            vf_driver = test_content.get('vf_driver')
+        return pf_driver, vf_driver
+
+    def __get_vf_test_content_from_cfg(self, test_content):
+        self.__suite = get_enum_name(self.suite_name, SUITE_NAME)
+        # pf driver
+        pf_driver, vf_driver = self.__get_suite_vf_pf_driver(test_content)
+        if pf_driver and isinstance(pf_driver, str):
+            self.__pf_driver = get_enum_name(pf_driver.lower(), NIC_DRV)
+        else:
+            self.__pf_driver = get_enum_name(self.drivername.lower(), NIC_DRV)
+            msg = (
+                f"pf driver use {self.__pf_driver}, "
+                f"{pf_driver} is set in cfg file")
+            self.logger.warning(msg)
+        # limit pf driver usage
+        if self.__pf_driver is NIC_DRV.IGB_UIO or \
+           self.__pf_driver is NIC_DRV.PCI_STUB:
+            pass
+        else:
+            msg = f"not support {self.__pf_driver.value} for pf nic !"
+            raise VerifyFailure(msg)
+        # vf driver
+        if vf_driver and isinstance(vf_driver, str) and \
+           vf_driver != NIC_DRV.PCI_STUB.value:
+            self.__vf_driver = get_enum_name(vf_driver.lower(), NIC_DRV)
+        else:
+            self.__vf_driver = get_enum_name(self.drivername.lower(), NIC_DRV)
+            msg = (
+                f"vf driver use {self.__vf_driver}, "
+                f"{vf_driver} is set in cfg file")
+            self.logger.warning(msg)
+        # limit vf driver usage
+        if self.__vf_driver is NIC_DRV.IGB_UIO or \
+           self.__vf_driver is NIC_DRV.VFIO_PCI:
+            pass
+        else:
+            msg = f"not support {self.__vf_driver.value} for vf nic !"
+            raise VerifyFailure(msg)
+        # set core offset
+        self.logger.info(
+             f"pf driver type: {self.__pf_driver}, "
+             f"vf driver type: {self.__vf_driver}")
+        if self.__pf_driver is not NIC_DRV.PCI_STUB:
+            self.__core_offset += max(2, len(self.__valports))
+
     def __get_test_content_from_cfg(self, test_content):
         self.logger.debug(pformat(test_content))
         # get flows configuration
-        suite_conf = SuiteConf('l3fwd_base')
+        conf_table = {
+            SUITE_TYPE.PF: 'l3fwd_base',
+            SUITE_TYPE.VF: 'vf_l3fwd_base',
+        }
+        conf_name = conf_table.get(self.__mode)
+        if not conf_name:
+            msg = f"{self.__mode} not supported"
+            raise VerifyFailure(msg)
+        suite_conf = SuiteConf(conf_name)
         flows = suite_conf.suite_cfg.get('l3fwd_flows')
         test_content['flows'] = flows
         # set stream format type
@@ -812,6 +1063,9 @@ class L3fwdBase(object):
         else:
             msg = f"use default stream format {self.__pkt_typ.value}"
             self.logger.warning(msg)
+        # get vf driver
+        if self.__mode is SUITE_TYPE.VF:
+            self.__get_vf_test_content_from_cfg(test_content)
         # binary file process setting
         self.__l3fwd_wait_up = test_content.get('l3fwd_wait_up', 0)
         self.__l3fwd_restart = test_content.get('l3fwd_restart', True)
@@ -827,14 +1081,27 @@ class L3fwdBase(object):
 
         return test_content
 
-    def __get_whitelist(self, port_list):
-        white_list = ''
-        for port_index in port_list:
-            pci = self.dut.ports_info[port_index].get('pci')
-            if not pci:
-                continue
-            white_list += '-w {} '.format(pci)
-        return white_list
+    def __get_l3fwd_whitelist(self, port_list=None):
+        whitelist = ''
+        if self.__mode is SUITE_TYPE.PF:
+            if not port_list:
+                return None
+            for port_index in port_list:
+                pci = self.dut.ports_info[port_index].get('pci')
+                if not pci:
+                    continue
+                whitelist += '-w {} '.format(pci)
+        else:
+            whitelist = ''.join(['-w {} '.format(pci)
+                                 for _, info in self.__vf_ports_info.items()
+                                 for pci in info.get('vfs_pci')])
+
+        return whitelist
+
+    def __get_testpmd_whitelist(self):
+        whitelist = ''.join(['-w {} '.format(info.get('pf_pci'))
+                             for _, info in self.__vf_ports_info.items()])
+        return whitelist
 
     def __preset_port_list(self, test_content):
         port_list = test_content.get('port_list')
@@ -848,20 +1115,40 @@ class L3fwdBase(object):
                     pformat(port_list))
                 self.logger.info(msg)
             self.__valports = port_list
-            self.__white_list = self.__get_whitelist(port_list)
+        return port_list
 
     def l3fwd_preset_test_environment(self, test_content):
         # if user set port list in cfg file, use
-        self.__preset_port_list(test_content)
+        port_list = self.__preset_port_list(test_content)
         # get test content
         self.__test_content = self.__get_test_content_from_cfg(test_content)
-        # binary process flag
-        self.__is_l3fwd_on = None
         # prepare target source code application
         self.__preset_compilation()
+        # set up testing environment
+        if self.__mode is SUITE_TYPE.VF:
+            self.__vf_init()
+            self.__vf_create()
+            self.__l3fwd_white_list = self.__get_l3fwd_whitelist()
+            if self.__pf_driver is not NIC_DRV.PCI_STUB:
+                self.__start_testpmd()
+        else:
+            self.__l3fwd_white_list = self.__get_l3fwd_whitelist(port_list)
         # config streams
         self.__streams = self.__preset_streams()
 
+    def l3fwd_destroy_resource(self):
+        if self.__mode is SUITE_TYPE.VF:
+            if self.__pf_driver is NIC_DRV.PCI_STUB:
+                pass
+            else:
+                self.__close_testpmd()
+                if self.__pmd_session:
+                    self.dut.close_session(self.__pmd_session)
+                    self.__pmd_session = None
+            self.__vf_destroy()
+        if self.__mode is SUITE_TYPE.PF:
+            self.__restore_compilation()
+
     def l3fwd_set_cur_case(self, name):
         self.__cur_case = name
 
-- 
2.21.0

Re: [dts] [PATCH V4 0/4] l3fwd: support l3fwd pf&vf test scenario

[Tu, Lijuan]

Applied the series, thanks

> -----Original Message-----
> From: Mo, YufengX
> Sent: Friday, April 10, 2020 3:52 PM
> To: dts@dpdk.org; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts][PATCH V4 0/4] l3fwd: support l3fwd pf&vf test scenario
> 
>  support l3fwd pf&vf test scenario.
> 
> v4:
>  -  rebase.
>  -
> 
> v3:
>  - close dpdk CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC compile flag when
> use vf.
> 
> v2:
>  -  change pf_driver to be kernel driver.
>  -  change vf_driver to be configured in execution.cfg.
>  -  remove cases
> test_perf_vf_rfc2544_ipv4_em/test_perf_vf_throughput_ipv4_em.
>  -  add case test_perf_vf_throughput_ipv4_lpm.
> 
> yufengmx (4):
>   conf/vf_l3fwd_base: stream flow configuration
>   conf/vf_l3fwd_kernelpf: suite testing configuration
>   tests/vf_l3fwd_kernelpf: upload automation script
>   vf_l3fwd: support l3fwd pf&vf test scenario
> 
>  conf/vf_l3fwd_base.cfg               |  14 +
>  conf/vf_l3fwd_kernelpf.cfg           | 104 +++++++
>  tests/TestSuite_vf_l3fwd_kernelpf.py |  90 ++++++
>  tests/l3fwd_base.py                  | 405 +++++++++++++++++++++++----
>  4 files changed, 554 insertions(+), 59 deletions(-)  create mode 100644
> conf/vf_l3fwd_base.cfg  create mode 100644 conf/vf_l3fwd_kernelpf.cfg
> create mode 100644 tests/TestSuite_vf_l3fwd_kernelpf.py
> 
> --
> 2.21.0