[dts] [PATCH V1 0/7] meson build reference

[dts] [PATCH V1 0/7] meson build reference

[lihong]

lihong (7):
  executions: add build_type field in all executions cfg
  conf: add configuration file of app_name.cfg
  framework/config: get configuration information from app_name.cfg
  framework: read and save the build_type from execution.cfg
  framework: add meson build method and get apps name of current build
    type
  framework/pmd_output: replace the hard code app path with the
    configured app path
  tests: replace the hard code app path with the configured app path

 conf/app_name.cfg                         | 119 +++++++++++++++++++++++++++
 executions/execution.cfg                  |   2 +
 executions/execution_FVL.cfg              |   2 +
 executions/execution_fm10k.cfg            |   2 +
 executions/execution_rxmode.cfg           |   3 +
 executions/execution_smoke.cfg            |   1 +
 framework/config.py                       |  29 +++++++
 framework/dts.py                          |   7 ++
 framework/dut.py                          |   8 +-
 framework/pmd_output.py                   |   3 +-
 framework/project_dpdk.py                 | 128 ++++++++++++++++++++++++++++--
 framework/settings.py                     |   1 +
 tests/TestSuite_multiprocess.py           |  53 ++++++++-----
 tests/TestSuite_vhost_multi_queue_qemu.py |  14 ++--
 14 files changed, 335 insertions(+), 37 deletions(-)
 create mode 100644 conf/app_name.cfg

-- 
2.7.4

[dts] [PATCH V1 1/7] executions: add build_type field in all executions cfg

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 executions/execution.cfg        | 2 ++
 executions/execution_FVL.cfg    | 2 ++
 executions/execution_fm10k.cfg  | 2 ++
 executions/execution_rxmode.cfg | 3 +++
 executions/execution_smoke.cfg  | 1 +
 5 files changed, 10 insertions(+)

diff --git a/executions/execution.cfg b/executions/execution.cfg
index 0683d74..0a1ac3c 100644
--- a/executions/execution.cfg
+++ b/executions/execution.cfg
@@ -1,6 +1,7 @@
 [Execution1]
 crbs=<CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci or mlx5_core or uio_pci_generic>
+build_type=<makefile or meson>
 test_suites=
     cmdline,
     hello_world,
@@ -35,6 +36,7 @@ parameters=nic_type=cfg:func=true
 [Execution2]
 crbs=<Performance CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci>
+build_type=<makefile or meson>
 test_suites=
     l2fwd,
     l3fwd,
diff --git a/executions/execution_FVL.cfg b/executions/execution_FVL.cfg
index d5e6cfb..84c1649 100644
--- a/executions/execution_FVL.cfg
+++ b/executions/execution_FVL.cfg
@@ -1,6 +1,7 @@
 [Execution1]
 crbs=<CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci or mlx5_core or uio_pci_generic>
+build_type=<makefile or meson>
 test_suites=
     checksum_offload,
     dynamic_config,
@@ -36,6 +37,7 @@ parameters=nic_type=cfg:func=true
 [Execution2]
 crbs=<Performance CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci>
+build_type=<makefile or meson>
 test_suites=
     l2fwd,
     l3fwd,
diff --git a/executions/execution_fm10k.cfg b/executions/execution_fm10k.cfg
index 971e4a4..dbd2b96 100644
--- a/executions/execution_fm10k.cfg
+++ b/executions/execution_fm10k.cfg
@@ -1,6 +1,7 @@
 [Execution1]
 crbs=<CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci or uio_pci_generic>
+build_type=<makefile or meson>
 test_suites=
     checksum_offload,
     vlan,
@@ -19,6 +20,7 @@ parameters=nic_type=cfg:func=true
 [Execution2]
 crbs=<Performance CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci>
+build_type=<makefile or meson>
 test_suites=
     l3fwd,
 targets=
diff --git a/executions/execution_rxmode.cfg b/executions/execution_rxmode.cfg
index cf46dbf..e1bf981 100644
--- a/executions/execution_rxmode.cfg
+++ b/executions/execution_rxmode.cfg
@@ -2,6 +2,7 @@
 crbs=<CRB IP Address>
 drivername=igb_uio
 rx_mode=scalar
+build_type=<makefile or meson>
 test_suites=
     fdir,
     jumboframes,
@@ -23,6 +24,7 @@ parameters=nic_type=cfg:func=true
 crbs=<CRB IP Address>
 drivername=igb_uio
 rx_mode=full
+build_type=<makefile or meson>
 test_suites=
     fdir,
     jumboframes,
@@ -44,6 +46,7 @@ parameters=nic_type=cfg:func=true
 crbs=<CRB IP Address>
 drivername=igb_uio
 rx_mode=vector
+build_type=<makefile or meson>
 test_suites=
     fdir,
     jumboframes,
diff --git a/executions/execution_smoke.cfg b/executions/execution_smoke.cfg
index f069365..ea5bd3f 100644
--- a/executions/execution_smoke.cfg
+++ b/executions/execution_smoke.cfg
@@ -1,6 +1,7 @@
 [Execution1]
 crbs=<CRB IP Address>
 drivername=<driver name igb_uio or vfio-pci or mlx5_core or uio_pci_generic>
+build_type=<makefile or meson>
 test_suites=
     unit_tests_cmdline,
     unit_tests_eal,
-- 
2.7.4

[dts] [PATCH V1 2/7] conf: add configuration file of app_name.cfg

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 conf/app_name.cfg | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)
 create mode 100644 conf/app_name.cfg

diff --git a/conf/app_name.cfg b/conf/app_name.cfg
new file mode 100644
index 0000000..928b2db
--- /dev/null
+++ b/conf/app_name.cfg
@@ -0,0 +1,119 @@
+# app path configuration
+# [meson/makefile] : the build type
+# app_name=app_path #about the app_path, you can use 'target' instead of build target in the path
+# eg: if the testpmd app_path is: x86_64-native-linuxapp-gcc/app/testpmd,
+#     you can use string 'target' instead of 'x86_64-native-linuxapp-gcc'
+# test-pmd=target/app/testpmd
+[meson]
+pdump=target/app/dpdk-pdump
+proc-info=target/app/dpdk-proc-info
+test=target/app/test/dpdk-test
+test-acl=target/app/dpdk-test-acl
+test-bbdev=target/app/dpdk-test-bbdev
+test-cmdline=target/app/dpdk-test-cmdline
+test-compress-perf=target/app/dpdk-test-compress-perf
+test-crypto-perf=target/app/dpdk-test-crypto-perf
+test-eventdev=target/app/dpdk-test-eventdev
+test-fib=target/app/dpdk-test-fib
+test-pipeline=target/app/dpdk-test-pipeline
+test-pmd=target/app/dpdk-testpmd
+test-sad=target/app/dpdk-test-sad
+ethtool=target/examples/dpdk-ethtool
+eventdev_pipeline=target/examples/dpdk-eventdev_pipeline
+fips_validation=target/examples/dpdk-fips_validation
+flow_classify=target/examples/dpdk-flow_classify
+flow_filtering=target/examples/dpdk-flow_filtering
+helloworld=target/examples/dpdk-helloworld
+ioat=target/examples/dpdk-ioat
+ip_fragmentation=target/examples/dpdk-ip_fragmentation
+ip_pipeline=target/examples/dpdk-ip_pipeline
+ip_reassembly=target/examples/dpdk-ip_reassembly
+ipsec-secgw=target/examples/dpdk-ipsec-secgw
+ipv4_multicast=target/examples/dpdk-ipv4_multicast
+kni=target/examples/dpdk-kni
+l2fwd=target/examples/dpdk-l2fwd
+l2fwd-crypto=target/examples/dpdk-l2fwd-crypto
+l2fwd-event=target/examples/dpdk-l2fwd-event
+l2fwd-jobstats=target/examples/dpdk-l2fwd-jobstats
+l2fwd-keepalive=target/examples/dpdk-l2fwd-keepalive
+l3fwd=target/examples/dpdk-l3fwd
+l3fwd-acl=target/examples/dpdk-l3fwd-acl
+l3fwd-power=target/examples/dpdk-l3fwd-power
+link_status_interrupt=target/examples/dpdk-link_status_interrupt
+ntb=target/examples/dpdk-ntb
+packet_ordering=target/examples/dpdk-packet_ordering
+ptpclient=target/examples/dpdk-ptpclient
+qos_meter=target/examples/dpdk-qos_meter
+qos_sched=target/examples/dpdk-qos_sched
+rxtx_callbacks=target/examples/dpdk-rxtx_callbacks
+service_cores=target/examples/dpdk-service_cores
+skeleton=target/examples/dpdk-skeleton
+tep_termination=target/examples/dpdk-tep_termination
+timer=target/examples/dpdk-timer
+vdpa=target/examples/dpdk-vdpa
+vhost=target/examples/dpdk-vhost
+vhost_blk=target/examples/dpdk-vhost_blk
+vhost_crypto=target/examples/dpdk-vhost_crypto
+vmdq=target/examples/dpdk-vmdq
+vmdq_dcb=target/examples/dpdk-vmdq_dcb
+mp_client=target/examples/dpdk-mp_client
+hotplug_mp=target/examples/dpdk-hotplug_mp
+mp_server=target/examples/dpdk-mp_server
+simple_mp=target/examples/dpdk-simple_mp
+symmetric_mp=target/examples/dpdk-symmetric_mp
+[makefile]
+pdump=target/app/dpdk-pdump
+proc-info=target/app/dpdk-procinfo
+test=target/app/test
+test-acl=target/app/testacl
+test-bbdev=target/app/testbbdev
+test-cmdline=target/app/cmdline_test
+test-compress-perf=target/app/dpdk-test-compress-perf
+test-crypto-perf=target/app/dpdk-test-crypto-perf
+test-eventdev=target/app/dpdk-test-eventdev
+test-pipeline=target/app/testpipeline
+test-pmd=target/app/testpmd
+test-sad=target/app/testsad
+ethtool=examples/ethtool/ethtool-app/target/ethtool
+eventdev_pipeline=examples/build/app/eventdev_pipeline
+fips_validation=examples/build/app/fips_validation
+flow_classify=examples/build/app/flow_classify
+flow_filtering=examples/build/app/flow
+helloworld=examples/build/app/helloworld
+ioat=examples/build/app/ioatfwd
+ip_fragmentation=examples/build/app/ip_fragmentation
+ip_pipeline=examples/build/app/ip_pipeline
+ip_reassembly=examples/build/app/ip_reassembly
+ipsec-secgw=examples/build/app/ipsec-secgw
+ipv4_multicast=examples/build/app/ipv4_multicast
+kni=examples/build/app/kni
+l2fwd=examples/build/app/l2fwd
+l2fwd-crypto=examples/build/app/l2fwd-crypto
+l2fwd-event=examples/build/app/l2fwd-event
+l2fwd-jobstats=examples/build/app/l2fwd-jobstats
+l2fwd-keepalive=examples/build/app/l2fwd-keepalive
+l3fwd=examples/build/app/l3fwd
+l3fwd-acl=examples/build/app/l3fwd-acl
+l3fwd-power=examples/build/app/l3fwd-power
+link_status_interrupt=examples/build/app/link_status_interrupt
+ntb=examples/build/app/ntb_fwd
+packet_ordering=examples/build/app/packet_ordering
+ptpclient=examples/build/app/ptpclient
+qos_meter=examples/build/app/qos_meter
+qos_sched=examples/build/app/qos_sched
+rxtx_callbacks=examples/build/app/rxtx_callbacks
+service_cores==examples/build/app/service_cores
+skeleton=examples/build/app/basicfwd
+tep_termination=examples/build/app/tep_termination
+timer=examples/build/app/timer
+vdpa=examples/build/app/vdpa
+vhost=examples/build/app/vhost-switch
+vhost_blk=examples/build/app/vhost-blk
+vhost_crypto=examples/build/app/vhost-crypto
+vmdq=examples/build/app/vmdq_app
+vmdq_dcb=examples/build/app/vmdq_dcb_app
+mp_client=examples/multi_process/client_server_mp/mp_client/target/app/mp_client
+hotplug_mp=examples/multi_process/hotplug_mp/target/app/hotplug_mp
+mp_server=examples/multi_process/client_server_mp/mp_client/target/app/mp_server
+simple_mp/=examples/multi_process/simple_mp/target/app/simple_mp/
+symmetric_mp=examples/multi_process/symmetric_mp/target/app/symmetric_mp
-- 
2.7.4

[dts] [PATCH V1 3/7] framework/config: get configuration information from app_name.cfg

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 framework/config.py | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/framework/config.py b/framework/config.py
index 4dc3f31..dd3bf30 100644
--- a/framework/config.py
+++ b/framework/config.py
@@ -48,6 +48,7 @@ IXIACONF = "%s/ixia.cfg" % CONFIG_ROOT_PATH
 PKTGENCONF = "%s/pktgen.cfg" % CONFIG_ROOT_PATH
 SUITECONF_SAMPLE = "%s/suite_sample.cfg" % CONFIG_ROOT_PATH
 GLOBALCONF = "%s/global_suite.cfg" % CONFIG_ROOT_PATH
+APPNAMECONF = "%s/app_name.cfg" % CONFIG_ROOT_PATH
 
 
 class UserConf():
@@ -471,6 +472,34 @@ class PktgenConf(UserConf):
 
         return self.pktgen_cfg
 
+class AppNameConf(UserConf):
+    def __init__(self, app_name_conf=APPNAMECONF):
+        self.config_file = app_name_conf
+        self.app_name_cfg = {}
+        try:
+            self.app_name_conf = UserConf(self.config_file)
+        except ConfigParseException:
+            self.app_name_conf = None
+            raise ConfigParseException
+
+    def load_app_name_conf(self):
+        sections = self.app_name_conf.get_sections()
+        if not sections:
+            return self.app_name_cfg
+
+        for build_type in sections:
+            cur_name_cfg = self.app_name_conf.load_section(build_type)
+            if not cur_name_cfg:
+                continue
+
+            name_cfg = {}
+            for cfg in cur_name_cfg:
+                key, value = cfg
+                name_cfg[key] = value
+
+            self.app_name_cfg[build_type.lower()]=name_cfg
+
+        return self.app_name_cfg
 
 if __name__ == '__main__':
     parser = argparse.ArgumentParser(
-- 
2.7.4

[dts] [PATCH V1 4/7] framework: read and save the build_type from execution.cfg

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 framework/dts.py      | 7 +++++++
 framework/settings.py | 1 +
 2 files changed, 8 insertions(+)

diff --git a/framework/dts.py b/framework/dts.py
index d6c4fe5..2c0c86a 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -88,6 +88,13 @@ def dts_parse_param(config, section):
 
     parameters = config.get(section, 'parameters').split(':')
     drivername = config.get(section, 'drivername').split('=')[-1]
+    # get the build method, default is makefile
+    try:
+        buildtype = config.get(section, 'build_type').split('=')[-1]
+    except:
+        buildtype = 'makefile'
+    buildtype = buildtype.lower()
+    settings.save_global_setting(settings.HOST_BUILD_TYPE_SETTING, buildtype)
 
     driver = drivername.split(':')
     if len(driver) == 2:
diff --git a/framework/settings.py b/framework/settings.py
index bb23077..f91452d 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -249,6 +249,7 @@ PERF_SETTING = "DTS_PERF_ONLY"
 FUNC_SETTING = "DTS_FUNC_ONLY"
 HOST_DRIVER_SETTING = "DTS_HOST_DRIVER"
 HOST_DRIVER_MODE_SETTING = "DTS_HOST_DRIVER_MODE"
+HOST_BUILD_TYPE_SETTING = "DTS_HOST_BUILD_TYPE"
 HOST_NIC_SETTING = "DTS_HOST_NIC"
 HOST_SHARED_LIB_SETTING = "DTS_HOST_SHARED_LIB"
 HOST_SHARED_LIB_PATH = "DTS_HOST_SHARED_LIB_PATH"
-- 
2.7.4

[dts] [PATCH V1 5/7] framework: add meson build method and get apps name of current build type

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 framework/dut.py          |   8 ++-
 framework/project_dpdk.py | 128 +++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 127 insertions(+), 9 deletions(-)

diff --git a/framework/dut.py b/framework/dut.py
index 2545621..bef0d15 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -33,7 +33,7 @@ import os
 import re
 import time
 import settings
-from config import PortConf
+from config import PortConf,AppNameConf
 from settings import NICS, LOG_NAME_SEP
 from ssh_connection import SSHConnection
 from crb import Crb
@@ -78,6 +78,8 @@ class Dut(Crb):
         self.prefix_subfix = str(os.getpid()) + '_' + time.strftime("%Y%m%d%H%M%S", time.localtime())
         self.prefix_list = []
         self.hugepage_path = None
+        self.apps_name_conf = {}
+        self.apps_name = {}
 
     def filter_cores_from_crb_cfg(self):
         # get core list from crbs.cfg
@@ -388,6 +390,10 @@ class Dut(Crb):
         # initialize virtualization resource pool
         self.virt_pool = VirtResource(self)
 
+        # load app name conf
+        name_cfg = AppNameConf()
+        self.apps_name_conf = name_cfg.load_app_name_conf()
+
     def restore_interfaces(self):
         """
         Restore all ports's interfaces.
diff --git a/framework/project_dpdk.py b/framework/project_dpdk.py
index c0dfd59..040d070 100644
--- a/framework/project_dpdk.py
+++ b/framework/project_dpdk.py
@@ -33,7 +33,7 @@ import os
 import re
 
 from settings import NICS, load_global_setting, accepted_nic
-from settings import DPDK_RXMODE_SETTING, HOST_DRIVER_SETTING, HOST_DRIVER_MODE_SETTING
+from settings import DPDK_RXMODE_SETTING, HOST_DRIVER_SETTING, HOST_DRIVER_MODE_SETTING, HOST_BUILD_TYPE_SETTING
 from settings import HOST_SHARED_LIB_SETTING, HOST_SHARED_LIB_PATH
 from ssh_connection import SSHConnection
 from crb import Crb
@@ -73,6 +73,16 @@ class DPDKdut(Dut):
 
         self.set_driver_specific_configurations(drivername)
 
+        # get apps name of current build type
+        build_type = load_global_setting(HOST_BUILD_TYPE_SETTING)
+        if build_type not in self.apps_name_conf:
+            raise Exception('please config the apps name in app_name.cfg of build type:%s' % build_type)
+        self.apps_name = self.apps_name_conf[build_type]
+        # use the dut target directory instead of 'target' string in app name
+        for app in self.apps_name:
+            cur_app_path = self.apps_name[app].replace('target', self.target)
+            self.apps_name[app] = cur_app_path + ' '
+
         if not self.skip_setup:
             self.build_install_dpdk(target)
 
@@ -214,10 +224,58 @@ class DPDKdut(Dut):
                              "config/common_base", '#')
         self.send_expect("sed -i 's/CONFIG_RTE_EAL_IGB_UIO=n/CONFIG_RTE_EAL_IGB_UIO=y/g' "
                         "config/common_base", '#')
-        build_install_dpdk = getattr(self, 'build_install_dpdk_%s' % self.get_os_type())
+        build_type = load_global_setting(HOST_BUILD_TYPE_SETTING)
+        build_install_dpdk = getattr(self, 'build_install_dpdk_%s_%s' % (self.get_os_type(), build_type))
         build_install_dpdk(target, extra_options)
 
-    def build_install_dpdk_linux(self, target, extra_options):
+    def build_install_dpdk_linux_meson(self, target, extra_options):
+        """
+        Build DPDK source code on linux use meson
+        """
+        build_time = 900
+        target_info = target.split('-')
+        arch = target_info[0]
+        machine = target_info[1]
+        execenv = target_info[2]
+        toolchain = target_info[3]
+
+        default_library = 'static'
+        use_shared_lib = load_global_setting(HOST_SHARED_LIB_SETTING)
+        if use_shared_lib == 'true' and 'Virt' not in str(self):
+            default_library = 'shared'
+        if arch == 'i686':
+            # find the pkg-config path and set the PKG_CONFIG_LIBDIR environmental variable to point it
+            out = self.send_expect("find /usr -type d -name pkgconfig", "# ")
+            pkg_path = ''
+            default_cflags = self.send_expect("echo $CFLAGS", "# ")
+            default_pkg_config = self.send_expect("echo $PKG_CONFIG_LIBDIR", "# ")
+            res_path = out.split('\r\n')
+            for cur_path in res_path:
+                if 'i386' in cur_path:
+                    pkg_path = cur_path
+                    break
+            assert(pkg_path is not ''), "please make sure you env have the i386 pkg-config path"
+
+            self.send_expect("export CFLAGS=-m32", "# ")
+            self.send_expect("export PKG_CONFIG_LIBDIR=%s" % pkg_path, "# ")
+
+        self.send_expect("rm -rf " + target, "#")
+        out = self.send_expect("CC=%s meson --werror -Denable_kmods=True -Dlibdir=lib --default-library=%s %s" % (
+                        toolchain, default_library, target), "# ", build_time)
+        assert ("Error" not in out), "meson setup failed ..."
+
+        out = self.send_expect("ninja -C %s -j %d" % (target, self.number_of_cores), "# ", build_time)
+        assert ("Error" not in out), "ninja complie failed ..."
+
+        # copy kmod file to the folder same as make
+        out = self.send_expect("find ./%s/kernel/ -name *.ko" % target, "# ", verify=True)
+        self.send_expect("mkdir -p %s/kmod" % target, "# ")
+        if not isinstance(out, int) and len(out) > 0:
+            kmod = out.split('\r\n')
+            for mod in kmod:
+                self.send_expect("cp %s %s/kmod/" % (mod, target), "# ")
+
+    def build_install_dpdk_linux_makefile(self, target, extra_options):
         """
         Build DPDK source code on linux with specified target.
         """
@@ -241,7 +299,11 @@ class DPDKdut(Dut):
         assert ("Error" not in out), "Compilation error..."
         assert ("No rule to make" not in out), "No rule to make error..."
 
-    def build_install_dpdk_freebsd(self, target, extra_options):
+    def build_install_dpdk_freebsd_meson(self, target, extra_options):
+        # meson build same as linux
+        self.build_install_dpdk_linux_meson(target, extra_options)
+
+    def build_install_dpdk_freebsd_makefile(self, target, extra_options):
         """
         Build DPDK source code on Freebsd with specified target.
         """
@@ -312,7 +374,6 @@ class DPDKdut(Dut):
                     self.send_expect("mkdir -p ~/QMP", "# ")
                 self.session.copy_file_to('dep/QMP/qemu-ga-client', '~/QMP/')
                 self.session.copy_file_to('dep/QMP/qmp.py', '~/QMP/')
-
             self.kill_all()
 
             # enable core dump
@@ -395,10 +456,57 @@ class DPDKdut(Dut):
         """
         Build dpdk sample applications.
         """
-        build_dpdk_apps = getattr(self, 'build_dpdk_apps_%s' % self.get_os_type())
+        build_type = load_global_setting(HOST_BUILD_TYPE_SETTING)
+        build_dpdk_apps = getattr(self, 'build_dpdk_apps_%s_%s' % (self.get_os_type(), build_type))
         return build_dpdk_apps(folder, extra_options)
 
-    def build_dpdk_apps_linux(self, folder, extra_options):
+    def build_dpdk_apps_linux_meson(self, folder, extra_options):
+        """
+        Build dpdk sample applications on linux use meson
+        """
+        # icc compile need more time
+        if 'icc' in self.target:
+            timeout = 300
+        else:
+            timeout = 90
+
+        target_info = self.target.split('-')
+        arch = target_info[0]
+        if arch == 'i686':
+            # find the pkg-config path and set the PKG_CONFIG_LIBDIR environmental variable to point it
+            out = self.send_expect("find /usr -type d -name pkgconfig", "# ")
+            pkg_path = ''
+            default_cflags = self.send_expect("echo $CFLAGS", "# ")
+            default_pkg_config = self.send_expect("echo $PKG_CONFIG_LIBDIR", "# ")
+            res_path = out.split('\r\n')
+            for cur_path in res_path:
+                if 'i386' in cur_path:
+                    pkg_path = cur_path
+                    break
+            assert(pkg_path is not ''), "please make sure you env have the i386 pkg-config path"
+
+            self.send_expect("export CFLAGS=-m32", "# ", alt_session=True)
+            self.send_expect("export PKG_CONFIG_LIBDIR=%s" % pkg_path, "# ", alt_session=True)
+
+        folder_info = folder.split('/')
+        name = folder_info[-1]
+        if name not in self.apps_name:
+            raise Exception('Please config %s file path on conf/app_name.cfg' % name)
+
+        example = '/'.join(folder_info[folder_info.index('examples')+1:])
+        self.send_expect("cd %s/%s" % (self.base_dir, self.target), "# ", alt_session=True)
+        out = self.send_expect("meson configure -Dexamples=%s" % example, "# ", alt_session=True)
+        assert ("Error" not in out), "Compilation error..."
+        out = self.send_expect("ninja", "# ", timeout, alt_session=True)
+        assert ("Error" not in out), "Compilation error..."
+
+        # verify the app build in the config path
+        out = self.send_expect('ls %s' % self.apps_name[name], "# ", verify=True)
+        assert(isinstance(out, str)), 'please confirm %s app path and name in app_name.cfg' % name
+
+        return out
+
+    def build_dpdk_apps_linux_makefile(self, folder, extra_options):
         """
         Build dpdk sample applications on linux.
         """
@@ -414,7 +522,11 @@ class DPDKdut(Dut):
                                                          folder, extra_options),
                                 "# ", timeout)
 
-    def build_dpdk_apps_freebsd(self, folder, extra_options):
+    def build_dpdk_apps_freebsd_meson(self, folder, extra_options):
+        # meson build same as linux
+        self.build_dpdk_apps_linux_meson(folder, extra_options)
+
+    def build_dpdk_apps_freebsd_makefile(self, folder, extra_options):
         """
         Build dpdk sample applications on Freebsd.
         """
-- 
2.7.4

[dts] [PATCH V1 6/7] framework/pmd_output: replace the hard code app path with the configured app path

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 framework/pmd_output.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/framework/pmd_output.py b/framework/pmd_output.py
index 2d66743..bba7676 100644
--- a/framework/pmd_output.py
+++ b/framework/pmd_output.py
@@ -178,7 +178,8 @@ class PmdOutput():
             part_eal_param = self.dut.create_eal_parameters(fixed_prefix=fixed_prefix, socket=socket, **config)
             all_eal_param = part_eal_param + ' ' + other_eal_str
 
-        command = "./%s/app/testpmd %s -- -i %s" % (self.dut.target, all_eal_param, param)
+        app_name = self.dut.apps_name['test-pmd']
+        command = app_name + " %s -- -i %s" % (all_eal_param, param)
         out = self.session.send_expect(command, "testpmd> ", 120)
         self.command = command
         # wait 10s to ensure links getting up before test start.
-- 
2.7.4

[dts] [PATCH V1 7/7] tests: replace the hard code app path with the configured app path

[lihong]

Signed-off-by: lihong <lihongx.ma@intel.com>
---
 tests/TestSuite_multiprocess.py           | 53 +++++++++++++++++++------------
 tests/TestSuite_vhost_multi_queue_qemu.py | 14 ++++----
 2 files changed, 40 insertions(+), 27 deletions(-)

diff --git a/tests/TestSuite_multiprocess.py b/tests/TestSuite_multiprocess.py
index 0ede8bf..5a99ee3 100644
--- a/tests/TestSuite_multiprocess.py
+++ b/tests/TestSuite_multiprocess.py
@@ -61,8 +61,20 @@ class TestMultiprocess(TestCase):
         self.tester.extend_external_packet_generator(TestMultiprocess, self)
         self.dut_ports = self.dut.get_ports()
         self.socket = self.dut.get_numa_id(self.dut_ports[0])
-        out = self.dut.build_dpdk_apps("./examples/multi_process/")
-        self.verify('Error' not in out, "Compilation failed")
+
+        out = self.dut.build_dpdk_apps("./examples/multi_process/client_server_mp/mp_client")
+        self.verify('Error' not in out, "Compilation mp_client failed")
+        out = self.dut.build_dpdk_apps("./examples/multi_process/client_server_mp/mp_server")
+        self.verify('Error' not in out, "Compilation mp_server failed")
+        out = self.dut.build_dpdk_apps("./examples/multi_process/simple_mp")
+        self.verify('Error' not in out, "Compilation simple_mp failed")
+        out = self.dut.build_dpdk_apps("./examples/multi_process/symmetric_mp")
+        self.verify('Error' not in out, "Compilation symmetric_mp failed")
+
+        self.app_mp_client = self.dut.apps_name['mp_client']
+        self.app_mp_server = self.dut.apps_name['mp_server']
+        self.app_simple_mp = self.dut.apps_name['simple_mp']
+        self.app_symmetric_mp = self.dut.apps_name['symmetric_mp']
 
         executions.append({'nprocs': 1, 'cores': '1S/1C/1T', 'pps': 0})
         executions.append({'nprocs': 2, 'cores': '1S/1C/2T', 'pps': 0})
@@ -74,6 +86,7 @@ class TestMultiprocess(TestCase):
         self.eal_param = ""
         for i in self.dut_ports:
             self.eal_param += " -w %s" % self.dut.ports_info[i]['pci']
+
         # start new session to run secondary
         self.session_secondary = self.dut.new_session()
 
@@ -100,12 +113,12 @@ class TestMultiprocess(TestCase):
         # Send message from secondary to primary
         cores = self.dut.get_core_list('1S/2C/1T', socket=self.socket)
         coremask = utils.create_mask(cores)
-        self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=primary" % (self.target, coremask, self.eal_param),
+        self.dut.send_expect(self.app_simple_mp + " -n 1 -c %s --proc-type=primary" % (coremask),
                              "Finished Process Init", 100)
         time.sleep(20)
         coremask = hex(int(coremask, 16) * 0x10).rstrip("L")
         self.session_secondary.send_expect(
-            "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=secondary" % (self.target, coremask, self.eal_param), "Finished Process Init",
+            self.app_simple_mp + " -n 1 -c %s --proc-type=secondary" % (coremask), "Finished Process Init",
             100)
 
         self.session_secondary.send_expect("send hello_primary", ">")
@@ -117,10 +130,10 @@ class TestMultiprocess(TestCase):
         cores = self.dut.get_core_list('1S/2C/1T', socket=self.socket)
         coremask = utils.create_mask(cores)
         self.session_secondary.send_expect(
-            "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=primary " % (self.target, coremask, self.eal_param), "Finished Process Init", 100)
+            self.app_simple_mp + " -n 1 -c %s --proc-type=primary " % (coremask), "Finished Process Init", 100)
         time.sleep(20)
         coremask = hex(int(coremask, 16) * 0x10).rstrip("L")
-        self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=secondary" % (self.target, coremask, self.eal_param),
+        self.dut.send_expect(self.app_simple_mp + " -n 1 -c %s --proc-type=secondary" % (coremask),
                              "Finished Process Init", 100)
         self.session_secondary.send_expect("send hello_secondary", ">")
         out = self.dut.get_session_output()
@@ -137,11 +150,11 @@ class TestMultiprocess(TestCase):
 
         cores = self.dut.get_core_list('1S/2C/1T', socket=self.socket)
         coremask = utils.create_mask(cores)
-        self.session_secondary.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=primary" % (self.target, coremask, self.eal_param),
+        self.session_secondary.send_expect(self.app_simple_mp + " -n 1 -c %s --proc-type=primary" % (coremask),
                                            "Finished Process Init", 100)
         time.sleep(20)
         coremask = hex(int(coremask, 16) * 0x10).rstrip("L")
-        self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=secondary" % (self.target, coremask, self.eal_param),
+        self.dut.send_expect(self.app_simple_mp + " -n 1 -c %s --proc-type=secondary" % (coremask),
                              "Finished Process Init", 100)
         stringsSent = 0
         for line in open('/usr/share/dict/words', 'r').readlines():
@@ -163,13 +176,13 @@ class TestMultiprocess(TestCase):
         # Send message from secondary to primary (auto process type)
         cores = self.dut.get_core_list('1S/2C/1T', socket=self.socket)
         coremask = utils.create_mask(cores)
-        out = self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=auto " % (self.target, coremask, self.eal_param),
+        out = self.dut.send_expect(self.app_simple_mp + " -n 1 -c %s --proc-type=auto " % (coremask),
                                    "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: PRIMARY" in out, "The type of process (PRIMARY) was not detected properly")
         time.sleep(20)
         coremask = hex(int(coremask, 16) * 0x10).rstrip("L")
         out = self.session_secondary.send_expect(
-            "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=auto" % (self.target, coremask, self.eal_param), "Finished Process Init", 100)
+            self.app_simple_mp + " -n 1 -c %s --proc-type=auto" % (coremask), "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: SECONDARY" in out,
                     "The type of process (SECONDARY) was not detected properly")
 
@@ -183,11 +196,11 @@ class TestMultiprocess(TestCase):
         cores = self.dut.get_core_list('1S/2C/1T', socket=self.socket)
         coremask = utils.create_mask(cores)
         out = self.session_secondary.send_expect(
-            "./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=auto" % (self.target, coremask, self.eal_param), "Finished Process Init", 100)
+            self.app_simple_mp + " -n 1 -c %s --proc-type=auto" % (coremask), "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: PRIMARY" in out, "The type of process (PRIMARY) was not detected properly")
         time.sleep(20)
         coremask = hex(int(coremask, 16) * 0x10).rstrip("L")
-        out = self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s --proc-type=auto" % (self.target, coremask, self.eal_param),
+        out = self.dut.send_expect(self.app_simple_mp + " -n 1 -c %s --proc-type=auto" % (coremask),
                                    "Finished Process Init", 100)
         self.verify("EAL: Auto-detected process type: SECONDARY" in out, "The type of process (SECONDARY) was not detected properly")
         self.session_secondary.send_expect("send hello_secondary", ">", 100)
@@ -205,10 +218,10 @@ class TestMultiprocess(TestCase):
 
         cores = self.dut.get_core_list('1S/2C/1T', socket=self.socket)
         coremask = utils.create_mask(cores)
-        self.session_secondary.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s -m 64" % (self.target, coremask, self.eal_param),
+        self.session_secondary.send_expect(self.app_simple_mp + " -n 1 -c %s -m 64" % (coremask),
                                            "Finished Process Init", 100)
         coremask = hex(int(coremask, 16) * 0x10).rstrip("L")
-        out = self.dut.send_expect("./examples/multi_process/simple_mp/%s/simple_mp -n 1 -c %s %s" % (self.target, coremask, self.eal_param), "# ", 100)
+        out = self.dut.send_expect(self.app_simple_mp + " -n 1 -c %s" % (coremask), "# ", 100)
 
         self.verify("Is another primary process running" in out,
                     "No other primary process detected")
@@ -248,8 +261,8 @@ class TestMultiprocess(TestCase):
             execution = validExecutions[n]
             coreMask = utils.create_mask(self.dut.get_core_list(execution['cores'], socket=self.socket))
             self.session_secondary.send_expect(
-                "./examples/multi_process/symmetric_mp/%s/symmetric_mp -c %s %s --proc-type=auto -- -p %s --num-procs=%d --proc-id=%d" % (
-                    self.target, coreMask, self.eal_param, portMask, execution['nprocs'], n), "Finished Process Init")
+                self.app_symmetric_mp + " -c %s --proc-type=auto -- -p %s --num-procs=%d --proc-id=%d" % (
+                    coreMask, portMask, execution['nprocs'], n), "Finished Process Init")
 
             # clear streams before add new streams
             self.tester.pktgen.clear_streams()
@@ -301,16 +314,16 @@ class TestMultiprocess(TestCase):
 
             coreMask = utils.create_mask(self.dut.get_core_list('1S/1C/1T'))
             portMask = utils.create_mask(self.dut_ports)
-            self.dut.send_expect("./examples/multi_process/client_server_mp/mp_server/%s/mp_server -n %d -c %s %s -- -p %s -n %d" % (
-                self.target, self.dut.get_memory_channels(), "0xA0", self.eal_param, portMask, execution['nprocs']), "Finished Process Init", 20)
+            self.dut.send_expect(self.app_mp_server + " -n %d -c %s -- -p %s -n %d" % (
+                self.dut.get_memory_channels(), "0xA0", portMask, execution['nprocs']), "Finished Process Init", 20)
             self.dut.send_expect("^Z", "\r\n")
             self.dut.send_expect("bg", "# ")
 
             for n in range(execution['nprocs']):
                 time.sleep(5)
                 coreMask = utils.create_mask([coreList[n]])
-                self.dut.send_expect("./examples/multi_process/client_server_mp/mp_client/%s/mp_client -n %d -c %s %s --proc-type=secondary -- -n %d" % (
-                    self.target, self.dut.get_memory_channels(), coreMask, self.eal_param, n), "Finished Process Init")
+                self.dut.send_expect(self.app_mp_client + " -n %d -c %s --proc-type=secondary -- -n %d" % (
+                    self.dut.get_memory_channels(), coreMask, n), "Finished Process Init")
                 self.dut.send_expect("^Z", "\r\n")
                 self.dut.send_expect("bg", "# ")
 
diff --git a/tests/TestSuite_vhost_multi_queue_qemu.py b/tests/TestSuite_vhost_multi_queue_qemu.py
index 88524d0..01d5826 100644
--- a/tests/TestSuite_vhost_multi_queue_qemu.py
+++ b/tests/TestSuite_vhost_multi_queue_qemu.py
@@ -79,6 +79,8 @@ class TestVhostMultiQueueQemu(TestCase):
         # create an instance to set stream field setting
         self.pktgen_helper = PacketGeneratorHelper()
         self.base_dir = self.dut.base_dir.replace('~', '/root')
+        self.app_path = self.dut.apps_name['test-pmd']
+        self.app_name = self.app_path[self.app_path.rfind('/')+1:]
 
     def set_up(self):
         """
@@ -86,8 +88,8 @@ class TestVhostMultiQueueQemu(TestCase):
         """
         self.dut.send_expect("rm -rf ./vhost.out", "#")
         self.dut.send_expect("rm -rf %s/vhost-net*" % self.base_dir, "#")
-        self.dut.send_expect("killall -s INT testpmd", "#")
-        self.vm_testpmd_vector = self.target + "/app/testpmd -c %s -n 3" + \
+        self.dut.send_expect("killall -s INT %s" % self.app_name, "#")
+        self.vm_testpmd_vector = self.app_path + "-c %s -n 3" + \
                                  " -- -i --tx-offloads=0x0 " + \
                                  " --rxq=%d --txq=%d --rss-ip --nb-cores=2" % (self.queue_number, self.queue_number)
 
@@ -95,11 +97,10 @@ class TestVhostMultiQueueQemu(TestCase):
         """
         Launch the vhost sample with different parameters
         """
-        testcmd = self.target + "/app/testpmd "
         vdev = [r"'net_vhost0,iface=%s/vhost-net,queues=%d'" % (self.base_dir, self.queue_number)]
         eal_params = self.dut.create_eal_parameters(cores=self.cores, ports=[self.pci_info], vdevs=vdev)
         para = " -- -i --rxq=%d --txq=%d --nb-cores=2" % (self.queue_number, self.queue_number)
-        testcmd_start = testcmd + eal_params + para
+        testcmd_start = self.app_path + eal_params + para
         self.dut.send_expect(testcmd_start, "testpmd> ", 120)
         self.dut.send_expect("set fwd mac", "testpmd> ", 120)
         self.dut.send_expect("start", "testpmd> ", 120)
@@ -257,7 +258,7 @@ class TestVhostMultiQueueQemu(TestCase):
         """
         self.launch_testpmd()
         self.start_onevm()
-        self.vm_testpmd_queue_1 = self.target + "/app/testpmd -c %s -n 3" + \
+        self.vm_testpmd_queue_1 = self.app_path + "-c %s -n 3" + \
                                   " -- -i --tx-offloads=0x0 " + \
                                   " --rxq=1 --txq=1 --rss-ip --nb-cores=1"
         self.get_vm_coremask()
@@ -294,11 +295,10 @@ class TestVhostMultiQueueQemu(TestCase):
         Test the performance for change vhost queue size
         """
         self.queue_number = 2
-        testcmd = self.target + "/app/testpmd "
         vdev = [r"'net_vhost0,iface=%s/vhost-net,queues=2'" % self.base_dir]
         eal_params = self.dut.create_eal_parameters(cores=self.cores, ports=[self.pci_info], vdevs=vdev)
         para = " -- -i --rxq=1 --txq=1 --nb-cores=1"
-        testcmd_start = testcmd + eal_params + para
+        testcmd_start = self.app_path + eal_params + para
         self.dut.send_expect(testcmd_start, "testpmd> ", 120)
         self.dut.send_expect("set fwd mac", "testpmd> ", 120)
         self.dut.send_expect("start", "testpmd> ", 120)
-- 
2.7.4

Re: [dts] [PATCH V1 0/7] meson build reference

[Xiao, QimaiX]

Tested-by: Xiao Qimai <qimaix.xiao@intel.com>

Regards,
Xiao Qimai

> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of lihong
> Sent: Tuesday, July 7, 2020 10:45 AM
> To: dts@dpdk.org
> Cc: Chen, Zhaoyan <zhaoyan.chen@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Subject: [dts] [PATCH V1 0/7] meson build reference
> 
> 
> lihong (7):
>   executions: add build_type field in all executions cfg
>   conf: add configuration file of app_name.cfg
>   framework/config: get configuration information from app_name.cfg
>   framework: read and save the build_type from execution.cfg
>   framework: add meson build method and get apps name of current build
>     type
>   framework/pmd_output: replace the hard code app path with the
>     configured app path
>   tests: replace the hard code app path with the configured app path
> 
>  conf/app_name.cfg                         | 119 +++++++++++++++++++++++++++
>  executions/execution.cfg                  |   2 +
>  executions/execution_FVL.cfg              |   2 +
>  executions/execution_fm10k.cfg            |   2 +
>  executions/execution_rxmode.cfg           |   3 +
>  executions/execution_smoke.cfg            |   1 +
>  framework/config.py                       |  29 +++++++
>  framework/dts.py                          |   7 ++
>  framework/dut.py                          |   8 +-
>  framework/pmd_output.py                   |   3 +-
>  framework/project_dpdk.py                 | 128
> ++++++++++++++++++++++++++++--
>  framework/settings.py                     |   1 +
>  tests/TestSuite_multiprocess.py           |  53 ++++++++-----
>  tests/TestSuite_vhost_multi_queue_qemu.py |  14 ++--
>  14 files changed, 335 insertions(+), 37 deletions(-)  create mode 100644
> conf/app_name.cfg
> 
> --
> 2.7.4

Re: [dts] [PATCH V1 0/7] meson build reference

[Tu, Lijuan]

Applied the series, thanks

> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of lihong
> Sent: 2020年7月7日 10:45
> To: dts@dpdk.org
> Cc: Chen, Zhaoyan <zhaoyan.chen@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Subject: [dts] [PATCH V1 0/7] meson build reference
> 
> 
> lihong (7):
>   executions: add build_type field in all executions cfg
>   conf: add configuration file of app_name.cfg
>   framework/config: get configuration information from app_name.cfg
>   framework: read and save the build_type from execution.cfg
>   framework: add meson build method and get apps name of current build
>     type
>   framework/pmd_output: replace the hard code app path with the
>     configured app path
>   tests: replace the hard code app path with the configured app path
> 
>  conf/app_name.cfg                         | 119 +++++++++++++++++++++++++++
>  executions/execution.cfg                  |   2 +
>  executions/execution_FVL.cfg              |   2 +
>  executions/execution_fm10k.cfg            |   2 +
>  executions/execution_rxmode.cfg           |   3 +
>  executions/execution_smoke.cfg            |   1 +
>  framework/config.py                       |  29 +++++++
>  framework/dts.py                          |   7 ++
>  framework/dut.py                          |   8 +-
>  framework/pmd_output.py                   |   3 +-
>  framework/project_dpdk.py                 | 128 ++++++++++++++++++++++++++++--
>  framework/settings.py                     |   1 +
>  tests/TestSuite_multiprocess.py           |  53 ++++++++-----
>  tests/TestSuite_vhost_multi_queue_qemu.py |  14 ++--
>  14 files changed, 335 insertions(+), 37 deletions(-)  create mode 100644
> conf/app_name.cfg
> 
> --
> 2.7.4