* [dts][PATCH V3 1/2] tests/meson_tests: add new testsuite for meson tests
@ 2022-05-26 9:19 Li, WeiyuanX
2022-05-26 9:19 ` [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan " Li, WeiyuanX
0 siblings, 1 reply; 5+ messages in thread
From: Li, WeiyuanX @ 2022-05-26 9:19 UTC (permalink / raw)
To: dts, lijuan.tu; +Cc: Weiyuan Li
From: Weiyuan Li <weiyuanx.li@intel.com>
Add new testsuite for meson tests.
Signed-off-by: Weiyuan Li <weiyuanx.li@intel.com>
---
v2:
-fix format issue.
v3:
-add verification that the test result is empty
tests/TestSuite_meson_tests.py | 233 +++++++++++++++++++++++++++++++++
1 file changed, 233 insertions(+)
create mode 100644 tests/TestSuite_meson_tests.py
diff --git a/tests/TestSuite_meson_tests.py b/tests/TestSuite_meson_tests.py
new file mode 100644
index 00000000..4b1f7df2
--- /dev/null
+++ b/tests/TestSuite_meson_tests.py
@@ -0,0 +1,233 @@
+# BSD LICENSE
+#
+# Copyright(c) 2022 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test Suite.
+
+meson autotest
+"""
+import os
+import re
+from mimetypes import init
+from sre_constants import SUCCESS
+
+import framework.utils as utils
+from framework.test_case import TestCase
+
+
+class TestMesonTests(TestCase):
+ def set_up_all(self):
+ """
+ Run at the start of each test suite.
+ """
+ # icc compilation cost long long time.
+ self.cores = self.dut.get_core_list("all")
+ self.dut_ip = self.dut.get_ip_address()
+ self.timeout = 600
+ # -t 2
+ self.ratio = 6
+ self.dut_pathlog = "fast-test.log"
+ self.execute_wait_time = self.ratio * self.timeout * 10
+ # skip scope
+ self.SKIP_SCOPE = ""
+ # Test log storage directory
+ self.base_output = os.path.join(
+ os.path.dirname(os.path.dirname(os.path.realpath(__file__))), "output"
+ )
+
+ def set_up(self):
+ """
+ Run before each test case.
+ """
+ self.meson_auto_test = {}
+
+ def delete_exists_files(self):
+ out = self.tester.send_command(
+ f"ls -l {self.base_output}/{self.dut_pathlog}", "# "
+ )
+ if self.dut_pathlog in out:
+ self.tester.send_command(f"rm -rf {self.base_output}/{self.dut_pathlog}")
+
+ def find_text(self, res):
+ pattern = r"(DPDK:fast-tests|DPDK:perf-tests|DPDK:debug-tests|DPDK:extra-tests|DPDK:driver-tests) / (\w+)\s+(\w+)\s+"
+ regex = re.compile(pattern)
+ mo = regex.search(res)
+
+ if mo is None:
+ return 0
+ case_name = mo.group(2)
+ test_result = mo.group(3)
+
+ self.meson_auto_test[case_name] = test_result
+
+ def check_sub_case(self):
+ success = []
+ skip = []
+ timeout = []
+ failed = []
+
+ self.verify(
+ bool(self.meson_auto_test),
+ f"Test failed the meson no test results were obtained.",
+ )
+
+ for k, v in self.meson_auto_test.items():
+ if v == "FAIL" and k not in self.SKIP_SCOPE:
+ failed.append(k)
+ elif v == "OK":
+ success.append(k)
+ elif v == "SKIP":
+ skip.append(k)
+ elif v == "TIMEOUT":
+ timeout.append(k)
+ else:
+ failed.append(k)
+
+ for item in skip:
+ self.logger.debug(utils.RED(f"Skip sub case:{item}"))
+
+ for item in timeout:
+ self.logger.debug(utils.RED(f"TimeOut sub case:{item}"))
+
+ if (len(failed) + len(timeout)) > 0:
+ sub_fail = ""
+ sub_timeout = ""
+ if len(failed) > 0:
+ sub_fail = "[" + ",".join(failed) + "] sub case failed. "
+ if len(timeout) > 0:
+ sub_timeout = "[" + ",".join(timeout) + "] sub case time out. "
+ self.verify(False, f"Test failed. {sub_fail}{sub_timeout}")
+
+ def check_scp_file_valid_between_dut(self):
+ out = self.tester.send_command(
+ f"scp root@{self.dut_ip}:/root/{self.dut_pathlog} {self.base_output}",
+ timeout=15,
+ )
+ if "Are you sure you want to continue connecting" in out:
+ out = self.tester.send_command("yes", timeout=20)
+ for item in range(30):
+ if "password" in out:
+ self.tester.send_command(self.dut.get_password(), timeout=20)
+ break
+
+ out = self.tester.send_command(
+ f"ls -l {self.base_output}/{self.dut_pathlog}", "# "
+ )
+ self.verify(
+ "No such file or directory" not in out, "No test result log was obtained!"
+ )
+
+ def insmod_kni(self):
+ out = self.dut.send_expect("lsmod | grep rte_kni", "# ")
+
+ if "rte_kni" in out:
+ self.dut.send_expect("rmmod rte_kni.ko", "# ")
+
+ out = self.dut.send_expect(
+ "insmod ./%s/kmod/rte_kni.ko lo_mode=lo_mode_fifo" % (self.target), "# "
+ )
+ self.verify("Error" not in out, "Error loading KNI module: " + out)
+
+ def check_meson_test_result(self, res=None):
+ if not os.path.exists(f"{self.base_output}/{self.dut_pathlog}"):
+ self.verify(False, "meson tests log file is not exists!!")
+ if res is None:
+ with open(f"{self.base_output}/{self.dut_pathlog}", "r") as file_obj:
+ for files in file_obj:
+ self.find_text(files)
+ else:
+ self.find_text(res)
+ self.check_sub_case()
+
+ def test_fasts(self):
+ # init file name
+ self.dut_pathlog = "fast-test.log"
+ self.delete_exists_files()
+ self.insmod_kni()
+ cmds = f'meson test -C x86_64-native-linuxapp-gcc/ --suite DPDK:fast-tests -t {self.ratio} --test-args="-c 0xff" |tee /root/{self.dut_pathlog}'
+ out = self.dut.send_expect(cmds, "# ", self.execute_wait_time)
+ self.logger.info(out)
+ self.check_scp_file_valid_between_dut()
+ self.check_meson_test_result()
+
+ def test_driver(self):
+ # init file name
+ self.dut_pathlog = "driver-test.log"
+ self.delete_exists_files()
+ cmds = f'meson test -C x86_64-native-linuxapp-gcc/ --suite DPDK:driver-tests -t {self.ratio} --test-args="-c 0xff" |tee /root/{self.dut_pathlog}'
+ out = self.dut.send_expect(cmds, "# ", self.execute_wait_time)
+ self.logger.info(out)
+ self.check_scp_file_valid_between_dut()
+ self.check_meson_test_result()
+
+ def test_debug(self):
+ self.logger.warning(
+ "Debug-tests all case failed commands are not registered, there is a bug https://bugs.dpdk.org/show_bug.cgi?id=1002"
+ )
+
+ def test_extra(self):
+ self.dut_pathlog = "extra-test.log"
+ # delete exists files
+ self.delete_exists_files()
+ cmds = f'meson test -C x86_64-native-linuxapp-gcc/ --suite DPDK:extra-tests -t {self.ratio} --test-args="-c 0xff" |tee /root/{self.dut_pathlog}'
+ out = self.dut.send_expect(cmds, "# ", self.execute_wait_time)
+ self.logger.info(out)
+ self.check_scp_file_valid_between_dut()
+ self.check_meson_test_result()
+ self.logger.warning(
+ "Extra-tests are know issues which are recorded in DPDK commit and meson.build (detail see test plan)"
+ )
+
+ def test_perf(self):
+ # init file name
+ self.dut_pathlog = "perf-test.log"
+ # delete exists files
+ self.delete_exists_files()
+ cmds = f'meson test -C x86_64-native-linuxapp-gcc/ --suite DPDK:perf-tests -t {self.ratio} --test-args="-c 0xff" |tee /root/{self.dut_pathlog}'
+ out = self.dut.send_expect(cmds, "# ", self.execute_wait_time)
+ self.logger.info(out)
+ self.check_scp_file_valid_between_dut()
+ self.check_meson_test_result()
+
+ def tear_down(self):
+ """
+ Run after each test case.
+ """
+ out = self.dut.send_expect("lsmod | grep rte_kni", "# ")
+
+ if "rte_kni" in out:
+ self.dut.send_expect("rmmod rte_kni.ko", "# ")
+
+ def tear_down_all(self):
+ """
+ Run after each test suite.
+ """
+ pass
--
2.17.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan for meson tests
2022-05-26 9:19 [dts][PATCH V3 1/2] tests/meson_tests: add new testsuite for meson tests Li, WeiyuanX
@ 2022-05-26 9:19 ` Li, WeiyuanX
2022-05-26 10:53 ` Chen, LingliX
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Li, WeiyuanX @ 2022-05-26 9:19 UTC (permalink / raw)
To: dts, lijuan.tu; +Cc: Weiyuan Li
From: Weiyuan Li <weiyuanx.li@intel.com>
Add new test plan for meson tests.
Signed-off-by: Weiyuan Li <weiyuanx.li@intel.com>
---
v2:
-fix format issue.
v3:
-add verification that the test result is empty
test_plans/index.rst | 1 +
test_plans/meson_tests_test_plan.rst | 185 +++++++++++++++++++++++++++
2 files changed, 186 insertions(+)
create mode 100644 test_plans/meson_tests_test_plan.rst
diff --git a/test_plans/index.rst b/test_plans/index.rst
index 6978a2ef..45a23e7e 100644
--- a/test_plans/index.rst
+++ b/test_plans/index.rst
@@ -281,6 +281,7 @@ The following are the test plans for the DPDK DTS automated test system.
unit_tests_ringpmd_test_plan
unit_tests_ring_test_plan
unit_tests_timer_test_plan
+ meson_tests_test_plan
cmdline_test_plan
hello_world_test_plan
diff --git a/test_plans/meson_tests_test_plan.rst b/test_plans/meson_tests_test_plan.rst
new file mode 100644
index 00000000..e199ec81
--- /dev/null
+++ b/test_plans/meson_tests_test_plan.rst
@@ -0,0 +1,185 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright (c) <2022>, Intel Corporation
+
+===========
+Meson tests
+===========
+
+Description
+===========
+This test plan describes how to run unit tests with meson.
+Test tests have been classified into five different groups.
+
+- Fast tests.
+- Performance tests.
+- Driver tests.
+- Extra tests.
+- Debug tests.
+
+For more details, please refer to `Running DPDK Unit Tests with Meson <http://doc.dpdk.org/guides/prog_guide/meson_ut.html>`_.
+
+Prerequisites
+=============
+
+Required Libraries
+------------------
+* crypto relative cases require dependencies `libIPSec_MB` and `libcrypto`::
+
+ cryptodev_aesni_mb_autotest
+ cryptodev_aesni_gcm_autotest
+ cryptodev_null_autotest
+ cryptodev_openssl_autotest
+ cryptodev_openssl_asym_autotest
+ cryptodev_qat_autotest
+ cryptodev_sw_kasumi_autotest
+ cryptodev_sw_snow3g_autotest
+ cryptodev_sw_zuc_autotest
+ cryptodev_scheduler_autotest
+
+ .. note::
+
+ For more details, please refer to `Crypto Device Drivers <http://doc.dpdk.org/guides/cryptodevs/index.html>`_
+
+* The following cases require dependencies `libpcap`::
+
+ bpf_convert_autotest
+ eal_flags_mem_autotest
+ flow_classify_autotest
+ efd_autotest
+ efd_perf_autotest
+
+General set up
+--------------
+
+* Compile DPDK::
+
+ # CC=gcc meson --werror -Denable_kmods=True -Dlibdir=lib --default-library=static <dpdk build dir>
+ # ninja -C <dpdk build dir> -j 110
+
+* Load rte_kni driver::
+
+ # lsmod | grep rte_kni
+ # rmmod rte_kni.ko
+ # insmod ./<dpdk build dir>/kmod/rte_kni.ko lo_mode=lo_mode_fifo
+
+ .. note::
+
+ Test case fast-tests/kni_autotest requires to load kni.ko.
+
+* Get the pci device id and interface of DUT and tester.
+ For example, 0000:18:00.0 and 0000:18:00.1 is pci device id,
+ ens785f0 and ens785f1 is interface::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -s
+ ...
+ 0000:18:00.0 'Device 159b' if=ens785f0 drv=ice unused=vfio-pci
+ 0000:18:00.1 'Device 159b' if=ens785f1 drv=ice unused=vfio-pci
+
+* Bind the DUT port to dpdk::
+
+ <dpdk dir># ./usertools/dpdk-devbind.py -b vfio-pci 0000:18:00.0 0000:18:00.1
+
+.. note::
+
+ kni_autotest requires at least one ethernet port, but other can be none.
+
+Test case
+=========
+
+Test case 1: test_fasts
+-------------------------------
+This section explains how to run the meson tests for fast-tests.
+
+test steps
+~~~~~~~~~~
+1. Run test on DUT::
+
+ meson test -C x86_64-native-linuxapp-gcc --suite DPDK:fast-tests -t 1 --test-args="-c 0xff"
+
+ * `-t` is used to specify the timeout of test case, unit is second.
+ * `--test-args option` is used to specify test specific parameters,
+ distributor_autotest and distributor_perf_autotest require core number less than 64.
+
+2. Open and view the results on DUT::
+
+ <dpdk dir># cat ./x86_64-native-linuxapp-gcc/meson-logs/testlog.txt
+
+3. Check the result output log. All case results are "OK", for example::
+
+ 1/99 DPDK:fast-tests / acl_autotest OK 3.43s
+ 2/99 DPDK:fast-tests / atomic_autotest OK 6.21s
+ 3/99 DPDK:fast-tests / bitmap_autotest OK 1.18s
+ 4/99 DPDK:fast-tests / bpf_autotest OK 1.17s
+ 5/99 DPDK:fast-tests / bpf_convert_autotest OK 1.19s
+
+Test case 2: test_driver
+-------------------------------
+This section explains how to run the meson tests for driver-tests.
+
+test steps
+~~~~~~~~~~
+1. Run test on DUT::
+
+ meson test -C x86_64-native-linuxapp-gcc --suite DPDK:driver-tests -t 1 --test-args="-c 0xff"
+
+2. Following test case 1 step 2 and step 3.
+
+Test case 3: test_debug
+-------------------------------
+This section explains how to run the meson tests for debug-tests.
+
+test steps
+~~~~~~~~~~
+1. Run test on DUT::
+
+ meson test -C x86_64-native-linuxapp-gcc --suite DPDK:debug-tests -t 1 --test-args="-c 0xff"
+
+2.Following test case 1 step 2 and step 3.
+
+.. warning::
+
+ `The bug <https://bugs.dpdk.org/show_bug.cgi?id=1002>_` impacts the following cases::
+
+ DPDK:debug-tests/dump_struct_sizes
+ DPDK:debug-tests/dump_mempool
+ DPDK:debug-tests/dump_malloc_stats
+ DPDK:debug-tests/dump_devargs
+ DPDK:debug-tests/dump_log_types
+ DPDK:debug-tests/dump_ring
+ DPDK:debug-tests/dump_physmem
+ DPDK:debug-tests/dump_memzone
+
+Test case 4: test_extra
+-------------------------------
+This section explains how to run the meson tests for extra-tests.
+
+test steps
+~~~~~~~~~~
+1. Run test on DUT::
+
+ meson test -C x86_64-native-linuxapp-gcc --suite DPDK:extra-tests -t 1 --test-args="-c 0xff"
+
+2. Following test case 1 step 2 and step 3.
+
+.. warning::
+
+ Extra-tests are know issues which are recorded in app/test/meson.build::
+
+ #Tests known to have issues or which don't belong in other tests lists.
+ extra_test_names = [
+ 'alarm_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
+ 'cycles_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
+ 'delay_us_sleep_autotest', # ee00af60170b ("test: remove strict timing requirements some tests")
+ 'red_autotest', # https://bugs.dpdk.org/show_bug.cgi?id=826]
+
+Test case 5: test_perf
+-------------------------------
+This section explains how to run the meson tests for perf-tests.
+
+test steps
+~~~~~~~~~~
+1. Run test on DUT::
+
+ meson test -C x86_64-native-linuxapp-gcc --suite DPDK:perf-tests -t 1 --test-args="-c 0xff"
+
+2. Following test case 1 step 2 and step 3.
\ No newline at end of file
--
2.17.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan for meson tests
2022-05-26 9:19 ` [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan " Li, WeiyuanX
@ 2022-05-26 10:53 ` Chen, LingliX
2022-05-28 8:57 ` lijuan.tu
2022-05-28 8:59 ` lijuan.tu
2 siblings, 0 replies; 5+ messages in thread
From: Chen, LingliX @ 2022-05-26 10:53 UTC (permalink / raw)
To: dts; +Cc: Li, WeiyuanX
> -----Original Message-----
> From: Li, WeiyuanX <weiyuanx.li@intel.com>
> Sent: Thursday, May 26, 2022 5:20 PM
> To: dts@dpdk.org; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: Li, WeiyuanX <weiyuanx.li@intel.com>
> Subject: [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan for meson
> tests
>
> From: Weiyuan Li <weiyuanx.li@intel.com>
>
> Add new test plan for meson tests.
>
> Signed-off-by: Weiyuan Li <weiyuanx.li@intel.com>
> ---
>
> v2:
> -fix format issue.
> v3:
> -add verification that the test result is empty
>
Tested-by: Lingli Chen <linglix.chen@intel.com>
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan for meson tests
2022-05-26 9:19 ` [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan " Li, WeiyuanX
2022-05-26 10:53 ` Chen, LingliX
@ 2022-05-28 8:57 ` lijuan.tu
2022-05-28 8:59 ` lijuan.tu
2 siblings, 0 replies; 5+ messages in thread
From: lijuan.tu @ 2022-05-28 8:57 UTC (permalink / raw)
To: dts, lijuan.tu, Li, WeiyuanX; +Cc: Weiyuan Li
On Thu, 26 May 2022 17:19:36 +0800, "Li, WeiyuanX" <weiyuanx.li@intel.com> wrote:
> From: Weiyuan Li <weiyuanx.li@intel.com>
>
> Add new test plan for meson tests.
>
> Signed-off-by: Weiyuan Li <weiyuanx.li@intel.com>
Series applied, thanks
^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan for meson tests
2022-05-26 9:19 ` [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan " Li, WeiyuanX
2022-05-26 10:53 ` Chen, LingliX
2022-05-28 8:57 ` lijuan.tu
@ 2022-05-28 8:59 ` lijuan.tu
2 siblings, 0 replies; 5+ messages in thread
From: lijuan.tu @ 2022-05-28 8:59 UTC (permalink / raw)
To: dts, lijuan.tu, Li, WeiyuanX; +Cc: Weiyuan Li
On Thu, 26 May 2022 17:19:36 +0800, "Li, WeiyuanX" <weiyuanx.li@intel.com> wrote:
> From: Weiyuan Li <weiyuanx.li@intel.com>
>
> Add new test plan for meson tests.
>
> Signed-off-by: Weiyuan Li <weiyuanx.li@intel.com>
Reviewed-by: Lijuan Tu <lijuan.tu@intel.com>
Series applied, thanks
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-05-28 8:59 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-26 9:19 [dts][PATCH V3 1/2] tests/meson_tests: add new testsuite for meson tests Li, WeiyuanX
2022-05-26 9:19 ` [dts][PATCH V3 2/2] test_plans/meson_tests: add test plan " Li, WeiyuanX
2022-05-26 10:53 ` Chen, LingliX
2022-05-28 8:57 ` lijuan.tu
2022-05-28 8:59 ` lijuan.tu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).