DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] app/test: enhance sanity script
@ 2018-04-27 13:20 Reshma Pattan
  2018-04-27 13:41 ` Burakov, Anatoly
                   ` (2 more replies)
  0 siblings, 3 replies; 25+ messages in thread
From: Reshma Pattan @ 2018-04-27 13:20 UTC (permalink / raw)
  To: john.mcnamara, dev; +Cc: jananeex.m.parthasarathy

1. Sanity Script is enhanced with additional test cases
being added to autotest_data.py
2. Fixed in autotest_test_funcs.py to handle test cases
which returns "Skipped" as result.
  The issue was skipped test cases got timed out,
causing delay in sanity script execution.
3. Enhanced support for FreeBSD to add the respective memory limit,
as FreeBSD doesn't support socket-mem, file-prefix options.

Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
---
 test/test/autotest_data.py       | 471 ++++++++++++++++++++++++++++++++++++++-
 test/test/autotest_runner.py     |   9 +-
 test/test/autotest_test_funcs.py |   6 +-
 3 files changed, 474 insertions(+), 12 deletions(-)
 mode change 100644 => 100755 test/test/autotest_data.py
 mode change 100644 => 100755 test/test/autotest_test_funcs.py

diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py
old mode 100644
new mode 100755
index aacfe0a..2c6828d
--- a/test/test/autotest_data.py
+++ b/test/test/autotest_data.py
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # Test data for autotests
 
@@ -70,6 +70,31 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Eventdev selftest octeontx",
+                "Command": "eventdev_selftest_octeontx",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Event ring autotest",
+                "Command": "event_ring_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Table autotest",
+                "Command": "table_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Flow classify autotest",
+                "Command": "flow_classify_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -125,6 +150,50 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Dump struct sizes",
+                "Command": "dump_struct_sizes",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump mempool",
+                "Command": "dump_mempool",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump malloc stats",
+                "Command": "dump_malloc_stats",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump devargs",
+                "Command": "dump_devargs",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump log types",
+                "Command": "dump_log_types",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump_ring",
+                "Command": "dump_ring",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Quit",
+                "Command": "quit",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
         ],
     },
     {
@@ -168,6 +237,26 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Set rxtx mode",
+                "Command": "set_rxtx_mode",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Set rxtx anchor",
+                "Command": "set_rxtx_anchor",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Set rxtx sc",
+                "Command": "set_rxtx_sc",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
         ]
     },
     {
@@ -176,12 +265,6 @@ def per_sockets(num):
         "Tests":
         [
             {
-                "Name":    "PCI autotest",
-                "Command": "pci_autotest",
-                "Func":    default_autotest,
-                "Report":  None,
-            },
-            {
                 "Name":    "Malloc autotest",
                 "Command": "malloc_autotest",
                 "Func":    default_autotest,
@@ -211,6 +294,37 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Event eth rx adapter autotest",
+                "Command": "event_eth_rx_adapter_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Rawdev autotest",
+                "Command": "rawdev_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Kvargs autotest",
+                "Command": "kvargs_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump physmem",
+                "Command": "dump_physmem",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Dump memzone",
+                "Command": "dump_memzone",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -248,6 +362,31 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Devargs autotest",
+                "Command": "devargs_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Link bonding autotest",
+                "Command": "link_bonding_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Link bonding mode4 autotest",
+                "Command": "link_bonding_mode4_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Link bonding rssconf autotest",
+                "Command": "link_bonding_rssconf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -285,6 +424,25 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Crc autotest",
+                "Command": "crc_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Distributor autotest",
+                "Command": "distributor_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Reorder autotest",
+                "Command": "reorder_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
         ]
     },
     {
@@ -310,8 +468,184 @@ def per_sockets(num):
                 "Func":    default_autotest,
                 "Report":  None,
             },
+            {
+                "Name":    "Barrier autotest",
+                "Command": "barrier_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "User delay",
+                "Command": "user_delay_us",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Bitmap test",
+                "Command": "bitmap_test",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Hash scaling autotest",
+                "Command": "hash_scaling_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Hash multiwriter autotest",
+                "Command": "hash_multiwriter_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Service autotest",
+                "Command": "service_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
+        ]
+    },
+    {
+        "Prefix":   "group_8",
+        "Memory":   "128",
+        "Tests":
+        [
+            {
+                "Name":    "Timer racecond autotest",
+                "Command": "timer_racecond_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Member autotest",
+                "Command": "member_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":   "Efd_autotest",
+                "Command": "efd_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Thash autotest",
+                "Command": "thash_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Hash function autotest",
+                "Command": "hash_functions_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":   "group_9",
+        "Memory":   "64",
+        "Tests":
+        [
+            {
+                "Name":    "Cryptodev sw mrvl autotest",
+                "Command": "cryptodev_sw_mrvl_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev dpaa2 sec autotest",
+                "Command": "cryptodev_dpaa2_sec_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev dpaa sec autotest",
+                "Command": "cryptodev_dpaa_sec_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev qat autotest",
+                "Command": "cryptodev_qat_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev aesni mb autotest",
+                "Command": "cryptodev_aesni_mb_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev openssl autotest",
+                "Command": "cryptodev_openssl_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev scheduler autotest",
+                "Command": "cryptodev_scheduler_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev aesni gcm autotest",
+                "Command": "cryptodev_aesni_gcm_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev null autotest",
+                "Command": "cryptodev_null_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev sw snow3g autotest",
+                "Command": "cryptodev_sw_snow3g_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev sw kasumi autotest",
+                "Command": "cryptodev_sw_kasumi_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Cryptodev_sw_zuc_autotest",
+                "Command": "cryptodev_sw_zuc_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+
+
         ]
     },
+    {
+        "Prefix":   "group_10",
+        "Memory":   "64",
+        "Tests":
+        [
+            {
+                "Name":    "Reciprocal division",
+                "Command": "reciprocal_division",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+            {
+                "Name":    "Red all",
+                "Command": "red_all",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+
 ]
 
 # tests that should not be run when any other tests are running
@@ -336,8 +670,8 @@ def per_sockets(num):
         "Tests":
         [
             {
-                "Name":    "Eventdev sw autotest",
-                "Command": "eventdev_sw_autotest",
+                "Name":    "Eventdev selftest sw",
+                "Command": "eventdev_selftest_sw",
                 "Func":    default_autotest,
                 "Report":  None,
             },
@@ -447,6 +781,125 @@ def per_sockets(num):
             },
         ]
     },
+    {
+        "Prefix":    "reciprocal_division_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Reciprocal division perf",
+                "Command": "reciprocal_division_perf",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "member_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Member perf autotest",
+                "Command": "member_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "efd_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Efd perf autotest",
+                "Command": "efd_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "lpm_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Lpm perf autotest",
+                "Command": "lpm_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "lpm6_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Lpm6 perf autotest",
+                "Command": "lpm6_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "red_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Red_perf",
+                "Command": "red_perf",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "distributor_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Distributor perf autotest",
+                "Command": "distributor_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "ring_pmd_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Ring pmd perf autotest",
+                "Command": "ring_pmd_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+    {
+        "Prefix":    "pmd_perf",
+        "Memory":    per_sockets(256),
+        "Tests":
+        [
+            {
+                "Name":    "Pmd perf autotest",
+                "Command": "pmd_perf_autotest",
+                "Func":    default_autotest,
+                "Report":  None,
+            },
+        ]
+    },
+
+
 
     #
     # Please always make sure that ring_perf is the last test!
diff --git a/test/test/autotest_runner.py b/test/test/autotest_runner.py
index a692f06..77f0ac7 100644
--- a/test/test/autotest_runner.py
+++ b/test/test/autotest_runner.py
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
@@ -201,6 +201,13 @@ def __init__(self, cmdline, target, blacklist, whitelist):
     # set up cmdline string
     def __get_cmdline(self, test):
         cmdline = self.cmdline
+        os_name = platform.system()
+
+        # append the respective memory limitation on FREEBSD
+        # as socket-mem, file-prefix is not supported
+        if os_name == "FreeBSD" :
+            cmdline += " -m " + str(sum(map(int, test["Memory"].split(","))))
+            return cmdline
 
         # append memory limitations for each test
         # otherwise tests won't run in parallel
diff --git a/test/test/autotest_test_funcs.py b/test/test/autotest_test_funcs.py
old mode 100644
new mode 100755
index 65fe335..219c208
--- a/test/test/autotest_test_funcs.py
+++ b/test/test/autotest_test_funcs.py
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # Test functions
 
@@ -12,12 +12,14 @@
 def default_autotest(child, test_name):
     child.sendline(test_name)
     result = child.expect(["Test OK", "Test Failed",
-                           "Command not found", pexpect.TIMEOUT], timeout=900)
+                           "Command not found", "Skipped", pexpect.TIMEOUT], timeout=900)
     if result == 1:
         return -1, "Fail"
     elif result == 2:
         return -1, "Fail [Not found]"
     elif result == 3:
+        return -1, "Fail [Test returns Skipped]"
+    elif result == 4:
         return -1, "Fail [Timeout]"
     return 0, "Success"
 
-- 
1.7.12.2

^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2018-05-23 11:41 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-27 13:20 [dpdk-dev] [PATCH] app/test: enhance sanity script Reshma Pattan
2018-04-27 13:41 ` Burakov, Anatoly
2018-04-27 13:56 ` Burakov, Anatoly
2018-05-03 10:03   ` Parthasarathy, JananeeX M
2018-05-03 10:54     ` Burakov, Anatoly
2018-05-10 10:36 ` [dpdk-dev] [PATCH v2] " Jananee Parthasarathy
2018-05-13 22:12   ` Thomas Monjalon
2018-05-14 15:26     ` Pattan, Reshma
2018-05-14 15:29       ` Thomas Monjalon
2018-05-15 16:14     ` Parthasarathy, JananeeX M
2018-05-15 16:17       ` Thomas Monjalon
2018-05-18 13:32         ` Parthasarathy, JananeeX M
2018-05-18 13:48           ` Thomas Monjalon
2018-05-20 17:41             ` Parthasarathy, JananeeX M
2018-05-20 19:21               ` Thomas Monjalon
2018-05-15 15:58   ` [dpdk-dev] [PATCH v3 0/3] enhance sanity scripts Jananee Parthasarathy
2018-05-15 15:58     ` [dpdk-dev] [PATCH v3 1/3] app/test: add new test cases to sanity script Jananee Parthasarathy
2018-05-15 16:12       ` Burakov, Anatoly
2018-05-15 15:58     ` [dpdk-dev] [PATCH v3 2/3] app/test: update result for skipped test cases Jananee Parthasarathy
2018-05-15 15:58     ` [dpdk-dev] [PATCH v3 3/3] app/test: enhance freebsd support in sanity script Jananee Parthasarathy
2018-05-23 11:41     ` [dpdk-dev] [PATCH v4 0/4] enhance autotest config and test targets Jananee Parthasarathy
2018-05-23 11:41       ` [dpdk-dev] [PATCH v4 1/4] app/test: enhance autotest config Jananee Parthasarathy
2018-05-23 11:41       ` [dpdk-dev] [PATCH v4 2/4] app/test: update result for skipped test cases Jananee Parthasarathy
2018-05-23 11:41       ` [dpdk-dev] [PATCH v4 3/4] app/test: enhance freebsd support in autotest config Jananee Parthasarathy
2018-05-23 11:41       ` [dpdk-dev] [PATCH v4 4/4] mk: update make targets for classified testcases Jananee Parthasarathy

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).