test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [PATCH V3 0/13] l3fwd: refactor script
@ 2020-01-16  7:27 yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 1/13] conf/l3fwd: l3fwd suite testing configuration yufengmx
                   ` (13 more replies)
  0 siblings, 14 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx

refactor script. 
 

v4: 
 - rebase for merge conflict. 
 -  

v3: 
 - support one port testing topo for ixia. 
 - add test case name in cfg file. 
 - rename module l3fwd.py to l3fwd_base.py. 
 - rename common_l3fwd.cfg to l3fwd_base.cfg. 
 - rename verify method names. 
 - rename test scripts suite name. 
 -  

v2: 
 - update script based on version 2 l3fwd test plan. 
 - separate script into five suites for different testing scenario. 
 - separate flows configuration in common_l3fwd.cfg. 
 - add test result data check process as new test plan requirement. 
 - save test results in json format file. 
 -  

v1: 
 - optimize packet creation time consumption. 
 - separate LPM/EM mode packet creation process. 
 - use new pktgen coding style to take the place of migration coding style. 
 - unify rfc2544/throughput result data display table methods. 
 - move test content configuration into cfg file, including flows/EM table/LPM table/framesizes/port configs. 

yufengmx (13):
  conf/l3fwd: l3fwd suite testing configuration
  conf/l3fwd_base: l3fwd base class flows configuration
  conf/l3fwd_em: suite testing configuration
  conf/l3fwd_lpm_ipv4_rfc2544: suite testing configuration
  conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 testing configuration
  conf/l3fwd_lpm_ipv6: suite testing configuration
  tests/l3fwd: upload automation script
  tests/l3fwd_em: upload automation script
  tests/l3fwd_lpm_ipv4_rfc2544: upload automation script
  tests/l3fwd_lpm_ipv4: upload automation script
  tests/l3fwd_lpm_ipv6: upload automation script
  tests/l3fwd_base: upload automation script
  framework/test_case: add wirespeed columbiaville nic

 conf/l3fwd.cfg                            | 400 ++++++++++++
 conf/l3fwd_base.cfg                       | 103 +++
 conf/l3fwd_em.cfg                         | 147 +++++
 conf/l3fwd_lpm_ipv4.cfg                   | 105 +++
 conf/l3fwd_lpm_ipv4_rfc2544.cfg           | 105 +++
 conf/l3fwd_lpm_ipv6.cfg                   | 105 +++
 framework/test_case.py                    |   4 +
 tests/TestSuite_l3fwd.py                  | 370 ++---------
 tests/TestSuite_l3fwd_em.py               | 258 +-------
 tests/TestSuite_l3fwd_lpm_ipv4.py         |  86 +++
 tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py |  86 +++
 tests/TestSuite_l3fwd_lpm_ipv6.py         |  85 +++
 tests/l3fwd_base.py                       | 744 ++++++++++++++++++++++
 13 files changed, 2044 insertions(+), 554 deletions(-)
 create mode 100644 conf/l3fwd.cfg
 create mode 100644 conf/l3fwd_base.cfg
 create mode 100644 conf/l3fwd_em.cfg
 create mode 100644 conf/l3fwd_lpm_ipv4.cfg
 create mode 100644 conf/l3fwd_lpm_ipv4_rfc2544.cfg
 create mode 100644 conf/l3fwd_lpm_ipv6.cfg
 create mode 100644 tests/TestSuite_l3fwd_lpm_ipv4.py
 create mode 100644 tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
 create mode 100644 tests/TestSuite_l3fwd_lpm_ipv6.py
 create mode 100644 tests/l3fwd_base.py

-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 1/13] conf/l3fwd: l3fwd suite testing configuration
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 2/13] conf/l3fwd_base: l3fwd base class flows configuration yufengmx
                   ` (12 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd.cfg | 400 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 400 insertions(+)
 create mode 100644 conf/l3fwd.cfg

diff --git a/conf/l3fwd.cfg b/conf/l3fwd.cfg
new file mode 100644
index 0000000..e566aff
--- /dev/null
+++ b/conf/l3fwd.cfg
@@ -0,0 +1,400 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g':
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == True, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the beginning, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as usual.
+
+[suite]
+update_expected = True
+
+test_duration = 12
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_throughput = {
+    'test_perf_throughput_ipv4_em': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},
+    'test_perf_throughput_ipv4_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},
+    'test_perf_throughput_ipv6_em': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},
+    'test_perf_throughput_ipv6_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},}
+expected_rfc2544 = {
+    'test_perf_rfc2544_ipv4_em': {
+        'niantic': {
+            '1C/1T/1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},
+    'test_perf_rfc2544_ipv4_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},
+    'test_perf_rfc2544_ipv6_em': {
+        'niantic': {
+            '1C/1T/1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},
+    'test_perf_rfc2544_ipv6_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},}
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 2/13] conf/l3fwd_base: l3fwd base class flows configuration
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 1/13] conf/l3fwd: l3fwd suite testing configuration yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 3/13] conf/l3fwd_em: suite testing configuration yufengmx
                   ` (11 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd base class flows configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_base.cfg | 103 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 conf/l3fwd_base.cfg

diff --git a/conf/l3fwd_base.cfg b/conf/l3fwd_base.cfg
new file mode 100644
index 0000000..ac2ab26
--- /dev/null
+++ b/conf/l3fwd_base.cfg
@@ -0,0 +1,103 @@
+[suite]
+l3fwd_flows = {
+    "ipv4": {
+        "em": [
+            {
+                "ipv4": {
+                    "dst": "101.0.0.0",
+                    "src": "100.10.0.1"
+                },
+                "udp": {
+                    "dst": 11,
+                    "src": 101
+                }
+            },
+            {
+                "ipv4": {
+                    "dst": "201.0.0.0",
+                    "src": "200.20.0.1"
+                },
+                "udp": {
+                    "dst": 12,
+                    "src": 102
+                }
+            },
+            {
+                "ipv4": {
+                    "dst": "111.0.0.0",
+                    "src": "100.30.0.1"
+                },
+                "udp": {
+                    "dst": 11,
+                    "src": 101
+                }
+            },
+            {
+                "ipv4": {
+                    "dst": "211.0.0.0",
+                    "src": "200.40.0.1"
+                },
+                "udp": {
+                    "dst": 11,
+                    "src": 102
+                }
+            }
+        ],
+        "lpm": [
+            "198.18.0.0/24",
+            "198.18.1.0/24",
+            "198.18.2.0/24",
+            "198.18.3.0/24",
+            "198.18.4.0/24",
+            "198.18.5.0/24",
+            "198.18.6.0/24",
+            "198.18.7.0/24"
+        ]
+    },
+    "ipv6": {
+        "em": [
+            {
+                "ipv6": {
+                    "dst": "fe80:0000:0000:0000:021e:67ff:fe00:0000",
+                    "src": "fe80:0000:0000:0000:021b:21ff:fe91:3805"
+                },
+                "udp": {
+                    "dst": 101,
+                    "src": 11 }
+            },
+            {
+                "ipv6": {
+                    "dst": "fe90:0000:0000:0000:021e:67ff:fe00:0000",
+                    "src": "fe90:0000:0000:0000:021b:21ff:fe91:3805" },
+                "udp": {
+                    "dst": 102,
+                    "src": 12 }
+            },
+            {
+                "ipv6": {
+                    "dst": "fea0:0000:0000:0000:021e:67ff:fe00:0000",
+                    "src": "fea0:0000:0000:0000:021b:21ff:fe91:3805" },
+                "udp": {
+                    "dst": 101,
+                    "src": 11 }
+            },
+            {
+                "ipv6": {
+                    "dst": "feb0:0000:0000:0000:021e:67ff:fe00:0000",
+                    "src": "feb0:0000:0000:0000:021b:21ff:fe91:3805" },
+                "udp": {
+                    "dst": 102,
+                    "src": 12 }
+            }
+        ],
+        "lpm": [
+            "3201:20:00:00:00:00:00:00/48",
+            "3201:20:00:00:01:00:00:00/48",
+            "3201:20:00:00:02:00:00:00/48",
+            "3201:20:00:00:03:00:00:00/48",
+            "3201:20:00:00:04:00:00:00/48",
+            "3201:20:00:00:05:00:00:00/48",
+            "3201:20:00:00:06:00:00:00/48",
+            "3201:20:00:00:07:00:00:00/48"
+        ]
+    } }
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 3/13] conf/l3fwd_em: suite testing configuration
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 1/13] conf/l3fwd: l3fwd suite testing configuration yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 2/13] conf/l3fwd_base: l3fwd base class flows configuration yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 4/13] conf/l3fwd_lpm_ipv4_rfc2544: " yufengmx
                   ` (10 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd_em suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_em.cfg | 147 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 147 insertions(+)
 create mode 100644 conf/l3fwd_em.cfg

diff --git a/conf/l3fwd_em.cfg b/conf/l3fwd_em.cfg
new file mode 100644
index 0000000..7f2408e
--- /dev/null
+++ b/conf/l3fwd_em.cfg
@@ -0,0 +1,147 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g':
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == True, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the beginning, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as usual.
+
+[suite]
+update_expected = True
+
+test_duration = 12
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_throughput = {
+    'test_perf_throughput_ipv4_em': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},
+    'test_perf_throughput_ipv6_em': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},},}
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 4/13] conf/l3fwd_lpm_ipv4_rfc2544: suite testing configuration
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (2 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 3/13] conf/l3fwd_em: suite testing configuration yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 5/13] conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 " yufengmx
                   ` (9 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd_lpm_ipv4_rfc2544 suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_lpm_ipv4_rfc2544.cfg | 105 ++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 conf/l3fwd_lpm_ipv4_rfc2544.cfg

diff --git a/conf/l3fwd_lpm_ipv4_rfc2544.cfg b/conf/l3fwd_lpm_ipv4_rfc2544.cfg
new file mode 100644
index 0000000..4ca40de
--- /dev/null
+++ b/conf/l3fwd_lpm_ipv4_rfc2544.cfg
@@ -0,0 +1,105 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g': 
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == Ture, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the begining, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as ususal.
+
+[suite]
+update_expected = True
+
+test_duration = 60
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_rfc2544 = {
+    'test_perf_rfc2544_ipv4_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T/2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T/4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},}
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 5/13] conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 testing configuration
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (3 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 4/13] conf/l3fwd_lpm_ipv4_rfc2544: " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 6/13] conf/l3fwd_lpm_ipv6: suite " yufengmx
                   ` (8 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd_lpm_ipv4 suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_lpm_ipv4.cfg | 105 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 conf/l3fwd_lpm_ipv4.cfg

diff --git a/conf/l3fwd_lpm_ipv4.cfg b/conf/l3fwd_lpm_ipv4.cfg
new file mode 100644
index 0000000..5935c65
--- /dev/null
+++ b/conf/l3fwd_lpm_ipv4.cfg
@@ -0,0 +1,105 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g': 
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == Ture, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the begining, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as ususal.
+
+[suite]
+update_expected = True
+
+test_duration = 60
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_throughput = {
+    'test_perf_throughput_ipv4_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},}, }
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 6/13] conf/l3fwd_lpm_ipv6: suite testing configuration
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (4 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 5/13] conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 7/13] tests/l3fwd: upload automation script yufengmx
                   ` (7 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd_lpm_ipv6 suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_lpm_ipv6.cfg | 105 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 conf/l3fwd_lpm_ipv6.cfg

diff --git a/conf/l3fwd_lpm_ipv6.cfg b/conf/l3fwd_lpm_ipv6.cfg
new file mode 100644
index 0000000..200ddcd
--- /dev/null
+++ b/conf/l3fwd_lpm_ipv6.cfg
@@ -0,0 +1,105 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g': 
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == Ture, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the begining, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as ususal.
+
+[suite]
+update_expected = True
+
+test_duration = 60
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_throughput = {
+    'test_perf_throughput_ipv6_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},}, }
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 7/13] tests/l3fwd: upload automation script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (5 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 6/13] conf/l3fwd_lpm_ipv6: suite " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 8/13] tests/l3fwd_em: " yufengmx
                   ` (6 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload l3fwd suite automation script.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd.py | 370 +++++----------------------------------
 1 file changed, 44 insertions(+), 326 deletions(-)

diff --git a/tests/TestSuite_l3fwd.py b/tests/TestSuite_l3fwd.py
index ae6a5cb..ea89cfe 100644
--- a/tests/TestSuite_l3fwd.py
+++ b/tests/TestSuite_l3fwd.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -34,77 +34,42 @@ DPDK Test suite.
 Layer-3 forwarding test script.
 """
 
-import utils
-import string
-import time
-import re
-import os
 from test_case import TestCase
-from settings import HEADER_SIZE
-from pktgen import PacketGeneratorHelper
+from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
 
 
-class TestL3fwd(TestCase):
-
-    path = "./examples/l3fwd/build/"
-    cmdline_2_ports = {
-        "1S/1C/1T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.0})'",
-        "1S/1C/2T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.1})'",
-        "1S/2C/1T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0})'",
-        "1S/4C/1T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}), (P0,1,C{1.3.0}), (P1,1,C{1.4.0})'"}
-
-    cmdline_4_ports = {
-        "1S/1C/1T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.0}), (P2,0,C{1.1.0}), (P3,0,C{1.1.0})'",
-        "1S/2C/2T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.1}), (P2,0,C{1.2.0}), (P3,0,C{1.2.1})'",
-        "1S/4C/1T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}), (P2,1,C{1.3.0}), (P3,1,C{1.4.0})'",
-        "1S/8C/1T": "%s -c %s -n %d -- -p %s --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}), (P2,0,C{1.3.0}), (P3,0,C{1,4,0}),\
-        (P0,1,C{1.5.0}), (P1,1,C{1.6.0}), (P2,1,C{1.7.0}), (P3,1,C{1,8,0})'"}
+class TestL3fwd(TestCase, L3fwdBase):
 
+    #
+    # Test cases.
+    #
     def set_up_all(self):
         """
         Run at the start of each test suite.
         L3fwd Prerequisites
         """
-        self.tester.extend_external_packet_generator(TestL3fwd, self)
         # Based on h/w type, choose how many ports to use
         self.dut_ports = self.dut.get_ports(self.nic)
-        global valports
-        valports = [_ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
-
-        # Verify that enough ports are available
-        self.verify(len(valports) == 2 or len(valports) == 4, "Port number must be 2 or 4.")
-
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
         # get socket and cores
-        self.socket = self.dut.get_numa_id(self.dut_ports[0])
-        self.cores = self.dut.get_core_list("1S/8C/1T", socket=self.socket)
-        self.verify(self.cores is not None, "Insufficient cores for speed testing")
-
-        self.frame_sizes = [64, 128, 256, 512, 1024, 1518]
-
-        # Update config file and rebuild to get best perf on FVL
-        if self.nic in ["fortville_sprit", "fortville_eagle", "fortville_25g"]:
-            self.dut.send_expect("sed -i -e 's/CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n/CONFIG_RTE_LIBRTE_"
-                                 "I40E_16BYTE_RX_DESC=y/' ./config/common_base", "#", 20)
-            self.dut.build_install_dpdk(self.target)
-
-        self.logger.info("Configure RX/TX descriptor to 2048, and re-build ./examples/l3fwd")
-        self.dut.send_expect("sed -i -e 's/define RTE_TEST_RX_DESC_DEFAULT.*$/"
-                             + "define RTE_TEST_RX_DESC_DEFAULT 2048/' ./examples/l3fwd/main.c", "#", 20)
-        self.dut.send_expect("sed -i -e 's/define RTE_TEST_TX_DESC_DEFAULT.*$/"
-                             + "define RTE_TEST_TX_DESC_DEFAULT 2048/' ./examples/l3fwd/main.c", "#", 20)
-
-        self.pat = re.compile("P([0123])")
-        self.test_results = {'header': [], 'data': []}
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
+        self.verify(cores is not None, "Insufficient cores for speed testing")
+        # init l3fwd common base class parameters
+        self.l3fwd_init(valports, socket)
+        # preset testing environment
+        self.l3fwd_preset_test_environment(self.get_suite_cfg())
 
-        # get dts output path
-        if self.logger.log_path.startswith(os.sep):
-            self.output_path = self.logger.log_path
-        else:
-            cur_path = os.path.dirname(
-                os.path.dirname(os.path.realpath(__file__)))
-            self.output_path = os.sep.join([cur_path, self.logger.log_path])
-        # create an instance to set stream field setting
-        self.pktgen_helper = PacketGeneratorHelper()
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        if self:
+            self.l3fwd_save_results()
+            self = None
 
     def set_up(self):
         """
@@ -112,288 +77,41 @@ class TestL3fwd(TestCase):
         """
         pass
 
-    def portRepl(self, match):
-        """
-        Function to replace P([0123]) pattern in tables
-        """
-        portid = match.group(1)
-        self.verify(int(portid) in range(4), "invalid port id")
-        if int(portid) >= len(valports):
-            return '0'
-        else:
-            return '%s' % valports[int(portid)]
-
-    def install_l3fwd_application(self, l3_proto, mode):
-        """
-        Prepare long prefix match table, replace P(x) port pattern
-        """
-        l3fwd_method = l3_proto + "_l3fwd_" + mode
-        self.dut.send_expect("make clean -C examples/l3fwd", "# ")
-        if "lpm" in l3fwd_method:
-            out = self.dut.build_dpdk_apps("./examples/l3fwd", "USER_FLAGS=-DAPP_LOOKUP_METHOD=1")
-        elif "em" in l3fwd_method:
-            out = self.dut.build_dpdk_apps("./examples/l3fwd", "USER_FLAGS=-DAPP_LOOKUP_METHOD=0")
-        self.verify("Error" not in out, "compilation error 1")
-        self.verify("No such file" not in out, "compilation error 2")
-
-        # Backup the l3fwd exe.
-        self.dut.send_expect("mv -f examples/l3fwd/build/l3fwd examples/l3fwd/build/%s" % l3fwd_method, "# ")
-
-    def flows(self):
-        """
-        Return a list of packets that implements the flows described in the
-        l3fwd test plan.
-        """
-        return {"ipv4": ['IP(src="1.2.3.4",dst="192.18.1.0")',
-                         'IP(src="1.2.3.4",dst="192.18.1.1")',
-                         'IP(src="1.2.3.4",dst="192.18.0.0")',
-                         'IP(src="1.2.3.4",dst="192.18.0.1")',
-                         'IP(src="1.2.3.4",dst="192.18.3.0")',
-                         'IP(src="1.2.3.4",dst="192.18.3.1")',
-                         'IP(src="1.2.3.4",dst="192.18.2.0")',
-                         'IP(src="1.2.3.4",dst="192.18.2.1")'],
-                "ipv6": [
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="fe80:0000:0000:0000:021b:21ff:fe91:3805")/UDP(sport=10,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="fe80:0000:0000:0000:031b:21ff:fe91:3805")/UDP(sport=10,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="2a80:0000:0000:0000:021b:21ff:fe91:3805")/UDP(sport=11,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="2a80:0000:0000:0000:031b:21ff:fe91:3805")/UDP(sport=11,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="2b80:0000:0000:0000:021b:21ff:fe91:3805")/UDP(sport=12,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="2b80:0000:0000:0000:031b:21ff:fe91:3805")/UDP(sport=12,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="2c80:0000:0000:0000:021b:21ff:fe91:3805")/UDP(sport=13,dport=1)',
-                    'IPv6(src="fe80:0000:0000:0000:021e:67ff:fe0d:b60a",dst="2c80:0000:0000:0000:031b:21ff:fe91:3805")/UDP(sport=13,dport=1)'
-                ]}
-
-    def repl(self, match):
-        pid = match.group(1)
-        qid = match.group(2)
-        self.logger.debug("%s\n" % match.group(3))
-        lcid = self.dut.get_lcore_id(match.group(3))
-        self.logger.debug("%s\n" % lcid)
-
-        global corelist
-        corelist.append(int(lcid))
-        self.verify(int(pid) in range(4), "invalid port id")
-        self.verify(lcid, "invalid thread id")
-        return '%s,%s,%s' % (str(valports[int(pid)]), qid, lcid)
-
-    def perpare_commandline(self, ports):
-        """
-        Generate the command line based on the number of ports
-        """
-        global corelist
-        pat = re.compile("P([0123]),([0123]),(C\{\d.\d.\d\})")
-        core_mask = {}
-        if ports == 2:
-            rtCmdLines = dict(TestL3fwd.cmdline_2_ports)
-        elif ports == 4:
-            rtCmdLines = dict(TestL3fwd.cmdline_4_ports)
-
-        for key in list(rtCmdLines.keys()):
-            corelist = []
-            while pat.search(rtCmdLines[key]):
-                print(rtCmdLines[key])
-                rtCmdLines[key] = pat.sub(self.repl, rtCmdLines[key])
-            core_mask[key] = utils.create_mask(set(corelist))
-        return rtCmdLines, core_mask
-
-    def create_pcap_file(self, frame_size, l3_proto):
-        """
-        Prepare traffic flow for packet generator
-        """
-        if l3_proto == 'ipv4':
-            payload_size = frame_size - HEADER_SIZE['eth'] - HEADER_SIZE['ip'] - HEADER_SIZE['udp']
-        else:
-            payload_size = frame_size - HEADER_SIZE['eth'] - HEADER_SIZE['ipv6'] - HEADER_SIZE['udp']
-
-        pcaps = {}
-        for _port in valports:
-            index = valports[_port]
-            dmac = self.dut.get_mac_address(index)
-            cnt = 0
-            layer3s = self.flows()[l3_proto][_port * 2:(_port + 1) * 2]
-            for l3 in layer3s:
-                flow = ['Ether(dst="%s")/%s/UDP()/("X"*%d)' % (dmac, l3, payload_size)]
-                pcap = os.sep.join([self.output_path, "dst{0}_{1}.pcap".format(index, cnt)])
-                self.tester.scapy_append('wrpcap("%s", [%s])' % (pcap, string.join(flow, ',')))
-                self.tester.scapy_execute()
-                if index not in pcaps:
-                    pcaps[index] = []
-                pcaps[index].append(pcap)
-                cnt += 1
-        return pcaps
-
-    def prepare_stream(self, pcaps):
-        """
-        create streams for ports, one port one stream
-        """
-        tgen_input = []
-        for rxPort in valports:
-            if rxPort % len(valports) == 0 or len(valports) % rxPort == 2:
-                txIntf = self.tester.get_local_port(valports[rxPort + 1])
-                port_id = valports[rxPort + 1]
-            else:
-                txIntf = self.tester.get_local_port(valports[rxPort - 1])
-                port_id = valports[rxPort - 1]
-            rxIntf = self.tester.get_local_port(valports[rxPort])
-            for pcap in pcaps[port_id]:
-                tgen_input.append((txIntf, rxIntf, pcap))
-        return tgen_input
-
-    def create_result_table(self, ttl, ttl1, ttl2, ttl3, ttl4):
-
-        header_row = [ttl, ttl1, ttl2, ttl3, ttl4]
-        self.test_results['header'] = header_row
-        self.result_table_create(header_row)
-        self.test_results['data'] = []
-
-    def measure_throughput(self, l3_proto, mode):
-        """
-        measure throughput according to Layer-3 Protocal and Lookup Mode
-        """
-        # create result table
-        self.create_result_table("Frame Size", "Mode", "S/C/T", "Mpps", "% Linerate")
-        # build application
-        self.install_l3fwd_application(l3_proto, mode)
-        # perpare commandline and core mask
-        rtCmdLines, core_mask = self.perpare_commandline(len(valports))
-
-        for frame_size in self.frame_sizes:
-            if l3_proto == "ipv6" and frame_size == 64:
-                frame_size += 2
-            for cores in list(rtCmdLines.keys()):
-                # Start L3fwd appliction
-                command_line = rtCmdLines[cores] % (TestL3fwd.path + l3_proto + "_l3fwd_" + mode, core_mask[cores],
-                                                    self.dut.get_memory_channels(), utils.create_mask(valports))
-                if self.nic == "niantic":
-                    command_line += " --parse-ptype"
-                if frame_size > 1518:
-                    command_line += " --enable-jumbo --max-pkt-len %d" % frame_size
-                self.dut.send_expect(command_line, "L3FWD:", 120)
-                self.logger.info("Executing l3fwd using %s mode, %d ports, %s and %d frame size"
-                                 % (mode, len(valports), cores, frame_size))
-                # crete traffic flow
-                pcaps = self.create_pcap_file(frame_size, l3_proto)
-                # send the traffic and Measure test
-                tgenInput = self.prepare_stream(pcaps)
-
-                vm_config = self.set_fields()
-                if l3_proto == "ipv6":
-                    vm_config = None
-                # clear streams before add new streams
-                self.tester.pktgen.clear_streams()
-                # run packet generator
-                streams = self.pktgen_helper.prepare_stream_from_tginput(tgenInput, 100, vm_config, self.tester.pktgen)
-                # set traffic option
-                traffic_opt = {'delay': 30}
-                _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
-                self.verify(pps > 0, "No traffic detected")
-                # statistical result
-                pps /= 1000000.0
-                linerate = self.wirespeed(self.nic, frame_size, len(valports))
-                percentage = pps * 100 / linerate
-                data_row = [frame_size, mode, cores, str(pps), str(percentage)]
-                self.result_table_add(data_row)
-                self.test_results['data'].append(data_row)
-                # Stop L3fwd
-                self.dut.send_expect("^C", "#")
-                time.sleep(1)
-        # Print result
-        self.result_table_print()
-
-    def measure_rfc2544(self, l3_proto, mode):
+    def tear_down(self):
         """
-        measure RFC2544 according to Layer-3 Protocal and Lookup Mode
+        Run after each test case.
         """
-        # create result table
-        self.create_result_table("Frame Size", "Mode", "S/C/T", "Zero Loss Throughput(Mpps)", " % Zero Loss Rate")
-        # build application
-        self.install_l3fwd_application(l3_proto, mode)
-        # perpare commandline and core mask
-        rtCmdLines, core_mask = self.perpare_commandline(len(valports))
-
-        for frame_size in self.frame_sizes:
-            if l3_proto == "ipv6" and frame_size == 64:
-                frame_size += 2
-            for cores in list(rtCmdLines.keys()):
-                # in order to save time, only some of the cases will be run.
-                if cores in ["1S/2C/1T", "1S/4C/1T"]:
-                    # Start L3fwd appliction
-                    command_line = rtCmdLines[cores] % (TestL3fwd.path + l3_proto + "_l3fwd_" + mode, core_mask[cores],
-                                                        self.dut.get_memory_channels(), utils.create_mask(valports))
-                    if self.nic == "niantic":
-                        command_line += " --parse-ptype"
-                    if frame_size > 1518:
-                        command_line += " --enable-jumbo --max-pkt-len %d" % frame_size
-                    self.dut.send_expect(command_line, "L3FWD:", 120)
-                    self.logger.info("Executing l3fwd using %s mode, %d ports, %s and %d frame size"
-                                     % (mode, len(valports), cores, frame_size))
-                    # crete traffic flow
-                    pcaps = self.create_pcap_file(frame_size, l3_proto)
-                    # send the traffic and Measure test
-                    tgenInput = self.prepare_stream(pcaps)
-
-                    vm_config = self.set_fields()
-                    if l3_proto == "ipv6":
-                        vm_config = None
-                    # clear streams before add new streams
-                    self.tester.pktgen.clear_streams()
-                    # run packet generator
-                    streams = self.pktgen_helper.prepare_stream_from_tginput(tgenInput, 100, vm_config, self.tester.pktgen)
-                    # set traffic option
-                    traffic_opt = {'duration': 15}
-                    zero_loss_rate, tx_pkts, rx_pkts = self.tester.pktgen.measure_rfc2544(stream_ids=streams,
-                                                                                          options=traffic_opt)
-                    # statistical result
-                    linerate = self.wirespeed(self.nic, frame_size, len(valports))
-                    zero_loss_throughput = (linerate * zero_loss_rate) / 100
-                    data_row = [frame_size, mode, cores, str(zero_loss_throughput), str(zero_loss_rate)]
-                    self.result_table_add(data_row)
-                    self.test_results['data'].append(data_row)
-                # Stop L3fwd
-                self.dut.send_expect("^C", "#")
-                time.sleep(1)
-        # Print result
-        self.result_table_print()
+        self.dut.kill_all()
+        self.l3fwd_reset_cur_case()
 
     def test_perf_rfc2544_ipv4_lpm(self):
-        self.measure_rfc2544(l3_proto="ipv4", mode="lpm")
+        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv4_lpm')
+        self.qt_rfc2544(l3_proto=L3_IPV4, mode=LPM)
 
     def test_perf_rfc2544_ipv4_em(self):
-        self.measure_rfc2544(l3_proto="ipv4", mode="em")
+        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv4_em')
+        self.qt_rfc2544(l3_proto=L3_IPV4, mode=EM)
 
     def test_perf_throughput_ipv4_lpm(self):
-        self.measure_throughput(l3_proto="ipv4", mode="lpm")
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_lpm')
+        self.ms_throughput(l3_proto=L3_IPV4, mode=LPM)
 
     def test_perf_throughput_ipv4_em(self):
-        self.measure_throughput(l3_proto="ipv4", mode="em")
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_em')
+        self.ms_throughput(l3_proto=L3_IPV4, mode=EM)
 
     def test_perf_rfc2544_ipv6_lpm(self):
-        self.measure_rfc2544(l3_proto="ipv6", mode="lpm")
+        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv6_lpm')
+        self.qt_rfc2544(l3_proto=L3_IPV6, mode=LPM)
 
     def test_perf_rfc2544_ipv6_em(self):
-        self.measure_rfc2544(l3_proto="ipv6", mode="em")
+        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv6_em')
+        self.qt_rfc2544(l3_proto=L3_IPV6, mode=EM)
 
     def test_perf_throughput_ipv6_lpm(self):
-        self.measure_throughput(l3_proto="ipv6", mode="lpm")
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_lpm')
+        self.ms_throughput(l3_proto=L3_IPV6, mode=LPM)
 
     def test_perf_throughput_ipv6_em(self):
-        self.measure_throughput(l3_proto="ipv6", mode="em")
-
-    def set_fields(self):
-        """
-        set ip protocol field behavior
-        """
-        fields_config = {'ip':  {'src': {'action': 'random'}, }, }
-        return fields_config
-
-    def tear_down(self):
-        """
-        Run after each test case.
-        """
-        self.dut.kill_all()
-
-    def tear_down_all(self):
-        """
-        Run after each test suite.
-        """
-        pass
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_em')
+        self.ms_throughput(l3_proto=L3_IPV6, mode=EM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 8/13] tests/l3fwd_em: upload automation script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (6 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 7/13] tests/l3fwd: upload automation script yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 9/13] tests/l3fwd_lpm_ipv4_rfc2544: " yufengmx
                   ` (5 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload l3fwd_em suite automation script.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_em.py | 258 +++++-------------------------------
 1 file changed, 30 insertions(+), 228 deletions(-)

diff --git a/tests/TestSuite_l3fwd_em.py b/tests/TestSuite_l3fwd_em.py
index 33acbd5..c586759 100644
--- a/tests/TestSuite_l3fwd_em.py
+++ b/tests/TestSuite_l3fwd_em.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -31,257 +31,59 @@
 
 """
 DPDK Test suite.
-Layer-3 forwarding exact-match test script.
+Layer-3 forwarding test script.
 """
-
-import os
-import utils
-import string
-import re
 from test_case import TestCase
-from exception import VerifyFailure
-from settings import HEADER_SIZE
-from utils import *
-from pktgen import PacketGeneratorHelper
-
-
-class TestL3fwdEM(TestCase):
-
-    path = "./examples/l3fwd/build/"
-
-    test_cases_2_ports = {"1S/1C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.1.0})' --hash-entry-num 0x400000",
-                          "1S/2C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0})' --hash-entry-num 0x400000",
-                          "1S/4C/1T": "%s -c %s -n %d -- -p %s -E --config '(P0,0,C{1.1.0}), (P1,0,C{1.2.0}),(P0,1,C{1.3.0}), (P1,1,C{1.4.0})' --hash-entry-num 0x400000"
-                          }
-
-    frame_sizes = [64, 65, 128, 256, 512, 1518]
-    methods = ['exact']
-
-    #
-    #
-    # Utility methods and other non-test code.
-    #
-    # Insert or move non-test functions here.
-    def portRepl(self, match):
-        """
-        Function to replace P([0123]) pattern in tables
-        """
+from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
 
-        portid = match.group(1)
-        self.verify(int(portid) in range(4), "invalid port id")
-        if int(portid) >= len(valports):
-            return '0'
-        else:
-            return '%s' % valports[int(portid)]
 
-    #
-    #
+class TestL3fwdEm(TestCase, L3fwdBase):
     #
     # Test cases.
     #
-
     def set_up_all(self):
         """
         Run at the start of each test suite.
-
-
         L3fwd Prerequisites
         """
         # Based on h/w type, choose how many ports to use
-        ports = self.dut.get_ports(socket=1)
-        if not ports:
-            ports = self.dut.get_ports(socket=0)
-
-        self.tester.extend_external_packet_generator(TestL3fwdEM, self)
-        # Verify that enough ports are available
-        self.verify(len(ports) >= 2, "Insufficient ports for speed testing")
-        
-        netdev = self.dut.ports_info[ports[0]]['port']
-        
-        self.port_socket = netdev.socket
-        
-        # Verify that enough threads are available
-        cores = self.dut.get_core_list("1S/8C/2T")
+        self.dut_ports = self.dut.get_ports(self.nic)
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
+        # get socket and cores
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
         self.verify(cores is not None, "Insufficient cores for speed testing")
+        # init l3fwd common base class parameters
+        self.l3fwd_init(valports, socket)
+        # preset testing environment
+        self.l3fwd_preset_test_environment(self.get_suite_cfg())
 
-        global valports
-        valports = [_ for _ in ports if self.tester.get_local_port(_) != -1]
-        
-        self.verify(len(valports) >= 2, "Insufficient active ports for speed testing")
-
-        pat = re.compile("P([0123])")
-
-        # Update config file and rebuild to get best perf on FVL
-        self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_CONFIG=n/CONFIG_RTE_PCI_CONFIG=y/' ./config/common_linuxapp", "#", 20)
-        self.dut.send_expect("sed -i -e 's/CONFIG_RTE_PCI_EXTENDED_TAG=.*$/CONFIG_RTE_PCI_EXTENDED_TAG=\"on\"/' ./config/common_linuxapp", "#", 20)
-        self.dut.build_install_dpdk(self.target)
-
-        out = self.dut.build_dpdk_apps("./examples/l3fwd")
-        self.verify("Error" not in out, "compilation error 1")
-        self.verify("No such file" not in out, "compilation error 2")
-
-        self.l3fwd_test_results = {'header': [],
-                                   'data': []}
-        # get dts output path
-        if self.logger.log_path.startswith(os.sep):
-            self.output_path = self.logger.log_path
-        else:
-            cur_path = os.path.dirname(
-                                os.path.dirname(os.path.realpath(__file__)))
-            self.output_path = os.sep.join([cur_path, self.logger.log_path])
-        # create an instance to set stream field setting
-        self.pktgen_helper = PacketGeneratorHelper()
-
-    def repl(self, match):
-        pid = match.group(1)
-        qid = match.group(2)
-        self.logger.debug("%s\n" % match.group(3))
-        lcid = self.dut.get_lcore_id(match.group(3))
-        self.logger.debug("%s\n" % lcid)
-
-        global corelist
-        corelist.append(int(lcid))
-
-        self.verify(int(pid) in range(4), "invalid port id")
-        self.verify(lcid, "invalid thread id")
-
-        return '%s,%s,%s' % (str(valports[int(pid)]), qid, lcid)
-
-    def set_up(self):
+    def tear_down_all(self):
         """
-        Run before each test case.
+        Run after each test suite.
         """
         pass
 
-    def test_perf_l3fwd_2ports(self):
+    def set_up(self):
         """
-        L3fwd main 2 ports.
+        Run before each test case.
         """
-
-        header_row = ["Frame", "mode", "S/C/T", "Mpps", "% linerate"]
-        self.l3fwd_test_results['header'] = header_row
-        self.result_table_create(header_row)
-        self.l3fwd_test_results['data'] = []
-
-        for frame_size in TestL3fwdEM.frame_sizes:
-
-            # Prepare traffic flow
-            payload_size = frame_size -  \
-                HEADER_SIZE['ip'] - HEADER_SIZE['eth'] - HEADER_SIZE['tcp']
-            
-            # Traffic for port0
-            dmac_port0 = self.dut.get_mac_address(valports[0])
-            flow1 = '[Ether(dst="%s")/IP(src="200.20.0.1",dst="201.0.0.0")/TCP(sport=12,dport=102)/("X"*%d)]' %(dmac_port0,payload_size)
-
-            pcaps = {}
-            pcap = os.sep.join([self.output_path, "dst0.pcap"])
-            pcaps[0] = pcap
-            self.tester.scapy_append('wrpcap("{0}",{1})'.format(pcap, flow1))
-
-            # Traffic for port1
-            dmac_port1 = self.dut.get_mac_address(valports[1])
-            flow2 = '[Ether(dst="%s")/IP(src="100.10.0.1",dst="101.0.0.0")/TCP(sport=11,dport=101)/("X"*%d)]' %(dmac_port1,payload_size)
-
-            pcap = os.sep.join([self.output_path, "dst1.pcap"])
-            pcaps[1] = pcap
-            self.tester.scapy_append('wrpcap("{0}",{1})'.format(pcap, flow2))
-            self.tester.scapy_execute()
-
-            # Prepare the command line
-            global corelist
-            pat = re.compile("P([0123]),([0123]),(C\{\d.\d.\d\})")
-            
-            pat2 = re.compile("C\{\d")
-            repl1 = "C{" + str(self.port_socket)
-
-            coreMask = {}
-            rtCmdLines = dict(TestL3fwdEM.test_cases_2_ports)
-            for key in list(rtCmdLines.keys()):
-                corelist = []
-                while pat.search(rtCmdLines[key]):
-                    # Change the socket to the NIC's socket
-                    if key.find('1S') >= 0:
-                        rtCmdLines[key] = pat2.sub(repl1, rtCmdLines[key])
-                    rtCmdLines[key] = pat.sub(self.repl, rtCmdLines[key])
-
-                self.logger.info("%s\n" % str(corelist))
-                coreMask[key] = utils.create_mask(set(corelist))
-
-            # measure by two different mode
-            for mode in TestL3fwdEM.methods:
-
-                # start l3fwd
-                index = 0
-                subtitle = []
-                for cores in list(rtCmdLines.keys()):
-
-                    info = "Executing l3fwd using %s mode, 2 ports, %s and %d frame size.\n" % (
-                           mode, cores, frame_size)
-
-                    self.logger.info(info)
-                    self.rst_report(info, annex=True)
-
-                    subtitle.append(cores)
-                    cmdline = rtCmdLines[cores] % (TestL3fwdEM.path + "l3fwd", coreMask[cores],
-                                                   self.dut.get_memory_channels(), utils.create_mask(valports[:2]))
-
-                    self.rst_report(cmdline + "\n", frame=True, annex=True)
-
-                    out = self.dut.send_expect(cmdline, "L3FWD: entering main loop", 120)
-                    
-                    # Measure test
-                    tgenInput = []
-                    for rxPort in range(2):
-                        # No use on rx/tx limitation
-                        if rxPort % 2 == 0:
-                            txIntf = self.tester.get_local_port(valports[rxPort + 1])
-                        else:
-                            txIntf = self.tester.get_local_port(valports[rxPort - 1])
-
-                        rxIntf = self.tester.get_local_port(valports[rxPort])
-                        port_index = valports[rxPort+1] if rxPort % 2 == 0 else valports[rxPort-1]
-                        tgenInput.append((txIntf, rxIntf, pcaps[port_index]))
-                    
-                    vm_config = self.set_fields()
-                    # clear streams before add new streams
-                    self.tester.pktgen.clear_streams()
-                    # run packet generator
-                    streams = self.pktgen_helper.prepare_stream_from_tginput(tgenInput, 100,
-                                            vm_config, self.tester.pktgen)
-                    traffic_opt = {'duration': 20, }
-                    _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams, options=traffic_opt)
-                    
-                    self.verify(pps > 0, "No traffic detected")
-                    pps /= 1000000.0
-                    linerate = self.wirespeed(self.nic, frame_size, 2)
-                    pct = pps * 100 / linerate
-                    index += 1
-
-                    # Stop l3fwd
-                    self.dut.send_expect("^C", "#")
-                    data_row = [frame_size, mode, cores, str(pps), str(pct)]
-                    self.result_table_add(data_row)
-                    self.l3fwd_test_results['data'].append(data_row)
-
-        self.result_table_print()
-
-    def set_fields(self):
-        ''' set ip protocol field behavior '''
-        fields_config = {
-        'ip':  {
-            'dst': {'range': 64, 'mask': '255.240.0.0', 'action': 'inc'}
-        },}
-
-        return fields_config
+        pass
 
     def tear_down(self):
         """
         Run after each test case.
         """
-        pass
+        self.dut.kill_all()
+        self.l3fwd_reset_cur_case()
 
-    def tear_down_all(self):
-        """
-        Run after each test suite.
-        """
-        pass
+    def test_perf_throughput_ipv4_em(self):
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_em')
+        self.ms_throughput(l3_proto=L3_IPV4, mode=EM)
+
+    def test_perf_throughput_ipv6_em(self):
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_em')
+        self.ms_throughput(l3_proto=L3_IPV6, mode=EM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 9/13] tests/l3fwd_lpm_ipv4_rfc2544: upload automation script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (7 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 8/13] tests/l3fwd_em: " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 0/13] tests/l3fwd_lpm_ipv4: " yufengmx
                   ` (4 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload l3fwd_lpm_ipv4_rfc2544 suite automation script.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py | 86 +++++++++++++++++++++++
 1 file changed, 86 insertions(+)
 create mode 100644 tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py

diff --git a/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py b/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
new file mode 100644
index 0000000..380d2e9
--- /dev/null
+++ b/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
@@ -0,0 +1,86 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+Layer-3 forwarding test script.
+"""
+from test_case import TestCase
+from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+
+
+class TestL3fwdLpmIpv4Rfc2544(TestCase, L3fwdBase):
+
+    #
+    # Test cases.
+    #
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        L3fwd Prerequisites
+        """
+        # Based on h/w type, choose how many ports to use
+        self.dut_ports = self.dut.get_ports(self.nic)
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
+        # get socket and cores
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
+        self.verify(cores is not None, "Insufficient cores for speed testing")
+        # init l3fwd common base class parameters
+        self.l3fwd_init(valports, socket)
+        # preset testing environment
+        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        pass
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        self.l3fwd_reset_cur_case()
+
+    def test_perf_rfc2544_ipv4_lpm(self):
+        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv4_lpm')
+        self.qt_rfc2544(l3_proto=L3_IPV4, mode=LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 0/13] tests/l3fwd_lpm_ipv4: upload automation script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (8 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 9/13] tests/l3fwd_lpm_ipv4_rfc2544: " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 1/13] tests/l3fwd_lpm_ipv6: " yufengmx
                   ` (3 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload l3fwd_lpm_ipv4 suite automation script.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_lpm_ipv4.py | 86 +++++++++++++++++++++++++++++++
 1 file changed, 86 insertions(+)
 create mode 100644 tests/TestSuite_l3fwd_lpm_ipv4.py

diff --git a/tests/TestSuite_l3fwd_lpm_ipv4.py b/tests/TestSuite_l3fwd_lpm_ipv4.py
new file mode 100644
index 0000000..dc6c0fd
--- /dev/null
+++ b/tests/TestSuite_l3fwd_lpm_ipv4.py
@@ -0,0 +1,86 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+Layer-3 forwarding test script.
+"""
+from test_case import TestCase
+from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+
+
+class TestL3fwdLpmIpv4(TestCase, L3fwdBase):
+
+    #
+    # Test cases.
+    #
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        L3fwd Prerequisites
+        """
+        # Based on h/w type, choose how many ports to use
+        self.dut_ports = self.dut.get_ports(self.nic)
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
+        # get socket and cores
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
+        self.verify(cores is not None, "Insufficient cores for speed testing")
+        # init l3fwd common base class parameters
+        self.l3fwd_init(valports, socket)
+        # preset testing environment
+        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        pass
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        self.l3fwd_reset_cur_case()
+
+    def test_perf_throughput_ipv4_lpm(self):
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_lpm')
+        self.ms_throughput(l3_proto=L3_IPV4, mode=LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 1/13] tests/l3fwd_lpm_ipv6: upload automation script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (9 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 0/13] tests/l3fwd_lpm_ipv4: " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 2/13] tests/l3fwd_base: " yufengmx
                   ` (2 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload l3fwd_lpm_ipv6 suite automation script.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_lpm_ipv6.py | 85 +++++++++++++++++++++++++++++++
 1 file changed, 85 insertions(+)
 create mode 100644 tests/TestSuite_l3fwd_lpm_ipv6.py

diff --git a/tests/TestSuite_l3fwd_lpm_ipv6.py b/tests/TestSuite_l3fwd_lpm_ipv6.py
new file mode 100644
index 0000000..c9dd0ab
--- /dev/null
+++ b/tests/TestSuite_l3fwd_lpm_ipv6.py
@@ -0,0 +1,85 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+Layer-3 forwarding test script.
+"""
+from test_case import TestCase
+from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+
+
+class TestL3fwdLpmIpv6(TestCase, L3fwdBase):
+    #
+    # Test cases.
+    #
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        L3fwd Prerequisites
+        """
+        # Based on h/w type, choose how many ports to use
+        self.dut_ports = self.dut.get_ports(self.nic)
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
+        # get socket and cores
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
+        self.verify(cores is not None, "Insufficient cores for speed testing")
+        # init l3fwd common base class parameters
+        self.l3fwd_init(valports, socket)
+        # preset testing environment
+        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        pass
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        self.l3fwd_reset_cur_case()
+
+    def test_perf_throughput_ipv6_lpm(self):
+        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_lpm')
+        self.ms_throughput(l3_proto=L3_IPV6, mode=LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 2/13] tests/l3fwd_base: upload automation script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (10 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 1/13] tests/l3fwd_lpm_ipv6: " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  7:27 ` [dts] [PATCH V3 3/13] framework/test_case: add wirespeed columbiaville nic yufengmx
  2020-01-16  8:43 ` [dts] [PATCH V3 0/13] l3fwd: refactor script Tu, Lijuan
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload l3fwd_base automation script for l3fwd relevant test suites.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/l3fwd_base.py | 744 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 744 insertions(+)
 create mode 100644 tests/l3fwd_base.py

diff --git a/tests/l3fwd_base.py b/tests/l3fwd_base.py
new file mode 100644
index 0000000..3fbf71f
--- /dev/null
+++ b/tests/l3fwd_base.py
@@ -0,0 +1,744 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+Layer-3 forwarding test script base class.
+"""
+import os
+import time
+import traceback
+import texttable
+import json
+from pprint import pformat
+from itertools import product
+from copy import deepcopy
+
+from config import SuiteConf
+from settings import HEADER_SIZE
+from packet import Packet
+from pktgen import TRANSMIT_CONT, PKTGEN_TREX, PKTGEN_IXIA
+from utils import convert_int2ip, convert_ip2int
+from exception import VerifyFailure
+import utils
+
+
+# LPM(longest prefix match) mode
+LPM = 'lpm'
+# EM(Exact-Match) mode
+EM = 'em'
+# stream types
+L3_IPV6 = 'ipv6'
+L3_IPV4 = 'ipv4'
+
+
+class L3fwdBase(object):
+
+    def l3fwd_init(self, valports, socket):
+        self.__valports = valports
+        self.__socket = socket
+        self.__nic_name = self.nic
+        self.__pkt_typ = 'udp'
+        # for result
+        self.__cur_case = None
+        self.__json_results = {}
+
+    @property
+    def output_path(self):
+        suiteName = self.suite_name
+        if self.logger.log_path.startswith(os.sep):
+            output_path = os.path.join(self.logger.log_path, suiteName)
+        else:
+            cur_path = os.sep.join(
+                os.path.realpath(__file__).split(os.sep)[:-3])
+            output_path = os.path.join(
+                cur_path, self.logger.log_path, suiteName)
+        if not os.path.exists(output_path):
+            os.makedirs(output_path)
+
+        return output_path
+
+    def d_con(self, cmd):
+        _cmd = [cmd, '# ', 10] if isinstance(cmd, (str, unicode)) else cmd
+        return self.dut.send_expect(*_cmd)
+
+    def __get_ipv4_lpm_vm_config(self, lpm_config):
+        netaddr, mask = lpm_config.split('/')
+        ip_range = int('1' * (32 - int(mask)), 2)
+        start_ip = convert_int2ip(convert_ip2int(netaddr) + 1)
+        end_ip = convert_int2ip(convert_ip2int(start_ip) + ip_range - 1)
+        layers = {'ipv4': {'src': start_ip, }, }
+        fields_config = {
+            'ip': {'dst': {
+                'src': start_ip,
+                'dst': end_ip,
+                'step': 1,
+                'action': 'random', }, }, }
+        return layers, fields_config
+
+    def __get_ipv6_lpm_vm_config(self, lpm_config):
+        netaddr, mask = lpm_config.split('/')
+        ip_range = int('1' * (128 - int(mask)), 2)
+        start_ip = convert_int2ip(
+            convert_ip2int(netaddr, ip_type=6) + 1, ip_type=6)
+        end_ip = convert_int2ip(
+            convert_ip2int(start_ip, ip_type=6) + ip_range - 1, ip_type=6)
+        layers = {'ipv6': {'src': start_ip, }, }
+        fields_config = {
+            'ipv6': {'dst': {
+                'src': start_ip,
+                'dst': end_ip,
+                'step': 1,
+                'action': 'random', }, }, }
+        return layers, fields_config
+
+    def __get_pkt_len(self, pkt_type, ip_type='ip', frame_size=64):
+        headers_size = sum(
+            map(lambda x: HEADER_SIZE[x], ['eth', ip_type, pkt_type]))
+        pktlen = frame_size - headers_size
+        return pktlen
+
+    def __get_frame_size(self, name, frame_size):
+        _frame_size = 66 if name == L3_IPV6 and frame_size == 64 else \
+            frame_size
+        return _frame_size
+
+    def __config_stream(self, stm_name, layers=None, frame_size=64):
+        _framesize = self.__get_frame_size(stm_name, frame_size)
+        payload_size = self.__get_pkt_len(
+            self.__pkt_typ,
+            'ip' if stm_name == L3_IPV4 else 'ipv6', _framesize)
+        # set streams for traffic
+        pkt_configs = {
+            L3_IPV4: {
+                'type': self.__pkt_typ.upper(),
+                'pkt_layers': {
+                    'raw': {'payload': ['58'] * payload_size}}},
+            L3_IPV6: {
+                'type': 'IPv6_' + self.__pkt_typ.upper(),
+                'pkt_layers': {
+                    'raw': {'payload': ['58'] * payload_size}}}, }
+        if stm_name not in pkt_configs.keys():
+            msg = '{} not set in table'.format(stm_name)
+            raise VerifyFailure(msg)
+        values = deepcopy(pkt_configs.get(stm_name))
+        if layers:
+            values['pkt_layers'].update(layers)
+        self.logger.debug(pformat(values))
+        pkt = self.__get_pkt_inst(values)
+
+        return pkt
+
+    def __get_pkt_inst(self, pkt_config):
+        pkt_type = pkt_config.get('type')
+        pkt_layers = pkt_config.get('pkt_layers')
+        pkt = Packet(pkt_type=pkt_type)
+        for layer in pkt_layers.keys():
+            pkt.config_layer(layer, pkt_layers[layer])
+        self.logger.debug(pformat(pkt.pktgen.pkt.command()))
+
+        return pkt.pktgen.pkt
+
+    def __preset_flows_configs(self):
+        flows = self.__test_content.get('flows')
+        if not flows:
+            msg = "flows not set in json cfg file"
+            raise VerifyFailure(msg)
+        flows_configs = {}
+        for name, mode_configs in flows.iteritems():
+            for mode, configs in mode_configs.iteritems():
+                for index, config in enumerate(configs):
+                    if mode == LPM:
+                        # under LPM mode, one port only set one stream
+                        if index >= len(self.__valports):
+                            break
+                        port_id = self.__valports[index]
+                        dmac = self.dut.get_mac_address(port_id)
+                        _layer = {'ether': {'dst': dmac, }, }
+                        _layer2, fields_config = \
+                            self.__get_ipv4_lpm_vm_config(config) \
+                            if name == L3_IPV4 else \
+                            self.__get_ipv6_lpm_vm_config(config)
+                        _layer.update(_layer2)
+                    else:
+                        if index >= 2 * len(self.__valports):
+                            break
+                        port_id = \
+                            self.__valports[(index / 2) % len(self.__valports)]
+                        dmac = self.dut.get_mac_address(port_id)
+                        _layer = {'ether': {'dst': dmac, }, }
+                        _layer.update(config)
+                        fields_config = None
+                    flows_configs.setdefault((name, mode), []).append(
+                        [_layer, fields_config])
+        return flows_configs
+
+    def __preset_streams(self):
+        frame_sizes = self.__test_content.get('frame_sizes')
+        if not frame_sizes:
+            msg = "frame sizes not set in json cfg file"
+            raise VerifyFailure(msg)
+        test_streams = {}
+        flows_configs = self.__preset_flows_configs()
+        for frame_size in frame_sizes:
+            for flow_key, flows_config in flows_configs.iteritems():
+                streams_key = flow_key + (frame_size, )
+                for flow_config in flows_config:
+                    _layers, fields_config = flow_config
+                    pkt = self.__config_stream(
+                        flow_key[0], _layers, frame_size)
+                    test_streams.setdefault(streams_key, []).append(
+                        [pkt, fields_config])
+        self.logger.debug(pformat(test_streams))
+        return test_streams
+
+    def __add_stream_to_pktgen(self, streams, option):
+        def port(index):
+            p = self.tester.get_local_port(self.__valports[index])
+            return p
+        topos = [[port(index), port(index - 1)]
+                 if index % 2 else
+                 [port(index), port(index + 1)]
+                 for index, _ in enumerate(self.__valports)] \
+                 if len(self.__valports) > 1 else [[port(0), port(0)]]
+        stream_ids = []
+        step = len(streams) / len(self.__valports)
+        for cnt, stream in enumerate(streams):
+            pkt, field_config = stream
+            index = cnt // step
+            txport, rxport = topos[index]
+            _option = deepcopy(option)
+            _option['pcap'] = pkt
+            if field_config:
+                _option['field_config'] = field_config
+            stream_id = self.tester.pktgen.add_stream(txport, rxport, pkt)
+            self.tester.pktgen.config_stream(stream_id, _option)
+            stream_ids.append(stream_id)
+        return stream_ids
+
+    def __send_packets_by_pktgen(self, option):
+        streams = option.get('stream')
+        rate = option.get('rate')
+        # set traffic option
+        traffic_opt = option.get('traffic_opt')
+        self.logger.debug(option)
+        # clear streams before add new streams
+        self.tester.pktgen.clear_streams()
+        # set stream into pktgen
+        stream_option = {
+            'stream_config': {
+                'txmode': {},
+                'transmit_mode': TRANSMIT_CONT,
+                'rate': rate, }}
+        stream_ids = self.__add_stream_to_pktgen(streams, stream_option)
+        # run packet generator
+        result = self.tester.pktgen.measure(stream_ids, traffic_opt)
+        return result
+
+    def __throughput(self, l3_proto, mode, frame_size):
+        """
+        measure __throughput according to Layer-3 Protocol and Lookup Mode
+        """
+        flow_key = (l3_proto, mode, frame_size)
+        if flow_key not in self.__streams.keys():
+            msg = "{} {} {}: expected streams failed to create".format(
+                *flow_key)
+            raise VerifyFailure(msg)
+        streams = self.__streams.get(flow_key)
+        # set traffic option
+        duration = self.__test_content.get('test_duration')
+        option = {
+            'stream': streams,
+            'rate': 100,
+            'traffic_opt': {
+                'method': 'throughput',
+                'duration': duration, }}
+        # run traffic
+        result = self.__send_packets_by_pktgen(option)
+        # statistics result
+        _, pps = result
+        self.verify(pps > 0, "No traffic detected")
+        return result
+
+    def __rfc2544(self, config, l3_proto, mode, frame_size):
+        """
+        measure RFC2544 according to Layer-3 Protocol and Lookup Mode
+        """
+        flow_key = (l3_proto, mode, frame_size)
+        if flow_key not in self.__streams.keys():
+            msg = "{} {} {}: expected streams failed to create".format(
+                *flow_key)
+            raise VerifyFailure(msg)
+        streams = self.__streams.get(flow_key)
+        # set traffic option
+        if not self.__cur_case:
+            msg = 'current test case name not set, use default traffic option'
+            self.logger.warning(msg)
+        conf_opt = self.__test_content.get('expected_rfc2544', {}).get(
+            self.__cur_case, {}).get(self.__nic_name, {}).get(config, {}).get(
+            str(frame_size), {}).get('traffic_opt', {})
+        max_rate = float(conf_opt.get('max_rate') or 100.0)
+        min_rate = float(conf_opt.get('min_rate') or 0.0)
+        accuracy = float(conf_opt.get('accuracy') or 0.001)
+        pdr = float(conf_opt.get('pdr') or 0.001)
+        duration = self.__test_content.get('test_duration')
+        option = {
+            'stream': streams,
+            'rate': max_rate,
+            'traffic_opt': {
+                'method': 'rfc2544_dichotomy',
+                'max_rate': max_rate,
+                'min_rate': min_rate,
+                'accuracy': accuracy,
+                'pdr': pdr,
+                'duration': duration, }}
+        # run traffic
+        result = self.__send_packets_by_pktgen(option)
+        # statistics result
+        if result:
+            _, tx_pkts, rx_pkts = result
+            self.verify(tx_pkts > 0, "No traffic detected")
+            self.verify(rx_pkts > 0, "No packet transfer detected")
+        else:
+            msg = 'failed to get zero loss rate percent with traffic option.'
+            self.logger.error(msg)
+            self.logger.info(pformat(option))
+
+        return result
+
+    def __preset_compilation(self):
+        # Update config file and rebuild to get best perf on FVL
+        if self.nic in ["fortville_sprit", "fortville_eagle", "fortville_25g"]:
+            self.d_con(
+                ("sed -i -e 's/"
+                 "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n/"
+                 "CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=y/' "
+                 "./config/common_base"))
+            self.dut.build_install_dpdk(self.target)
+        # init l3fwd binary file
+        self.logger.info(
+            "Configure RX/TX descriptor to 2048, re-build ./examples/l3fwd")
+        self.d_con((
+            "sed -i -e 's/"
+            "define RTE_TEST_RX_DESC_DEFAULT.*$/"
+            "define RTE_TEST_RX_DESC_DEFAULT 2048/' "
+            "./examples/l3fwd/main.c"))
+        self.d_con((
+            "sed -i -e 's/"
+            "define RTE_TEST_TX_DESC_DEFAULT.*$/"
+            "define RTE_TEST_TX_DESC_DEFAULT 2048/' "
+            "./examples/l3fwd/main.c"))
+        self.__l3fwd_em = self.__init_l3fwd(EM)
+        self.__l3fwd_lpm = self.__init_l3fwd(LPM)
+
+    def __init_l3fwd(self, mode):
+        """
+        Prepare long prefix match table, __replace P(x) port pattern
+        """
+        l3fwd_method = '_'.join(['l3fwd', mode])
+        self.d_con("make clean -C examples/l3fwd")
+        flg = 1 if LPM in l3fwd_method else 0
+        out = self.dut.build_dpdk_apps(
+            "./examples/l3fwd",
+            "USER_FLAGS=-DAPP_LOOKUP_METHOD={}".format(flg))
+        self.verify("Error" not in out, "compilation error 1")
+        self.verify("No such file" not in out, "compilation error 2")
+        # rename binary file
+        self.d_con(
+            ("mv -f examples/l3fwd/build/l3fwd "
+             "examples/l3fwd/build/{}").format(l3fwd_method))
+        l3fwd_bin = os.path.join("./examples/l3fwd/build/", l3fwd_method)
+        return l3fwd_bin
+
+    def __start_l3fwd(self, mode, core_mask, config, frame_size):
+        bin = self.__l3fwd_em if mode == EM else self.__l3fwd_lpm
+        # Start L3fwd application
+        command_line = (
+            "{bin} "
+            "-c {cores} "
+            "-n {channel} "
+            "-- "
+            "-p {port_mask} "
+            "--config '{config}'"
+            "").format(**{
+                'bin': bin,
+                'cores': core_mask,
+                'channel': self.dut.get_memory_channels(),
+                'port_mask': utils.create_mask(self.__valports),
+                'config': config, })
+        if self.nic == "niantic":
+            command_line += " --parse-ptype"
+        if frame_size > 1518:
+            command_line += " --enable-jumbo --max-pkt-len %d" % frame_size
+        self.d_con([command_line, "L3FWD:", 120])
+        self.__is_l3fwd_on = True
+        # wait several second for l3fwd checking ports link status.
+        # It is aimed to make sure trex detect link up status.
+        time.sleep(2 * len(self.__valports))
+
+    def __close_l3fwd(self):
+        if not self.__is_l3fwd_on:
+            return
+        self.d_con("^C")
+        self.__is_l3fwd_on = False
+
+    def __json_rfc2544(self, value):
+        return {"unit": "Mpps", "name": "Rfc2544",
+                "value": value[0], "delta": value[1]}
+
+    def __json_throughput(self, value):
+        return {"unit": "Mpps", "name": "Throughput",
+                "value": value[0], "delta": value[1]}
+
+    def __json_line_rate(self, value):
+        return {"unit": "", "name": "% of Line Rate", "value": value}
+
+    def __json_port_config(self, value):
+        return {"unit": "", "name": "Number of Cores/Queues/Threads",
+                "value": value}
+
+    def __json_frame_size(self, value):
+        return {"unit": "bytes", "name": "Frame size",
+                "value": value}
+
+    def __save_throughput_result(self, case_name, result):
+        suite_results = {}
+        case_result = suite_results[case_name] = []
+        for sub_result in result:
+            status, throughput, line_rate, port_config, frame_size = sub_result
+            one_result = {
+                "status": status,
+                "performance": [
+                    self.__json_throughput(throughput),
+                    self.__json_line_rate(line_rate), ],
+                "parameters": [
+                    self.__json_port_config(port_config),
+                    self.__json_frame_size(frame_size), ]}
+            case_result.append(one_result)
+        self.logger.debug(pformat(suite_results))
+        self.__json_results[case_name] = suite_results
+
+    def __save_rfc2544_result(self, case_name, result):
+        suite_results = {}
+        case_result = suite_results[case_name] = []
+        for sub_result in result:
+            status, rfc2544, line_rate, port_config, frame_size = sub_result
+            one_result = {
+                "status": status,
+                "performance": [
+                    self.__json_rfc2544(rfc2544),
+                    self.__json_line_rate(line_rate), ],
+                "parameters": [
+                    self.__json_port_config(port_config),
+                    self.__json_frame_size(frame_size), ]}
+            case_result.append(one_result)
+        self.logger.debug(pformat(suite_results))
+        self.__json_results[case_name] = suite_results
+
+    def l3fwd_save_results(self, json_file=None):
+        if not self.__json_results:
+            msg = 'json results data is empty'
+            self.logger.error(msg)
+            return
+        _js_file = os.path.join(
+            self.output_path,
+            json_file if json_file else 'l3fwd_result.json')
+        with open(_js_file, 'w') as fp:
+            json.dump(self.__json_results, fp, indent=4,
+                      separators=(',', ': '),
+                      encoding="utf-8", sort_keys=True)
+
+    def __display_suite_result(self, data, mode):
+        values = data.get('values')
+        title = data.get('title')
+        max_length = sum([len(item) + 5 for item in title])
+        self.result_table_create(title)
+        self._result_table.table = texttable.Texttable(max_width=max_length)
+        for value in values:
+            self.result_table_add(value)
+        self.result_table_print()
+
+    def __check_throughput_result(self, stm_name, data, mode):
+        if not data:
+            msg = 'no result data'
+            raise VerifyFailure(msg)
+        values = []
+        js_results = []
+        bias = float(self.__test_content.get('accepted_tolerance') or 1.0)
+        for sub_data in data:
+            config, frame_size, result = sub_data
+            _, pps = result
+            pps /= 1000000.0
+            _frame_size = self.__get_frame_size(stm_name, frame_size)
+            linerate = self.wirespeed(
+                self.nic, _frame_size, len(self.__valports))
+            percentage = pps * 100 / linerate
+            # data for display
+            values.append(
+                [config, frame_size, mode.upper(), str(pps), str(percentage)])
+            # check data with expected values
+            expected_rate = None if not self.__cur_case else \
+                self.__test_content.get('expected_throughput', {}).get(
+                    self.__cur_case, {}).get(self.__nic_name, {}).get(
+                        config, {}).get(str(frame_size))
+            if expected_rate and float(expected_rate):
+                expected = float(expected_rate)
+                gap = 100 * (pps - expected) / expected
+                if abs(gap) < bias:
+                    status = 'pass'
+                else:
+                    status = 'failed'
+                    msg = ('expected <{}>, '
+                           'current <{}> is '
+                           '{}% over accepted tolerance').format(
+                        expected, pps, round(gap, 2))
+                    self.logger.error(msg)
+            else:
+                msg = ('{0} {1} expected throughput value is not set, '
+                       'ignore check').format(config, frame_size)
+                self.logger.warning(msg)
+                status = 'pass'
+            js_results.append([status, result, linerate, config, frame_size])
+        # save data with json format
+        self.__save_throughput_result(self.__cur_case, js_results)
+        # display result table
+        title = [
+            'Total Cores/Threads/Queues per port',
+            'Frame Size',
+            "Mode",
+            'Throughput Rate {} Mode mpps'.format(mode.upper()),
+            'Throughput Rate {} Mode Linerate%'.format(mode.upper()), ]
+
+        _data = {
+            'title': title,
+            'values': values}
+        self.__display_suite_result(_data, mode)
+
+    def __check_rfc2544_result(self, stm_name, data, mode):
+        if not data:
+            msg = 'no result data'
+            raise Exception(msg)
+        bias = self.__test_content.get('accepted_tolerance')
+        values = []
+        js_results = []
+        for sub_data in data:
+            config, frame_size, result = sub_data
+            expected_cfg = {} if not self.__cur_case else \
+                self.__test_content.get('expected_rfc2544', {}).get(
+                self.__cur_case, {}).get(self.__nic_name, {}).get(
+                config, {}).get(str(frame_size), {})
+            zero_loss_rate, tx_pkts, rx_pkts = result if result else [None] * 3
+            # expected line rate
+            _frame_size = self.__get_frame_size(stm_name, frame_size)
+            linerate = self.wirespeed(
+                self.nic, _frame_size, len(self.__valports))
+            throughput = linerate * zero_loss_rate / 100
+            # append data for display
+            pdr = expected_cfg.get('traffic_opt', {}).get('pdr')
+            values.append([
+                config, frame_size, mode.upper(),
+                str(throughput),
+                str(zero_loss_rate),
+            ])
+            # check data with expected values
+            expected_rate = float(expected_cfg.get('rate') or 100.0)
+            status = 'pass' \
+                if zero_loss_rate and zero_loss_rate > expected_rate \
+                else 'failed'
+            js_results.append(
+                [status, [zero_loss_rate, 0], linerate, config, frame_size])
+        # save data in json file
+        self.__save_rfc2544_result(self.__cur_case, js_results)
+        # display result table
+        title = [
+            'Total Cores/Threads/Queues per port',
+            "Frame Size",
+            "Mode",
+            'Theory line rate (Mpps) '.format(mode.upper()),
+            '{} Mode Zero Loss Rate % '.format(mode.upper()),
+        ]
+
+        _data = {
+            'title': title,
+            'values': values}
+        self.__display_suite_result(_data, mode)
+
+    def ms_throughput(self, l3_proto, mode):
+        except_content = None
+        try:
+            test_content = self.__test_content.get('port_configs')
+            results = []
+            for config, core_mask, port_conf, frame_size in test_content:
+                # Start L3fwd application
+                self.logger.info(
+                    ("Executing l3fwd with {0} mode, {1} ports, "
+                     "{2} and {3} frame size").format(
+                        mode, len(self.__valports), config, frame_size))
+                self.__start_l3fwd(mode, core_mask, port_conf, frame_size)
+                result = self.__throughput(l3_proto, mode, frame_size)
+                # Stop L3fwd
+                self.__close_l3fwd()
+                if result:
+                    results.append([config, frame_size, result])
+            self.__check_throughput_result(l3_proto, results, mode)
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.__close_l3fwd()
+
+        # re-raise verify exception result
+        if except_content:
+            raise VerifyFailure(except_content)
+
+    def qt_rfc2544(self, l3_proto, mode):
+        except_content = None
+        try:
+            test_content = self.__test_content.get('port_configs')
+            results = []
+            for config, core_mask, port_conf, frame_size in test_content:
+                # Start L3fwd application
+                self.logger.info(
+                    ("Executing l3fwd with {0} mode, {1} ports, "
+                     "{2} and {3} frame size").format(
+                        mode, len(self.__valports), config, frame_size))
+                self.__start_l3fwd(mode, core_mask, port_conf, frame_size)
+                result = self.__rfc2544(config, l3_proto, mode, frame_size)
+                # Stop L3fwd
+                self.__close_l3fwd()
+                if result:
+                    results.append([config, frame_size, result])
+            self.__check_rfc2544_result(l3_proto, results, mode)
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            except_content = e
+        finally:
+            self.__close_l3fwd()
+
+        # re-raise verify exception result
+        if except_content:
+            raise VerifyFailure(except_content)
+
+    def __parse_port_config(self, config):
+        cores, total_threads, queue = config.split('/')
+        _thread = str(int(total_threads[:-1]) / int(cores[:-1])) + 'T'
+        _cores = str(int(cores[:-1]) * len(self.__valports)) + 'C'
+        # only use one socket
+        cores_config = '/'.join(['1S', _cores, _thread])
+        queues_per_port = int(queue[:-1])
+        return cores_config, queues_per_port
+
+    def __get_test_configs(self, options, ports, socket):
+        if not options:
+            msg = "'test_parameters' not set in suite configuration file"
+            raise VerifyFailure(msg)
+        configs = []
+        frame_sizes_grp = []
+        for test_item, frame_sizes in sorted(options.iteritems()):
+            _frame_sizes = [int(frame_size) for frame_size in frame_sizes]
+            frame_sizes_grp.extend([int(item) for item in _frame_sizes])
+            cores, queues_per_port = self.__parse_port_config(test_item)
+            grp = [list(item)
+                   for item in product(range(queues_per_port), range(ports))]
+            corelist = self.dut.get_core_list(
+                cores, socket if cores.startswith('1S') else -1)
+            corelist = [str(int(core) + 2) for core in corelist]
+            cores_mask = utils.create_mask(corelist)
+            total = len(grp)
+            _corelist = (corelist * (total // len(corelist) + 1))[:total]
+            # ignore first 2 cores
+            [grp[index].append(core)
+             for index, core in enumerate(_corelist)]
+            # (port,queue,lcore)
+            [configs.append([
+                test_item,
+                cores_mask,
+                ','.join(["({0},{1},{2})".format(port, queue, core)
+                          for queue, port, core in grp]),
+                frame_size, ]) for frame_size in _frame_sizes]
+        return configs, sorted(set(frame_sizes_grp))
+
+    def __get_test_content_from_cfg(self, test_content):
+        self.logger.debug(pformat(test_content))
+        # get flows configuration
+        suite_conf = SuiteConf('l3fwd_base')
+        flows = suite_conf.suite_cfg.get('l3fwd_flows')
+        test_content['flows'] = flows
+        # parse port config of l3fwd
+        port_configs, frame_sizes = self.__get_test_configs(
+            test_content.get('test_parameters'),
+            len(self.__valports), self.__socket)
+        test_content['port_configs'] = port_configs
+        test_content['frame_sizes'] = frame_sizes
+        self.logger.debug(pformat(test_content))
+
+        return test_content
+
+    def l3fwd_preset_test_environment(self, test_content):
+        # get test content
+        self.__test_content = self.__get_test_content_from_cfg(test_content)
+        # binary process flag
+        self.__is_l3fwd_on = None
+        # prepare target source code application
+        self.__preset_compilation()
+        # config streams
+        self.__streams = self.__preset_streams()
+
+    def l3fwd_set_cur_case(self, name):
+        self.__cur_case = name
+
+    def l3fwd_reset_cur_case(self):
+        self.__cur_case = None
+
+    @property
+    def is_pktgen_on(self):
+        return hasattr(self.tester, 'is_pktgen') and self.tester.is_pktgen
+
+    @property
+    def pktgen_type(self):
+        if self.is_pktgen_on:
+            return self.tester.pktgen.pktgen_type
+        else:
+            return 'scapy'
+
+    def verify_ports_number(self, port_num):
+        supported_num = {
+            PKTGEN_TREX: [2, 4],
+            PKTGEN_IXIA: [1, 2, 4],
+        }
+        if not self.is_pktgen_on:
+            msg = 'not using pktgen'
+            self.logger.warning(msg)
+            return
+        # verify that enough ports are available
+        _supported_num = supported_num.get(self.pktgen_type)
+        msg = "Port number must be {} when using pktgen <{}>".format(
+            pformat(_supported_num), self.pktgen_type)
+        self.verify(len(port_num) in _supported_num, msg)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 3/13] framework/test_case: add wirespeed columbiaville nic
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (11 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 2/13] tests/l3fwd_base: " yufengmx
@ 2020-01-16  7:27 ` yufengmx
  2020-01-16  8:43 ` [dts] [PATCH V3 0/13] l3fwd: refactor script Tu, Lijuan
  13 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  7:27 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add wirespeed columbiaville nic.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/test_case.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/framework/test_case.py b/framework/test_case.py
index 0b91fed..282c96d 100644
--- a/framework/test_case.py
+++ b/framework/test_case.py
@@ -483,5 +483,9 @@ class TestCase(object):
             bitrate *= 10
         elif nic == 'fortville_25g':
             bitrate *=25
+        elif nic == 'columbiaville_25g':
+            bitrate *= 25
+        elif nic == 'columbiaville_100g':
+            bitrate *= 100
 
         return bitrate * num_ports / 8 / (frame_size + 20)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dts] [PATCH V3 0/13] l3fwd: refactor script
  2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
                   ` (12 preceding siblings ...)
  2020-01-16  7:27 ` [dts] [PATCH V3 3/13] framework/test_case: add wirespeed columbiaville nic yufengmx
@ 2020-01-16  8:43 ` Tu, Lijuan
  13 siblings, 0 replies; 16+ messages in thread
From: Tu, Lijuan @ 2020-01-16  8:43 UTC (permalink / raw)
  To: Mo, YufengX, dts

Applied the series.
thanks

> -----Original Message-----
> From: Mo, YufengX
> Sent: Thursday, January 16, 2020 3:27 PM
> To: dts@dpdk.org; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts][PATCH V3 0/13] l3fwd: refactor script
> 
> refactor script.
> 
> 
> v4:
>  - rebase for merge conflict.
>  -
> 
> v3:
>  - support one port testing topo for ixia.
>  - add test case name in cfg file.
>  - rename module l3fwd.py to l3fwd_base.py.
>  - rename common_l3fwd.cfg to l3fwd_base.cfg.
>  - rename verify method names.
>  - rename test scripts suite name.
>  -
> 
> v2:
>  - update script based on version 2 l3fwd test plan.
>  - separate script into five suites for different testing scenario.
>  - separate flows configuration in common_l3fwd.cfg.
>  - add test result data check process as new test plan requirement.
>  - save test results in json format file.
>  -
> 
> v1:
>  - optimize packet creation time consumption.
>  - separate LPM/EM mode packet creation process.
>  - use new pktgen coding style to take the place of migration coding style.
>  - unify rfc2544/throughput result data display table methods.
>  - move test content configuration into cfg file, including flows/EM table/LPM
> table/framesizes/port configs.
> 
> yufengmx (13):
>   conf/l3fwd: l3fwd suite testing configuration
>   conf/l3fwd_base: l3fwd base class flows configuration
>   conf/l3fwd_em: suite testing configuration
>   conf/l3fwd_lpm_ipv4_rfc2544: suite testing configuration
>   conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 testing configuration
>   conf/l3fwd_lpm_ipv6: suite testing configuration
>   tests/l3fwd: upload automation script
>   tests/l3fwd_em: upload automation script
>   tests/l3fwd_lpm_ipv4_rfc2544: upload automation script
>   tests/l3fwd_lpm_ipv4: upload automation script
>   tests/l3fwd_lpm_ipv6: upload automation script
>   tests/l3fwd_base: upload automation script
>   framework/test_case: add wirespeed columbiaville nic
> 
>  conf/l3fwd.cfg                            | 400 ++++++++++++
>  conf/l3fwd_base.cfg                       | 103 +++
>  conf/l3fwd_em.cfg                         | 147 +++++
>  conf/l3fwd_lpm_ipv4.cfg                   | 105 +++
>  conf/l3fwd_lpm_ipv4_rfc2544.cfg           | 105 +++
>  conf/l3fwd_lpm_ipv6.cfg                   | 105 +++
>  framework/test_case.py                    |   4 +
>  tests/TestSuite_l3fwd.py                  | 370 ++---------
>  tests/TestSuite_l3fwd_em.py               | 258 +-------
>  tests/TestSuite_l3fwd_lpm_ipv4.py         |  86 +++
>  tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py |  86 +++
>  tests/TestSuite_l3fwd_lpm_ipv6.py         |  85 +++
>  tests/l3fwd_base.py                       | 744 ++++++++++++++++++++++
>  13 files changed, 2044 insertions(+), 554 deletions(-)  create mode 100644
> conf/l3fwd.cfg  create mode 100644 conf/l3fwd_base.cfg  create mode
> 100644 conf/l3fwd_em.cfg  create mode 100644 conf/l3fwd_lpm_ipv4.cfg
> create mode 100644 conf/l3fwd_lpm_ipv4_rfc2544.cfg  create mode 100644
> conf/l3fwd_lpm_ipv6.cfg  create mode 100644
> tests/TestSuite_l3fwd_lpm_ipv4.py  create mode 100644
> tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
>  create mode 100644 tests/TestSuite_l3fwd_lpm_ipv6.py  create mode
> 100644 tests/l3fwd_base.py
> 
> --
> 2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dts] [PATCH V3 5/13] conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 testing configuration
  2020-01-16  5:28 yufengmx
@ 2020-01-16  5:28 ` yufengmx
  0 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2020-01-16  5:28 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add l3fwd_lpm_ipv4 suite testing configuration.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_lpm_ipv4.cfg | 105 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100644 conf/l3fwd_lpm_ipv4.cfg

diff --git a/conf/l3fwd_lpm_ipv4.cfg b/conf/l3fwd_lpm_ipv4.cfg
new file mode 100644
index 0000000..5935c65
--- /dev/null
+++ b/conf/l3fwd_lpm_ipv4.cfg
@@ -0,0 +1,105 @@
+# Throughput numbers vary in different environment.
+# Users could change these configuration on demand:
+#
+#  - test_parameters defines the combination of frame size and descriptor
+# numbers, and the pattern is
+#   {
+#       'Numbers of Cores/Sockets/Queues #1': ['frame_size #1', 'frame_size #2',...],
+#       'Numbers of Cores/Sockets/Queues #2': ['frame_size #1', 'frame_size #2',...],
+#       ......
+#   }
+#
+#  - test_duration is how many seconds each combination performance will
+#  be recorded, default is 60s
+#
+#  - accepted_tolerance defines the accepted tolerance between test
+# results and expected numbers, unit is percentage, (actual number - expected number)/expected number/100
+#
+#  - expected_throughput is a dictionary defining expected throughput
+# numbers based on NIC, and the pattern is
+#   {
+#       'columbiaville_100g': 
+#           {
+#               'Numbers of Cores/Sockets/Queues #1':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#               'Numbers of Cores/Sockets/Queues #2':
+#                   {
+#                       'frame_size #1': 'expected number',
+#                       'frame_size #2': 'expected number',
+#                       ...
+#                   },
+#           }
+#       ......
+#   }
+# Every user should fill it out with your actual numbers. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+#==========this feature supported is P4.======================
+#  - if update_expected == Ture, and add argument "--update-expected" in
+# bash command, all objects in this file will changed after the run
+# for example: ./dts --update-expected
+# Highlights:
+# At the begining, please change test_parameters according to your
+# requirements, then run ./dts --update-expected to get the absolute
+# results which will replace the default numbers 0.00 in this configuration.
+# So you will have your own private configuration, and could start your
+# tests as ususal.
+
+[suite]
+update_expected = True
+
+test_duration = 60
+accepted_tolerance = 1
+test_parameters = {
+    '1C/1T/1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T/2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T/4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+expected_throughput = {
+    'test_perf_throughput_ipv4_lpm': {
+        'niantic': {
+            '1C/1T/1Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '1C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/2T/2Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '2C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },
+            '4C/4T/4Q': {
+                 '64': '0.00',
+                 '128': '0.00',
+                 '256': '0.00',
+                 '512': '0.00',
+                 '1024': '0.00',
+                 '1280': '0.00',
+                 '1518': '0.00', },},}, }
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2020-01-16  8:43 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-16  7:27 [dts] [PATCH V3 0/13] l3fwd: refactor script yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 1/13] conf/l3fwd: l3fwd suite testing configuration yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 2/13] conf/l3fwd_base: l3fwd base class flows configuration yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 3/13] conf/l3fwd_em: suite testing configuration yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 4/13] conf/l3fwd_lpm_ipv4_rfc2544: " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 5/13] conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 6/13] conf/l3fwd_lpm_ipv6: suite " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 7/13] tests/l3fwd: upload automation script yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 8/13] tests/l3fwd_em: " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 9/13] tests/l3fwd_lpm_ipv4_rfc2544: " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 0/13] tests/l3fwd_lpm_ipv4: " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 1/13] tests/l3fwd_lpm_ipv6: " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 2/13] tests/l3fwd_base: " yufengmx
2020-01-16  7:27 ` [dts] [PATCH V3 3/13] framework/test_case: add wirespeed columbiaville nic yufengmx
2020-01-16  8:43 ` [dts] [PATCH V3 0/13] l3fwd: refactor script Tu, Lijuan
  -- strict thread matches above, loose matches on Subject: below --
2020-01-16  5:28 yufengmx
2020-01-16  5:28 ` [dts] [PATCH V3 5/13] conf/l3fwd_lpm_ipv4: l3fwd_lpm_ipv4 testing configuration yufengmx

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).