test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing
@ 2021-01-25  8:43 yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 01/27] framework/pktgen: return trex tx stats yufengmx
                   ` (27 more replies)
  0 siblings, 28 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


v1: 
 - enable IxNetwork packet generator. 
 - add testpmd rfc2544 testing scenario. 
 - enhance pktgen/trex to support return tx_pps and tx_bps from pktgen.measure_throughput. 

yufengmx (27):
  framework/pktgen: return trex tx stats
  framework/pktgen: return throughput tx stats
  framework/pktgen: return throughput tx stats
  conf/pktgen: enable ixNetwork
  conf/pktgen: enable ixNetwork
  conf/l3fwd: add packet types comment
  conf/testpmd: testpmd perf config
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  conf/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  framework/pktgen: enable ixNetwork
  tests/perf_test: rename l3fwd_base module
  tests/perf_test: cover testpmd testing scenario
  tests/perf_test: save rfc2544 expected throughput
  tests/l3fwd_em: update script
  tests/lpm_ipv4_rfc2544: update script
  tests/l3fwd_lpm_ipv4: update script
  tests/l3fwd_lpm_ipv6: update script
  tests/l3fwd: update script
  tests/vf_l3fwd_kernelpf: update script
  tests/testpmd_perf: upload script.

 conf/crbs.cfg                              |   2 +-
 conf/l3fwd_base.cfg                        |   2 +
 conf/pktgen.cfg                            |  12 +
 conf/testpmd_perf.cfg                      | 190 +++++
 framework/config.py                        |   9 +-
 framework/ixia_network/__init__.py         | 182 +++++
 framework/ixia_network/ixnet.py            | 844 +++++++++++++++++++++
 framework/ixia_network/ixnet_config.py     |  42 +
 framework/ixia_network/ixnet_stream.py     | 317 ++++++++
 framework/ixia_network/packet_parser.py    |  93 +++
 framework/pktgen.py                        |   9 +-
 framework/pktgen_base.py                   | 103 ++-
 framework/pktgen_ixia_network.py           | 224 ++++++
 framework/pktgen_trex.py                   |   4 +-
 framework/settings.py                      |   5 +-
 tests/TestSuite_l3fwd.py                   |  45 +-
 tests/TestSuite_l3fwd_em.py                |  20 +-
 tests/TestSuite_l3fwd_lpm_ipv4.py          |  16 +-
 tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py  |  16 +-
 tests/TestSuite_l3fwd_lpm_ipv6.py          |  16 +-
 tests/TestSuite_testpmd_perf.py            | 102 +++
 tests/TestSuite_vf_l3fwd_kernelpf.py       |  16 +-
 tests/{l3fwd_base.py => perf_test_base.py} | 483 ++++++++----
 23 files changed, 2512 insertions(+), 240 deletions(-)
 create mode 100644 conf/testpmd_perf.cfg
 create mode 100644 framework/ixia_network/__init__.py
 create mode 100644 framework/ixia_network/ixnet.py
 create mode 100644 framework/ixia_network/ixnet_config.py
 create mode 100644 framework/ixia_network/ixnet_stream.py
 create mode 100644 framework/ixia_network/packet_parser.py
 create mode 100644 framework/pktgen_ixia_network.py
 create mode 100644 tests/TestSuite_testpmd_perf.py
 rename tests/{l3fwd_base.py => perf_test_base.py} (75%)

-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 01/27] framework/pktgen: return trex tx stats
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 02/27] framework/pktgen: return throughput " yufengmx
                   ` (26 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


support return tx_pps and tx_bps from pktgen.measure_throughput for trex.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_trex.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py
index 8da0c849..c0b13428 100644
--- a/framework/pktgen_trex.py
+++ b/framework/pktgen_trex.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -696,7 +696,7 @@ class TrexPacketGenerator(PacketGenerator):
         self.logger.debug(pformat(port_stats))
         self.logger.debug(os.linesep.join(msg))
 
-        return rx_bps, rx_pps
+        return (tx_bps, rx_bps), (tx_pps, rx_pps)
 
     def _loss_rate_stats(self, stream, stats):
         # tx packet
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 02/27] framework/pktgen: return throughput tx stats
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 01/27] framework/pktgen: return trex tx stats yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 03/27] " yufengmx
                   ` (25 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


support return tx_pps and tx_bps from pktgen.measure_throughput for trex.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_base.py | 77 ++++++++++++++++++++++++++++++----------
 1 file changed, 59 insertions(+), 18 deletions(-)

diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py
index 8003e01c..81c8830c 100644
--- a/framework/pktgen_base.py
+++ b/framework/pktgen_base.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -35,6 +35,7 @@ from abc import abstractmethod
 from copy import deepcopy
 from logger import getLogger
 from pprint import pformat
+from enum import Enum, unique
 
 from config import PktgenConf
 # packet generator name
@@ -46,6 +47,12 @@ TRANSMIT_M_BURST = 'multi_burst'
 TRANSMIT_S_BURST = 'single_burst'
 
 
+@unique
+class STAT_TYPE(Enum):
+    RX = 'rx'
+    TXRX = 'txrx'
+
+
 class PacketGenerator(object):
     """
     Basic class for packet generator, define basic function for each kinds of
@@ -106,8 +113,8 @@ class PacketGenerator(object):
         return port
 
     def add_stream(self, tx_port, rx_port, pcap_file):
-        pktgen_tx_port  = self._convert_tester_port(tx_port)
-        pktgen_rx_port  = self._convert_tester_port(rx_port)
+        pktgen_tx_port = self._convert_tester_port(tx_port)
+        pktgen_rx_port = self._convert_tester_port(rx_port)
 
         stream_id = len(self.__streams)
         stream = {'tx_port': pktgen_tx_port,
@@ -168,28 +175,50 @@ class PacketGenerator(object):
         time.sleep(delay)
         self._stop_transmission(stream_ids)
 
-    def __get_single_throughput_statistic(self, stream_ids):
+    def __get_single_throughput_statistic(self, stream_ids, stat_type=None):
         bps_rx = []
         pps_rx = []
+        bps_tx = []
+        pps_tx = []
         used_rx_port = []
         msg = 'begin get port statistic ...'
         self.logger.info(msg)
         for stream_id in stream_ids:
             if self.__streams[stream_id]['rx_port'] not in used_rx_port:
-                rxbps_rates, rxpps_rates = self._retrieve_port_statistic(
+                bps_rate, pps_rate = self._retrieve_port_statistic(
                     stream_id, 'throughput')
                 used_rx_port.append(self.__streams[stream_id]['rx_port'])
-                bps_rx.append(rxbps_rates)
-                pps_rx.append(rxpps_rates)
-        bps_rx_total = self._summary_statistic(bps_rx)
-        pps_rx_total = self._summary_statistic(pps_rx)
-        self.logger.info(
-            "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
+                if stat_type and stat_type is STAT_TYPE.TXRX:
+                    bps_tx.append(bps_rate[0])
+                    pps_tx.append(pps_rate[0])
+
+                if isinstance(bps_rate, tuple) and isinstance(pps_rate, tuple):
+                    bps_rx.append(bps_rate[1])
+                    pps_rx.append(pps_rate[1])
+                else:
+                    bps_rx.append(bps_rate)
+                    pps_rx.append(pps_rate)
+        if stat_type and stat_type is STAT_TYPE.TXRX:
+            bps_tx_total = self._summary_statistic(bps_tx)
+            pps_tx_total = self._summary_statistic(pps_tx)
+            bps_rx_total = self._summary_statistic(bps_rx)
+            pps_rx_total = self._summary_statistic(pps_rx)
+            self.logger.info(
+                "throughput: pps_tx %f, bps_tx %f" % (pps_tx_total, bps_tx_total))
+            self.logger.info(
+                "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
+
+            return (bps_tx_total, bps_rx_total), (pps_tx_total, pps_rx_total)
+        else:
+            bps_rx_total = self._summary_statistic(bps_rx)
+            pps_rx_total = self._summary_statistic(pps_rx)
+            self.logger.info(
+                "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
 
-        return bps_rx_total, pps_rx_total
+            return bps_rx_total, pps_rx_total
 
     def __get_multi_throughput_statistic(
-            self, stream_ids, duration, interval, callback=None):
+            self, stream_ids, duration, interval, callback=None, stat_type=None):
         """
         duration: traffic duration (second)
         interval: interval of get throughput statistics (second)
@@ -202,7 +231,7 @@ class PacketGenerator(object):
         stats = []
         while time_elapsed < duration:
             time.sleep(interval)
-            stats.append(self.__get_single_throughput_statistic(stream_ids))
+            stats.append(self.__get_single_throughput_statistic(stream_ids, stat_type))
             if callback and callable(callback):
                 callback()
             time_elapsed += interval
@@ -237,23 +266,35 @@ class PacketGenerator(object):
 
             duration:
                 traffic lasting time(second). Default value is 10 second.
+
+            stat_type(for trex only):
+                STAT_TYPE.RX  return (rx bps, rx_pps)
+                STAT_TYPE.TXRX return ((tx bps, rx_bps), (tx pps, rx_pps))
         """
         interval = options.get('interval')
         callback = options.get('callback')
         duration = options.get('duration') or 10
         delay = options.get('delay')
+        if self.pktgen_type == PKTGEN_TREX:
+            stat_type = options.get('stat_type') or STAT_TYPE.RX
+        else:
+            if options.get('stat_type') is not None:
+                msg = ("'stat_type' option is only for trex, "
+                       "should not set when use other pktgen tools")
+                raise Exception(msg)
+            stat_type = STAT_TYPE.RX
         self._prepare_transmission(stream_ids=stream_ids)
         # start warm up traffic
         self.__warm_up_pktgen(stream_ids, options, delay)
         # main traffic
-        self._start_transmission(stream_ids)
+        self._start_transmission(stream_ids, options)
         # keep traffic within a duration time and get throughput statistic
         if interval and duration:
             stats = self.__get_multi_throughput_statistic(
-                stream_ids, duration, interval, callback)
+                stream_ids, duration, interval, callback, stat_type)
         else:
             time.sleep(duration)
-            stats = self.__get_single_throughput_statistic(stream_ids)
+            stats = self.__get_single_throughput_statistic(stream_ids, stat_type)
         self._stop_transmission(stream_ids)
         return stats
 
@@ -366,7 +407,7 @@ class PacketGenerator(object):
             return True
 
     def measure_rfc2544(self, stream_ids=[], options={}):
-        """ check loss rate with rate percent dropping 
+        """ check loss rate with rate percent dropping
 
         options usage:
             rate:
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 03/27] framework/pktgen: return throughput tx stats
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 01/27] framework/pktgen: return trex tx stats yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 02/27] framework/pktgen: return throughput " yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 04/27] conf/pktgen: enable ixNetwork yufengmx
                   ` (24 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


support return tx_pps and tx_bps from pktgen.measure_throughput for trex.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/framework/pktgen.py b/framework/pktgen.py
index d00cf3bf..92d17784 100644
--- a/framework/pktgen.py
+++ b/framework/pktgen.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -42,7 +42,7 @@ from scapy.utils import rdpcap
 from utils import (convert_int2ip, convert_ip2int,
                    convert_mac2long, convert_mac2str)
 
-from pktgen_base import (PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA,
+from pktgen_base import (PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, STAT_TYPE,
                          TRANSMIT_CONT, TRANSMIT_M_BURST, TRANSMIT_S_BURST)
 from pktgen_base import DpdkPacketGenerator
 from pktgen_ixia import IxiaPacketGenerator
@@ -208,7 +208,7 @@ def getPacketGenerator(tester, pktgen_type=PKTGEN_IXIA):
     pktgen_cls = {
         PKTGEN_DPDK: DpdkPacketGenerator,
         PKTGEN_IXIA: IxiaPacketGenerator,
-        PKTGEN_TREX: TrexPacketGenerator,}
+        PKTGEN_TREX: TrexPacketGenerator, }
 
     if pktgen_type in list(pktgen_cls.keys()):
         CLS = pktgen_cls.get(pktgen_type)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 04/27] conf/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (2 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 03/27] " yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 05/27] " yufengmx
                   ` (23 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add ixNetwork configuration key.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/crbs.cfg | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/conf/crbs.cfg b/conf/crbs.cfg
index 2e6c9483..0ba9d1fc 100644
--- a/conf/crbs.cfg
+++ b/conf/crbs.cfg
@@ -7,7 +7,7 @@
 #  tester_ip: Tester ip address
 #  tester_passwd: Tester password
 #  ixia_group: IXIA group name
-#  pktgen_group: packet generator group name
+#  pktgen_group: packet generator group name: ixia/trex/ixia_network
 #  channels: Board channel number
 #  bypass_core0: Whether by pass core0
 #  dut_cores: DUT core list, eg: 1,2,3,4,5,18-22
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 05/27] conf/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (3 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 04/27] conf/pktgen: enable ixNetwork yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 06/27] conf/l3fwd: add packet types comment yufengmx
                   ` (22 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add ixNetwork configuration content.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/pktgen.cfg | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg
index e1114596..6e0825b7 100644
--- a/conf/pktgen.cfg
+++ b/conf/pktgen.cfg
@@ -45,3 +45,15 @@ ixia_ports=
     card=1,port=3;
     card=1,port=4;
 ixia_force100g=disable
+[IXIA_NETWORK]
+ixia_version=9.0
+ixia_ip=xxx.xxx.xxx.xxx
+# ixia network REST server ip address
+ixnet_api_server_ip=xxx.xxx.xxx.xxx
+# maximum re-run traffic times, default is 5 times. (optional)
+#max_retry=5
+ixia_ports=
+    card=1,port=1;
+    card=1,port=2;
+    card=1,port=3;
+    card=1,port=4;
\ No newline at end of file
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 06/27] conf/l3fwd: add packet types comment
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (4 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 05/27] " yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 07/27] conf/testpmd: testpmd perf config yufengmx
                   ` (21 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add packet types comment.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/l3fwd_base.cfg | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/conf/l3fwd_base.cfg b/conf/l3fwd_base.cfg
index ac2ab261..8d281144 100644
--- a/conf/l3fwd_base.cfg
+++ b/conf/l3fwd_base.cfg
@@ -1,3 +1,5 @@
+# stream_type=  "tcp"/"udp"/"raw". If not set this key, default is "raw"
+#
 [suite]
 l3fwd_flows = {
     "ipv4": {
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 07/27] conf/testpmd: testpmd perf config
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (5 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 06/27] conf/l3fwd: add packet types comment yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 08/27] framework/pktgen: enable ixNetwork yufengmx
                   ` (20 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload testpmd_perf suite config file.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 conf/testpmd_perf.cfg | 190 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 190 insertions(+)
 create mode 100644 conf/testpmd_perf.cfg

diff --git a/conf/testpmd_perf.cfg b/conf/testpmd_perf.cfg
new file mode 100644
index 00000000..a39d3604
--- /dev/null
+++ b/conf/testpmd_perf.cfg
@@ -0,0 +1,190 @@
+# Users could change these configuration here:
+#
+# - compile_rx_desc
+#     compile switch flag CONFIG_RTE_LIBRTE_<nic driver>_<number>BYTE_RX_DESC, support 32/16
+#
+#     e.g.
+#         compile_rx_desc=16
+#
+# - compile_avx
+#     compile switch flag CONFIG_RTE_ENABLE_<XXXX>, support 'AVX512' or 'AVX2'
+#
+#     e.g.
+#         compile_avx='AVX512' or 'AVX2'
+#
+# - forwarding_mode
+#     testpmd set the packet forwarding mode, support 'io'/'mac'
+#
+#     e.g.
+#         forwarding_mode = 'io'
+#
+# - descriptor_numbers
+#    testpmd TX/RX descriptor for a port TX/RX queue
+#
+#     e.g.
+#         descriptor_numbers = {
+#           'txd': 2048,
+#           'rxd': 2048, }
+#
+# - cores_for_all
+#     True:  'Numbers of Cores/Thread-Queues #1' define for all ports.
+#     False: 'Numbers of Cores/Thread-Queues #1' define for each port.
+#
+#     e.g.
+#         cores_for_all=True
+#
+# - port_list
+#     defines which ports are used to run testing If not set this key, use default dut port list of dts framework.
+#
+#     e.g.
+#         port_list=[0,1,...]
+#
+# - test_parameters
+#     defines the combination of frame size and hardware resources, and the pattern is
+#
+#    {
+#      'Numbers of Cores/Thread-Queues #1': ['frame_size #1', 'frame_size #2',...],
+#      'Numbers of Cores/Thread-Queues #2': ['frame_size #1', 'frame_size #2',...],
+#      ......
+#    }
+#
+# - test_duration
+#     how long(seconds) each combination performance will be executed, default is 60s
+#
+# - accepted_tolerance
+#     defines the accepted tolerance between test results and expected values, unit is percentage
+#     (actual value - expected value)/expected value/100
+#
+# - expected_rfc2544
+#     a dictionary defining expected rfc2544 values based on NIC, and the pattern is
+#
+#     {
+#      'columbiaville_100g':
+#          {
+#              'values of Cores/Thread-Queues #1':
+#                  {
+#                      'frame_size #1': 'expected value',
+#                      'frame_size #2': 'expected value',
+#                      ...
+#                  },
+#              'values of Cores/Thread-Queues #2':
+#                  {
+#                      'frame_size #1': 'expected value',
+#                      'frame_size #2': 'expected value',
+#                      ...
+#                  },
+#          }
+#      ......
+#    }
+#
+#
+# Every user should fill it out with your actual values. To keep the
+# expected throughput private, dts takes 0.00 as default.
+#
+
+[suite]
+# compile_rx_desc=16
+# compile_avx='AVX512' or 'AVX2'
+# 'io', 'mac'
+forwarding_mode = 'io'
+descriptor_numbers = {
+    'txd': 2048,
+    'rxd': 2048, }
+
+# port_list=None
+cores_for_all=True
+test_duration = 60
+accepted_tolerance = 1
+
+test_parameters = {
+    '1C/1T-1Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '1C/2T-2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/2T-2Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '2C/4T-4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],
+    '4C/4T-4Q': ['64', '128', '256', '512', '1024', '1280', '1518',],}
+
+expected_rfc2544 = {
+    'test_perf_rfc2544_ipv4_lpm': {
+        'niantic': {
+            '1C/1T-1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T-2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T-2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T-4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T-4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},
+    'test_perf_rfc2544_ipv6_lpm': {
+        'niantic': {
+            '1C/1T-1Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '1C/2T-2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/2T-2Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '2C/4T-4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },
+            '4C/4T-4Q': {
+                '64': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '128': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '256': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '512': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1024': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1280': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } },
+                '1518': { 'rate' :'100.00', 'traffic_opt': {'min_rate': '10.0', 'max_rate': '100.0', 'pdr': '0.01', 'accuracy': '5', } } },},},}
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 08/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (6 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 07/27] conf/testpmd: testpmd perf config yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 09/27] " yufengmx
                   ` (19 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


create ixia_network package.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/ixia_network/__init__.py | 182 +++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
 create mode 100644 framework/ixia_network/__init__.py

diff --git a/framework/ixia_network/__init__.py b/framework/ixia_network/__init__.py
new file mode 100644
index 00000000..086fb5af
--- /dev/null
+++ b/framework/ixia_network/__init__.py
@@ -0,0 +1,182 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+"""
+ixNetwork package
+"""
+import os
+import time
+import traceback
+from pprint import pformat
+
+from .ixnet import IxnetTrafficGenerator
+from .ixnet_config import IxiaNetworkConfig
+
+__all__ = [
+    "IxNetwork",
+]
+
+
+class IxNetwork(IxnetTrafficGenerator):
+    """
+    ixNetwork performance measurement class.
+    """
+
+    def __init__(self, name, config, logger):
+        self.NAME = name
+        self.logger = logger
+        ixiaRef = self.NAME
+        if ixiaRef not in config:
+            return
+        _config = config.get(ixiaRef, {})
+        self.ixiaVersion = _config.get("Version")
+        self.ports = _config.get("Ports")
+        ixia_ip = _config.get("IP")
+        rest_server_ip = _config.get("ixnet_api_server_ip")
+        self.max_retry = int(_config.get("max_retry") or '5') # times
+        self.logger.debug(locals())
+        rest_config = IxiaNetworkConfig(
+            ixia_ip,
+            rest_server_ip,
+            '11009',
+            [[ixia_ip, p.get('card'), p.get('port')] for p in self.ports],
+            )
+        super(IxNetwork, self).__init__(rest_config, logger)
+        self._traffic_list = []
+        self._result = None
+
+    @property
+    def OUTPUT_DIR(self):
+        # get dts output folder path
+        if self.logger.log_path.startswith(os.sep):
+            output_path = self.logger.log_path
+        else:
+            cur_path = os.sep.join(
+                os.path.realpath(__file__).split(os.sep)[:-2])
+            output_path = os.path.join(cur_path, self.logger.log_path)
+        if not os.path.exists(output_path):
+            os.makedirs(output_path)
+
+        return output_path
+
+    def get_ports(self):
+        """
+        get ixNetwork ports for dts `ports_info`
+        """
+        plist = []
+        for p in self.ports:
+            plist.append({
+                'type': 'ixia',
+                'pci': 'IXIA:%d.%d' % (p['card'], p['port']),
+                })
+        return plist
+
+    def send_ping6(self, pci, mac, ipv6):
+        return '64 bytes from'
+
+    def disconnect(self):
+        ''' quit from ixNetwork api server '''
+        self.tear_down()
+        msg = 'close ixNetwork session done !'
+        self.logger.info(msg)
+
+    def prepare_ixia_network_stream(self, traffic_list):
+        self._traffic_list = []
+        for txPort, rxPort, pcapFile, option in traffic_list:
+            stream = self.configure_streams(pcapFile, option.get('fields_config'))
+            tx_p = self.tg_vports[txPort]
+            rx_p = self.tg_vports[rxPort]
+            self._traffic_list.append((tx_p, rx_p, stream))
+
+    def start(self, options):
+        ''' start ixNetwork measurement '''
+        test_mode = options.get('method')
+        options['traffic_list'] = self._traffic_list
+        self.logger.debug(pformat(options))
+        if test_mode == 'rfc2544_dichotomy':
+            cnt = 0
+            while cnt < self.max_retry:
+                try:
+                    result = self.send_rfc2544_throughput(options)
+                    if result:
+                        break
+                except Exception as e:
+                    msg = "failed to run rfc2544".format(cnt)
+                    self.logger.error(msg)
+                    self.logger.error(traceback.format_exc())
+                cnt += 1
+                msg = "No.{} rerun ixNetwork rfc2544".format(cnt)
+                self.logger.warning(msg)
+                time.sleep(10)
+            else:
+                result = []
+        else:
+            msg = "not support measurement {}".format(test_mode)
+            self.logger.error(msg)
+            self._result = None
+            return None
+        self.logger.info('measure <{}> completed'.format(test_mode))
+        self.logger.info(result)
+        self._result = result
+        return result
+
+    def get_rfc2544_stat(self, port_list):
+        """
+        Get RX/TX packet statistics.
+        """
+        if not self._result:
+            return [0] * 3
+
+        result = self._result
+        _ixnet_stats = {}
+        for item in result:
+            port_id = int(item.get('Trial')) - 1
+            _ixnet_stats[port_id] = dict(item)
+        port_stat = _ixnet_stats.get(0, {})
+        rx_packets = float(port_stat.get('Agg Rx Count (frames)') or '0.0')
+        tx_packets = float(port_stat.get('Agg Tx Count (frames)') or '0.0')
+        rx_pps = float(port_stat.get('Agg Rx Throughput (fps)') or '0.0')
+        return tx_packets, rx_packets, rx_pps
+
+    def get_stats(self, ports, mode):
+        '''
+        get statistics of custom mode
+        '''
+        methods = {
+            'rfc2544': self.get_rfc2544_stat,
+        }
+        if mode not in list(methods.keys()):
+            msg = "not support mode <{0}>".format(mode)
+            raise Exception(msg)
+        # get custom mode stat
+        func = methods.get(mode)
+        stats = func(ports)
+
+        return stats
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 09/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (7 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 08/27] framework/pktgen: enable ixNetwork yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 10/27] " yufengmx
                   ` (18 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


ixNetwork packet generator tool configuration content.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/ixia_network/ixnet_config.py | 42 ++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 framework/ixia_network/ixnet_config.py

diff --git a/framework/ixia_network/ixnet_config.py b/framework/ixia_network/ixnet_config.py
new file mode 100644
index 00000000..5c6aea46
--- /dev/null
+++ b/framework/ixia_network/ixnet_config.py
@@ -0,0 +1,42 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+"""
+Misc functions.
+"""
+
+from typing import List, NamedTuple
+
+
+class IxiaNetworkConfig(NamedTuple):
+    ixia_ip: str
+    tg_ip: str
+    tg_ip_port: str
+    tg_ports: List
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 10/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (8 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 09/27] " yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 11/27] " yufengmx
                   ` (17 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


set ixNetwork stream data format.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/ixia_network/ixnet_stream.py | 317 +++++++++++++++++++++++++
 1 file changed, 317 insertions(+)
 create mode 100644 framework/ixia_network/ixnet_stream.py

diff --git a/framework/ixia_network/ixnet_stream.py b/framework/ixia_network/ixnet_stream.py
new file mode 100644
index 00000000..56cd53b6
--- /dev/null
+++ b/framework/ixia_network/ixnet_stream.py
@@ -0,0 +1,317 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import os
+import json
+
+from utils import convert_int2ip, convert_ip2int
+
+
+class IxnetConfigStream(object):
+
+    def __init__(self, packetLayers, field_config=None, frame_size=64,
+                 trafficItem=1, configElement=1):
+        self.traffic_item_id = f'trafficItem[{trafficItem}]'
+        self.config_element_id = f'configElement[{configElement}]'
+
+        self.packetLayers = packetLayers
+        self.layer_names = [name for name in packetLayers]
+        self.field_config = field_config or {}
+        self.frame_size = frame_size
+
+    def action_key(self, action):
+        if not action:
+            msg = "action not set !!!"
+            print(msg)
+
+        ret = {
+            'inc': 'increment',
+            'dec': 'decrement',
+            }.get(action or 'inc')
+
+        if ret:
+            msg = f'action <{action}> not supported, using increment action now'
+            print(msg)
+
+        return ret or 'increment'
+
+    @property
+    def ethernet(self):
+        layer_name = 'Ethernet'
+        default_config = self.packetLayers.get(layer_name)
+
+        index = self.layer_names.index(layer_name) + 1
+        tag = f'{layer_name.lower()}-{index}'
+
+        src_mac = default_config.get('src')
+        dst_mac = default_config.get('dst')
+        # mac src config
+        src_config = {
+            "singleValue": src_mac}
+        src_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'ethernet.header.sourceAddress-2']"
+        # mac dst config
+        dst_config = {
+            "singleValue": dst_mac}
+        dst_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'ethernet.header.destinationAddress-1']"
+        # ixNetwork stream configuration table
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']",
+            "field": [
+                src_config,
+                dst_config,
+            ]
+        }
+        return element
+
+    @property
+    def ip(self):
+        layer_name = 'IP'
+        default_config = self.packetLayers.get(layer_name)
+        vm_config = self.field_config.get(layer_name.lower()) or {}
+
+        index = self.layer_names.index(layer_name) + 1
+        tag = f'ipv4-{index}'
+
+        src_ip = default_config.get('src')
+        dst_ip = default_config.get('dst')
+
+        # ip src config
+        ip_src_vm = vm_config.get('src', {})
+        start_ip = ip_src_vm.get('start') or src_ip
+        end_ip = ip_src_vm.get('end') or '255.255.255.255'
+        src_config = {
+            "startValue": start_ip,
+            "stepValue": convert_int2ip(ip_src_vm.get('step')) or '0.0.0.1',
+            "countValue": str(abs(convert_ip2int(end_ip) - convert_ip2int(start_ip)) + 1),
+            "valueType": self.action_key(ip_src_vm.get('action')), } \
+            if ip_src_vm else \
+            {"singleValue": src_ip}
+        src_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'ipv4.header.srcIp-27']"
+        # ip dst config
+        ip_dst_vm = vm_config.get('dst', {})
+        start_ip = ip_dst_vm.get('start') or dst_ip
+        end_ip = ip_dst_vm.get('end') or '255.255.255.255'
+        dst_config = {
+            "startValue": start_ip,
+            "stepValue": convert_int2ip(ip_dst_vm.get('step')) or '0.0.0.1',
+            "countValue": str(abs(convert_ip2int(end_ip) - convert_ip2int(start_ip)) + 1),
+            "valueType": self.action_key(ip_dst_vm.get('action')), } \
+            if ip_dst_vm else \
+            {"singleValue": dst_ip}
+        dst_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'ipv4.header.dstIp-28']"
+        # ixNetwork stream configuration table
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']",
+            "field": [
+                src_config,
+                dst_config,
+            ]
+        }
+        return element
+
+    @property
+    def ipv6(self):
+        layer_name = 'IPv6'
+        default_config = self.packetLayers.get(layer_name)
+        vm_config = self.field_config.get(layer_name.lower()) or {}
+
+        index = self.layer_names.index(layer_name) + 1
+        tag = f'{layer_name.lower()}-{index}'
+
+        src_ip = default_config.get('src')
+        dst_ip = default_config.get('dst')
+        # ip src config
+        ip_src_vm = vm_config.get('src', {})
+        start_ip = ip_src_vm.get('start') or src_ip
+        end_ip = ip_src_vm.get('end') or 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff'
+        src_config = {
+            "startValue": start_ip,
+            "stepValue": convert_int2ip(ip_src_vm.get('step'), ip_type=6) or '0:0:0:0:0:0:0:1',
+            "countValue": str(min(abs(convert_ip2int(end_ip, ip_type=6) - convert_ip2int(start_ip, ip_type=6)) + 1, 2147483647)),
+            "valueType": self.action_key(ip_src_vm.get('action')), } \
+            if ip_src_vm else \
+            {"singleValue": src_ip}
+        header_src = "srcIP-7"
+        src_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'ipv6.header.{header_src}']"
+        # ip dst config
+        ip_dst_vm = vm_config.get('dst', {})
+        start_ip = ip_dst_vm.get('start') or dst_ip
+        end_ip = ip_dst_vm.get('end') or 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff'
+        dst_config = {
+            "startValue": start_ip,
+            "stepValue": convert_int2ip(ip_dst_vm.get('step'), ip_type=6) or '0:0:0:0:0:0:0:1',
+            "countValue": str(min(abs(convert_ip2int(end_ip, ip_type=6) - convert_ip2int(start_ip, ip_type=6)) + 1, 2147483647)),
+            "valueType": self.action_key(ip_dst_vm.get('action')), } \
+            if ip_dst_vm else \
+            {"singleValue": dst_ip}
+        header_dst = "dstIP-8"
+        dst_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'ipv6.header.{header_dst}']"
+        # ixNetwork stream configuration table
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']",
+            "field": [
+                src_config,
+                dst_config,
+            ]
+        }
+        return element
+
+    @property
+    def udp(self):
+        layer_name = 'UDP'
+        default_config = self.packetLayers.get(layer_name)
+
+        index = self.layer_names.index(layer_name) + 1
+        tag = f'{layer_name.lower()}-{index}'
+
+        sport = default_config.get('sport')
+        dport = default_config.get('dport')
+        # udp src config
+        src_config = {
+            "singleValue": str(sport)}
+        header_src = "srcPort-1"
+        src_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'udp.header.{header_src}']"
+        # udp dst config
+        dst_config = {
+            "singleValue": str(dport)}
+        header_dst = "dstPort-2"
+        dst_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'udp.header.{header_dst}']"
+        # ixNetwork stream configuration table
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']",
+            "field": [
+                src_config,
+                dst_config,
+            ]
+        }
+
+        return element
+
+    @property
+    def tcp(self):
+        layer_name = 'TCP'
+        default_config = self.packetLayers.get(layer_name)
+
+        index = self.layer_names.index(layer_name) + 1
+        tag = f'{layer_name.lower()}-{index}'
+
+        sport = default_config.get('sport')
+        dport = default_config.get('dport')
+        # tcp src config
+        src_config = {
+            "singleValue": str(sport)}
+        header_src = "srcPort-1"
+        src_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'tcp.header.{header_src}']"
+        # tcp dst config
+        dst_config = {
+            "singleValue": str(dport)}
+        header_dst = "dstPort-2"
+        dst_config["xpath"] = \
+            f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']/field[@alias = 'tcp.header.{header_dst}']"
+        # ixNetwork stream configuration table
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/stack[@alias = '{tag}']",
+            "field": [
+                src_config,
+                dst_config,
+            ]
+        }
+
+        return element
+
+    @property
+    def framePayload(self):
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/framePayload",
+            "type": "incrementByte",
+            "customRepeat": 'true',
+            "customPattern": "",
+        }
+        return element
+
+    @property
+    def stack(self):
+        element = [getattr(self, name.lower()) for name in self.packetLayers if name.lower() != 'raw']
+        return element
+
+    @property
+    def frameSize(self):
+        element = {
+            "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}/frameSize",
+            "fixedSize": self.frame_size,
+        }
+        return element
+
+    @property
+    def configElement(self):
+        element = [
+            {
+                "xpath": f"/traffic/{self.traffic_item_id}/{self.config_element_id}",
+                "stack": self.stack,
+                "frameSize": self.frameSize,
+                "framePayload": self.framePayload,
+            }
+        ]
+        return element
+
+    @property
+    def trafficItem(self):
+        element = [
+            {
+                "xpath": f"/traffic/{self.traffic_item_id}",
+                "configElement": self.configElement,
+            }
+        ]
+        return element
+
+    @property
+    def traffic(self):
+        element = {
+            "xpath": "/traffic",
+            "trafficItem": self.trafficItem,
+        }
+        return element
+
+    @property
+    def ixnet_packet(self):
+        element = {
+            "xpath": "/",
+            "traffic": self.traffic,
+        }
+        return element
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 11/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (9 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 10/27] " yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:43 ` [dts] [PATCH V1 12/27] " yufengmx
                   ` (16 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


ixNetwork api server restful interface.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/ixia_network/ixnet.py | 844 ++++++++++++++++++++++++++++++++
 1 file changed, 844 insertions(+)
 create mode 100644 framework/ixia_network/ixnet.py

diff --git a/framework/ixia_network/ixnet.py b/framework/ixia_network/ixnet.py
new file mode 100644
index 00000000..a8de15ca
--- /dev/null
+++ b/framework/ixia_network/ixnet.py
@@ -0,0 +1,844 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+"""
+This module implant from pei,yulong ixNetwork tool.
+"""
+
+import os
+import time
+import re
+import requests
+import json
+import csv
+from collections import OrderedDict
+from datetime import datetime
+
+
+# local lib deps
+from .packet_parser import PacketParser
+from .ixnet_stream import IxnetConfigStream
+
+
+class IxnetTrafficGenerator(object):
+    """ixNetwork Traffic Generator."""
+    json_header = {'content-type': 'application/json'}
+
+    def __init__(self, config, logger):
+        # disable SSL warnings
+        requests.packages.urllib3.disable_warnings()
+        self.logger = logger
+        self.tg_ip = config.tg_ip
+        self.tg_ports = config.tg_ports
+        port = config.tg_ip_port or '11009'
+        # id will always be 1 when using windows api server
+        self.api_server = 'http://{0}:{1}'.format(self.tg_ip, port)
+        self.session = requests.session()
+        self.session_id = self.get_session_id(self.api_server)
+        self.session_url = "{0}/api/v1/sessions/{1}".format(
+            self.api_server, self.session_id)
+        # initialize ixNetwork
+        self.new_blank_config()
+        self.tg_vports = self.assign_ports(self.tg_ports)
+
+    def get_session_id(self, api_server):
+        url = '{server}/api/v1/sessions'.format(server=api_server)
+        response = self.session.post(
+            url, headers=self.json_header, verify=False)
+        session_id = response.json()['links'][0]['href'].split('/')[-1]
+        msg = "{0}: Session ID is {1}".format(api_server, session_id)
+        self.logger.info(msg)
+        return session_id
+
+    def destroy_config(self, name):
+        json_header = {
+            'content-type': 'application/json',
+            'X-HTTP-Method-Override': 'DELETE',
+        }
+        response = self.session.post(name, headers=json_header, verify=False)
+        return response
+
+    def __get_ports(self):
+        """Return available tg vports list"""
+        return self.tg_vports
+
+    def disable_port_misdirected(self):
+        msg = 'close mismatched flag'
+        self.logger.debug(msg)
+        url = "{0}/ixnetwork/traffic".format(self.session_url)
+        data = {
+            "detectMisdirectedOnAllPorts": False,
+            "disablePortLevelMisdirected": True,
+        }
+        response = self.session.patch(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+    def delete_session(self):
+        """delete session after test done"""
+        try:
+            url = self.session_url
+            response = self.destroy_config(url)
+            self.logger.debug("STATUS CODE: %s" % response.status_code)
+        except requests.exceptions.RequestException as err_msg:
+            raise Exception('DELETE error: {0}\n'.format(err_msg))
+
+    def configure_streams(self, pkt, field_config=None):
+        hParser = PacketParser()
+        hParser._parse_pcap(pkt)
+        hConfig = IxnetConfigStream(
+            hParser.packetLayers, field_config, hParser.framesize)
+        return hConfig.ixnet_packet
+
+    def regenerate_trafficitems(self, trafficItemList):
+        """
+        Parameter
+            trafficItemList: ['/api/v1/sessions/1/ixnetwork/traffic/trafficItem/1', ...]
+        """
+        url = "{0}/ixnetwork/traffic/trafficItem/operations/generate".format(
+            self.session_url)
+        data = {"arg1": trafficItemList}
+        self.logger.info('Regenerating traffic items: %s' % trafficItemList)
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        self.wait_for_complete(response, url + '/' + response.json()['id'])
+
+    def apply_traffic(self):
+        """Apply the configured traffic."""
+        url = "{0}/ixnetwork/traffic/operations/apply".format(self.session_url)
+        data = {"arg1": f"/api/v1/sessions/{self.session_id}/ixnetwork/traffic"}
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        self.wait_for_complete(response, url + '/' + response.json()['id'])
+
+    def start_traffic(self):
+        """start the configured traffic."""
+        self.logger.info("Traffic starting...")
+        url = "{0}/ixnetwork/traffic/operations/start".format(self.session_url)
+        data = {"arg1": f"/api/v1/sessions/{self.session_id}/ixnetwork/traffic"}
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        self.check_traffic_state(
+            expectedState=['started', 'startedWaitingForStats'], timeout=45)
+        self.logger.info("Traffic started Successfully.")
+
+    def stop_traffic(self):
+        """stop the configured traffic."""
+        url = "{0}/ixnetwork/traffic/operations/stop".format(self.session_url)
+        data = {"arg1": f"/api/v1/sessions/{self.session_id}/ixnetwork/traffic"}
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        self.check_traffic_state(
+            expectedState=['stopped', 'stoppedWaitingForStats'])
+        time.sleep(5)
+
+    def check_traffic_state(self, expectedState=['stopped'], timeout=45):
+        """
+        Description
+            Check the traffic state for the expected state.
+
+        Traffic states are:
+            startedWaitingForStats, startedWaitingForStreams, started, stopped,
+            stoppedWaitingForStats, txStopWatchExpected, locked, unapplied
+
+        Parameters
+            expectedState = Input a list of expected traffic state.
+                            Example: ['started', startedWaitingForStats']
+            timeout = The amount of seconds you want to wait for the expected traffic state.
+                      Defaults to 45 seconds.
+                      In a situation where you have more than 10 pages of stats, you will
+                      need to increase the timeout time.
+        """
+        if type(expectedState) != list:
+            expectedState.split(' ')
+
+        self.logger.info(
+            'check_traffic_state: expecting traffic state {0}'.format(expectedState))
+        for counter in range(1, timeout + 1):
+            url = "{0}/ixnetwork/traffic".format(self.session_url)
+            response = self.session.get(
+                url, headers=self.json_header, verify=False)
+            current_traffic_state = response.json()['state']
+            self.logger.info('check_traffic_state: {trafficstate}: Waited {counter}/{timeout} seconds'.format(
+                trafficstate=current_traffic_state,
+                counter=counter,
+                timeout=timeout))
+            if counter < timeout and current_traffic_state not in expectedState:
+                time.sleep(1)
+                continue
+            if counter < timeout and current_traffic_state in expectedState:
+                time.sleep(8)
+                self.logger.info(
+                    'check_traffic_state: got expected [ %s ], Done' % current_traffic_state)
+                return 0
+
+        raise Exception(
+            'Traffic state did not reach the expected state (%s):' % expectedState)
+
+    def _get_stats(self, viewName='Flow Statistics', csvFile=None, csvEnableFileTimestamp=False):
+        """
+         sessionUrl: http://10.219.x.x:11009/api/v1/sessions/1/ixnetwork
+
+         csvFile = None or <filename.csv>.
+                   None will not create a CSV file.
+                   Provide a <filename>.csv to record all stats to a CSV file.
+                   Example: _get_stats(sessionUrl, csvFile='Flow_Statistics.csv')
+
+         csvEnableFileTimestamp = True or False. If True, timestamp will be appended to the filename.
+
+         viewName options (Not case sensitive):
+
+            'Port Statistics'
+            'Tx-Rx Frame Rate Statistics'
+            'Port CPU Statistics'
+            'Global Protocol Statistics'
+            'Protocols Summary'
+            'Port Summary'
+            'OSPFv2-RTR Drill Down'
+            'OSPFv2-RTR Per Port'
+            'IPv4 Drill Down'
+            'L2-L3 Test Summary Statistics'
+            'Flow Statistics'
+            'Traffic Item Statistics'
+            'IGMP Host Drill Down'
+            'IGMP Host Per Port'
+            'IPv6 Drill Down'
+            'MLD Host Drill Down'
+            'MLD Host Per Port'
+            'PIMv6 IF Drill Down'
+            'PIMv6 IF Per Port'
+
+         Note: Not all of the viewNames are listed here. You have to get the exact names from
+               the IxNetwork GUI in statistics based on your protocol(s).
+
+         Return you a dictionary of all the stats: statDict[rowNumber][columnName] == statValue
+           Get stats on row 2 for 'Tx Frames' = statDict[2]['Tx Frames']
+        """
+        url = "{0}/ixnetwork/statistics/view".format(self.session_url)
+        viewList = self.session.get(
+            url, headers=self.json_header, verify=False)
+        views = ['{0}/{1}'.format(url, str(i['id'])) for i in viewList.json()]
+
+        for view in views:
+            # GetAttribute
+            response = self.session.get(
+                view, headers=self.json_header, verify=False)
+            if response.status_code != 200:
+                raise Exception('getStats: Failed: %s' % response.text)
+            captionMatch = re.match(viewName, response.json()['caption'], re.I)
+            if captionMatch:
+                # viewObj: sessionUrl + /statistics/view/11'
+                viewObj = view
+                break
+
+        self.logger.info("viewName: %s, %s" % (viewName, viewObj))
+
+        try:
+            response = self.session.patch(viewObj, data=json.dumps(
+                {'enabled': 'true'}), headers=self.json_header, verify=False)
+        except Exception as e:
+            raise Exception('get_stats error: No stats available')
+
+        for counter in range(0, 31):
+            response = self.session.get(
+                viewObj + '/page', headers=self.json_header, verify=False)
+            totalPages = response.json()['totalPages']
+            if totalPages == 'null':
+                self.logger.info(
+                    'Getting total pages is not ready yet. Waiting %d/30 seconds' % counter)
+                time.sleep(1)
+            if totalPages != 'null':
+                break
+            if totalPages == 'null' and counter == 30:
+                raise Exception('getStats: failed to get total pages')
+
+        if csvFile is not None:
+            csvFileName = csvFile.replace(' ', '_')
+            if csvEnableFileTimestamp:
+                timestamp = datetime.now().strftime('%H%M%S')
+                if '.' in csvFileName:
+                    csvFileNameTemp = csvFileName.split('.')[0]
+                    csvFileNameExtension = csvFileName.split('.')[1]
+                    csvFileName = csvFileNameTemp + '_' + \
+                        timestamp + '.' + csvFileNameExtension
+                else:
+                    csvFileName = csvFileName + '_' + timestamp
+
+            csvFile = open(csvFileName, 'w')
+            csvWriteObj = csv.writer(csvFile)
+
+        # Get the stat column names
+        columnList = response.json()['columnCaptions']
+        if csvFile is not None:
+            csvWriteObj.writerow(columnList)
+
+        statDict = {}
+        flowNumber = 1
+        # Get the stat values
+        for pageNumber in range(1, totalPages + 1):
+            self.session.patch(viewObj + '/page', data=json.dumps(
+                {'currentPage': pageNumber}), headers=self.json_header, verify=False)
+            response = self.session.get(
+                viewObj + '/page', headers=self.json_header, verify=False)
+            statValueList = response.json()['pageValues']
+            for statValue in statValueList:
+                if csvFile is not None:
+                    csvWriteObj.writerow(statValue[0])
+
+                self.logger.info('Row: %d' % flowNumber)
+                statDict[flowNumber] = {}
+                index = 0
+                for statValue in statValue[0]:
+                    statName = columnList[index]
+                    statDict[flowNumber].update({statName: statValue})
+                    self.logger.info('%s: %s' % (statName, statValue))
+                    index += 1
+                flowNumber += 1
+
+        if csvFile is not None:
+            csvFile.close()
+        return statDict
+        # Flow Statistics dictionary output example
+        """
+        Flow: 50
+            Tx Port: Ethernet - 002
+            Rx Port: Ethernet - 001
+            Traffic Item: OSPF T1 to T2
+            Source/Dest Value Pair: 2.0.21.1-1.0.21.1
+            Flow Group: OSPF T1 to T2-FlowGroup-1 - Flow Group 0002
+            Tx Frames: 35873
+            Rx Frames: 35873
+            Frames Delta: 0
+            Loss %: 0
+            Tx Frame Rate: 3643.5
+            Rx Frame Rate: 3643.5
+            Tx L1 Rate (bps): 4313904
+            Rx L1 Rate (bps): 4313904
+            Rx Bytes: 4591744
+            Tx Rate (Bps): 466368
+            Rx Rate (Bps): 466368
+            Tx Rate (bps): 3730944
+            Rx Rate (bps): 3730944
+            Tx Rate (Kbps): 3730.944
+            Rx Rate (Kbps): 3730.944
+            Tx Rate (Mbps): 3.731
+            Rx Rate (Mbps): 3.731
+            Store-Forward Avg Latency (ns): 0
+            Store-Forward Min Latency (ns): 0
+            Store-Forward Max Latency (ns): 0
+            First TimeStamp: 00:00:00.722
+            Last TimeStamp: 00:00:10.568
+        """
+
+    def new_blank_config(self):
+        """
+        Start a new blank configuration.
+        """
+        url = "{0}/ixnetwork/operations/newconfig".format(self.session_url)
+        self.logger.info('newBlankConfig: %s' % url)
+        response = self.session.post(url, verify=False)
+        url = "{0}/{1}".format(url, response.json()['id'])
+        self.wait_for_complete(response, url)
+
+    def wait_for_complete(self, response='', url='', timeout=120):
+        """
+        Wait for an operation progress to complete.
+        response: The POST action response.
+        """
+        if response.json() == '' and response.json()['state'] == 'SUCCESS':
+            self.logger.info('State: SUCCESS')
+            return
+
+        if response.json() == []:
+            raise Exception('waitForComplete: response is empty.')
+
+        if 'errors' in response.json():
+            raise Exception(response.json()["errors"][0])
+
+        if response.json()['state'] in ["ERROR", "EXCEPTION"]:
+            raise Exception('WaitForComplete: STATE=%s: %s' %
+                            (response.json()['state'], response.text))
+
+        self.logger.info("%s" % url)
+        self.logger.info("State: %s" % (response.json()["state"]))
+        while response.json()["state"] == "IN_PROGRESS" or response.json()["state"] == "down":
+            if timeout == 0:
+                raise Exception('%s' % response.text)
+            time.sleep(1)
+            response = self.session.get(
+                url, headers=self.json_header, verify=False)
+            self.logger.info("State: %s" % (response.json()["state"]))
+            if response.json()["state"] == 'SUCCESS':
+                return
+            timeout = timeout - 1
+
+    def create_vports(self, portList=None, rawTrafficVport=True):
+        """
+        This creates virtual ports based on a portList.
+        portList:  Pass in a list of ports in the format of ixChassisIp, slotNumber, portNumber
+          portList = [[ixChassisIp, '1', '1'],
+                      [ixChassisIp, '2', '1']]
+        rawTrafficVport = For raw Traffic Item src/dest endpoints, vports must be in format:
+                               /api/v1/sessions1/vport/{id}/protocols
+        Next step is to call assign_port.
+        Return: A list of vports
+        """
+        createdVportList = []
+        for index in range(0, len(portList)):
+            url = "{0}/ixnetwork/vport".format(self.session_url)
+
+            card = portList[index][1]
+            port = portList[index][2]
+            portNumber = str(card) + '/' + str(port)
+            self.logger.info('Name: %s' % portNumber)
+            data = {'name': portNumber}
+            response = self.session.post(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+            vportObj = response.json()['links'][0]['href']
+            self.logger.info('createVports: %s' % vportObj)
+            if rawTrafficVport:
+                createdVportList.append(vportObj + '/protocols')
+            else:
+                createdVportList.append(vportObj)
+
+        if createdVportList == []:
+            raise Exception('No vports created')
+
+        self.logger.info('createVports: %s' % createdVportList)
+        return createdVportList
+
+    def assign_ports(self, portList, createVports=True, rawTraffic=True, timeout=90):
+        """
+        Description
+            Use this to assign physical ports to the virtual ports.
+
+        Parameters
+            portList: [ [ixChassisIp, '1','1'], [ixChassisIp, '1','2'] ]
+            vportList: list return by create_vports.
+            timeout: Timeout for port up.
+
+        Syntaxes
+            POST: http://{apiServerIp:port}/api/v1/sessions/{id}/ixnetwork/operations/assignports
+                  data={arg1: [{arg1: ixChassisIp, arg2: 1, arg3: 1}, {arg1: ixChassisIp, arg2: 1, arg3: 2}],
+                        arg2: [],
+                        arg3: ['/api/v1/sessions/{1}/ixnetwork/vport/1',
+                               '/api/v1/sessions/{1}/ixnetwork/vport/2'],
+                        arg4: true}  <-- True will clear port ownership
+                  headers={'content-type': 'application/json'}
+            GET:  http://{apiServerIp:port}/api/v1/sessions/{id}/ixnetwork/operations/assignports/1
+                  data={}
+                  headers={}
+            Expecting:   RESPONSE:  SUCCESS
+        """
+        if createVports:
+            vportList = self.create_vports(portList, rawTrafficVport=False)
+        url = "{0}/ixnetwork/operations/assignports".format(self.session_url)
+        data = {"arg1": [], "arg2": [], "arg3": vportList, "arg4": "true"}
+        [data["arg1"].append({"arg1": str(chassis), "arg2": str(
+            card), "arg3": str(port)}) for chassis, card, port in portList]
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        self.logger.info('%s' % response.json())
+        url = "{0}/{1}".format(url, response.json()['id'])
+        self.wait_for_complete(response, url)
+
+        for vport in vportList:
+            url = "{0}{1}/l1Config".format(self.api_server, vport)
+            response = self.session.get(
+                url, headers=self.json_header, verify=False)
+            url = url + '/' + response.json()['currentType']
+            data = {"enabledFlowControl": False}
+            response = self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        if rawTraffic:
+            vportList_protocol = []
+            for vport in vportList:
+                vportList_protocol.append(vport + '/protocols')
+            self.logger.info('vports: %s' % vportList_protocol)
+            return vportList_protocol
+        else:
+            self.logger.info('vports: %s' % vportList)
+            return vportList
+
+    def destroy_assign_ports(self, vportList):
+        msg = "release {}".format(vportList)
+        self.logger.info(msg)
+        for vport_url in vportList:
+            url = self.api_server + "/".join(vport_url.split("/")[:-1])
+            self.destroy_config(url)
+
+    def config_config_elements(self, config_element_obj, config_elements):
+        """
+        Parameters
+        config_element_obj: /api/v1/sessions/1/ixnetwork/traffic/trafficItem/{id}/configElement/{id}
+        """
+        url = self.api_server + config_element_obj + '/transmissionControl'
+        if 'transmissionType' in config_elements:
+            data = {'type': config_elements['transmissionType']}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        if 'burstPacketCount' in config_elements:
+            data = {
+                'burstPacketCount': int(config_elements['burstPacketCount'])}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        if 'frameCount' in config_elements:
+            data = {'frameCount': int(config_elements['frameCount'])}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        if 'duration' in config_elements:
+            data = {'duration': int(config_elements['duration'])}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        url = self.api_server + config_element_obj + '/frameRate'
+        if 'frameRate' in config_elements:
+            data = {'rate': int(config_elements['frameRate'])}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        if 'frameRateType' in config_elements:
+            data = {'type': config_elements['frameRateType']}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        url = self.api_server + config_element_obj + '/frameSize'
+        if 'frameSize' in config_elements:
+            data = {'fixedSize': int(config_elements['frameSize'])}
+            self.session.patch(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+    def import_json_config_obj(self, data_obj):
+        """
+        Parameter
+            data_obj: The JSON config object.
+        Note
+            arg2 value must be a string of JSON data: '{"xpath": "/traffic/trafficItem[1]", "enabled": false}'
+        """
+        data = {"arg1": "/api/v1/sessions/1/ixnetwork/resourceManager",
+                "arg2": json.dumps(data_obj),
+                "arg3": False}
+        url = "{0}/ixnetwork/resourceManager/operations/importconfig".format(
+            self.session_url)
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        url = "{0}/{1}".format(url, response.json()['id'])
+        self.wait_for_complete(response, url)
+
+    def send_rfc2544_throughput(self, options):
+        """Send traffic per RFC2544 throughput test specifications.
+        Send packets at a variable rate, using ``traffic_list`` configuration,
+        until minimum rate at which no packet loss is detected is found.
+        """
+        # new added parameters
+        duration = options.get('duration') or 10
+        initialBinaryLoadRate = max_rate = options.get('max_rate') or 100.0
+        min_rate = options.get('min_rate') or 0.0
+        accuracy = options.get('accuracy') or 0.001
+        permit_loss_rate = options.get('pdr') or 0.0
+        # old parameters
+        traffic_list = options.get('traffic_list')
+        if traffic_list is None:
+            raise Exception('traffic_list is empty.')
+
+        # close port mismatched statistics
+        self.disable_port_misdirected()
+
+        url = "{0}/ixnetwork/traffic/trafficItem".format(self.session_url)
+        response = self.session.get(
+            url, headers=self.json_header, verify=False)
+        if response.json() != []:
+            for item in response.json():
+                url = "{0}{1}".format(
+                    self.api_server, item['links'][0]['href'])
+                response = self.destroy_config(url)
+                if response.status_code != 200:
+                    raise Exception("remove trafficitem failed")
+
+        trafficitem_list = []
+        index = 0
+        for traffic in traffic_list:
+            index = index + 1
+            # create trafficitem
+            url = "{0}/ixnetwork/traffic/trafficItem".format(self.session_url)
+            data = {"name": "Traffic Item " + str(index), "trafficType": "raw"}
+            response = self.session.post(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+            trafficitem_obj = response.json()['links'][0]['href']
+            self.logger.info('create traffic item: %s' % trafficitem_obj)
+            trafficitem_list.append(trafficitem_obj)
+            # create endpointset
+            url = "{0}{1}/endpointSet".format(self.api_server, trafficitem_obj)
+            data = {
+                "sources": [traffic[0]],
+                "destinations": [traffic[1]]
+            }
+            response = self.session.post(
+                url, data=json.dumps(data), headers=self.json_header, verify=False)
+            # packet config
+            config_stack_obj = eval(
+                str(traffic[2]).replace('trafficItem[1]', 'trafficItem[' + str(index) + ']'))
+            self.import_json_config_obj(config_stack_obj)
+            # get framesize
+            url = "{0}{1}/configElement/1/frameSize".format(
+                self.api_server, trafficitem_obj)
+            response = self.session.get(
+                url, headers=self.json_header, verify=False)
+            frame_size = response.json()['fixedSize']
+
+        self.regenerate_trafficitems(trafficitem_list)
+
+        # query existing quick test
+        url = "{0}/ixnetwork/quickTest/rfc2544throughput".format(
+            self.session_url)
+        response = self.session.get(
+            url, headers=self.json_header, verify=False)
+        if response.json() != []:
+            for qt in response.json():
+                url = "{0}{1}".format(self.api_server, qt['links'][0]['href'])
+                response = self.destroy_config(url)
+                if response.status_code != 200:
+                    raise Exception("remove quick test failed")
+        # create quick test
+        url = "{0}/ixnetwork/quickTest/rfc2544throughput".format(
+            self.session_url)
+        data = [{"name": "QuickTest1", "mode": "existingMode"}]
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        quicktest_obj = response.json()['links'][0]['href']
+        self.logger.info('create quick test: %s' % quicktest_obj)
+        # add trafficitems
+        url = "{0}{1}/trafficSelection".format(self.api_server, quicktest_obj)
+        data = [{"__id__": item_obj} for item_obj in trafficitem_list]
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        self.logger.info("add traffic item status: %s" % response.content)
+        # modify quick test config
+        url = "{0}{1}/testConfig".format(self.api_server, quicktest_obj)
+        data = {
+            # If Enabled, The minimum size of the frame is used .
+            "enableMinFrameSize": True,
+            # This attribute is the frame size mode for the Quad Gaussian.
+            # Possible values includes:
+            "frameSizeMode": "custom",
+            # The list of the available frame size.
+            "framesizeList": [str(frame_size)],
+            # The minimum delay between successive packets.
+            "txDelay": 5,
+            # Specifies the amount of delay after every transmit
+            "delayAfterTransmit": 5,
+            # sec
+            "duration": duration,
+            # The initial binary value of the load rate
+            "initialBinaryLoadRate": initialBinaryLoadRate,
+            # The upper bound of the iteration rates for each frame size during
+            # a binary search
+            "maxBinaryLoadRate": max_rate,
+            # Specifies the minimum rate of the binary algorithm.
+            "minBinaryLoadRate": min_rate,
+            # The frame loss unit for traffic in binary.
+            # Specifies the resolution of the iteration. The difference between
+            # the real rate transmission in two consecutive iterations, expressed
+            # as a percentage, is compared with the resolution value. When the
+            # difference is smaller than the value specified for the
+            # resolution, the test stops .
+            "resolution": accuracy * 100,
+            # The load unit value in binary.
+            "binaryFrameLossUnit": "%",
+            # The binary tolerance level.
+            "binaryTolerance": permit_loss_rate,
+        }
+        response = self.session.patch(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        if response.status_code != 200:
+            raise Exception("change quick test config failed")
+        # run the quick test
+        url = "{0}{1}/operations/run".format(self.api_server, quicktest_obj)
+        data = {"arg1": quicktest_obj, "arg2": ""}
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+        url = url + '/' + response.json()['id']
+        state = response.json()["state"]
+        self.logger.info("Quicktest State: %s" % state)
+        while state == "IN_PROGRESS":
+            response = self.session.get(
+                url, headers=self.json_header, verify=False)
+            state = response.json()["state"]
+            self.logger.info("Quicktest State: %s" % state)
+            time.sleep(5)
+
+        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
+        copy_to_path = os.sep.join([
+            self.OUTPUT_DIR,
+            'ixnet' + datetime.now().strftime("%Y%m%d_%H%M%S")])
+        if not os.path.exists(copy_to_path):
+            os.makedirs(copy_to_path)
+        self.get_quicktest_csvfiles(quicktest_obj, copy_to_path, csvfile='all')
+        qt_result_csv = "{0}/AggregateResults.csv".format(copy_to_path)
+        return self.parse_quicktest_results(qt_result_csv)
+
+    def parse_quicktest_results(self, path_file):
+        """ parse csv filte and return quicktest result """
+        results = OrderedDict()
+
+        if not os.path.exists(path_file):
+            msg = "failed to get result file from windows api server"
+            self.logger.error(msg)
+            return results
+
+        ret_result = []
+        with open(path_file, "r") as f:
+            qt_result = csv.DictReader(f)
+            for row in qt_result:
+                ret_result.append(row)
+                results['framesize'] = row['Framesize']
+                results['throughput'] = row['Agg Rx Throughput (fps)']
+                results['linerate%'] = row['Agg Rx Throughput (% Line Rate)']
+                results['min_latency'] = row['Min Latency (ns)']
+                results['max_latency'] = row['Max Latency (ns)']
+                results['avg_latency'] = row['Avg Latency (ns)']
+
+        return ret_result
+
+    def get_quicktest_resultpath(self, quicktest_obj):
+        """
+        quicktest_obj = /api/v1/sessions/1/ixnetwork/quickTest/rfc2544throughput/2
+        """
+        url = "{0}{1}/results".format(self.api_server, quicktest_obj)
+        response = self.session.get(
+            url, headers=self.json_header, verify=False)
+        return response.json()['resultPath']
+
+    def get_quicktest_csvfiles(self, quicktest_obj, copy_to_path, csvfile='all'):
+        """
+        Description
+            Copy Quick Test CSV result files to a specified path on either Windows or Linux.
+            Note: Currently only supports copying from Windows.
+        quicktest_obj: The Quick Test handle.
+        copy_to_path: The destination path to copy to.
+                    If copy to Windows: c:\\Results\\Path
+                    If copy to Linux: /home/user1/results/path
+        csvfile: A list of CSV files to get: 'all', one or more CSV files to get:
+                 AggregateResults.csv, iteration.csv, results.csv, logFile.txt, portMap.csv
+        """
+        results_path = self.get_quicktest_resultpath(quicktest_obj)
+        self.logger.info('get_quickTest_csvfiles: %s' % results_path)
+        if csvfile == 'all':
+            get_csv_files = [
+                'AggregateResults.csv', 'iteration.csv', 'results.csv', 'logFile.txt', 'portMap.csv']
+        else:
+            if type(csvfile) is not list:
+                get_csv_files = [csvfile]
+            else:
+                get_csv_files = csvfile
+
+        for each_csvfile in get_csv_files:
+            # Backslash indicates the results resides on a Windows OS.
+            if '\\' in results_path:
+                cnt = 0
+                while cnt < 5:
+                    try:
+                        self.copyfile_windows2linux(
+                            results_path + '\\{0}'.format(each_csvfile), copy_to_path)
+                        break
+                    except Exception as e:
+                        time.sleep(5)
+                        cnt += 1
+                        msg = "No.{} retry to get result from windows".format(cnt)
+                        self.logger.warning(msg)
+                        continue
+            else:
+                # TODO:Copy from Linux to Windows and Linux to Linux.
+                pass
+
+    def copyfile_windows2linux(self, winPathFile, linuxPath, includeTimestamp=False):
+        """
+        Description
+            Copy files from the IxNetwork API Server c: drive to local Linux filesystem.
+            You could also include a timestamp for the destination file.
+        Parameters
+            winPathFile: (str): The full path and filename to retrieve from Windows client.
+            linuxPath: (str): The Linux destination path to put the file to.
+            includeTimestamp: (bool):  If False, each time you copy the same file will be overwritten.
+        Syntax
+            post: /api/v1/sessions/1/ixnetwork/operations/copyfile
+            data: {'arg1': winPathFile, 'arg2': '/api/v1/sessions/1/ixnetwork/files/'+fileName'}
+        """
+        self.logger.info('copyfile From: %s to %s' % (winPathFile, linuxPath))
+        fileName = winPathFile.split('\\')[-1]
+        fileName = fileName.replace(' ', '_')
+        destinationPath = '/api/v1/sessions/1/ixnetwork/files/' + fileName
+        currentTimestamp = datetime.now().strftime('%H%M%S')
+
+        # Step 1 of 2:
+        url = "{0}/ixnetwork/operations/copyfile".format(self.session_url)
+        data = {"arg1": winPathFile, "arg2": destinationPath}
+        response = self.session.post(
+            url, data=json.dumps(data), headers=self.json_header, verify=False)
+
+        # Step 2 of 2:
+        url = "{0}/ixnetwork/files/{1}".format(self.session_url, fileName)
+        requestStatus = self.session.get(
+            url, stream=True, headers=self.json_header, verify=False)
+        if requestStatus.status_code == 200:
+            contents = requestStatus.raw.read()
+
+            if includeTimestamp:
+                tempFileName = fileName.split('.')
+                if len(tempFileName) > 1:
+                    extension = fileName.split('.')[-1]
+                    fileName = tempFileName[0] + '_' + currentTimestamp + '.' + extension
+                else:
+                    fileName = tempFileName[0] + '_' + currentTimestamp
+
+                linuxPath = linuxPath + '/' + fileName
+            else:
+                linuxPath = linuxPath + '/' + fileName
+
+            with open(linuxPath, 'wb') as downloadedFileContents:
+                downloadedFileContents.write(contents)
+
+            url = "{0}/ixnetwork/files".format(self.session_url)
+            response = self.session.get(
+                url, headers=self.json_header, verify=False)
+            self.logger.info('A copy of saved file is in: %s' % (winPathFile))
+            self.logger.info(
+                'copyfile_windows2linux: The copyfile is in %s' % linuxPath)
+        else:
+            raise Exception(
+                "copyfile_windows2linux: Failed to download file from IxNetwork API Server.")
+
+    def tear_down(self):
+        """do needed clean up"""
+        self.destroy_assign_ports(self.tg_vports)
+        self.session.close()
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 12/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (10 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 11/27] " yufengmx
@ 2021-01-25  8:43 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 13/27] conf/pktgen: " yufengmx
                   ` (15 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:43 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


parse pcap file/scapy Packet layers content.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/ixia_network/packet_parser.py | 93 +++++++++++++++++++++++++
 1 file changed, 93 insertions(+)
 create mode 100644 framework/ixia_network/packet_parser.py

diff --git a/framework/ixia_network/packet_parser.py b/framework/ixia_network/packet_parser.py
new file mode 100644
index 00000000..6bb0e2f3
--- /dev/null
+++ b/framework/ixia_network/packet_parser.py
@@ -0,0 +1,93 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import os
+from collections import OrderedDict
+
+from scapy.all import conf
+from scapy.packet import NoPayload
+from scapy.packet import Packet as scapyPacket
+from scapy.fields import ConditionalField
+from scapy.utils import rdpcap
+
+
+class PacketParser(object):
+    ''' parse packet full layers information '''
+
+    def __init__(self):
+        self.packetLayers = OrderedDict()
+        self.framesize = 64
+
+    def _parse_packet_layer(self, pkt_object):
+        ''' parse one packet every layers' fields and value '''
+        if pkt_object is None:
+            return
+
+        self.packetLayers[pkt_object.name] = OrderedDict()
+        for curfield in pkt_object.fields_desc:
+            if isinstance(curfield, ConditionalField) and \
+                    not curfield._evalcond(pkt_object):
+                continue
+            field_value = pkt_object.getfieldval(curfield.name)
+            if isinstance(field_value, scapyPacket) or (curfield.islist and
+                                                        curfield.holds_packets and type(field_value) is list):
+                continue
+            repr_value = curfield.i2repr(pkt_object, field_value)
+            if isinstance(repr_value, str):
+                repr_value = repr_value.replace(os.linesep,
+                                                os.linesep + " " * (len(curfield.name) + 4))
+            self.packetLayers[pkt_object.name][curfield.name] = repr_value
+
+        if isinstance(pkt_object.payload, NoPayload):
+            return
+        else:
+            self._parse_packet_layer(pkt_object.payload)
+
+    def _parse_pcap(self, pcapFile, number=0):
+        ''' parse one packet content '''
+        self.packetLayers = OrderedDict()
+        pcap_pkts = []
+        if isinstance(pcapFile, str):
+            if os.path.exists(pcapFile) is False:
+                warning = "{0} is not exist !".format(pcapFile)
+                raise Exception(warning)
+            pcap_pkts = rdpcap(pcapFile)
+        else:
+            pcap_pkts = pcapFile
+        # parse packets' every layers and fields
+        if len(pcap_pkts) == 0:
+            warning = "{0} is empty".format(pcapFile)
+            raise Exception(warning)
+        elif number >= len(pcap_pkts):
+            warning = "{0} is missing No.{1} packet".format(pcapFile, number)
+            raise Exception(warning)
+        else:
+            self._parse_packet_layer(pcap_pkts[number])
+            self.framesize = len(pcap_pkts[number]) + 4
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 13/27] conf/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (11 preceding siblings ...)
  2021-01-25  8:43 ` [dts] [PATCH V1 12/27] " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 14/27] framework/pktgen: " yufengmx
                   ` (14 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


support ixNetwork configuration content parsing process.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/config.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/framework/config.py b/framework/config.py
index dd3bf306..30604773 100644
--- a/framework/config.py
+++ b/framework/config.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -36,7 +36,7 @@ import os
 import re
 import configparser  # config parse module
 import argparse      # parse arguments module
-from settings import (IXIA, PKTGEN, PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA,
+from settings import (IXIA, PKTGEN, PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN_IXIA_NETWORK,
                       CONFIG_ROOT_PATH, SUITE_SECTION_NAME)
 from settings import load_global_setting, DTS_CFG_FOLDER
 from exception import ConfigParseException, VirtConfigParseException, PortConfigParseException
@@ -429,6 +429,8 @@ class PktgenConf(UserConf):
                 ixia_group['Ports'] = ixia_ports
             elif key == 'ixia_enable_rsfec':
                 ixia_group['enable_rsfec'] = value
+            else:
+                ixia_group[key] = value
 
         if 'Version' not in ixia_group:
             print('ixia configuration file request ixia_version option!!!')
@@ -466,7 +468,8 @@ class PktgenConf(UserConf):
                 for conf in pktgen_confs:
                     key, value = conf
                     self.pktgen_cfg[key] = value
-            elif self.pktgen_type == PKTGEN_IXIA and section.lower() == PKTGEN_IXIA:
+            elif (self.pktgen_type == PKTGEN_IXIA and section.lower() == PKTGEN_IXIA) or \
+                 (self.pktgen_type == PKTGEN_IXIA_NETWORK and section.lower() == PKTGEN_IXIA_NETWORK):
                 # covert file configuration to dts pktgen cfg
                 self.load_pktgen_ixia_config(section)
 
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 14/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (12 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 13/27] conf/pktgen: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 15/27] " yufengmx
                   ` (13 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add ixNetwork constants.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/settings.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/framework/settings.py b/framework/settings.py
index c5edbb7a..23eb0075 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -235,7 +235,8 @@ PKTGEN = "pktgen"
 PKTGEN_DPDK = "dpdk"
 PKTGEN_TREX = "trex"
 PKTGEN_IXIA = "ixia"
-PKTGEN_GRP = frozenset([PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA])
+PKTGEN_IXIA_NETWORK = "ixia_network"
+PKTGEN_GRP = frozenset([PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN_IXIA_NETWORK])
 """
 The log name seperater.
 """
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 15/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (13 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 14/27] framework/pktgen: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 16/27] " yufengmx
                   ` (12 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


add ixNetwork packet generator support in pktgen.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/framework/pktgen.py b/framework/pktgen.py
index 92d17784..a00bc090 100644
--- a/framework/pktgen.py
+++ b/framework/pktgen.py
@@ -42,10 +42,11 @@ from scapy.utils import rdpcap
 from utils import (convert_int2ip, convert_ip2int,
                    convert_mac2long, convert_mac2str)
 
-from pktgen_base import (PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, STAT_TYPE,
+from pktgen_base import (PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN_IXIA_NETWORK, STAT_TYPE,
                          TRANSMIT_CONT, TRANSMIT_M_BURST, TRANSMIT_S_BURST)
 from pktgen_base import DpdkPacketGenerator
 from pktgen_ixia import IxiaPacketGenerator
+from pktgen_ixia_network import IxNetworkPacketGenerator
 from pktgen_trex import TrexPacketGenerator
 
 
@@ -208,7 +209,9 @@ def getPacketGenerator(tester, pktgen_type=PKTGEN_IXIA):
     pktgen_cls = {
         PKTGEN_DPDK: DpdkPacketGenerator,
         PKTGEN_IXIA: IxiaPacketGenerator,
-        PKTGEN_TREX: TrexPacketGenerator, }
+        PKTGEN_IXIA_NETWORK: IxNetworkPacketGenerator,
+        PKTGEN_TREX: TrexPacketGenerator,
+    }
 
     if pktgen_type in list(pktgen_cls.keys()):
         CLS = pktgen_cls.get(pktgen_type)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 16/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (14 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 15/27] " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 17/27] " yufengmx
                   ` (11 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


enable pktgen/ixNetwork rfc2544 testing scenario.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_base.py | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py
index 81c8830c..f30785d0 100644
--- a/framework/pktgen_base.py
+++ b/framework/pktgen_base.py
@@ -39,7 +39,7 @@ from enum import Enum, unique
 
 from config import PktgenConf
 # packet generator name
-from settings import PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN
+from settings import PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN_IXIA_NETWORK, PKTGEN
 
 # macro definition
 TRANSMIT_CONT = 'continuous'
@@ -355,6 +355,27 @@ class PacketGenerator(object):
         # In real testing scenario, this method can offer more data than it
         return list(result.values())[0]
 
+    def _measure_rfc2544_ixnet(self, stream_ids=[], options={}):
+        """
+        used for ixNetwork
+        """
+        # main traffic
+        self._prepare_transmission(stream_ids=stream_ids)
+        self._start_transmission(stream_ids, options)
+        self._stop_transmission(None)
+        # parsing test result
+        stats = self._retrieve_port_statistic(stream_ids[0], 'rfc2544')
+        tx_pkts, rx_pkts, pps = stats
+        lost_p = tx_pkts - rx_pkts
+        if tx_pkts <= 0:
+            loss_rate = 0
+        else:
+            loss_rate = float(lost_p) / float(tx_pkts)
+            if loss_rate < 0:
+                loss_rate = 0
+        result = (loss_rate, tx_pkts, rx_pkts, pps)
+        return result
+
     def measure_latency(self, stream_ids=[], options={}):
         """
         Measure latency on each tx/rx ports
@@ -533,6 +554,9 @@ class PacketGenerator(object):
             accuracy :
                 dichotomy algorithm accuracy, default 0.001.
         """
+        if self.pktgen_type == PKTGEN_IXIA_NETWORK:
+            return self._measure_rfc2544_ixnet(stream_ids, options)
+
         max_rate = options.get('max_rate') or 100.0
         min_rate = options.get('min_rate') or 0.0
         accuracy = options.get('accuracy') or 0.001
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 17/27] framework/pktgen: enable ixNetwork
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (15 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 16/27] " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 18/27] tests/perf_test: rename l3fwd_base module yufengmx
                   ` (10 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


enable ixNetwork rfc2544 testing scenario.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_ixia_network.py | 224 +++++++++++++++++++++++++++++++
 1 file changed, 224 insertions(+)
 create mode 100644 framework/pktgen_ixia_network.py

diff --git a/framework/pktgen_ixia_network.py b/framework/pktgen_ixia_network.py
new file mode 100644
index 00000000..3cca2578
--- /dev/null
+++ b/framework/pktgen_ixia_network.py
@@ -0,0 +1,224 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import os
+import time
+import traceback
+from pprint import pformat
+
+from pktgen_base import PacketGenerator, PKTGEN_IXIA_NETWORK
+
+
+class IxNetworkPacketGenerator(PacketGenerator):
+    """
+    ixNetwork packet generator
+    """
+    def __init__(self, tester):
+        self.pktgen_type = PKTGEN_IXIA_NETWORK
+        self._conn = None
+        # ixNetwork configuration information of dts
+        conf_inst = self._get_generator_conf_instance()
+        self.conf = conf_inst.load_pktgen_config()
+        # ixNetwork port configuration
+        self._traffic_ports = []
+        self._ports = []
+        self._rx_ports = []
+
+        super(IxNetworkPacketGenerator, self).__init__(tester)
+
+    def get_ports(self):
+        ''' used for ixNetwork packet generator '''
+        return self._conn.get_ports()
+
+    def _prepare_generator(self):
+        ''' connect with ixNetwork api server '''
+        try:
+            self._connect(self.conf)
+        except Exception as e:
+            msg = "failed to connect to ixNetwork api server"
+            raise Exception(msg)
+
+    def _connect(self, conf):
+        # initialize ixNetwork class
+        from ixia_network import IxNetwork
+        self._conn = IxNetwork(self.pktgen_type, conf, self.logger)
+        for p in self._conn.get_ports():
+            self._ports.append(p)
+
+        self.logger.debug(self._ports)
+
+    def _disconnect(self):
+        '''
+        disconnect with ixNetwork api server
+        '''
+        try:
+            self._remove_all_streams()
+            self._conn.disconnect()
+        except Exception as e:
+            msg = 'Error disconnecting: %s' % e
+            self.logger.error(msg)
+        self._conn = None
+
+    def quit_generator(self):
+        ''' close ixNetwork session '''
+        if self._conn is not None:
+            self._disconnect()
+
+    def _get_port_pci(self, port_id):
+        '''
+        get ixNetwork port pci address
+        '''
+        for pktgen_port_id, info in enumerate(self._ports):
+            if pktgen_port_id == port_id:
+                _pci = info.get('pci')
+                return _pci
+        else:
+            return None
+
+    def _get_gen_port(self, pci):
+        '''
+        get port management id of the packet generator
+        '''
+        for pktgen_port_id, info in enumerate(self._ports):
+            _pci = info.get('pci')
+            if _pci == pci:
+                return pktgen_port_id
+        else:
+            return -1
+
+    def _is_gen_port(self, pci):
+        '''
+        check if a pci address is managed by the packet generator
+        '''
+        for name, _port_obj in self._conn.ports.items():
+            _pci = _port_obj.info['pci_addr']
+            self.logger.debug((_pci, pci))
+            if _pci == pci:
+                return True
+        else:
+            return False
+
+    def _get_ports(self):
+        """
+        Return self ports information
+        """
+        ports = []
+        for idx in range(len(self._ports)):
+            ports.append('IXIA:%d' % idx)
+        return ports
+
+    def send_ping6(self, pci, mac, ipv6):
+        ''' Send ping6 packet from IXIA ports. '''
+        return self._conn.send_ping6(pci, mac, ipv6)
+
+    def _clear_streams(self):
+        ''' clear streams in `PacketGenerator` '''
+        # if streams has been attached, remove them from ixNetwork api server.
+        self._remove_all_streams()
+
+    def _remove_all_streams(self):
+        '''
+        remove all stream deployed on the packet generator
+        '''
+        if not self.get_streams():
+            return
+
+    def _check_options(self, opts={}):
+        return True
+
+    def _retrieve_port_statistic(self, stream_id, mode):
+        ''' ixNetwork traffic statistics '''
+        stats = self._conn.get_stats(self._traffic_ports, mode)
+        stream = self._get_stream(stream_id)
+        self.logger.debug(pformat(stream))
+        self.logger.debug(pformat(stats))
+        if mode == 'rfc2544':
+            return stats
+        else:
+            msg = "not support mode <{0}>".format(mode)
+            raise Exception(msg)
+
+    ##########################################################################
+    #
+    #  class ``PacketGenerator`` abstract methods should be implemented here
+    #
+    ##########################################################################
+    def _prepare_transmission(self, stream_ids=[], latency=False):
+        ''' add one/multiple streams in one/multiple ports '''
+        port_config = {}
+
+        for stream_id in stream_ids:
+            stream = self._get_stream(stream_id)
+            tx_port = stream.get('tx_port')
+            rx_port = stream.get('rx_port')
+            pcap_file = stream.get('pcap_file')
+            # save port id list
+            if tx_port not in self._traffic_ports:
+                self._traffic_ports.append(tx_port)
+            if rx_port not in self._traffic_ports:
+                self._traffic_ports.append(rx_port)
+            if rx_port not in self._rx_ports:
+                self._rx_ports.append(rx_port)
+            # set all streams in one port to do batch configuration
+            options = stream['options']
+            if tx_port not in list(port_config.keys()):
+                port_config[tx_port] = []
+            config = {}
+            config.update(options)
+            # get stream rate percent
+            stream_config = options.get('stream_config')
+            rate_percent = stream_config.get('rate')
+            # set port list input parameter of ixNetwork class
+            ixia_option = [tx_port, rx_port, pcap_file, options]
+            port_config[tx_port].append(ixia_option)
+
+        self.rate_percent = rate_percent
+        if not port_config:
+            msg = 'no stream options for ixNetwork packet generator'
+            raise Exception(msg)
+
+        port_lists = []
+        for port_id, option in port_config.items():
+            port_lists += option
+        self._conn.prepare_ixia_network_stream(port_lists)
+
+    def _start_transmission(self, stream_ids, options={}):
+        # run ixNetwork api server
+        try:
+            # Start traffic on port(s)
+            self.logger.info("begin traffic ......")
+            self._conn.start(options)
+        except Exception as e:
+            self.logger.error(traceback.format_exc())
+            self.logger.error(e)
+
+    def _stop_transmission(self, stream_id):
+        if self._traffic_ports:
+            self.logger.info("traffic completed. ")
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 18/27] tests/perf_test: rename l3fwd_base module
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (16 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 17/27] " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 19/27] tests/perf_test: cover testpmd testing scenario yufengmx
                   ` (9 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


rename l3fwd_base to perf_test_base to support l3fwd/testpmd binary.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/{l3fwd_base.py => perf_test_base.py} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename tests/{l3fwd_base.py => perf_test_base.py} (100%)

diff --git a/tests/l3fwd_base.py b/tests/perf_test_base.py
similarity index 100%
rename from tests/l3fwd_base.py
rename to tests/perf_test_base.py
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 19/27] tests/perf_test: cover testpmd testing scenario
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (17 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 18/27] tests/perf_test: rename l3fwd_base module yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 20/27] tests/perf_test: save rfc2544 expected throughput yufengmx
                   ` (8 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


new traffic setting requirement:
    default IP: dst ip fixed to 198.18.0.0(based on topology), src ip is random.
    Packet type can be configured to L3/L4.
    L3: IPv4, IPv6
    L4: UDP, TCP, None

Extend develop and test to cover testpmd rfc2544 testing scenario, support below content and
set them in config file:
    forwarding mode io/mac
    burst mode AVX2/AVX512
    descriptor size 16B/32B(RTE_LIBRTE_<XXX>_<XX>BYTE_RX_DESC)
    descriptor numbers txd/rxd

support ixNetwork packet generator usage.
unify l3fwd/testpmd common parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/perf_test_base.py | 480 ++++++++++++++++++++++++++++------------
 1 file changed, 336 insertions(+), 144 deletions(-)

diff --git a/tests/perf_test_base.py b/tests/perf_test_base.py
index 235ee278..3d5cae85 100644
--- a/tests/perf_test_base.py
+++ b/tests/perf_test_base.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -45,12 +45,18 @@ from copy import deepcopy
 
 from config import SuiteConf
 from packet import Packet
-from pktgen import TRANSMIT_CONT, PKTGEN_TREX, PKTGEN_IXIA
+from pktgen import TRANSMIT_CONT, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN_IXIA_NETWORK
 from utils import convert_int2ip, convert_ip2int
 from exception import VerifyFailure
 import utils
 
 
+@unique
+class BIN_TYPE(Enum):
+    L3FWD = 'l3fwd'
+    PMD = 'testpmd'
+
+
 @unique
 class SUITE_TYPE(Enum):
     VF = 'vf_l3fwd'
@@ -60,6 +66,7 @@ class SUITE_TYPE(Enum):
 @unique
 class SUITE_NAME(Enum):
     VF_KERNELPF = 'vf_l3fwd_kernelpf'
+    TESTPMD_PERF = 'testpmd_perf'
 
 
 @unique
@@ -77,12 +84,6 @@ class MATCH_MODE(Enum):
     EM = 'em'
 
 
-# LPM(longest prefix match) mode
-LPM = MATCH_MODE.LPM
-# EM(Exact-Match) mode
-EM = MATCH_MODE.EM
-
-
 # stream internet protocol layer types
 @unique
 class IP_TYPE(Enum):
@@ -90,12 +91,9 @@ class IP_TYPE(Enum):
     V4 = 'ipv4'
 
 
-L3_IPV6 = IP_TYPE.V6
-L3_IPV4 = IP_TYPE.V4
-
-
 @unique
 class STREAM_TYPE(Enum):
+    TCP = 'TCP'
     UDP = 'UDP'
     RAW = 'RAW'
 
@@ -118,9 +116,13 @@ def get_enum_name(value, enum_cls):
         raise Exception(msg)
 
 
-class L3fwdBase(object):
+class PerfTestBase(object):
+
 
-    def l3fwd_init(self, valports, socket, mode=None):
+    def __init__(self, valports, socket, mode=None, bin_type=None):
+        self.__bin_type = bin_type or BIN_TYPE.L3FWD
+        self.__compile_rx_desc = None
+        self.__compile_avx = None
         self.__mode = mode or SUITE_TYPE.PF
         self.__suite = None
         self.__valports = valports
@@ -131,11 +133,12 @@ class L3fwdBase(object):
         self.__vf_driver = None
         self.__pf_driver = None
         self.__vf_ports_info = None
-        self.__is_l3fwd_on = None
-        self.__l3fwd_white_list = None
-        self.__l3fwd_restart = True
-        self.__pre_l3fwd_cmd = None
-        self.__l3fwd_wait_up = 0
+
+        self.__is_bin_ps_on = None
+        self.__bin_ps_white_list = None
+        self.__bin_ps_restart = True
+        self.__pre_bin_ps_cmd = None
+        self.__bin_ps_wait_up = 0
         self.__traffic_stop_wait_time = 0
         self.__is_pmd_on = None
         self.__pmd_session = None
@@ -186,7 +189,7 @@ class L3fwdBase(object):
             raise VerifyFailure(msg)
         return thread_index//core_index
 
-    def __pmd_con(self, cmd):
+    def __host_pmd_con(self, cmd):
         if not self.__pmd_session:
             return
         _cmd = [cmd, '# ', 10] if isinstance(cmd, str) else cmd
@@ -204,11 +207,14 @@ class L3fwdBase(object):
         end_ip = convert_int2ip(convert_ip2int(start_ip) + ip_range - 1)
         layers = {'ipv4': {'src': start_ip, }, }
         fields_config = {
-            'ip': {'dst': {
-                'src': start_ip,
-                'dst': end_ip,
-                'step': 1,
-                'action': 'random', }, }, }
+            'ip': {
+                'src': {
+                    'start': start_ip,
+                    'end': end_ip,
+                    'step': 1,
+                    'action': 'random', },
+                },
+        }
         return layers, fields_config
 
     def __get_ipv6_lpm_vm_config(self, lpm_config):
@@ -220,14 +226,17 @@ class L3fwdBase(object):
             convert_ip2int(start_ip, ip_type=6) + ip_range - 1, ip_type=6)
         layers = {'ipv6': {'src': start_ip, }, }
         fields_config = {
-            'ipv6': {'dst': {
-                'src': start_ip,
-                'dst': end_ip,
-                'step': 1,
-                'action': 'random', }, }, }
+            'ipv6': {
+                'src': {
+                    'start': start_ip,
+                    'end': end_ip,
+                    'step': 1,
+                    'action': 'random', },
+                },
+        }
         return layers, fields_config
 
-    def ___get_pkt_layers(self, pkt_type):
+    def __get_pkt_layers(self, pkt_type):
         if pkt_type in list(Packet.def_packet.keys()):
             return deepcopy(Packet.def_packet.get(pkt_type).get('layers'))
         local_def_packet = {
@@ -240,7 +249,7 @@ class L3fwdBase(object):
         return layers
 
     def __get_pkt_len(self, pkt_type, ip_type='ip', frame_size=64):
-        layers = self.___get_pkt_layers(pkt_type)
+        layers = self.__get_pkt_layers(pkt_type)
         if 'raw' in layers:
             layers.remove('raw')
         headers_size = sum(map(lambda x: HEADER_SIZE[x], layers))
@@ -248,8 +257,14 @@ class L3fwdBase(object):
         return pktlen
 
     def __get_frame_size(self, name, frame_size):
-        _frame_size = 66 if name is IP_TYPE.V6 and frame_size == 64 else \
-            frame_size
+        if self.pktgen_type is PKTGEN_IXIA_NETWORK:
+            # ixNetwork api server will set ipv6 packet size to 78 bytes when
+            # set frame size < 78.
+            _frame_size = 78 if name is IP_TYPE.V6 and frame_size == 64 else \
+                frame_size
+        else:
+            _frame_size = 66 if name is IP_TYPE.V6 and frame_size == 64 else \
+                frame_size
         return _frame_size
 
     def __get_pkt_type_name(self, ip_layer):
@@ -289,7 +304,7 @@ class L3fwdBase(object):
     def __get_pkt_inst(self, pkt_config):
         pkt_type = pkt_config.get('type')
         pkt_layers = pkt_config.get('pkt_layers')
-        _layers = self.___get_pkt_layers(pkt_type)
+        _layers = self.__get_pkt_layers(pkt_type)
         if pkt_type not in Packet.def_packet.keys():
             pkt = Packet()
             pkt.pkt_cfgload = True
@@ -310,8 +325,18 @@ class L3fwdBase(object):
             dmac = self.__vf_ports_info[port_id]['vf_mac']
             layer = {'ether': {'src': smac, 'dst': dmac, }, }
         else:
-            dmac = self.dut.get_mac_address(port_id)
-            layer = {'ether': {'dst': dmac, }, }
+            if self.__bin_type is BIN_TYPE.PMD and self.kdriver == 'ice':
+                # testpmd can't use port real mac address to create traffic packet when use CVL nics.
+                dmacs = [
+                    "52:00:00:00:00:00",
+                    "52:00:00:00:00:01",
+                    "52:00:00:00:00:02",
+                    "52:00:00:00:00:03",
+                    ]
+                layer = {'ether': {'dst': dmacs[self.__valports.index(port_id)], }, }
+            else:
+                dmac = self.dut.get_mac_address(port_id)
+                layer = {'ether': {'dst': dmac, }, }
         return layer
 
     def __preset_flows_configs(self):
@@ -540,25 +565,51 @@ class L3fwdBase(object):
         self.__vf_ports_info = None
 
     def __preset_dpdk_compilation(self):
-        # Update config file and rebuild to get best perf on FVL
-        if self.__mode is SUITE_TYPE.PF:
-            if self.nic in ["fortville_spirit", "fortville_eagle", "fortville_25g"]:
-                self.dut.set_build_options({'RTE_LIBRTE_I40E_16BYTE_RX_DESC': 'y'})
-            elif self.nic in ["columbiaville_100g", "columbiaville_25g", "columbiaville_25gx2"]:
-                self.dut.set_build_options({'RTE_LIBRTE_ICE_16BYTE_RX_DESC': 'y'})
-            self.dut.build_install_dpdk(self.target)
+        # set compile flag and rebuild to get best perf
+        compile_flags = {}
+        # RX_DESC
+        if self.__compile_rx_desc:
+            rx_desc_flag = 'RTE_LIBRTE_{nic_drv}_{bit}BYTE_RX_DESC'.format(**{
+                'nic_drv': self.kdriver.upper(),
+                'bit': self.__compile_rx_desc or 32,
+            })
+            if self.__compile_rx_desc == 32:
+                msg = f"{rx_desc_flag} is dpdk default compile flag, ignore this compile flag"
+                self.logger.warning(msg)
+            else:
+                compile_flags[rx_desc_flag] = 'y'
+        # AVX flag
+        if self.__bin_type is BIN_TYPE.PMD:
+            if self.__compile_avx:
+                compile_flags[f"RTE_ENABLE_{self.__compile_avx.upper()}"] = 'y'
+        if not compile_flags:
+            return
+        self.dut.set_build_options(compile_flags)
+        self.dut.build_install_dpdk(self.target)
 
     def __restore_compilation(self):
-        if self.__mode is SUITE_TYPE.PF:
-            if self.nic in ["fortville_spirit", "fortville_eagle", "fortville_25g"]:
-                self.dut.set_build_options({'RTE_LIBRTE_ICE_16BYTE_RX_DESC': 'n'})
-            elif self.nic in ["columbiaville_100g", "columbiaville_25g", "columbiaville_25gx2"]:
-                self.dut.set_build_options({'RTE_LIBRTE_ICE_16BYTE_RX_DESC': 'n'})
-            self.dut.build_install_dpdk(self.target)
+        compile_flags = {}
+        # RX_DESC
+        if self.__compile_rx_desc:
+            rx_desc_flag = 'RTE_LIBRTE_{nic_drv}_{bit}BYTE_RX_DESC'.format(**{
+                'nic_drv': self.kdriver.upper(),
+                'bit': self.__compile_rx_desc or 32,
+            })
+            if self.__compile_rx_desc == 32:
+                msg = f"{rx_desc_flag} is dpdk default compile flag, ignore this compile flag"
+                self.logger.warning(msg)
+            else:
+                compile_flags[rx_desc_flag] = 'n'
+        # AVX flag
+        if self.__bin_type is BIN_TYPE.PMD:
+            if self.__compile_avx:
+                compile_flags[f"RTE_ENABLE_{self.__compile_avx.upper()}"] = 'n'
+        if not compile_flags:
+            return
+        self.dut.set_build_options(compile_flags)
+        self.dut.build_install_dpdk(self.target)
 
-    def __preset_compilation(self):
-        # Update config file and rebuild to get best perf on FVL
-        self.__preset_dpdk_compilation()
+    def __preset_l3fwd_compilation(self):
         # init l3fwd binary file
         if self.nic not in ["columbiaville_100g", "columbiaville_25g", "columbiaville_25gx2"]:
             self.logger.info(
@@ -575,22 +626,33 @@ class L3fwdBase(object):
                 "./examples/l3fwd/l3fwd.h"))
         if self.__mode is SUITE_TYPE.VF:
             self.__l3fwd_lpm = self.__l3fwd_em = \
-                self.__init_l3fwd(MATCH_MODE.EM, rename=False)
+                self.__l3fwd_init(MATCH_MODE.EM, rename=False)
             # init testpmd
             if self.__pf_driver is not NIC_DRV.PCI_STUB:
-                self.__init_testpmd()
+                self.__init_host_testpmd()
         else:
-            self.__l3fwd_em = self.__init_l3fwd(MATCH_MODE.EM)
-            self.__l3fwd_lpm = self.__init_l3fwd(MATCH_MODE.LPM)
+            self.__l3fwd_em = self.__l3fwd_init(MATCH_MODE.EM)
+            self.__l3fwd_lpm = self.__l3fwd_init(MATCH_MODE.LPM)
 
-    def __init_testpmd(self):
+    def __preset_compilation(self):
+        # Update compile config file and rebuild to get best perf on different nics
+        self.__preset_dpdk_compilation()
+        if self.__bin_type is BIN_TYPE.L3FWD:
+            self.__preset_l3fwd_compilation()
+
+    def __init_host_testpmd(self):
+        '''
+        apply under vf testing scenario
+        '''
         self.__pmd_session_name = 'testpmd'
         self.__pmd_session = self.dut.create_session(self.__pmd_session_name)
-        self.__testpmd = "{}/{}/app/testpmd".format(
-            self.__target_dir, self.dut.target)
+        self.__host_testpmd = os.path.join(self.__target_dir,
+            self.dut.apps_name['test-pmd'])
 
-    def __start_testpmd(self):
+    def __start_host_testpmd(self):
         """
+        apply under vf testing scenario
+
         require enough PF ports,using kernel or dpdk driver, create 1 VF from each PF.
         """
         corelist = self.dut.get_core_list(
@@ -607,32 +669,91 @@ class L3fwdBase(object):
             "--file-prefix={prefix} "
             "{whitelist} "
             "-- -i ").format(**{
-                'bin': self.__testpmd,
+                'bin': self.__host_testpmd,
                 'core_mask': core_mask,
                 'mem_channel': self.dut.get_memory_channels(),
                 'memsize': mem_size,
                 'whitelist': self.__get_testpmd_whitelist(),
                 'prefix': 'pf', })
-        self.__pmd_con([cmd, "testpmd> ", 120])
+        self.__host_pmd_con([cmd, "testpmd> ", 120])
         self.__is_pmd_on = True
         index = 0
         for _, info in self.__vf_ports_info.items():
             cmd = 'set vf mac addr %d 0 %s' % (index, info.get('vf_mac'))
-            self.__pmd_con([cmd, "testpmd> ", 15])
+            self.__host_pmd_con([cmd, "testpmd> ", 15])
             index += 1
-        self.__pmd_con(['start', "testpmd> ", 15])
+        self.__host_pmd_con(['start', "testpmd> ", 15])
         time.sleep(1)
 
-    def __close_testpmd(self):
+    def __close_host_testpmd(self):
         """
+        apply under vf testing scenario
+
         destroy the setup VFs
         """
         if not self.__is_pmd_on:
             return
-        self.__pmd_con(['quit', '# ', 15])
+        self.__host_pmd_con(['quit', '# ', 15])
         self.__is_pmd_on = False
 
-    def __init_l3fwd(self, mode, rename=True):
+    def __get_topo_option(self):
+        port_num = len(re.findall('-w', self.__bin_ps_white_list)) \
+            if self.__bin_ps_white_list else len(self.__valports)
+        return 'loop' if port_num == 1 else 'chained'
+
+    def __testpmd_start(self, mode, core_mask, config, frame_size):
+        # use test pmd
+        bin = os.path.join(self.__target_dir,
+                           self.dut.apps_name['test-pmd'])
+        fwd_mode, _config = config
+        port_topo = "--port-topology={}".format(self.__get_topo_option())
+        command_line = (
+            "{bin} "
+            "-v "
+            "-c {cores} "
+            "-n {channel} "
+            "{whitelist}"
+            "-- -i "
+            "{config} "
+            "{port_topo} "
+            "").format(**{
+                'bin': bin,
+                'cores': core_mask,
+                'channel': self.dut.get_memory_channels(),
+                'whitelist': self.__bin_ps_white_list if self.__bin_ps_white_list else '',
+                'port_mask': utils.create_mask(self.__valports),
+                'config': _config,
+                'port_topo': port_topo,
+                })
+        if self.__bin_ps_restart_check(command_line):
+            return
+        self.d_con([command_line, 'testpmd>', 60])
+        self.__is_bin_ps_on = True
+        wait_time = self.__bin_ps_wait_up if self.__bin_ps_wait_up else \
+                    2 * len(self.__valports)
+        self.logger.debug(
+            f"wait {wait_time} seconds for port link up")
+        time.sleep(wait_time)
+        _cmds = [
+            'set fwd {}'.format(fwd_mode),
+            'start',
+        ]
+        [self.dut.send_expect(cmd, 'testpmd>') for cmd in _cmds]
+
+    def __testpmd_close(self):
+        if not self.__is_bin_ps_on:
+            return
+        if not self.__bin_ps_restart and self.__pre_bin_ps_cmd:
+            return
+        _cmds = [
+            'show port stats all',
+            'stop',
+        ]
+        [self.dut.send_expect(cmd, 'testpmd>') for cmd in _cmds]
+        self.dut.send_expect('quit', '# ')
+        self.__is_bin_ps_on = False
+
+    def __l3fwd_init(self, mode, rename=True):
         """
         Prepare long prefix match table, __replace P(x) port pattern
         """
@@ -652,7 +773,7 @@ class L3fwdBase(object):
         l3fwd_bin = os.path.join("./" + self.app_name + l3fwd_method)
         return l3fwd_bin
 
-    def __start_l3fwd(self, mode, core_mask, config, frame_size):
+    def __l3fwd_start(self, mode, core_mask, config, frame_size):
         bin = self.__l3fwd_em if mode is MATCH_MODE.EM else self.__l3fwd_lpm
         # Start L3fwd application
         command_line = (
@@ -668,50 +789,70 @@ class L3fwdBase(object):
                 'bin': bin,
                 'cores': core_mask,
                 'channel': self.dut.get_memory_channels(),
-                'whitelist': self.__l3fwd_white_list if self.__l3fwd_white_list else '',
+                'whitelist': self.__bin_ps_white_list if self.__bin_ps_white_list else '',
                 'port_mask': utils.create_mask(self.__valports),
                 'config': config, })
-        if self.nic in ["niantic", "columbiaville_100g", "columbiaville_25g", "columbiaville_25gx2"]:
+        suppored_nics = [
+            "niantic",
+            "columbiaville_100g", "columbiaville_25g", "columbiaville_25gx2",
+        ]
+        if self.nic in suppored_nics or self.__mode is SUITE_TYPE.VF:
             command_line += " --parse-ptype"
         if frame_size > 1518:
             command_line += " --enable-jumbo --max-pkt-len %d" % frame_size
         # ignore duplicate start binary with the same option
-        if self.__l3fwd_restart_check(command_line):
+        if self.__bin_ps_restart_check(command_line):
             return
         self.d_con([command_line, "L3FWD:", 120])
-        self.__is_l3fwd_on = True
+        self.__is_bin_ps_on = True
         # wait several second for l3fwd checking ports link status.
         # It is aimed to make sure packet generator detect link up status.
-        wait_time = self.__l3fwd_wait_up if self.__l3fwd_wait_up else \
+        wait_time = self.__bin_ps_wait_up if self.__bin_ps_wait_up else \
                     2 * len(self.__valports)
         self.logger.debug(
             f"wait {wait_time} seconds for port link up")
         time.sleep(wait_time)
 
-    def __l3fwd_restart_check(self, command_line):
-        if self.__l3fwd_restart:
-            self.__pre_l3fwd_cmd = None
+    def __l3fwd_close(self):
+        if not self.__is_bin_ps_on:
+            return
+        if not self.__bin_ps_restart and self.__pre_bin_ps_cmd:
+            return
+        self.d_con(["^C", '# ', 25])
+        self.__is_bin_ps_on = False
+
+    def __bin_ps_start(self, mode, core_mask, config, frame_size):
+        bin_methods = {
+            BIN_TYPE.PMD: self.__testpmd_start,
+            BIN_TYPE.L3FWD: self.__l3fwd_start,
+        }
+        start_func = bin_methods.get(self.__bin_type)
+        start_func(mode, core_mask, config, frame_size)
+
+    def __bin_ps_close(self):
+        bin_methods = {
+            BIN_TYPE.PMD: self.__testpmd_close,
+            BIN_TYPE.L3FWD: self.__l3fwd_close,
+        }
+        close_func = bin_methods.get(self.__bin_type)
+        close_func()
+
+    def __bin_ps_restart_check(self, command_line):
+        if self.__bin_ps_restart:
+            self.__pre_bin_ps_cmd = None
             return False
 
-        if self.__pre_l3fwd_cmd and self.__pre_l3fwd_cmd == command_line:
+        if self.__pre_bin_ps_cmd and self.__pre_bin_ps_cmd == command_line:
             self.logger.debug(
-                ('<{}> is the same command as previous one, '
-                 'ignore re-start l3fwd').format(command_line))
+                (f'<{command_line}> is the same command as previous one, '
+                 f'ignore re-start {self.__bin_type.value}'))
             return True
         else:
-            self.__pre_l3fwd_cmd = None
-            self.__close_l3fwd()
-            self.__pre_l3fwd_cmd = command_line
+            self.__pre_bin_ps_cmd = None
+            self.__bin_ps_close()
+            self.__pre_bin_ps_cmd = command_line
             return False
 
-    def __close_l3fwd(self):
-        if not self.__is_l3fwd_on:
-            return
-        if not self.__l3fwd_restart and self.__pre_l3fwd_cmd:
-            return
-        self.d_con(["^C", '# ', 25])
-        self.__is_l3fwd_on = False
-
     def __json_rfc2544(self, value):
         return {"unit": "Mpps", "name": "Rfc2544",
                 "value": round(value[0], 3),
@@ -768,19 +909,30 @@ class L3fwdBase(object):
         self.__json_results[case_name] = case_result
 
     def l3fwd_save_results(self, json_file=None):
+        _json_file = json_file if json_file else 'l3fwd_result.json'
+        self.perf_test_save_results(_json_file)
+
+    def perf_test_save_results(self, json_file=None):
         if not self.__json_results:
             msg = 'json results data is empty'
             self.logger.error(msg)
             return
         _js_file = os.path.join(
             self.__output_path,
-            json_file if json_file else 'l3fwd_result.json')
+            json_file if json_file else f'{self.suite_name}_result.json')
         with open(_js_file, 'w') as fp:
             json.dump(self.__json_results, fp, indent=4,
                       separators=(',', ': '),
                       sort_keys=True)
 
-    def __display_suite_result(self, data, mode):
+    def __display_mode_name(self, mode):
+        if self.__bin_type is BIN_TYPE.PMD:
+            mode_name = 'L3 fixed' if mode is MATCH_MODE.EM else "L3 random"
+        else:
+            mode_name = mode.value
+        return mode_name
+
+    def __display_suite_result(self, data):
         values = data.get('values')
         title = data.get('title')
         max_length = sum([len(item) + 5 for item in title])
@@ -845,7 +997,7 @@ class L3fwdBase(object):
         _data = {
             'title': title,
             'values': values}
-        self.__display_suite_result(_data, mode)
+        self.__display_suite_result(_data)
 
     def __check_rfc2544_result(self, stm_name, data, mode):
         if not data:
@@ -860,7 +1012,7 @@ class L3fwdBase(object):
                 self.__test_content.get('expected_rfc2544', {}).get(
                 self.__cur_case, {}).get(self.__nic_name, {}).get(
                 config, {}).get(str(frame_size), {})
-            zero_loss_rate, tx_pkts, rx_pkts, pps = result if result else [None] * 3
+            zero_loss_rate, tx_pkts, rx_pkts, pps = result if result else [None] * 4
             zero_loss_rate = zero_loss_rate or 0
             mpps = pps / 1000000.0
             # expected line rate
@@ -873,7 +1025,7 @@ class L3fwdBase(object):
             expected_rate = float(expected_cfg.get('rate') or 100.0)
             values.append([
                 config, frame_size, mode.upper(),
-                str(linerate * expected_rate /100),
+                str(linerate * expected_rate / 100),
                 str(mpps),
                 str(expected_rate),
                 str(actual_rate_percent),
@@ -897,7 +1049,7 @@ class L3fwdBase(object):
         # Mode: LPM/EM
         # Expected Throughput (Mpps)  :  Max linerate throughput value *  'Expected LineRate %'
         # Actual Throughput (Mpps)  :  actual run throughput value on the zero loss rate
-        # Expected LineRate %  :  which config in l3fwd_lpm_ipv4_rfc2544.cfg
+        # Expected LineRate %  :  which config in <suite name>.cfg
         # Actual LineRate %  :  actual run zero loss rate
         # tx_pkts :  send pkts num
         # rx_pkts :  received pkts num
@@ -916,7 +1068,7 @@ class L3fwdBase(object):
         _data = {
             'title': title,
             'values': values}
-        self.__display_suite_result(_data, mode)
+        self.__display_suite_result(_data)
 
     def ms_throughput(self, l3_proto, mode):
         except_content = None
@@ -924,24 +1076,27 @@ class L3fwdBase(object):
             test_content = self.__test_content.get('port_configs')
             results = []
             for config, core_mask, port_conf, frame_size in test_content:
-                # Start L3fwd application
+                # Start application binary process
                 self.logger.info(
-                    ("Executing l3fwd with {0} mode, {1} ports, "
+                    ("Executing {4} with {0} mode, {1} ports, "
                      "{2} and {3} frame size").format(
-                        mode, len(self.__valports), config, frame_size))
-                self.__start_l3fwd(mode, core_mask, port_conf, frame_size)
+                        self.__display_mode_name(mode),
+                        len(self.__valports), config, frame_size,
+                        self.__bin_type.value,
+                ))
+                self.__bin_ps_start(mode, core_mask, port_conf, frame_size)
                 result = self.__throughput(l3_proto, mode, frame_size)
-                # Stop L3fwd
-                self.__close_l3fwd()
+                # Stop binary process
+                self.__bin_ps_close()
                 if result:
                     results.append([config, frame_size, result])
-            self.__check_throughput_result(l3_proto, results, mode.value)
+            self.__check_throughput_result(l3_proto, results, self.__display_mode_name(mode))
         except Exception as e:
             self.logger.error(traceback.format_exc())
             except_content = e
         finally:
-            self.__pre_l3fwd_cmd = None
-            self.__close_l3fwd()
+            self.__pre_bin_ps_cmd = None
+            self.__bin_ps_close()
 
         # re-raise verify exception result
         if except_content:
@@ -953,24 +1108,27 @@ class L3fwdBase(object):
             test_content = self.__test_content.get('port_configs')
             results = []
             for config, core_mask, port_conf, frame_size in test_content:
-                # Start L3fwd application
+                # Start application binary process
                 self.logger.info(
-                    ("Executing l3fwd with {0} mode, {1} ports, "
+                    ("Executing {4} with {0} mode, {1} ports, "
                      "{2} and {3} frame size").format(
-                        mode, len(self.__valports), config, frame_size))
-                self.__start_l3fwd(mode, core_mask, port_conf, frame_size)
+                        self.__display_mode_name(mode),
+                        len(self.__valports), config, frame_size,
+                        self.__bin_type.value,
+                ))
+                self.__bin_ps_start(mode, core_mask, port_conf, frame_size)
                 result = self.__rfc2544(config, l3_proto, mode, frame_size)
-                # Stop L3fwd
-                self.__close_l3fwd()
+                # Stop binary process
+                self.__bin_ps_close()
                 if result:
                     results.append([config, frame_size, result])
-            self.__check_rfc2544_result(l3_proto, results, mode.value)
+            self.__check_rfc2544_result(l3_proto, results, self.__display_mode_name(mode))
         except Exception as e:
             self.logger.error(traceback.format_exc())
             except_content = e
         finally:
-            self.__pre_l3fwd_cmd = None
-            self.__close_l3fwd()
+            self.__pre_bin_ps_cmd = None
+            self.__bin_ps_close()
 
         # re-raise verify exception result
         if except_content:
@@ -979,7 +1137,7 @@ class L3fwdBase(object):
     def __parse_port_config(self, config, cores_for_all):
         '''
         [n]C/[mT]-[i]Q
-        
+
             n: how many physical core use for polling.
             m: how many cpu thread use for polling, if Hyper-threading disabled
                 in BIOS, m equals n, if enabled, m is 2 times as n.
@@ -1020,12 +1178,15 @@ class L3fwdBase(object):
     def __get_core_list(self, thread_num, cores, socket):
         corelist = self.dut.get_core_list(
             cores, socket if cores.startswith('1S') else -1)
-        corelist = corelist[self.__core_offset*thread_num:]
+        if self.__bin_type is BIN_TYPE.PMD:
+            corelist = corelist[(self.__core_offset - 1)*thread_num:]
+        else:
+            corelist = corelist[self.__core_offset*thread_num:]
         if '2T' in cores:
             corelist = corelist[0::2] + corelist[1::2]
         return corelist
 
-    def __get_test_configs(self, options, ports, socket, cores_for_all):
+    def __get_test_configs(self, options, ports, socket, cores_for_all, fixed_config=None):
         if not options:
             msg = "'test_parameters' not set in suite configuration file"
             raise VerifyFailure(msg)
@@ -1038,19 +1199,33 @@ class L3fwdBase(object):
             grp = [list(item)
                    for item in product(range(ports), range(queues_per_port))]
             corelist = self.__get_core_list(thread_num, cores, socket)
-            cores_mask = utils.create_mask(corelist)
-            total = len(grp)
-            _corelist = (corelist * (total // len(corelist) + 1))[:total]
-            # ignore first 2 cores
-            [grp[index].append(core)
-             for index, core in enumerate(_corelist)]
+
+            if self.__bin_type == BIN_TYPE.PMD:
+                # keep only one logic core of the main core
+                cores_mask = utils.create_mask(corelist[thread_num - 1:])
+                [configs.append([
+                    test_item,
+                    cores_mask,
+                    [fixed_config[0],
+                     fixed_config[1] +
+                     ' --rxq={0} --txq={0}'.format(queues_per_port) +
+                     " --nb-cores={}".format(len(corelist) - thread_num)
+                     ],
+                    frame_size, ]) for frame_size in _frame_sizes]
             # (port,queue,lcore)
-            [configs.append([
-                test_item,
-                cores_mask,
-                ','.join(["({0},{1},{2})".format(port, queue, core)
-                          for port, queue, core in grp]),
-                frame_size, ]) for frame_size in _frame_sizes]
+            else:
+                cores_mask = utils.create_mask(corelist)
+                total = len(grp)
+                # ignore first 2 cores
+                _corelist = (corelist * (total // len(corelist) + 1))[:total]
+                [grp[index].append(core)
+                 for index, core in enumerate(_corelist)]
+                [configs.append([
+                    test_item,
+                    cores_mask,
+                    ','.join(["({0},{1},{2})".format(port, queue, core)
+                              for port, queue, core in grp]),
+                    frame_size, ]) for frame_size in _frame_sizes]
         return configs, sorted(set(frame_sizes_grp))
 
     def __get_suite_vf_pf_driver(self, test_content):
@@ -1130,24 +1305,40 @@ class L3fwdBase(object):
         if self.__mode is SUITE_TYPE.VF:
             self.__get_vf_test_content_from_cfg(test_content)
         # binary file process setting
-        self.__l3fwd_wait_up = test_content.get('l3fwd_wait_up', 0)
-        self.__l3fwd_restart = test_content.get('l3fwd_restart', True)
+        if self.__bin_type is BIN_TYPE.L3FWD:
+            self.__bin_ps_wait_up = test_content.get('l3fwd_wait_up', 0)
+            self.__bin_ps_restart = test_content.get('l3fwd_restart', True)
+        else:
+            self.__bin_ps_wait_up = test_content.get('bin_ps_wait_up', 0)
+            self.__bin_ps_restart = test_content.get('bin_ps_restart', True)
         self.__traffic_stop_wait_time = \
             test_content.get('traffic_stop_wait_time', 0)
-        # parse port config of l3fwd suite
+        # parse port config of binary process suite
         cores_for_all = test_content.get('cores_for_all', False)
+        self.__compile_rx_desc = test_content.get('compile_rx_desc')
+        if self.__bin_type == BIN_TYPE.PMD:
+            self.__compile_avx = test_content.get('compile_avx')
+            forwarding_mode = test_content.get('forwarding_mode') or 'io'
+            descriptor_numbers = test_content.get('descriptor_numbers') or {'txd': 2048, 'rxd': 2048}
+            fixed_config = [
+                forwarding_mode,
+                "--rxd={rxd} --txd={txd}".format(**descriptor_numbers)]
+            self.__core_offset += 1
+        else:
+            fixed_config = None
         port_configs, frame_sizes = self.__get_test_configs(
             test_content.get('test_parameters'),
             len(self.__valports),
             self.__socket,
-            cores_for_all)
+            cores_for_all,
+            fixed_config=fixed_config)
         test_content['port_configs'] = port_configs
         test_content['frame_sizes'] = frame_sizes
         self.logger.debug(pformat(test_content))
 
         return test_content
 
-    def __get_l3fwd_whitelist(self, port_list=None):
+    def __get_bin_ps_whitelist(self, port_list=None):
         whitelist = ''
         if self.__mode is SUITE_TYPE.PF:
             if not port_list:
@@ -1183,7 +1374,7 @@ class L3fwdBase(object):
             self.__valports = port_list
         return port_list
 
-    def l3fwd_preset_test_environment(self, test_content):
+    def perf_preset_test_environment(self, test_content):
         # if user set port list in cfg file, use
         port_list = self.__preset_port_list(test_content)
         # get test content
@@ -1194,20 +1385,20 @@ class L3fwdBase(object):
         if self.__mode is SUITE_TYPE.VF:
             self.__vf_init()
             self.__vf_create()
-            self.__l3fwd_white_list = self.__get_l3fwd_whitelist()
+            self.__bin_ps_white_list = self.__get_bin_ps_whitelist()
             if self.__pf_driver is not NIC_DRV.PCI_STUB:
-                self.__start_testpmd()
+                self.__start_host_testpmd()
         else:
-            self.__l3fwd_white_list = self.__get_l3fwd_whitelist(port_list)
+            self.__bin_ps_white_list = self.__get_bin_ps_whitelist(port_list)
         # config streams
         self.__streams = self.__preset_streams()
 
-    def l3fwd_destroy_resource(self):
+    def perf_destroy_resource(self):
         if self.__mode is SUITE_TYPE.VF:
             if self.__pf_driver is NIC_DRV.PCI_STUB:
                 pass
             else:
-                self.__close_testpmd()
+                self.__close_host_testpmd()
                 if self.__pmd_session:
                     self.dut.close_session(self.__pmd_session)
                     self.__pmd_session = None
@@ -1215,10 +1406,10 @@ class L3fwdBase(object):
         if self.__mode is SUITE_TYPE.PF:
             self.__restore_compilation()
 
-    def l3fwd_set_cur_case(self, name):
+    def perf_set_cur_case(self, name):
         self.__cur_case = name
 
-    def l3fwd_reset_cur_case(self):
+    def perf_reset_cur_case(self):
         self.__cur_case = None
 
     @property
@@ -1236,6 +1427,7 @@ class L3fwdBase(object):
         supported_num = {
             PKTGEN_TREX: [2, 4],
             PKTGEN_IXIA: [1, 2, 4],
+            PKTGEN_IXIA_NETWORK: [1, 2, 4],
         }
         if not self.is_pktgen_on:
             msg = 'not using pktgen'
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 20/27] tests/perf_test: save rfc2544 expected throughput
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (18 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 19/27] tests/perf_test: cover testpmd testing scenario yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 21/27] tests/l3fwd_em: update script yufengmx
                   ` (7 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


save rfc2544 expected throughput in json file.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/perf_test_base.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/perf_test_base.py b/tests/perf_test_base.py
index 3d5cae85..20f3e152 100644
--- a/tests/perf_test_base.py
+++ b/tests/perf_test_base.py
@@ -857,6 +857,7 @@ class PerfTestBase(object):
         return {"unit": "Mpps", "name": "Rfc2544",
                 "value": round(value[0], 3),
                 "delta": round(value[1], 3),
+                "Expected Throughput": round(value[2], 3),
                 }
 
     def __json_throughput(self, value):
@@ -1039,7 +1040,7 @@ class PerfTestBase(object):
                 else 'failed'
             js_results.append(
                 [status,
-                 [mpps, actual_rate_percent - expected_rate], actual_rate_percent,
+                 [mpps, actual_rate_percent - expected_rate, linerate], actual_rate_percent,
                  config, frame_size])
         # save data in json file
         self.__save_rfc2544_result(self.__cur_case, js_results)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 21/27] tests/l3fwd_em: update script
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (19 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 20/27] tests/perf_test: save rfc2544 expected throughput yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 22/27] tests/lpm_ipv4_rfc2544: " yufengmx
                   ` (6 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


update perf testing enum parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_em.py | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/TestSuite_l3fwd_em.py b/tests/TestSuite_l3fwd_em.py
index 2373aa32..4879b503 100644
--- a/tests/TestSuite_l3fwd_em.py
+++ b/tests/TestSuite_l3fwd_em.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -34,10 +34,10 @@ DPDK Test suite.
 Layer-3 forwarding test script.
 """
 from test_case import TestCase
-from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+from perf_test_base import PerfTestBase, MATCH_MODE, IP_TYPE
 
 
-class TestL3fwdEm(TestCase, L3fwdBase):
+class TestL3fwdEm(TestCase, PerfTestBase):
     #
     # Test cases.
     #
@@ -57,9 +57,9 @@ class TestL3fwdEm(TestCase, L3fwdBase):
         cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
         self.verify(cores is not None, "Insufficient cores for speed testing")
         # init l3fwd common base class parameters
-        self.l3fwd_init(valports, socket)
+        PerfTestBase.__init__(self, valports, socket)
         # preset testing environment
-        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+        self.perf_preset_test_environment(self.get_suite_cfg())
 
     def tear_down_all(self):
         """
@@ -78,12 +78,12 @@ class TestL3fwdEm(TestCase, L3fwdBase):
         Run after each test case.
         """
         self.dut.kill_all()
-        self.l3fwd_reset_cur_case()
+        self.perf_reset_cur_case()
 
     def test_perf_throughput_ipv4_em(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_em')
-        self.ms_throughput(l3_proto=L3_IPV4, mode=EM)
+        self.perf_set_cur_case('test_perf_throughput_ipv4_em')
+        self.ms_throughput(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.EM)
 
     def test_perf_throughput_ipv6_em(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_em')
-        self.ms_throughput(l3_proto=L3_IPV6, mode=EM)
+        self.perf_set_cur_case('test_perf_throughput_ipv6_em')
+        self.ms_throughput(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.EM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 22/27] tests/lpm_ipv4_rfc2544: update script
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (20 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 21/27] tests/l3fwd_em: update script yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 23/27] tests/l3fwd_lpm_ipv4: " yufengmx
                   ` (5 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


update perf testing enum parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py b/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
index ed352fd6..5b4a0aa3 100644
--- a/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
+++ b/tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -34,10 +34,10 @@ DPDK Test suite.
 Layer-3 forwarding test script.
 """
 from test_case import TestCase
-from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+from perf_test_base import PerfTestBase, MATCH_MODE, IP_TYPE
 
 
-class TestL3fwdLpmIpv4Rfc2544(TestCase, L3fwdBase):
+class TestL3fwdLpmIpv4Rfc2544(TestCase, PerfTestBase):
 
     #
     # Test cases.
@@ -58,9 +58,9 @@ class TestL3fwdLpmIpv4Rfc2544(TestCase, L3fwdBase):
         cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
         self.verify(cores is not None, "Insufficient cores for speed testing")
         # init l3fwd common base class parameters
-        self.l3fwd_init(valports, socket)
+        PerfTestBase.__init__(self, valports, socket)
         # preset testing environment
-        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+        self.perf_preset_test_environment(self.get_suite_cfg())
 
     def tear_down_all(self):
         """
@@ -79,8 +79,8 @@ class TestL3fwdLpmIpv4Rfc2544(TestCase, L3fwdBase):
         Run after each test case.
         """
         self.dut.kill_all()
-        self.l3fwd_reset_cur_case()
+        self.perf_reset_cur_case()
 
     def test_perf_rfc2544_ipv4_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv4_lpm')
-        self.qt_rfc2544(l3_proto=L3_IPV4, mode=LPM)
+        self.perf_set_cur_case('test_perf_rfc2544_ipv4_lpm')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 23/27] tests/l3fwd_lpm_ipv4: update script
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (21 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 22/27] tests/lpm_ipv4_rfc2544: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 24/27] tests/l3fwd_lpm_ipv6: " yufengmx
                   ` (4 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


update perf testing enum parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_lpm_ipv4.py | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/TestSuite_l3fwd_lpm_ipv4.py b/tests/TestSuite_l3fwd_lpm_ipv4.py
index 942a6a71..3b881e0a 100644
--- a/tests/TestSuite_l3fwd_lpm_ipv4.py
+++ b/tests/TestSuite_l3fwd_lpm_ipv4.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -34,10 +34,10 @@ DPDK Test suite.
 Layer-3 forwarding test script.
 """
 from test_case import TestCase
-from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+from perf_test_base import PerfTestBase, MATCH_MODE, IP_TYPE
 
 
-class TestL3fwdLpmIpv4(TestCase, L3fwdBase):
+class TestL3fwdLpmIpv4(TestCase, PerfTestBase):
 
     #
     # Test cases.
@@ -58,9 +58,9 @@ class TestL3fwdLpmIpv4(TestCase, L3fwdBase):
         cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
         self.verify(cores is not None, "Insufficient cores for speed testing")
         # init l3fwd common base class parameters
-        self.l3fwd_init(valports, socket)
+        PerfTestBase.__init__(self, valports, socket)
         # preset testing environment
-        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+        self.perf_preset_test_environment(self.get_suite_cfg())
 
     def tear_down_all(self):
         """
@@ -79,8 +79,8 @@ class TestL3fwdLpmIpv4(TestCase, L3fwdBase):
         Run after each test case.
         """
         self.dut.kill_all()
-        self.l3fwd_reset_cur_case()
+        self.perf_reset_cur_case()
 
     def test_perf_throughput_ipv4_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_lpm')
-        self.ms_throughput(l3_proto=L3_IPV4, mode=LPM)
+        self.perf_set_cur_case('test_perf_throughput_ipv4_lpm')
+        self.ms_throughput(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 24/27] tests/l3fwd_lpm_ipv6: update script
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (22 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 23/27] tests/l3fwd_lpm_ipv4: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 25/27] tests/l3fwd: " yufengmx
                   ` (3 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


update perf testing enum parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd_lpm_ipv6.py | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/TestSuite_l3fwd_lpm_ipv6.py b/tests/TestSuite_l3fwd_lpm_ipv6.py
index 890479e1..e53e4f77 100644
--- a/tests/TestSuite_l3fwd_lpm_ipv6.py
+++ b/tests/TestSuite_l3fwd_lpm_ipv6.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -34,10 +34,10 @@ DPDK Test suite.
 Layer-3 forwarding test script.
 """
 from test_case import TestCase
-from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+from perf_test_base import PerfTestBase, MATCH_MODE, IP_TYPE
 
 
-class TestL3fwdLpmIpv6(TestCase, L3fwdBase):
+class TestL3fwdLpmIpv6(TestCase, PerfTestBase):
     #
     # Test cases.
     #
@@ -57,9 +57,9 @@ class TestL3fwdLpmIpv6(TestCase, L3fwdBase):
         cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
         self.verify(cores is not None, "Insufficient cores for speed testing")
         # init l3fwd common base class parameters
-        self.l3fwd_init(valports, socket)
+        PerfTestBase.__init__(self, valports, socket)
         # preset testing environment
-        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+        self.perf_preset_test_environment(self.get_suite_cfg())
 
     def tear_down_all(self):
         """
@@ -78,8 +78,8 @@ class TestL3fwdLpmIpv6(TestCase, L3fwdBase):
         Run after each test case.
         """
         self.dut.kill_all()
-        self.l3fwd_reset_cur_case()
+        self.perf_reset_cur_case()
 
     def test_perf_throughput_ipv6_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_lpm')
-        self.ms_throughput(l3_proto=L3_IPV6, mode=LPM)
+        self.perf_set_cur_case('test_perf_throughput_ipv6_lpm')
+        self.ms_throughput(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 25/27] tests/l3fwd: update script
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (23 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 24/27] tests/l3fwd_lpm_ipv6: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 26/27] tests/vf_l3fwd_kernelpf: " yufengmx
                   ` (2 subsequent siblings)
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


update perf testing enum parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_l3fwd.py | 45 ++++++++++++++++++++--------------------
 1 file changed, 22 insertions(+), 23 deletions(-)

diff --git a/tests/TestSuite_l3fwd.py b/tests/TestSuite_l3fwd.py
index f82d0b69..4efcca09 100644
--- a/tests/TestSuite_l3fwd.py
+++ b/tests/TestSuite_l3fwd.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2010-2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -33,12 +33,11 @@
 DPDK Test suite.
 Layer-3 forwarding test script.
 """
-
 from test_case import TestCase
-from l3fwd_base import L3fwdBase, LPM, EM, L3_IPV6, L3_IPV4
+from perf_test_base import PerfTestBase, MATCH_MODE, IP_TYPE
 
 
-class TestL3fwd(TestCase, L3fwdBase):
+class TestL3fwd(TestCase, PerfTestBase):
 
     #
     # Test cases.
@@ -59,9 +58,9 @@ class TestL3fwd(TestCase, L3fwdBase):
         cores = self.dut.get_core_list("1S/8C/1T", socket=socket)
         self.verify(cores is not None, "Insufficient cores for speed testing")
         # init l3fwd common base class parameters
-        self.l3fwd_init(valports, socket)
+        PerfTestBase.__init__(self, valports, socket)
         # preset testing environment
-        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+        self.perf_preset_test_environment(self.get_suite_cfg())
 
     def tear_down_all(self):
         """
@@ -80,36 +79,36 @@ class TestL3fwd(TestCase, L3fwdBase):
         Run after each test case.
         """
         self.dut.kill_all()
-        self.l3fwd_reset_cur_case()
+        self.perf_reset_cur_case()
 
     def test_perf_rfc2544_ipv4_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv4_lpm')
-        self.qt_rfc2544(l3_proto=L3_IPV4, mode=LPM)
+        self.perf_set_cur_case('test_perf_rfc2544_ipv4_lpm')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
 
     def test_perf_rfc2544_ipv4_em(self):
-        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv4_em')
-        self.qt_rfc2544(l3_proto=L3_IPV4, mode=EM)
+        self.perf_set_cur_case('test_perf_rfc2544_ipv4_em')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.EM)
 
     def test_perf_throughput_ipv4_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_lpm')
-        self.ms_throughput(l3_proto=L3_IPV4, mode=LPM)
+        self.perf_set_cur_case('test_perf_throughput_ipv4_lpm')
+        self.ms_throughput(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
 
     def test_perf_throughput_ipv4_em(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv4_em')
-        self.ms_throughput(l3_proto=L3_IPV4, mode=EM)
+        self.perf_set_cur_case('test_perf_throughput_ipv4_em')
+        self.ms_throughput(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.EM)
 
     def test_perf_rfc2544_ipv6_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv6_lpm')
-        self.qt_rfc2544(l3_proto=L3_IPV6, mode=LPM)
+        self.perf_set_cur_case('test_perf_rfc2544_ipv6_lpm')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.LPM)
 
     def test_perf_rfc2544_ipv6_em(self):
-        self.l3fwd_set_cur_case('test_perf_rfc2544_ipv6_em')
-        self.qt_rfc2544(l3_proto=L3_IPV6, mode=EM)
+        self.perf_set_cur_case('test_perf_rfc2544_ipv6_em')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.EM)
 
     def test_perf_throughput_ipv6_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_lpm')
-        self.ms_throughput(l3_proto=L3_IPV6, mode=LPM)
+        self.perf_set_cur_case('test_perf_throughput_ipv6_lpm')
+        self.ms_throughput(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.LPM)
 
     def test_perf_throughput_ipv6_em(self):
-        self.l3fwd_set_cur_case('test_perf_throughput_ipv6_em')
-        self.ms_throughput(l3_proto=L3_IPV6, mode=EM)
+        self.perf_set_cur_case('test_perf_throughput_ipv6_em')
+        self.ms_throughput(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.EM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 26/27] tests/vf_l3fwd_kernelpf: update script
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (24 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 25/27] tests/l3fwd: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-01-25  8:44 ` [dts] [PATCH V1 27/27] tests/testpmd_perf: upload script yufengmx
  2021-02-19  7:02 ` [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing Tu, Lijuan
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


update perf testing enum parameters and methods name.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_vf_l3fwd_kernelpf.py | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tests/TestSuite_vf_l3fwd_kernelpf.py b/tests/TestSuite_vf_l3fwd_kernelpf.py
index b2ae7ec0..4cd44aaa 100644
--- a/tests/TestSuite_vf_l3fwd_kernelpf.py
+++ b/tests/TestSuite_vf_l3fwd_kernelpf.py
@@ -1,6 +1,6 @@
 # BSD LICENSE
 #
-# Copyright(c) 2020 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -32,10 +32,10 @@
 import os
 import time
 from test_case import TestCase
-from l3fwd_base import L3fwdBase, IP_TYPE, MATCH_MODE, SUITE_TYPE
+from perf_test_base import PerfTestBase, IP_TYPE, MATCH_MODE, SUITE_TYPE
 
 
-class TestVfL3fwdKernelPf(TestCase, L3fwdBase):
+class TestVfL3fwdKernelPf(TestCase, PerfTestBase):
     #
     # Test cases.
     #
@@ -62,15 +62,15 @@ class TestVfL3fwdKernelPf(TestCase, L3fwdBase):
         cores = self.dut.get_core_list("1S/6C/1T", socket=socket)
         self.verify(cores, "Requested 6 cores failed")
         # init l3fwd common base class parameters
-        self.l3fwd_init(valports, socket, mode=SUITE_TYPE.VF)
+        PerfTestBase.__init__(self, valports, socket, mode=SUITE_TYPE.VF)
         # preset testing environment
-        self.l3fwd_preset_test_environment(self.get_suite_cfg())
+        self.perf_preset_test_environment(self.get_suite_cfg())
 
     def tear_down_all(self):
         """
         Run after each test suite.
         """
-        self.l3fwd_destroy_resource()
+        self.perf_destroy_resource()
         self.l3fwd_save_results(json_file="{}.json".format(self.suite_name))
 
     def set_up(self):
@@ -84,8 +84,8 @@ class TestVfL3fwdKernelPf(TestCase, L3fwdBase):
         Run after each test case.
         """
         self.dut.kill_all()
-        self.l3fwd_reset_cur_case()
+        self.perf_reset_cur_case()
 
     def test_perf_vf_throughput_ipv4_lpm(self):
-        self.l3fwd_set_cur_case('test_perf_vf_throughput_ipv4_lpm')
+        self.perf_set_cur_case('test_perf_vf_throughput_ipv4_lpm')
         self.ms_throughput(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* [dts] [PATCH V1 27/27] tests/testpmd_perf: upload script.
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (25 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 26/27] tests/vf_l3fwd_kernelpf: " yufengmx
@ 2021-01-25  8:44 ` yufengmx
  2021-02-19  7:02 ` [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing Tu, Lijuan
  27 siblings, 0 replies; 29+ messages in thread
From: yufengmx @ 2021-01-25  8:44 UTC (permalink / raw)
  To: dts, lijuan.tu; +Cc: yufengmx


upload testpmd_perf suite to support testpmd rfc2544 testing scenario.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 tests/TestSuite_testpmd_perf.py | 102 ++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 tests/TestSuite_testpmd_perf.py

diff --git a/tests/TestSuite_testpmd_perf.py b/tests/TestSuite_testpmd_perf.py
new file mode 100644
index 00000000..db2eb3c7
--- /dev/null
+++ b/tests/TestSuite_testpmd_perf.py
@@ -0,0 +1,102 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2021 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+testpmd perf test script.
+"""
+
+from test_case import TestCase
+from perf_test_base import PerfTestBase, BIN_TYPE, MATCH_MODE, IP_TYPE
+
+
+class TestPmdPerf(TestCase, PerfTestBase):
+
+    #
+    # Test cases.
+    #
+    @property
+    def supported_nics(self):
+        return [
+            'niantic',
+            'fortville_25g',
+            'fortville_spirit',
+            'columbiaville_100g',
+            'columbiaville_25g',
+            'ConnectX5_MT4121',
+            'ConnectX4_LX_MT4117',
+        ]
+
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        """
+        self.verify(self.nic in self.supported_nics, "Not required NIC ")
+        # Based on h/w type, choose how many ports to use
+        self.dut_ports = self.dut.get_ports(self.nic)
+        valports = [
+            _ for _ in self.dut_ports if self.tester.get_local_port(_) != -1]
+        self.logger.debug(valports)
+        self.verify_ports_number(valports)
+        # get socket and cores
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+        # init common base class parameters
+        PerfTestBase.__init__(self, valports, socket, bin_type=BIN_TYPE.PMD)
+        # preset testing environment
+        self.perf_preset_test_environment(self.get_suite_cfg())
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        self.perf_test_save_results()
+        self.perf_destroy_resource()
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        self.perf_reset_cur_case()
+
+    def test_perf_rfc2544_ipv4_lpm(self):
+        self.perf_set_cur_case('test_perf_rfc2544_ipv4_lpm')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V4, mode=MATCH_MODE.LPM)
+
+    def test_perf_rfc2544_ipv6_lpm(self):
+        self.perf_set_cur_case('test_perf_rfc2544_ipv6_lpm')
+        self.qt_rfc2544(l3_proto=IP_TYPE.V6, mode=MATCH_MODE.LPM)
-- 
2.21.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

* Re: [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing
  2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
                   ` (26 preceding siblings ...)
  2021-01-25  8:44 ` [dts] [PATCH V1 27/27] tests/testpmd_perf: upload script yufengmx
@ 2021-02-19  7:02 ` Tu, Lijuan
  27 siblings, 0 replies; 29+ messages in thread
From: Tu, Lijuan @ 2021-02-19  7:02 UTC (permalink / raw)
  To: Mo, YufengX, dts



> -----Original Message-----
> From: Mo, YufengX <yufengx.mo@intel.com>
> Sent: 2021年1月25日 16:44
> To: dts@dpdk.org; Tu, Lijuan <lijuan.tu@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts][PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing
> 
> 
> v1:
>  - enable IxNetwork packet generator.
>  - add testpmd rfc2544 testing scenario.
>  - enhance pktgen/trex to support return tx_pps and tx_bps from
> pktgen.measure_throughput.
> 
> yufengmx (27):
>   framework/pktgen: return trex tx stats
>   framework/pktgen: return throughput tx stats
>   framework/pktgen: return throughput tx stats
>   conf/pktgen: enable ixNetwork
>   conf/pktgen: enable ixNetwork
>   conf/l3fwd: add packet types comment
>   conf/testpmd: testpmd perf config
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   conf/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   framework/pktgen: enable ixNetwork
>   tests/perf_test: rename l3fwd_base module
>   tests/perf_test: cover testpmd testing scenario
>   tests/perf_test: save rfc2544 expected throughput
>   tests/l3fwd_em: update script
>   tests/lpm_ipv4_rfc2544: update script
>   tests/l3fwd_lpm_ipv4: update script
>   tests/l3fwd_lpm_ipv6: update script
>   tests/l3fwd: update script
>   tests/vf_l3fwd_kernelpf: update script
>   tests/testpmd_perf: upload script.
> 
>  conf/crbs.cfg                              |   2 +-
>  conf/l3fwd_base.cfg                        |   2 +
>  conf/pktgen.cfg                            |  12 +
>  conf/testpmd_perf.cfg                      | 190 +++++
>  framework/config.py                        |   9 +-
>  framework/ixia_network/__init__.py         | 182 +++++
>  framework/ixia_network/ixnet.py            | 844 +++++++++++++++++++++
>  framework/ixia_network/ixnet_config.py     |  42 +
>  framework/ixia_network/ixnet_stream.py     | 317 ++++++++
>  framework/ixia_network/packet_parser.py    |  93 +++
>  framework/pktgen.py                        |   9 +-
>  framework/pktgen_base.py                   | 103 ++-
>  framework/pktgen_ixia_network.py           | 224 ++++++
>  framework/pktgen_trex.py                   |   4 +-
>  framework/settings.py                      |   5 +-
>  tests/TestSuite_l3fwd.py                   |  45 +-
>  tests/TestSuite_l3fwd_em.py                |  20 +-
>  tests/TestSuite_l3fwd_lpm_ipv4.py          |  16 +-
>  tests/TestSuite_l3fwd_lpm_ipv4_rfc2544.py  |  16 +-
>  tests/TestSuite_l3fwd_lpm_ipv6.py          |  16 +-
>  tests/TestSuite_testpmd_perf.py            | 102 +++
>  tests/TestSuite_vf_l3fwd_kernelpf.py       |  16 +-
>  tests/{l3fwd_base.py => perf_test_base.py} | 483 ++++++++----
>  23 files changed, 2512 insertions(+), 240 deletions(-)  create mode 100644
> conf/testpmd_perf.cfg  create mode 100644
> framework/ixia_network/__init__.py
>  create mode 100644 framework/ixia_network/ixnet.py  create mode 100644
> framework/ixia_network/ixnet_config.py
>  create mode 100644 framework/ixia_network/ixnet_stream.py
>  create mode 100644 framework/ixia_network/packet_parser.py
>  create mode 100644 framework/pktgen_ixia_network.py  create mode 100644
> tests/TestSuite_testpmd_perf.py  rename tests/{l3fwd_base.py =>
> perf_test_base.py} (75%)
> 
> --
> 2.21.0

Applied, thanks


^ permalink raw reply	[flat|nested] 29+ messages in thread

end of thread, other threads:[~2021-02-19  7:03 UTC | newest]

Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-25  8:43 [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 01/27] framework/pktgen: return trex tx stats yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 02/27] framework/pktgen: return throughput " yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 03/27] " yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 04/27] conf/pktgen: enable ixNetwork yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 05/27] " yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 06/27] conf/l3fwd: add packet types comment yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 07/27] conf/testpmd: testpmd perf config yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 08/27] framework/pktgen: enable ixNetwork yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 09/27] " yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 10/27] " yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 11/27] " yufengmx
2021-01-25  8:43 ` [dts] [PATCH V1 12/27] " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 13/27] conf/pktgen: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 14/27] framework/pktgen: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 15/27] " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 16/27] " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 17/27] " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 18/27] tests/perf_test: rename l3fwd_base module yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 19/27] tests/perf_test: cover testpmd testing scenario yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 20/27] tests/perf_test: save rfc2544 expected throughput yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 21/27] tests/l3fwd_em: update script yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 22/27] tests/lpm_ipv4_rfc2544: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 23/27] tests/l3fwd_lpm_ipv4: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 24/27] tests/l3fwd_lpm_ipv6: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 25/27] tests/l3fwd: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 26/27] tests/vf_l3fwd_kernelpf: " yufengmx
2021-01-25  8:44 ` [dts] [PATCH V1 27/27] tests/testpmd_perf: upload script yufengmx
2021-02-19  7:02 ` [dts] [PATCH V1 00/27] dts: enable IxNetwork and enhance perf testing Tu, Lijuan

test suite reviews and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ https://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git