test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs
@ 2019-08-06  7:51 yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 1/7] framework/logger: add pktgen logger and remove duplicate yufengmx
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx

 These commits are internal bugs fixed, which are reported by li,wenjie. 
 
 *. move tester/dut logger close after session close for logger used by session. 
 *. remove duplicate ixia logger setting. 
 *. add pktgen logger setting in logger module. 
 *. set pktgen modules part of information logger display to debug level. 
 *. set PacketGenerator class logger as Ixia class logger. 
 *. remove logger format setting after import libs to fix dts redundant logs. 
 *. remove un-used libs. 
 *. fix typo. 
 *. fix pep8 issue. 

yufengmx (7):
  framework/logger: add pktgen logger and remove duplicate code
  framework/dut: fix logger quit issue
  framework/tester: fix logger quit issue
  framework/pktgen_base: fix internal bug
  framework/pktgen_ixia: fix internal bug
  framework/pktgen_trex: fix internal bug
  framework/pktgen: fix pep8 issue

 framework/dut.py         |   8 +-
 framework/logger.py      |  25 ++++++-
 framework/pktgen.py      |  29 +++-----
 framework/pktgen_base.py |  36 ++++-----
 framework/pktgen_ixia.py | 190 ++++++++++++++++++++++++-----------------------
 framework/pktgen_trex.py | 146 ++++++++++++++++++------------------
 framework/tester.py      |   2 +-
 7 files changed, 225 insertions(+), 211 deletions(-)

-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 1/7] framework/logger: add pktgen logger and remove duplicate
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
@ 2019-08-06  7:51 ` yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 2/7] framework/dut: fix logger quit issue yufengmx
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx

 code

*. remove duplicate ixia logger setting.
*. add pktgen logger setting.
*. fix typo.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/logger.py | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/framework/logger.py b/framework/logger.py
index 603d3aa..32dd954 100644
--- a/framework/logger.py
+++ b/framework/logger.py
@@ -64,6 +64,9 @@ logging.DTS_IXIA_OUTPUT = logging.DEBUG + 5
 logging.DTS_VIRTDUT_CMD = logging.INFO + 6
 logging.DTS_VIRTDUT_OUTPUT = logging.DEBUG + 6
 
+logging.DTS_PKTGEN_CMD = logging.INFO + 7
+logging.DTS_PKTGEN_OUTPUT = logging.DEBUG + 7
+
 logging.addLevelName(logging.DTS_DUT_CMD, 'DTS_DUT_CMD')
 logging.addLevelName(logging.DTS_DUT_OUTPUT, 'DTS_DUT_OUTPUT')
 logging.addLevelName(logging.DTS_DUT_RESULT, 'DTS_DUT_RESULT')
@@ -84,8 +87,8 @@ logging.addLevelName(logging.SUITE_DUT_OUTPUT, 'SUITE_DUT_OUTPUT')
 logging.addLevelName(logging.SUITE_TESTER_CMD, 'SUITE_TESTER_CMD')
 logging.addLevelName(logging.SUITE_TESTER_OUTPUT, 'SUITE_TESTER_OUTPUT')
 
-logging.addLevelName(logging.DTS_IXIA_CMD, 'DTS_IXIA_CMD')
-logging.addLevelName(logging.DTS_IXIA_OUTPUT, 'DTS_IXIA_OUTPUT')
+logging.addLevelName(logging.DTS_PKTGEN_CMD, 'DTS_PKTGEN_CMD')
+logging.addLevelName(logging.DTS_PKTGEN_OUTPUT, 'DTS_PKTGEN_OUTPUT')
 
 date_fmt = '%d/%m/%Y %H:%M:%S'
 RESET_COLOR = '\033[0m'
@@ -149,6 +152,12 @@ class BaseLoggerAdapter(logging.LoggerAdapter):
     def dts_virtdut_output(self, msg, *args, **kwargs):
         self.log(logging.DTS_VIRTDUT_OUTPUT, msg, *args, **kwargs)
 
+    def dts_pktgen_cmd(self, msg, *args, **kwargs):
+        self.log(logging.DTS_PKTGEN_CMD, msg, *args, **kwargs)
+
+    def dts_pktgen_output(self, msg, *args, **kwargs):
+        self.log(logging.DTS_PKTGEN_OUTPUT, msg, *args, **kwargs)
+
 
 class ColorHandler(logging.StreamHandler):
     """
@@ -167,6 +176,8 @@ class ColorHandler(logging.StreamHandler):
         logging.SUITE_TESTER_CMD: '',  # SYSTEM
         logging.DTS_IXIA_CMD: '',  # SYSTEM
         logging.DTS_IXIA_OUTPUT: '',  # SYSTEM
+        logging.DTS_PKTGEN_CMD: '',  # SYSTEM
+        logging.DTS_PKTGEN_OUTPUT: '',  # SYSTEM
         logging.DTS_VIRTDUT_CMD: '',  # SYSTEM
         logging.DTS_VIRTDUT_OUTPUT: '',  # SYSTEM
         logging.WARN: '\033[01;33m',  # BOLD YELLOW
@@ -228,7 +239,7 @@ class DTSLOG(BaseLoggerAdapter):
         fh.setFormatter(logging.Formatter(message_fmt, date_fmt))
         ch.setFormatter(logging.Formatter(stream_fmt, date_fmt))
 
-        fh.setLevel(logging.DEBUG)   # file hander default level
+        fh.setLevel(logging.DEBUG)   # file handler default level
         global verbose
         if verbose is True:
             ch.setLevel(logging.DEBUG)
@@ -310,6 +321,9 @@ class DTSLOG(BaseLoggerAdapter):
         elif crb.startswith('ixia'):
             self.info_lvl = logging.DTS_IXIA_CMD
             self.debug_lvl = logging.DTS_IXIA_OUTPUT
+        elif crb.startswith('pktgen'):
+            self.info_lvl = logging.DTS_PKTGEN_CMD
+            self.debug_lvl = logging.DTS_PKTGEN_OUTPUT
         elif crb.startswith('virtdut'):
             self.info_lvl = logging.DTS_VIRTDUT_CMD
             self.debug_lvl = logging.DTS_VIRTDUT_OUTPUT
@@ -342,6 +356,9 @@ class DTSLOG(BaseLoggerAdapter):
         elif crb == 'ixia':
             self.info_lvl = logging.DTS_IXIA_CMD
             self.debug_lvl = logging.DTS_IXIA_OUTPUT
+        elif crb == 'pktgen':
+            self.info_lvl = logging.DTS_PKTGEN_CMD
+            self.debug_lvl = logging.DTS_PKTGEN_OUTPUT
         elif crb == 'virtdut':
             self.info_lvl = logging.DTS_VIRTDUT_CMD
             self.debug_lvl = logging.DTS_VIRTDUT_OUTPUT
@@ -428,7 +445,7 @@ class LogParser(object):
             # only handle case log
             m = self.case_pattern.match(line.values()[0])
             if m:
-                # not determine case will start from begining
+                # not determine case will start from beginning
                 if case_name is None:
                     begin = self.loglist.index(line)
                 # start from the determined case
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 2/7] framework/dut: fix logger quit issue
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 1/7] framework/logger: add pktgen logger and remove duplicate yufengmx
@ 2019-08-06  7:51 ` yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 3/7] framework/tester: " yufengmx
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx


*. move dut logger close after session close for logger used by session.
*. fix typo.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/dut.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/framework/dut.py b/framework/dut.py
index fc4a2e6..448031d 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -207,7 +207,7 @@ class Dut(Crb):
         self.restore_interfaces()
         # rescan ports after interface up
         self.rescan_ports()
-        # load port infor from config file
+        # load port information from config file
         self.load_portconf()
         self.mount_procfs()
         # auto detect network topology
@@ -932,8 +932,8 @@ class Dut(Crb):
 
     def load_portconf(self):
         """
-        Load port configurations for ports_info. If manually configured infor
-        not same as auto scanned, still use infor in configuration file.
+        Load port configurations for ports_info. If manually configured info
+        not same as auto scanned, still use information in configuration file.
         """
         for port in self.ports_info:
             pci_bus = port['pci']
@@ -1101,6 +1101,6 @@ class Dut(Crb):
         """
         Recover all resource before crb exit
         """
-        self.logger.logger_exit()
         self.enable_tester_ipv6()
         self.close()
+        self.logger.logger_exit()
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 3/7] framework/tester: fix logger quit issue
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 1/7] framework/logger: add pktgen logger and remove duplicate yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 2/7] framework/dut: fix logger quit issue yufengmx
@ 2019-08-06  7:51 ` yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 4/7] framework/pktgen_base: fix internal bug yufengmx
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx


move tester logger close after session close for logger used by session.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/tester.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/framework/tester.py b/framework/tester.py
index 68ef2f9..174d0b6 100644
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -874,5 +874,5 @@ class Tester(Crb):
         """
         Close all resource before crb exit
         """
-        self.logger.logger_exit()
         self.close()
+        self.logger.logger_exit()
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 4/7] framework/pktgen_base: fix internal bug
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
                   ` (2 preceding siblings ...)
  2019-08-06  7:51 ` [dts] [PATCH V1 3/7] framework/tester: " yufengmx
@ 2019-08-06  7:51 ` yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 5/7] framework/pktgen_ixia: " yufengmx
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx


*. remove logger format setting after import libs to eliminate dts redundant logs.
*. set part of information logger display to debug level.
*. remove un-used libs import.
*. fix pep8 issue.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_base.py | 36 ++++++++++++++++++------------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py
index eec4e6f..8915ac5 100644
--- a/framework/pktgen_base.py
+++ b/framework/pktgen_base.py
@@ -29,27 +29,22 @@
 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-import os
 import time
 import logging
 from abc import abstractmethod
 from copy import deepcopy
-from logger import getLogger
 from pprint import pformat
 
 from config import PktgenConf
+from logger import getLogger, set_verbose
 # packet generator name
-from settings import PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN
+from settings import (PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN,
+                      load_global_setting, DEBUG_SETTING)
 
 # macro definition
 TRANSMIT_CONT = 'continuous'
 TRANSMIT_M_BURST = 'multi_burst'
 TRANSMIT_S_BURST = 'single_burst'
-# set logger
-FORMAT = '%(message)s'
-logging.basicConfig(format=FORMAT)
-logger = logging.getLogger(os.path.basename(__file__)[:-3].upper())
-logger.setLevel(logging.INFO)
 
 
 class PacketGenerator(object):
@@ -57,8 +52,11 @@ class PacketGenerator(object):
     Basic class for packet generator, define basic function for each kinds of
     generators
     """
+
     def __init__(self, tester):
         self.logger = getLogger(PKTGEN)
+        if load_global_setting(DEBUG_SETTING) == "yes":
+            set_verbose()
         self.tester = tester
         self.__streams = []
         self._ports_map = []
@@ -82,8 +80,8 @@ class PacketGenerator(object):
                 tester_pci = info['pci']
                 if tester_pci == gen_pci:
                     msg = "gen port {0} map test port {1}".format(
-                                                        port_id, port_idx)
-                    self.logger.info(msg)
+                        port_id, port_idx)
+                    self.logger.debug(msg)
                     return port_idx
             else:
                 port = -1
@@ -105,15 +103,15 @@ class PacketGenerator(object):
             tester_pci = info['pci']
             port = self._get_gen_port(tester_pci)
             msg = "test port {0} map gen port {1}".format(port_id, port)
-            self.logger.info(msg)
+            self.logger.debug(msg)
         except:
             port = -1
 
         return port
 
     def add_stream(self, tx_port, rx_port, pcap_file):
-        pktgen_tx_port  = self._convert_tester_port(tx_port)
-        pktgen_rx_port  = self._convert_tester_port(rx_port)
+        pktgen_tx_port = self._convert_tester_port(tx_port)
+        pktgen_rx_port = self._convert_tester_port(rx_port)
 
         stream_id = len(self.__streams)
         stream = {'tx_port': pktgen_tx_port,
@@ -183,13 +181,14 @@ class PacketGenerator(object):
         for stream_id in stream_ids:
             if self.__streams[stream_id]['rx_port'] not in used_rx_port:
                 rxbps_rates, rxpps_rates = self._retrieve_port_statistic(
-                                                        stream_id, 'throughput')
+                    stream_id, 'throughput')
                 used_rx_port.append(self.__streams[stream_id]['rx_port'])
                 bps_rx.append(rxbps_rates)
                 pps_rx.append(rxpps_rates)
         bps_rx_total = self._summary_statistic(bps_rx)
         pps_rx_total = self._summary_statistic(pps_rx)
-        self.logger.info("throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
+        self.logger.info(
+            "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
 
         return bps_rx_total, pps_rx_total
 
@@ -379,7 +378,7 @@ class PacketGenerator(object):
         result = self._measure_loss(stream_ids, options)
         status = self._check_loss_rate(result, permit_loss_rate)
         loss_rate_table.append(
-                    [options.get('rate') or rate_percent, result])
+            [options.get('rate') or rate_percent, result])
         # if first time loss rate is ok, ignore left flow
         if status:
             # return data is the same with dts/etgen format
@@ -439,7 +438,7 @@ class PacketGenerator(object):
                 traffic_pps_min = pps
             if traffic_pps_max - traffic_pps_min < step:
                 break
-            pps = (traffic_pps_max - traffic_pps_min)/2 + traffic_pps_min
+            pps = (traffic_pps_max - traffic_pps_min) / 2 + traffic_pps_min
 
         self.logger.info("zero loss pps is %f" % last_no_lost_mult)
         # use last result as return data to keep the same with dts/etgen format
@@ -516,4 +515,5 @@ class PacketGenerator(object):
         pass
 
 
-class DpdkPacketGenerator(PacketGenerator): pass # not implemented
\ No newline at end of file
+class DpdkPacketGenerator(PacketGenerator):
+    pass  # not implemented
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 5/7] framework/pktgen_ixia: fix internal bug
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
                   ` (3 preceding siblings ...)
  2019-08-06  7:51 ` [dts] [PATCH V1 4/7] framework/pktgen_base: fix internal bug yufengmx
@ 2019-08-06  7:51 ` yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 6/7] framework/pktgen_trex: " yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 7/7] framework/pktgen: fix pep8 issue yufengmx
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx


*. set part of information logger display to debug level.
*. set PacketGenerator class logger as Ixia class logger.
*. remove un-used libs import.
*. fix pep8 issue.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_ixia.py | 190 ++++++++++++++++++++++++-----------------------
 1 file changed, 97 insertions(+), 93 deletions(-)

diff --git a/framework/pktgen_ixia.py b/framework/pktgen_ixia.py
index 467188c..22d1f2c 100644
--- a/framework/pktgen_ixia.py
+++ b/framework/pktgen_ixia.py
@@ -36,7 +36,6 @@ from pprint import pformat
 
 from ssh_connection import SSHConnection
 from settings import SCAPY2IXIA
-from logger import getLogger
 from utils import (convert_int2ip, convert_ip2int,
                    convert_mac2long, convert_mac2str)
 
@@ -52,15 +51,15 @@ class Ixia(SSHConnection):
     IXIA performance measurement class.
     """
 
-    def __init__(self, tester, ixiaPorts):
+    def __init__(self, tester, ixiaPorts, logger):
         self.tester = tester
         self.NAME = PKTGEN_IXIA
-        self.logger = getLogger(self.NAME)
         super(Ixia, self).__init__(
-                            self.get_ip_address(),
-                            self.NAME,
-                            self.tester.get_username(),
-                            self.get_password())
+            self.get_ip_address(),
+            self.NAME,
+            self.tester.get_username(),
+            self.get_password())
+        self.logger = logger
         super(Ixia, self).init_log(self.logger)
 
         self.tcl_cmds = []
@@ -177,8 +176,8 @@ class Ixia(SSHConnection):
         """
         macAddr = macAddr.upper()
         return "%s %s %s %s %s %s" % (
-                                macAddr[:2], macAddr[3:5], macAddr[6:8],
-                                macAddr[9:11], macAddr[12:14], macAddr[15:17])
+            macAddr[:2], macAddr[3:5], macAddr[6:8],
+            macAddr[9:11], macAddr[12:14], macAddr[15:17])
 
     def set_ether_fields(self, fields, default_fields):
         """
@@ -204,15 +203,15 @@ class Ixia(SSHConnection):
             prefix = 'sa' if name == 'src' else 'da'
             if action == 'dec' and mac_end:
                 cmds.append('stream config -{0} "{1}"'.format(
-                                                            prefix, mac_end))
+                    prefix, mac_end))
             else:
                 cmds.append('stream config -{0} "{1}"'.format(
-                                                            prefix, mac_start))
+                    prefix, mac_start))
             if step:
                 cmds.append('stream config -{0}Step {1}'.format(prefix, step))
             if action:
                 cmds.append('stream config -{0}RepeatCounter {1}'.format(
-                                                prefix, addr_mode.get(action)))
+                    prefix, addr_mode.get(action)))
             if mac_end:
                 mac_start_int = convert_mac2long(mac_start)
                 mac_end_int = convert_mac2long(mac_end)
@@ -225,7 +224,7 @@ class Ixia(SSHConnection):
 
             if flow_num:
                 cmds.append('stream config -num{0} {1}'.format(
-                                                prefix.upper(), flow_num))
+                    prefix.upper(), flow_num))
             # clear default field after it has been set
             default_fields.pop(name)
         # if some filed not set, set it here
@@ -233,7 +232,8 @@ class Ixia(SSHConnection):
             for name, config in default_fields.iteritems():
                 ip_start = config.get('start')
                 prefix = 'sa' if name == 'src' else 'da'
-                cmds.append('stream config -{0} "{1}"'.format(prefix, ip_start))
+                cmds.append(
+                    'stream config -{0} "{1}"'.format(prefix, ip_start))
 
         return cmds
 
@@ -250,8 +250,8 @@ class Ixia(SSHConnection):
         # if vm has been set, pick pcap fields' as default value
         if fields:
             default_fields = {
-                'src': { 'action': 'default', 'start': src,},
-                'dst': { 'action': 'default', 'start': dst,},}
+                'src': {'action': 'default', 'start': src, },
+                'dst': {'action': 'default', 'start': dst, }, }
             # set custom setting for field actions
             cmds = self.set_ether_fields(fields, default_fields)
             # set them in tcl commands group
@@ -273,7 +273,7 @@ class Ixia(SSHConnection):
             # no change to IP address regardless of IpAddrRepeatCount
             'idle':     'ipIdle',
             # set default
-            'default':  'ipIdle',}
+            'default':  'ipIdle', }
         cmds = []
         for name, config in fields.iteritems():
             default_config = default_fields.get(name)
@@ -290,9 +290,10 @@ class Ixia(SSHConnection):
             else:
                 flow_num = None
 
-            mask =  config.get('mask')
+            mask = config.get('mask')
             _step = config.get('step')
-            step = int(_step) if _step and isinstance(_step, (str, unicode)) else \
+            step = int(_step) \
+                   if _step and isinstance(_step, (str, unicode)) else \
                    _step or 1
             action = config.get('action')
             # get ixia command prefix
@@ -300,16 +301,16 @@ class Ixia(SSHConnection):
             # set command
             if action == 'dec' and ip_end:
                 cmds.append('ip config -{0}IpAddr "{1}"'.format(
-                                                        prefix, ip_end))
+                    prefix, ip_end))
             else:
                 cmds.append('ip config -{0}IpAddr "{1}"'.format(
-                                                        prefix, ip_start))
+                    prefix, ip_start))
             if flow_num:
                 cmds.append('ip config -{0}IpAddrRepeatCount {1}'.format(
-                                                            prefix, flow_num))
+                    prefix, flow_num))
 
             cmds.append('ip config -{0}IpAddrMode {1}'.format(
-                                  prefix, addr_mode.get(action or 'default')))
+                prefix, addr_mode.get(action or 'default')))
 
             if mask:
                 cmds.append("ip config -{0}IpMask '{1}'".format(prefix, mask))
@@ -324,7 +325,7 @@ class Ixia(SSHConnection):
             prefix = 'source' if name == 'src' else 'dest'
             cmds.append('ip config -{0}IpAddr "{1}"'.format(prefix, ip_start))
             cmds.append('ip config -{0}IpAddrMode {1}'.format(
-                                  prefix, addr_mode.get('default')))
+                prefix, addr_mode.get('default')))
 
         return cmds
 
@@ -340,8 +341,8 @@ class Ixia(SSHConnection):
         if fields:
             # pick pcap fields' as default value
             default_fields = {
-                'src': { 'action': 'default', 'start': src,},
-                'dst': { 'action': 'default', 'start': dst,},}
+                'src': {'action': 'default', 'start': src, },
+                'dst': {'action': 'default', 'start': dst, }, }
             # set custom setting for field actions
             cmds = self.set_ip_fields(fields, default_fields)
             # append custom setting
@@ -366,9 +367,9 @@ class Ixia(SSHConnection):
         self.add_tcl_cmd("protocol config -name ipV6")
         self.add_tcl_cmd('ipV6 setDefault')
         self.add_tcl_cmd('ipV6 config -destAddr "%s"' %
-                                                self.ipv6_to_tcl_format(dst))
+                         self.ipv6_to_tcl_format(dst))
         self.add_tcl_cmd('ipV6 config -sourceAddr "%s"' %
-                                                self.ipv6_to_tcl_format(src))
+                         self.ipv6_to_tcl_format(src))
         self.add_tcl_cmd('ipV6 config -flowLabel %d' % fl)
         self.add_tcl_cmd('ipV6 config -nextHeader %d' % nh)
         self.add_tcl_cmd('ipV6 config -hopLimit %d' % hlim)
@@ -425,7 +426,7 @@ class Ixia(SSHConnection):
             # Generate random VlanID tag for each frame
             'random':   'vCtrRandom',
             # No change to VlanID tag regardless of repeat
-            'idle':     'vIdle',}
+            'idle':     'vIdle', }
         cmds = []
         for name, config in fields.iteritems():
             fv_name = '8021Q.{0}'.format(name)
@@ -440,7 +441,6 @@ class Ixia(SSHConnection):
                 flow_num = None
             step = config.get('step') or 1
             action = config.get('action')
-            #------------------------------------------------
             # set command
             if step:
                 cmds.append('vlan config -step {0}'.format(step))
@@ -448,7 +448,7 @@ class Ixia(SSHConnection):
                 cmds.append('vlan config -repeat {0}'.format(flow_num))
             if action:
                 cmds.append('vlan config -mode {0}'.format(
-                                                        addr_mode.get(action)))
+                    addr_mode.get(action)))
         return cmds
 
     def dot1q(self, port, vm, prio, id, vlan, type):
@@ -490,7 +490,7 @@ class Ixia(SSHConnection):
         transmit_mode = stream_config.get('transmit_mode') or TRANSMIT_CONT
         if transmit_mode == TRANSMIT_S_BURST:
             cmds = self.config_single_burst_stream(
-                                    stream_config.get('txmode'), rate_percent)
+                stream_config.get('txmode'), rate_percent)
             self.add_tcl_cmds(cmds)
         else:
             self.config_ixia_stream(
@@ -521,20 +521,21 @@ class Ixia(SSHConnection):
     def config_single_burst_stream(self, txmode, rate_percent):
         """ configure burst stream. """
         gapUnits = {
-        # (default) Sets units of time for gap to nanoseconds
-        'ns': 'gapNanoSeconds',
-        # Sets units of time for gap to microseconds
-        'us': 'gapMicroSeconds',
-        # Sets units of time for gap to milliseconds
-        'm': 'gapMilliSeconds',
-        # Sets units of time for gap to seconds
-        's': 'gapSeconds',}
+            # (default) Sets units of time for gap to nanoseconds
+            'ns': 'gapNanoSeconds',
+            # Sets units of time for gap to microseconds
+            'us': 'gapMicroSeconds',
+            # Sets units of time for gap to milliseconds
+            'm': 'gapMilliSeconds',
+            # Sets units of time for gap to seconds
+            's': 'gapSeconds', }
         pkt_count = 1
         burst_count = txmode.get('total_pkts', 32)
         frameType = txmode.get('frameType') or {}
         time_unit = frameType.get('type', 'ns')
         gapUnit = gapUnits.get(time_unit) \
-                        if time_unit in gapUnits.keys() else gapUnits.get('ns')
+                  if time_unit in gapUnits.keys() else \
+                  gapUnits.get('ns')
         # The inter-stream gap is the delay in clock ticks between stream.
         # This delay comes after the receive trigger is enabled. Setting this
         # option to 0 means no delay. (default = 960.0)
@@ -556,11 +557,11 @@ class Ixia(SSHConnection):
             "stream config -numBursts {0}".format(burst_count),
             "stream config -ifg {0}".format(ifg),
             "stream config -ifgType gapFixed",
-#             "stream config -enableIbg true",   # reserve
-#             "stream config -ibg {0}".format(ibg), # reserve
-#             "stream config -enableIsg true", # reserve
-#             "stream config -isg {0}".format(isg), # reserve
-            "stream config -frameSizeType sizeFixed",]
+            #"stream config -enableIbg true",   # reserve
+            #"stream config -ibg {0}".format(ibg), # reserve
+            #"stream config -enableIsg true", # reserve
+            #"stream config -isg {0}".format(isg), # reserve
+            "stream config -frameSizeType sizeFixed", ]
 
         return frame_cmds
 
@@ -598,7 +599,7 @@ class Ixia(SSHConnection):
             return False
 
         out = self.send_expect(
-                    "set chasId [ixGetChassisID %s]" % self.tclServerIP, "% ")
+            "set chasId [ixGetChassisID %s]" % self.tclServerIP, "% ")
         self.chasId = int(out.strip())
 
         self.send_expect("ixClearOwnership [list %s]" % string.join(
@@ -636,10 +637,10 @@ class Ixia(SSHConnection):
                 self.add_tcl_cmd("port config -autonegotiate false")
                 self.add_tcl_cmd("port config -enableRsFec true")
                 self.add_tcl_cmd("port set %d %d %d" % (
-                                        self.chasId,item['card'], item['port']))
+                    self.chasId, item['card'], item['port']))
 
             pl.append('[list %d %d %d]' % (
-                                       self.chasId, item['card'], item['port']))
+                self.chasId, item['card'], item['port']))
 
         self.add_tcl_cmd("set portList [list %s]" % string.join(pl, ' '))
 
@@ -727,7 +728,7 @@ class Ixia(SSHConnection):
         return port_info
 
     def get_ixia_port(self, port):
-        port_info= self.get_ixia_port_info(port)
+        port_info = self.get_ixia_port_info(port)
         ixia_port = "%d %d %d" % (self.chasId,
                                   port_info['card'], port_info['port'])
         return ixia_port
@@ -771,7 +772,7 @@ class Ixia(SSHConnection):
         Run latency performance test and return latency statistics.
         """
         rxPortlist, txPortlist = self._configure_everything(
-                                                portList, ratePercent, True)
+            portList, ratePercent, True)
         return self.get_packet_latency(rxPortlist)
 
     def get_packet_latency(self, rxPortlist):
@@ -795,7 +796,7 @@ class Ixia(SSHConnection):
         Run throughput performance test and return throughput statistics.
         """
         rxPortlist, txPortlist = self._configure_everything(
-                                                    port_list, rate_percent)
+            port_list, rate_percent)
         return self.get_transmission_results(rxPortlist, txPortlist, delay)
 
     def is_packet_ordered(self, port_list, delay):
@@ -831,7 +832,7 @@ class Ixia(SSHConnection):
         Prepare and configure IXIA ports for performance test.
         """
         rxPortlist, txPortlist = self.prepare_port_list(
-                                            port_list, rate_percent, latency)
+            port_list, rate_percent, latency)
         self.prepare_ixia_for_transmission(txPortlist, rxPortlist)
         self.configure_transmission()
         self.start_transmission()
@@ -872,7 +873,7 @@ class Ixia(SSHConnection):
 
         # port init
         self.config_port([self.get_ixia_port_info(port)
-                                for port in txPortlist.union(rxPortlist)])
+                          for port in txPortlist.union(rxPortlist)])
 
         # calculate total streams of ports
         for (txPort, rxPort, pcapFile, option) in portList:
@@ -908,7 +909,7 @@ class Ixia(SSHConnection):
         """
         self.add_tcl_cmd("ixClearStats portList")
         self.set_ixia_port_list([self.get_ixia_port(port)
-                                                 for port in txPortlist])
+                                 for port in txPortlist])
         self.add_tcl_cmd("ixWriteConfigToHardware portList")
         # Wait for changes to take affect and make sure links are up
         self.add_tcl_cmd("after 1000")
@@ -957,12 +958,12 @@ class Ixia(SSHConnection):
         """
         self.send_expect("source ./ixTcl1.0/ixiaDCB.tcl", "% ")
         self.send_expect("configIxia %d %s" % (
-                            self.chasId,
-                            string.join(["%s" % (
-                                repr(self.conRelation[port][n]))
-                                    for port in [rxPort, txPort]
-                                            for n in range(3)])),
-                         "% ", 100)
+            self.chasId,
+            string.join(["%s" % (
+                repr(self.conRelation[port][n]))
+                for port in [rxPort, txPort]
+                for n in range(3)])),
+            "% ", 100)
 
     def config_port_dcb(self, direction, tc):
         """
@@ -993,24 +994,26 @@ class Ixia(SSHConnection):
                 "pauseControl config -pauseControlType ieee8023x",
                 'pauseControl config -da "{0}"'.format(dst_mac),
                 "pauseControl config -pauseTime {0}".format(pause_time),
-                "pauseControl set {0}".format(ixia_port),]
+                "pauseControl set {0}".format(ixia_port), ]
         self.add_tcl_cmds(flow_ctrl_cmds)
 
     def cfgStreamDcb(self, stream, rate, prio, types):
         """
         Configure Stream for DCB.
         """
-        self.send_expect("configStream %s %s %s %s" % (stream, rate, prio, types), "% ", 100)
+        self.send_expect("configStream %s %s %s %s" %
+                         (stream, rate, prio, types), "% ", 100)
 
     def get_connection_relation(self, dutPorts):
         """
         Get the connect relations between DUT and Ixia.
         """
         for port in dutPorts:
-            info = self.tester.get_pci(self.tester.get_local_port(port)).split('.')
+            info = self.tester.get_pci(
+                self.tester.get_local_port(port)).split('.')
             self.conRelation[port] = [
-                            int(info[0]), int(info[1]),
-                            repr(self.tester.dut.get_mac_address(port).replace(':', ' ').upper())]
+                int(info[0]), int(info[1]),
+                repr(self.tester.dut.get_mac_address(port).replace(':', ' ').upper())]
         return self.conRelation
 
     def config_pktGroup_rx(self, ixia_port):
@@ -1018,7 +1021,8 @@ class Ixia(SSHConnection):
         Sets the transmit Packet Group configuration of the stream
         Default streamID is 1
         """
-        self.add_tcl_cmd("port config -receiveMode $::portRxModeWidePacketGroup")
+        self.add_tcl_cmd(
+            "port config -receiveMode $::portRxModeWidePacketGroup")
         self.add_tcl_cmd("port set %s" % ixia_port)
         self.add_tcl_cmd("packetGroup setDefault")
         self.add_tcl_cmd("packetGroup config -latencyControl cutThrough")
@@ -1046,7 +1050,8 @@ class Ixia(SSHConnection):
         """
         ixia_port = self.get_ixia_port(port_number)
         self.send_expect("ixStopPortPacketGroups %s" % ixia_port, "%", 100)
-        self.send_expect("packetGroupStats get %s 0 16384" % ixia_port, "%", 100)
+        self.send_expect(
+            "packetGroupStats get %s 0 16384" % ixia_port, "%", 100)
         self.send_expect("packetGroupStats getGroup 0", "%", 100)
 
     def close(self):
@@ -1089,7 +1094,7 @@ class Ixia(SSHConnection):
         """
         ixia_port = self.get_ixia_port(port_number)
         command = 'captureBuffer get {0} {1} {2}'.format(
-                                ixia_port, first_frame, last_frame)
+            ixia_port, first_frame, last_frame)
         self.send_expect(command, '%', 60)
 
     def ixia_export_buffer_to_file(self, frames_filename):
@@ -1194,7 +1199,7 @@ class Ixia(SSHConnection):
         nic_single_core_perf test case use
         """
         rxPortlist, txPortlist = self.prepare_port_list(
-                                            port_list, rate_percent, latency)
+            port_list, rate_percent, latency)
         self.prepare_ixia_for_transmission(txPortlist, rxPortlist)
         self.start_transmission()
         self.clear_tcl_commands()
@@ -1233,9 +1238,9 @@ class Ixia(SSHConnection):
 
         return rxPackets
 
-    #---------------------------------------------------------
+    #
     # extend methods for pktgen subclass `IxiaPacketGenerator
-    #---------------------------------------------------------
+    #
     def disconnect(self):
         ''' quit from ixia server '''
         pass
@@ -1251,7 +1256,7 @@ class Ixia(SSHConnection):
             return
         for item in self.ports:
             cmd = 'port reset {0} {1} {2}'.format(
-                                self.chasId, item['card'], item['port'])
+                self.chasId, item['card'], item['port'])
             self.send_expect(cmd, "%", 10)
 
     def reset(self, ports=None):
@@ -1302,7 +1307,7 @@ class Ixia(SSHConnection):
                 'rx_bps': 0,
                 'rx_pps': 0,
                 'tx_bps': 0,
-                'tx_pps': 0,}
+                'tx_pps': 0, }
             time.sleep(0.5)
         return stats
 
@@ -1331,7 +1336,7 @@ class Ixia(SSHConnection):
                 'rx_bps': bpsRate,
                 'rx_pps': rate,
                 'tx_bps': 0,
-                'tx_pps': 0,}
+                'tx_pps': 0, }
 
         return stats
 
@@ -1342,7 +1347,7 @@ class Ixia(SSHConnection):
         methods = {
             'throughput':   self.get_throughput_stat,
             'loss':         self.get_loss_stat,
-            'latency':      self.get_latency_stat,}
+            'latency':      self.get_latency_stat, }
         if mode not in methods.keys():
             msg = "not support mode <{0}>".format(mode)
             raise Exception(msg)
@@ -1357,6 +1362,7 @@ class IxiaPacketGenerator(PacketGenerator):
     """
     Ixia packet generator
     """
+
     def __init__(self, tester):
         # ixia management
         self.pktgen_type = PKTGEN_IXIA
@@ -1374,8 +1380,8 @@ class IxiaPacketGenerator(PacketGenerator):
         # check configuration options
         self.options_keys = [
             'txmode', 'ip', 'vlan', 'transmit_mode', 'rate']
-        self.ip_keys = ['start', 'end','action', 'step', 'mask',]
-        self.vlan_keys = ['start', 'end', 'action', 'step', 'count',]
+        self.ip_keys = ['start', 'end', 'action', 'step', 'mask', ]
+        self.vlan_keys = ['start', 'end', 'action', 'step', 'count', ]
 
         super(IxiaPacketGenerator, self).__init__(tester)
         self.tester = tester
@@ -1394,7 +1400,7 @@ class IxiaPacketGenerator(PacketGenerator):
 
     def _connect(self, tester, conf):
         # initialize ixia class
-        self._conn = Ixia(tester, conf)
+        self._conn = Ixia(tester, conf, self.logger)
         for p in self._conn.get_ports():
             self._ports.append(p)
 
@@ -1440,7 +1446,7 @@ class IxiaPacketGenerator(PacketGenerator):
         '''
         for name, _port_obj in self._conn.ports.iteritems():
             _pci = _port_obj.info['pci_addr']
-            self.logger.info((_pci, pci))
+            self.logger.debug((_pci, pci))
             if _pci == pci:
                 return True
         else:
@@ -1460,7 +1466,7 @@ class IxiaPacketGenerator(PacketGenerator):
         # close it and wait for more discussion about pktgen framework
         return None
         conf = {}
-        #get the subnet range of src and dst ip
+        # get the subnet range of src and dst ip
         if self.conf.has_key("ip_src"):
             conf['src'] = {}
             ip_src = self.conf['ip_src']
@@ -1529,8 +1535,8 @@ class IxiaPacketGenerator(PacketGenerator):
         msg = [
             "Tx Port %d stats: " % (tx_port_id),
             "tx_port: %d,  tx_bps: %f, tx_pps: %f " % (
-                                            tx_port_id, tx_bps, tx_pps)]
-        self.logger.info(pformat(port_stats))
+                tx_port_id, tx_bps, tx_pps)]
+        self.logger.debug(pformat(port_stats))
         self.logger.info(os.linesep.join(msg))
         # rx bps/pps
         rx_port_id = stream["rx_port"]
@@ -1543,9 +1549,9 @@ class IxiaPacketGenerator(PacketGenerator):
         msg = [
             "Rx Port %d stats: " % (rx_port_id),
             "rx_port: %d,  rx_bps: %f, rx_pps: %f" % (
-                                        rx_port_id, rx_bps, rx_pps)]
+                rx_port_id, rx_bps, rx_pps)]
 
-        self.logger.info(pformat(port_stats))
+        self.logger.debug(pformat(port_stats))
         self.logger.info(os.linesep.join(msg))
 
         return rx_bps, rx_pps
@@ -1585,7 +1591,7 @@ class IxiaPacketGenerator(PacketGenerator):
         latency_stats = {
             'min':    port_stats.get('total_min'),
             'max':    port_stats.get('total_max'),
-            'average':port_stats.get('average'),}
+            'average': port_stats.get('average'), }
 
         return latency_stats
 
@@ -1625,7 +1631,7 @@ class IxiaPacketGenerator(PacketGenerator):
             self._traffic_opt['flow_control'] = options.get('flow_control') or {}
             # if vm config by pktgen config file, set it here to take the place
             # of setting on suite
-            if self._vm_conf: # TBD, remove this process later
+            if self._vm_conf:  # TBD, remove this process later
                 config['fields_config'] = self._vm_conf
             # get stream rate percent
             stream_config = options.get('stream_config')
@@ -1636,13 +1642,12 @@ class IxiaPacketGenerator(PacketGenerator):
         if not port_config:
             msg = 'no stream options for ixia packet generator'
             raise Exception(msg)
-        #-------------------------------------------------------------------
         port_lists = []
         for port_id, option in port_config.iteritems():
             port_lists += option
         self._conn.clear_tcl_buffer()
         rxPortlist, txPortlist = self._conn.prepare_port_list(
-                                    port_lists, rate_percent or 100, latency)
+            port_lists, rate_percent or 100, latency)
         self._conn.prepare_ixia_for_transmission(txPortlist, rxPortlist)
         # preset port status before running traffic
         self._preset_ixia_port()
@@ -1652,7 +1657,6 @@ class IxiaPacketGenerator(PacketGenerator):
         rate_percent = options.get('rate') or '100'
         # run ixia server
         try:
-            ###########################################
             # Start traffic on port(s)
             self.logger.info("begin traffic ......")
             run_opt = {
@@ -1672,8 +1676,8 @@ class IxiaPacketGenerator(PacketGenerator):
         ''' ixia traffic statistics '''
         stats = self._conn.get_stats(self._traffic_ports, mode)
         stream = self._get_stream(stream_id)
-        self.logger.info(pformat(stream))
-        self.logger.info(pformat(stats))
+        self.logger.debug(pformat(stream))
+        self.logger.debug(pformat(stats))
         if mode == 'throughput':
             return self._throughput_stats(stream, stats)
         elif mode == 'loss':
@@ -1724,4 +1728,4 @@ class IxiaPacketGenerator(PacketGenerator):
         ''' close ixia session '''
         if self._conn is not None:
             self._disconnect()
-        return
\ No newline at end of file
+        return
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 6/7] framework/pktgen_trex: fix internal bug
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
                   ` (4 preceding siblings ...)
  2019-08-06  7:51 ` [dts] [PATCH V1 5/7] framework/pktgen_ixia: " yufengmx
@ 2019-08-06  7:51 ` yufengmx
  2019-08-06  7:51 ` [dts] [PATCH V1 7/7] framework/pktgen: fix pep8 issue yufengmx
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx


*. use tester alt session to close trex.
*. set part of information logger display to debug level.
*. remove logger format setting after import libs to eliminate dts redundant logs.
*. fix pep8 issue.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen_trex.py | 146 +++++++++++++++++++++++------------------------
 1 file changed, 72 insertions(+), 74 deletions(-)

diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py
index 26a802c..e47cf84 100644
--- a/framework/pktgen_trex.py
+++ b/framework/pktgen_trex.py
@@ -35,21 +35,18 @@ import time
 import logging
 from pprint import pformat
 
-from pktgen_base import (PacketGenerator, PKTGEN_TREX,
+from pktgen_base import (PacketGenerator, PKTGEN_TREX, PKTGEN,
                          TRANSMIT_CONT, TRANSMIT_M_BURST, TRANSMIT_S_BURST)
 
-FORMAT = '%(message)s'
-logging.basicConfig(format=FORMAT)
-logger = logging.getLogger(os.path.basename(__file__)[:-3].upper())
-logger.setLevel(logging.INFO)
-
 
 class TrexConfigVm(object):
     '''
     config one stream vm format of trex
     '''
+
     def __init__(self):
-        from trex_stl_lib.api import (ipv4_str_to_num, mac2str, is_valid_ipv4_ret)
+        from trex_stl_lib.api import (
+            ipv4_str_to_num, mac2str, is_valid_ipv4_ret)
         self.ipv4_str_to_num = ipv4_str_to_num
         self.is_valid_ipv4_ret = is_valid_ipv4_ret
         self.mac2str = mac2str
@@ -74,7 +71,7 @@ class TrexConfigVm(object):
             'max_value': max_value,
             'size': 4,
             'step': step,
-            'op': mode,},
+            'op': mode, },
             {'write': {'add_val': add_val, 'offset_fixup': 2}}]
 
         return var
@@ -86,7 +83,8 @@ class TrexConfigVm(object):
         _ip_start = self.ipv4_str_to_num(self.is_valid_ipv4_ret(ip_start))
         _ip_end = self.ipv4_str_to_num(self.is_valid_ipv4_ret(ip_end))
         _step = self.ipv4_str_to_num(self.is_valid_ipv4_ret(step)) \
-                                if isinstance(step, (str, unicode)) else step
+                if isinstance(step, (str, unicode)) else \
+                step
         if mode == 'inc' or mode == 'dec':
             min_value = _ip_start
             max_value = _ip_end
@@ -101,7 +99,7 @@ class TrexConfigVm(object):
             'max_value': max_value,
             'size': 4,
             'step': _step,
-            'op': mode,},
+            'op': mode, },
             {'write': {'add_val': add_val},
              'fix_chksum': {}}]
 
@@ -120,12 +118,11 @@ class TrexConfigVm(object):
                 mac_end = config.get('end') or 'FF:FF:FF:FF:FF:FF'
                 step = config.get('step') or 1
                 mode = config.get('action') or 'inc'
-                #-----------------
                 fv_name = 'Ethernet.{0}'.format(name)
                 # layer/field name
                 vm_var[fv_name] = self._mac_var(fv_name,
-                                            mac_start, mac_end,
-                                            step, mode)
+                                                mac_start, mac_end,
+                                                step, mode)
         ###################################################################
         # src ip mask inc/dec/random
         if 'ip' in option:
@@ -134,11 +131,10 @@ class TrexConfigVm(object):
                 ip_end = config.get('end') or '0.0.0.255'
                 step = config.get('step') or 1
                 mode = config.get('action') or 'inc'
-                #-----------------
                 fv_name = 'IP.{0}'.format(name)
                 # layer/field name
                 vm_var[fv_name] = self._ip_vm_var(fv_name, ip_start, ip_end,
-                                                 step, mode)
+                                                  step, mode)
         ###################################################################
         #  merge var1/var2/random/cache into one method
         ###################################################################
@@ -150,7 +146,6 @@ class TrexConfigVm(object):
                 port_end = config.get('end') or 255
                 step = config.get('step') or 1
                 mode = config.get('action') or 'inc'
-                #-----------------
                 fv_name = '{0}.{1}'.format(protocol.upper(), name)
                 # layer/field name
                 vm_var[fv_name] = {
@@ -159,17 +154,17 @@ class TrexConfigVm(object):
                     'max_value': port_end,
                     'size': 2,
                     'step': step,
-                    'op': mode,}
+                    'op': mode, }
         ###################################################################
         # vlan field inc/dec/random
         if 'vlan' in option:
             for name, config in option['vlan'].iteritems():
                 vlan_start = config.get('start') \
-                                if config.get('start') != None else 0
+                             if config.get('start') != None else \
+                             0
                 vlan_end = config.get('end') or 256
                 step = config.get('step') or 1
                 mode = config.get('action') or 'inc'
-                #-----------------
                 fv_name = '802|1Q:{0}.vlan'.format(name)
                 # vlan layer/field name
                 vm_var[fv_name] = {
@@ -178,7 +173,7 @@ class TrexConfigVm(object):
                     'max_value': vlan_end,
                     'size': 2,
                     'step': step,
-                    'op': mode,}
+                    'op': mode, }
         ###################################################################
         # payload change with custom sizes
         if 'pkt_size' in option:
@@ -189,7 +184,6 @@ class TrexConfigVm(object):
             mode = 'random'
             min_pkt_size = option['pkt_size']['start']
             max_pkt_size = option['pkt_size']['end']
-            #-----------------
             l3_len_fix = -len(Ether())
             l4_len_fix = l3_len_fix - len(IP())
 
@@ -200,17 +194,17 @@ class TrexConfigVm(object):
                 'max_value': max_pkt_size - 4,
                 'size': 2,
                 'step': step,
-                'op': mode,}
+                'op': mode, }
 
             vm_var = {
-            'IP.len': [
-                var, {'write': {'add_val': l3_len_fix},
-                      'trim': {},
-                      'fix_chksum': {}}],
-            'UDP.len': [
-                var, {'write': {'add_val': l4_len_fix},
-                      'trim': {},
-                      'fix_chksum': {}}]}
+                'IP.len': [
+                    var, {'write': {'add_val': l3_len_fix},
+                          'trim': {},
+                          'fix_chksum': {}}],
+                'UDP.len': [
+                    var, {'write': {'add_val': l4_len_fix},
+                          'trim': {},
+                          'fix_chksum': {}}]}
 
         return vm_var
 
@@ -219,9 +213,9 @@ class TrexConfigStream(object):
 
     def __init__(self):
         from trex_stl_lib.api import (
-                    STLTXCont, STLTXSingleBurst, STLTXMultiBurst,
-                    STLPktBuilder, STLProfile, STLVM,
-                    STLStream, STLFlowLatencyStats)
+            STLTXCont, STLTXSingleBurst, STLTXMultiBurst,
+            STLPktBuilder, STLProfile, STLVM,
+            STLStream, STLFlowLatencyStats)
 
         # set trex class
         self.STLStream = STLStream
@@ -270,8 +264,9 @@ class TrexConfigStream(object):
             if isinstance(layer, (tuple, list)):
                 vm_var.tuple_var(**config)
                 for offset in layer:
-                    fv_name = name+'.ip' if offset.startswith('IP') else \
-                              name+'.port'
+                    fv_name = name + '.ip' \
+                              if offset.startswith('IP') else \
+                              name + '.port'
                     _vars = {'fv_name': fv_name, 'pkt_offset': offset}
                     if op_config and 'write' in op_config:
                         _vars.update(op_config['write'])
@@ -324,16 +319,16 @@ class TrexConfigStream(object):
             burst_pkts = txmode_opt.get('burst_pkts') or 32
             bursts_count = txmode_opt.get('bursts_count') or 2
             ibg = txmode_opt.get('ibg') or 10
-            mode_inst = self.STLTXMultiBurst(pkts_per_burst = burst_pkts,
-                                        count = bursts_count,
-                                        ibg = ibg)
+            mode_inst = self.STLTXMultiBurst(pkts_per_burst=burst_pkts,
+                                             count=bursts_count,
+                                             ibg=ibg)
         else:
             msg = 'not support format {0}'.format(mode)
             raise Exception(msg)
 
         pkt = self.STLPktBuilder(pkt=_pkt, vm=vm)
         _stream = self.STLStream(packet=pkt, mode=mode_inst, isg=isg,
-                            flow_stats=flow_stats)
+                                 flow_stats=flow_stats)
 
         return _stream
 
@@ -369,7 +364,7 @@ class TrexConfigStream(object):
             # configure trex vm
             vm_var = self._generate_vm(vm_conf)
             # create
-            streams.append(self._create_stream( _pkt, _stream_op, vm_var))
+            streams.append(self._create_stream(_pkt, _stream_op, vm_var))
         _streams = self.STLProfile(streams).get_streams()
 
         return _streams
@@ -388,8 +383,8 @@ class TrexConfigStream(object):
             latency_opt = streams_config[0]
             _pkt = latency_opt.get('pcap')
             _stream_op = latency_opt.get('stream_config')
-            _stream = self._create_stream( _pkt, _stream_op,
-                                           flow_stats=flow_stats)
+            _stream = self._create_stream(_pkt, _stream_op,
+                                          flow_stats=flow_stats)
             streams.append(_stream)
         else:
             streams = _streams
@@ -402,6 +397,7 @@ class TrexPacketGenerator(PacketGenerator):
     Trex packet generator, detail usage can be seen at
     https://trex-tgn.cisco.com/trex/doc/trex_manual.html
     """
+
     def __init__(self, tester):
         self.pktgen_type = PKTGEN_TREX
         self.trex_app = "t-rex-64"
@@ -418,27 +414,29 @@ class TrexPacketGenerator(PacketGenerator):
 
         self.options_keys = [
             'txmode', 'ip', 'vlan', 'transmit_mode', 'rate']
-        self.ip_keys = ['start', 'end','action', 'mask', 'step']
+        self.ip_keys = ['start', 'end', 'action', 'mask', 'step']
         self.vlan_keys = ['start', 'end', 'action', 'step', 'count']
 
         super(TrexPacketGenerator, self).__init__(tester)
 
         # check trex binary file
-        trex_bin = os.sep.join([self.conf.get('trex_root_path'), self.trex_app])
+        trex_bin = os.sep.join(
+            [self.conf.get('trex_root_path'), self.trex_app])
         if not os.path.exists(trex_bin):
             msg = "{0} is not existed, please check {1} content".format(
-                                    trex_bin, conf_inst.config_file)
+                trex_bin, conf_inst.config_file)
             raise Exception(msg)
         # if `trex_lib_path` is not set, use a default relative directory.
         trex_lib_dir = \
-                self.conf.get('trex_lib_path') \
-                                if self.conf.get('trex_lib_path') else \
-                "{0}/automation/trex_control_plane/stl".format(
-                                            self.conf.get('trex_root_path'))
+            self.conf.get('trex_lib_path') \
+            if self.conf.get('trex_lib_path') else \
+            "{0}/automation/trex_control_plane/stl".format(
+                self.conf.get('trex_root_path'))
         # check trex lib root directory
         if not os.path.exists(trex_lib_dir):
             msg = ("{0} is not existed, please check {1} content and "
-                   "set `trex_lib_path`").format(trex_lib_dir, conf_inst.config_file)
+                   "set `trex_lib_path`").format(
+                    trex_lib_dir, conf_inst.config_file)
             raise Exception(msg)
         # check if trex lib is existed
         trex_lib = os.sep.join([trex_lib_dir, 'trex_stl_lib'])
@@ -504,7 +502,7 @@ class TrexPacketGenerator(PacketGenerator):
         '''
         for name, _port_obj in self._conn.ports.iteritems():
             _pci = _port_obj.info['pci_addr']
-            self.logger.info((_pci, pci))
+            self.logger.debug((_pci, pci))
             if _pci == pci:
                 return True
         else:
@@ -523,7 +521,7 @@ class TrexPacketGenerator(PacketGenerator):
                 'intf': 'TREX:%d' % idx,
                 'mac': mac,
                 'pci': pci,
-                'type': 'trex',})
+                'type': 'trex', })
         return ports
 
     def _clear_streams(self):
@@ -597,13 +595,11 @@ class TrexPacketGenerator(PacketGenerator):
                     app_param_temp = app_param_temp + " --cfg " + self.conf[key]
                 elif key == 'core_num':
                     app_param_temp = app_param_temp + " -c " + self.conf[key]
-            self.control_session = \
-                        self.tester.create_session('trex_control_session')
-
+            self.control_session = self.tester.create_session(PKTGEN)
             self.control_session.send_expect(
-               ';'.join(['cd ' + self.conf['trex_root_path'],
-                './' + self.trex_app + " " + app_param_temp]),
-                                        '-Per port stats table', 30)
+                ';'.join(['cd ' + self.conf['trex_root_path'],
+                          './' + self.trex_app + " " + app_param_temp]),
+                '-Per port stats table', 30)
         try:
             self._connect()
         except Exception as e:
@@ -614,7 +610,7 @@ class TrexPacketGenerator(PacketGenerator):
     def _vm_conf(self):
         return None # close it and wait for more discussion about pktgen framework
         conf = {}
-        #get the subnet range of src and dst ip
+        # get the subnet range of src and dst ip
         if self.conf.has_key("ip_src"):
             conf['src'] = {}
             ip_src = self.conf['ip_src']
@@ -640,7 +636,7 @@ class TrexPacketGenerator(PacketGenerator):
         if port_id not in ports:
             return None
         features = self._conn.ports[port_id].get_formatted_info()
-        self.logger.info(pformat(features))
+        self.logger.debug(pformat(features))
 
         return features
 
@@ -660,13 +656,14 @@ class TrexPacketGenerator(PacketGenerator):
         # for trex design requirement, all ports of trex should be the same type
         # nic, here use first port to check flow control attribute
         flow_ctrl = self._traffic_opt.get('flow_control') \
-                        if self._is_support_flow_control(rx_ports[0]) else None
+                    if self._is_support_flow_control(rx_ports[0]) else \
+                    None
         flow_ctrl_flag = flow_ctrl.get('flag') or 1 if flow_ctrl else None
         # flow control of port running trex traffic
-        self._conn.set_port_attr( rx_ports,
-                                  promiscuous=True,
-                                  link_up=True,
-                                  flow_ctrl = flow_ctrl_flag)
+        self._conn.set_port_attr(rx_ports,
+                                 promiscuous=True,
+                                 link_up=True,
+                                 flow_ctrl=flow_ctrl_flag)
 
     def _throughput_stats(self, stream, stats):
         # tx packet
@@ -680,8 +677,8 @@ class TrexPacketGenerator(PacketGenerator):
         msg = [
             "Tx Port %d stats: " % (tx_port_id),
             "tx_port: %d,  tx_bps: %f, tx_pps: %f " % (
-                                            tx_port_id, tx_bps, tx_pps)]
-        self.logger.info(pformat(port_stats))
+                tx_port_id, tx_bps, tx_pps)]
+        self.logger.debug(pformat(port_stats))
         self.logger.info(os.linesep.join(msg))
         # rx bps/pps
         rx_port_id = stream["rx_port"]
@@ -694,9 +691,9 @@ class TrexPacketGenerator(PacketGenerator):
         msg = [
             "Rx Port %d stats: " % (rx_port_id),
             "rx_port: %d,  rx_bps: %f, rx_pps: %f" % (
-                                        rx_port_id, rx_bps, rx_pps)]
+                rx_port_id, rx_bps, rx_pps)]
 
-        self.logger.info(pformat(port_stats))
+        self.logger.debug(pformat(port_stats))
         self.logger.info(os.linesep.join(msg))
 
         return rx_bps, rx_pps
@@ -735,7 +732,7 @@ class TrexPacketGenerator(PacketGenerator):
         latency_stats = {
             'min':    port_stats.get('total_min'),
             'max':    port_stats.get('total_max'),
-            'average':port_stats.get('average'),}
+            'average': port_stats.get('average'), }
 
         return latency_stats
 
@@ -763,7 +760,7 @@ class TrexPacketGenerator(PacketGenerator):
             # the same.
             stream_config = options.get('stream_config') or {}
             self._traffic_opt['rate'] = stream_config.get('rate') or 100
-            if stream_config.get('pps'): # reserve feature
+            if stream_config.get('pps'):  # reserve feature
                 self._traffic_opt['pps'] = stream_config.get('pps')
             # flow control option is deployed on all ports by design
             self._traffic_opt['flow_control'] = options.get('flow_control') or {}
@@ -802,6 +799,7 @@ class TrexPacketGenerator(PacketGenerator):
                 'core_mask': self.core_mask,
                 'force':    True,}
             self.logger.info("begin traffic ......")
+            self.logger.debug(run_opt)
             self._conn.start(**run_opt)
         except Exception as e:
             self.logger.error(e)
@@ -816,8 +814,8 @@ class TrexPacketGenerator(PacketGenerator):
         '''
         stats = self._conn.get_stats()
         stream = self._get_stream(stream_id)
-        self.logger.info(pformat(stream))
-        self.logger.info(pformat(stats))
+        self.logger.debug(pformat(stream))
+        self.logger.debug(pformat(stats))
         if mode == 'throughput':
             return self._throughput_stats(stream, stats)
         elif mode == 'loss':
@@ -831,7 +829,7 @@ class TrexPacketGenerator(PacketGenerator):
         if self._conn is not None:
             self._disconnect()
         if self.control_session is not None:
-            self.tester.send_expect('pkill -f _t-rex-64', '# ')
+            self.tester.alt_session.send_expect('pkill -f _t-rex-64', '# ')
             time.sleep(5)
             self.tester.destroy_session(self.control_session)
             self.control_session = None
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [dts] [PATCH V1 7/7] framework/pktgen: fix pep8 issue
  2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
                   ` (5 preceding siblings ...)
  2019-08-06  7:51 ` [dts] [PATCH V1 6/7] framework/pktgen_trex: " yufengmx
@ 2019-08-06  7:51 ` yufengmx
  6 siblings, 0 replies; 8+ messages in thread
From: yufengmx @ 2019-08-06  7:51 UTC (permalink / raw)
  To: dts; +Cc: yufengmx


*. remove un-used libs import.
*. remove set dts libs directory in system path.
*. fix pep8 issue.

Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
 framework/pktgen.py | 29 ++++++++++++-----------------
 1 file changed, 12 insertions(+), 17 deletions(-)

diff --git a/framework/pktgen.py b/framework/pktgen.py
index f87baa3..8132fe5 100644
--- a/framework/pktgen.py
+++ b/framework/pktgen.py
@@ -29,7 +29,7 @@
 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-import os, sys
+import os
 from copy import deepcopy
 
 from scapy.all import conf
@@ -38,13 +38,6 @@ from scapy.packet import Packet as scapyPacket
 from scapy.fields import ConditionalField
 from scapy.utils import rdpcap
 
-# import dts libs
-cwd = os.getcwd()
-sys.path.append(cwd + '/nics')
-sys.path.append(cwd + '/framework')
-sys.path.append(cwd + '/tests')
-sys.path.append(cwd + '/dep')
-
 # dts libs
 from utils import (convert_int2ip, convert_ip2int,
                    convert_mac2long, convert_mac2str)
@@ -59,13 +52,13 @@ from pktgen_trex import TrexPacketGenerator
 class PacketGeneratorHelper(object):
     ''' default packet generator stream option for all streams '''
     default_opt = {
-        'stream_config':{
-            'txmode' : {},
+        'stream_config': {
+            'txmode': {},
             'transmit_mode': TRANSMIT_CONT,
             # for temporary usage because current pktgen design don't support
             # port level configuration, here using stream configuration to pass
             # rate percent
-            'rate': 100,}}
+            'rate': 100, }}
 
     def __init__(self):
         self.packetLayers = dict()
@@ -78,7 +71,7 @@ class PacketGeneratorHelper(object):
         self.packetLayers[pkt_object.name] = dict()
         for curfield in pkt_object.fields_desc:
             if isinstance(curfield, ConditionalField) and \
-                not curfield._evalcond(pkt_object):
+               not curfield._evalcond(pkt_object):
                 continue
             field_value = pkt_object.getfieldval(curfield.name)
             if isinstance(field_value, scapyPacket) or (curfield.islist and \
@@ -87,7 +80,7 @@ class PacketGeneratorHelper(object):
             repr_value = curfield.i2repr(pkt_object, field_value)
             if isinstance(repr_value, str):
                 repr_value = repr_value.replace(os.linesep,
-                                    os.linesep + " "*(len(curfield.name) +4))
+                                                os.linesep + " " * (len(curfield.name) + 4))
             self.packetLayers[pkt_object.name][curfield.name] = repr_value
 
         if isinstance(pkt_object.payload, NoPayload):
@@ -107,7 +100,7 @@ class PacketGeneratorHelper(object):
         if len(pcap_pkts) == 0:
             warning = "{0} is empty".format(pcapFile)
             raise Exception(warning)
-        elif number>= len(pcap_pkts):
+        elif number >= len(pcap_pkts):
             warning = "{0} is missing No.{1} packet".format(pcapFile, number)
             raise Exception(warning)
         else:
@@ -136,7 +129,8 @@ class PacketGeneratorHelper(object):
                 range = config.get('range') or 64
                 step = config.get('step') or 1
                 start_mac = pcap_fields.get(name)
-                end_mac = convert_mac2str(convert_mac2long(start_mac) + range-1)
+                end_mac = convert_mac2str(
+                    convert_mac2long(start_mac) + range - 1)
                 fields_config[layer_name][name] = {}
                 fields_config[layer_name][name]['start'] = start_mac
                 fields_config[layer_name][name]['end'] = end_mac
@@ -203,6 +197,7 @@ class PacketGeneratorHelper(object):
             stream_ids.append(stream_id)
         return stream_ids
 
+
 def getPacketGenerator(tester, pktgen_type=PKTGEN_IXIA):
     """
     Get packet generator object
@@ -212,11 +207,11 @@ def getPacketGenerator(tester, pktgen_type=PKTGEN_IXIA):
     pktgen_cls = {
         PKTGEN_DPDK: DpdkPacketGenerator,
         PKTGEN_IXIA: IxiaPacketGenerator,
-        PKTGEN_TREX: TrexPacketGenerator,}
+        PKTGEN_TREX: TrexPacketGenerator, }
 
     if pktgen_type in pktgen_cls.keys():
         CLS = pktgen_cls.get(pktgen_type)
         return CLS(tester)
     else:
         msg = "not support <{0}> packet generator".format(pktgen_type)
-        raise Exception(msg)
\ No newline at end of file
+        raise Exception(msg)
-- 
1.9.3


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2019-08-06  8:01 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-06  7:51 [dts] [PATCH V1 0/7] dts/pktgen: fix internal bugs yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 1/7] framework/logger: add pktgen logger and remove duplicate yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 2/7] framework/dut: fix logger quit issue yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 3/7] framework/tester: " yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 4/7] framework/pktgen_base: fix internal bug yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 5/7] framework/pktgen_ixia: " yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 6/7] framework/pktgen_trex: " yufengmx
2019-08-06  7:51 ` [dts] [PATCH V1 7/7] framework/pktgen: fix pep8 issue yufengmx

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).