[dts] [PATCH for-next v3 0/7] Fixes and modifications for T-Rex integration and nic_single_core_perf

[dts] [PATCH for-next v3 0/7] Fixes and modifications for T-Rex integration and nic_single_core_perf

[Patrick MacArthur]

The first four patches of this series are bug fixes necessary to run the
nic_single_core_perf test in its current state in the UNH-IOL
environment.

Patch 5 automates starting T-Rex as part of the prepare_generator()
setup step, if the "start_trex" configuration option is enabled.

Patch 6 adds a section to the DTS getting started guide which describes
how T-Rex should be configured for relevant tests.

Patch 7 adds JSON output to the nic_single_core_perf test, which can be
more easily consumed by automated tools.

This patch set targets the next branch, and I have uploaded these
patches to our local open-source git server:

The following changes since commit 41685e25bedcbb1bf895c1094913aab3bf42de1e:

  framework/dut: adjust memory allocation for arm64 (2018-02-08 21:27:50 +0800)

are available in the git repository at:

  https://oss.iol.unh.edu/dpdk/dts.git tags/for-next-v3-trex-single-core-perf

for you to fetch changes up to 9aec13369928c8c62257ad78fb8abb7f535863b3:

  tests/TestSuite_nic_single_core_perf: Report the results in JSON format (2018-03-29 14:54:21 -0400)

Patrick MacArthur (7):
  framework/texttable: Update to latest upstream version
  framework: Do not attempt ping6 on T-Rex ports
  tests/TestSuite_nic_single_core_perf: Fix config parsing
  tests/TestSuite_nic_single_core_perf: Use user-specified output dir
  framework/pktgen: Start T-Rex during prepare_generator()
  doc: Add T-Rex section to documentation
  tests/TestSuite_nic_single_core_perf: Report the results in JSON
    format

Changes from v2:
 - Remove STLClient.resolve() call; assume that T-Rex is in Layer 2
   stateless mode.
 - Add trex section to manual describing expected Layer 2 stateless
   configuration.
 - Fix missing 'os' import (identified by Ali Alnubani).
 - Add JSON output file for use by UNH-IOL test lab.

Changes from v1:
 - Add a configuration option to pktgen.cfg; trex is only started by DTS
   if the option is present and non-empty.
 - Drop patch 6 which renamed the test case.

 conf/pktgen.cfg                               |   2 +
 doc/dts_gsg/index.rst                         |   1 +
 doc/dts_gsg/trex.rst                          |  27 +++++
 framework/dut.py                              |   4 +-
 framework/pktgen.py                           |  34 ++++--
 framework/tester.py                           |  11 +-
 framework/texttable.py                        | 145 +++++++++++++++++---------
 framework/virt_dut.py                         |   2 +-
 test_plans/nic_single_core_perf_test_plan.rst |   2 +
 tests/TestSuite_nic_single_core_perf.py       |  53 ++++++----
 10 files changed, 191 insertions(+), 90 deletions(-)
 create mode 100644 doc/dts_gsg/trex.rst

-- 
2.14.1

[dts] [PATCH for-next v3 1/7] framework/texttable: Update to latest upstream version

[Patrick MacArthur]

Both DTS and the T-Rex Python API use the Python texttable module. The
texttable module that is currently bundled with DTS is old and does not
support methods required by t-rex. Fix this by replacing it with an
updated version of the module [1]. Both the old and new version of this
module are made available by the author under the LGPL-2.1+ license.

[1]: https://github.com/foutaise/texttable/blob/master/texttable.py

Tested-by: Ali Alnubani <alialnu@mellanox.com>
Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 framework/texttable.py | 145 ++++++++++++++++++++++++++++++++-----------------
 1 file changed, 96 insertions(+), 49 deletions(-)

diff --git a/framework/texttable.py b/framework/texttable.py
index 2f37a0c7ba58..9f049035cf3e 100644
--- a/framework/texttable.py
+++ b/framework/texttable.py
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 #
 # texttable - module for creating simple ASCII tables
-# Copyright (C) 2003-2011 Gerome Fournier <jef(at)foutaise.org>
+# Copyright (C) 2003-2015 Gerome Fournier <jef(at)foutaise.org>
 #
 # This library is free software; you can redistribute it and/or
 # modify it under the terms of the GNU Lesser General Public
@@ -15,7 +15,7 @@
 #
 # You should have received a copy of the GNU Lesser General Public
 # License along with this library; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
 
 """module for creating simple ASCII tables
 
@@ -25,9 +25,9 @@ Example:
     table = Texttable()
     table.set_cols_align(["l", "r", "c"])
     table.set_cols_valign(["t", "m", "b"])
-    table.add_rows([ ["Name", "Age", "Nickname"],
-                     ["Mr\\nXavier\\nHuon", 32, "Xav'"],
-                     ["Mr\\nBaptiste\\nClement", 1, "Baby"] ])
+    table.add_rows([["Name", "Age", "Nickname"],
+                    ["Mr\\nXavier\\nHuon", 32, "Xav'"],
+                    ["Mr\\nBaptiste\\nClement", 1, "Baby"]])
     print table.draw() + "\\n"
 
     table = Texttable()
@@ -70,8 +70,8 @@ Result:
 __all__ = ["Texttable", "ArraySizeError"]
 
 __author__ = 'Gerome Fournier <jef(at)foutaise.org>'
-__license__ = 'GPL'
-__version__ = '0.8.1'
+__license__ = 'LGPL'
+__version__ = '0.8.4'
 __credits__ = """\
 Jeff Kowalczyk:
     - textwrap improved import
@@ -88,11 +88,17 @@ Roger Lew:
 
 Brian Peterson:
     - better handling of unicode errors
+
+Frank Sachsenheim:
+    - add Python 2/3-compatibility
+
+Maximilian Hils:
+    - fix minor bug for Python 3 compatibility
 """
 
 import sys
 import string
-from functools import reduce
+import re
 
 try:
     if sys.version >= '2.3':
@@ -105,11 +111,8 @@ except ImportError:
     sys.stderr.write("Can't import textwrap module!\n")
     raise
 
-try:
-    True, False
-except NameError:
-    (True, False) = (1, 0)
-
+if sys.version >= '2.7':
+    from functools import reduce
 
 def len(iterable):
     """Redefining len here so it will be able to work with non-ASCII characters
@@ -118,13 +121,45 @@ def len(iterable):
         return iterable.__len__()
 
     try:
-        return len(unicode(iterable, 'utf'))
+        if sys.version >= '3.0':
+            return len(str)
+        else:
+            return len(unicode(iterable, 'utf'))
     except:
         return iterable.__len__()
 
 
-class ArraySizeError(Exception):
+TEXT_CODES = {'bold': {'start': '\x1b[1m',
+                       'end': '\x1b[22m'},
+              'cyan': {'start': '\x1b[36m',
+                       'end': '\x1b[39m'},
+              'blue': {'start': '\x1b[34m',
+                       'end': '\x1b[39m'},
+              'red': {'start': '\x1b[31m',
+                      'end': '\x1b[39m'},
+              'magenta': {'start': '\x1b[35m',
+                          'end': '\x1b[39m'},
+              'green': {'start': '\x1b[32m',
+                        'end': '\x1b[39m'},
+              'yellow': {'start': '\x1b[33m',
+                         'end': '\x1b[39m'},
+              'underline': {'start': '\x1b[4m',
+                            'end': '\x1b[24m'}}
+
+class TextCodesStripper:
+    keys = [re.escape(v['start']) for k,v in TEXT_CODES.items()]
+    keys += [re.escape(v['end']) for k,v in TEXT_CODES.items()]
+    pattern = re.compile("|".join(keys))
+
+    @staticmethod
+    def strip (s):
+        return re.sub(TextCodesStripper.pattern, '', s)
+
+def ansi_len (iterable):
+    return len(TextCodesStripper.strip(iterable))
+
 
+class ArraySizeError(Exception):
     """Exception raised when specified rows don't fit the required size
     """
 
@@ -185,7 +220,7 @@ class Texttable:
 
         if len(array) != 4:
             raise ArraySizeError("array should contain 4 characters")
-        array = [x[:1] for x in [str(s) for s in array]]
+        array = [ x[:1] for x in [ str(s) for s in array ] ]
         (self._char_horiz, self._char_vert,
             self._char_corner, self._char_header) = array
 
@@ -262,7 +297,7 @@ class Texttable:
 
         self._check_row_size(array)
         try:
-            array = map(int, array)
+            array = list(map(int, array))
             if reduce(min, array) <= 0:
                 raise ValueError
         except ValueError:
@@ -278,7 +313,7 @@ class Texttable:
         - default value is set to 3
         """
 
-        if not isinstance(width, int) or width < 0:
+        if not type(width) is int or width < 0:
             raise ValueError('width must be an integer greater then 0')
         self._precision = width
 
@@ -287,7 +322,7 @@ class Texttable:
         """
 
         self._check_row_size(array)
-        self._header = map(str, array)
+        self._header = list(map(str, array))
 
     def add_row(self, array):
         """Add a row in the rows stack
@@ -361,7 +396,11 @@ class Texttable:
         try:
             f = float(x)
         except:
-            return str(x)
+            try:
+                return str(x)
+            except:
+                return x.encode('utf-8')
+
 
         n = self._precision
         dtype = self._dtype[i]
@@ -393,8 +432,8 @@ class Texttable:
         if not self._row_size:
             self._row_size = len(array)
         elif self._row_size != len(array):
-            raise ArraySizeError("array should contain %d elements"
-                                 % self._row_size)
+            raise ArraySizeError("array should contain %d elements" \
+                % self._row_size)
 
     def _has_vlines(self):
         """Return a boolean, if vlines are required or not
@@ -443,13 +482,13 @@ class Texttable:
             horiz = self._char_header
         # compute cell separator
         s = "%s%s%s" % (horiz, [horiz, self._char_corner][self._has_vlines()],
-                        horiz)
+            horiz)
         # build the line
-        l = string.join([horiz * n for n in self._width], s)
+        l = s.join([horiz * n for n in self._width])
         # add border if needed
         if self._has_border():
             l = "%s%s%s%s%s\n" % (self._char_corner, horiz, l, horiz,
-                                  self._char_corner)
+                self._char_corner)
         else:
             l += "\n"
         return l
@@ -466,10 +505,10 @@ class Texttable:
         for line in cell_lines:
             length = 0
             parts = line.split('\t')
-            for part, i in zip(parts, range(1, len(parts) + 1)):
+            for part, i in zip(parts, list(range(1, len(parts) + 1))):
                 length = length + len(part)
                 if i < len(parts):
-                    length = (length / 8 + 1) * 8
+                    length = (length//8 + 1) * 8
             maxi = max(maxi, length)
         return maxi
 
@@ -485,17 +524,17 @@ class Texttable:
             return
         maxi = []
         if self._header:
-            maxi = [self._len_cell(x) for x in self._header]
+            maxi = [ self._len_cell(x) for x in self._header ]
         for row in self._rows:
-            for cell, i in zip(row, range(len(row))):
+            for cell,i in zip(row, list(range(len(row)))):
                 try:
                     maxi[i] = max(maxi[i], self._len_cell(cell))
                 except (TypeError, IndexError):
                     maxi.append(self._len_cell(cell))
         items = len(maxi)
-        length = reduce(lambda x, y: x + y, maxi)
+        length = reduce(lambda x, y: x+y, maxi)
         if self._max_width and length + items * 3 + 1 > self._max_width:
-            maxi = [(self._max_width - items * 3 - 1) / items
+            maxi = [(self._max_width - items * 3 -1) // items \
                     for n in range(items)]
         self._width = maxi
 
@@ -524,14 +563,14 @@ class Texttable:
             for cell, width, align in zip(line, self._width, self._align):
                 length += 1
                 cell_line = cell[i]
-                fill = width - len(cell_line)
+                fill = width - ansi_len(cell_line)
                 if isheader:
-                    align = "l"
+                    align = "c"
                 if align == "r":
                     out += "%s " % (fill * space + cell_line)
                 elif align == "c":
-                    out += "%s " % (fill / 2 * space + cell_line
-                                    + (fill / 2 + fill % 2) * space)
+                    out += "%s " % (int(fill/2) * space + cell_line \
+                            + int(fill/2 + fill%2) * space)
                 else:
                     out += "%s " % (cell_line + fill * space)
                 if length < len(line):
@@ -551,26 +590,34 @@ class Texttable:
             array = []
             for c in cell.split('\n'):
                 try:
-                    c = unicode(c, 'utf')
+                    c = str(c)
                 except UnicodeDecodeError as strerror:
                     sys.stderr.write("UnicodeDecodeError exception for string '%s': %s\n" % (c, strerror))
-                    c = unicode(c, 'utf', 'replace')
-                array.extend(textwrap.wrap(c, width))
+                    if sys.version >= '3.0':
+                        c = str(c, 'utf', 'replace')
+                    else:
+                        c = unicode(c, 'utf', 'replace')
+
+                # imarom - no wrap for now
+                #array.extend(textwrap.wrap(c, width))
+                array.extend([c])
+
             line_wrapped.append(array)
-        max_cell_lines = reduce(max, map(len, line_wrapped))
+        max_cell_lines = reduce(max, list(map(len, line_wrapped)))
         for cell, valign in zip(line_wrapped, self._valign):
             if isheader:
                 valign = "t"
             if valign == "m":
                 missing = max_cell_lines - len(cell)
-                cell[:0] = [""] * (missing / 2)
-                cell.extend([""] * (missing / 2 + missing % 2))
+                cell[:0] = [""] * int(missing / 2)
+                cell.extend([""] * int(missing / 2 + missing % 2))
             elif valign == "b":
                 cell[:0] = [""] * (max_cell_lines - len(cell))
             else:
                 cell.extend([""] * (max_cell_lines - len(cell)))
         return line_wrapped
 
+
 if __name__ == '__main__':
     table = Texttable()
     table.set_cols_align(["l", "r", "c"])
@@ -578,7 +625,7 @@ if __name__ == '__main__':
     table.add_rows([["Name", "Age", "Nickname"],
                     ["Mr\nXavier\nHuon", 32, "Xav'"],
                     ["Mr\nBaptiste\nClement", 1, "Baby"]])
-    print table.draw() + "\n"
+    print(table.draw() + "\n")
 
     table = Texttable()
     table.set_deco(Texttable.HEADER)
@@ -586,11 +633,11 @@ if __name__ == '__main__':
                           'f',  # float (decimal)
                           'e',  # float (exponent)
                           'i',  # integer
-                          'a'])  # automatic
+                          'a']) # automatic
     table.set_cols_align(["l", "r", "r", "r", "l"])
-    table.add_rows([["text", "float", "exp", "int", "auto"],
-                    ["abcd", "67", 654, 89, 128.001],
-                    ["efghijk", 67.5434, .654, 89.6, 12800000000000000000000.00023],
-                    ["lmn", 5e-78, 5e-78, 89.4, .000000000000128],
-                    ["opqrstu", .023, 5e+78, 92., 12800000000000000000000]])
-    print table.draw()
+    table.add_rows([["text",    "float", "exp", "int", "auto"],
+                    ["abcd",    "67",    654,   89,    128.001],
+                    ["efghijk", 67.5434, .654,  89.6,  12800000000000000000000.00023],
+                    ["lmn",     5e-78,   5e-78, 89.4,  .000000000000128],
+                    ["opqrstu", .023,    5e+78, 92.,   12800000000000000000000]])
+    print(table.draw())
-- 
2.14.1

[dts] [PATCH for-next v3 2/7] framework: Do not attempt ping6 on T-Rex ports

[Patrick MacArthur]

DTS uses ping6 to determine whether the DUT and tester node ports are
connected.  Currently the IXIA ports are excluded from this test because
they are matched explicitly with DUT ports in the ports.cfg file and the
traffic generator will not respond to pings in the usual fashion.

For the same reason, exclude T-Rex ports from ping6 matching and
connectivity testing. This avoids a traceback due to the T-Rex ports not
having enough available information to complete the ping6 check.

Tested-by: Ali Alnubani <alialnu@mellanox.com>
Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 framework/dut.py      |  4 ++--
 framework/tester.py   | 10 ++++++----
 framework/virt_dut.py |  2 +-
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/framework/dut.py b/framework/dut.py
index 31b3daaaebcb..5b72c7a523e3 100644
--- a/framework/dut.py
+++ b/framework/dut.py
@@ -980,13 +980,13 @@ class Dut(Crb):
 
     def disable_tester_ipv6(self):
         for tester_port in self.ports_map:
-            if self.tester.ports_info[tester_port]['type'] != 'ixia':
+            if self.tester.ports_info[tester_port]['type'].lower() not in ('ixia', 'trex'):
                 port = self.tester.ports_info[tester_port]['port']
                 port.disable_ipv6()
 
     def enable_tester_ipv6(self):
         for tester_port in range(len(self.tester.ports_info)):
-            if self.tester.ports_info[tester_port]['type'] != 'ixia':
+            if self.tester.ports_info[tester_port]['type'].lower() not in ('ixia', 'trex'):
                 port = self.tester.ports_info[tester_port]['port']
                 port.enable_ipv6()
 
diff --git a/framework/tester.py b/framework/tester.py
index 151200872338..be5b0e13897c 100755
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -200,7 +200,7 @@ class Tester(Crb):
         if localPort == -1:
             raise ParameterInvalidException("local port should not be -1")
 
-        if self.ports_info[localPort]['type'] == 'ixia':
+        if self.ports_info[localPort]['type'] in ('ixia', 'trex'):
             return "00:00:00:00:00:01"
         else:
             return self.ports_info[localPort]['mac']
@@ -305,7 +305,7 @@ class Tester(Crb):
             return
 
         for port_info in self.ports_info:
-            if port_info['type'] == 'ixia':
+            if port_info['type'].lower() in ('ixia', 'trex'):
                 continue
 
             addr_array = port_info['pci'].split(':')
@@ -394,7 +394,7 @@ class Tester(Crb):
         """
         Send ping6 packet from local port with destination ipv4 address.
         """
-        if self.ports_info[localPort]['type'] == 'ixia':
+        if self.ports_info[localPort]['type'].lower() in ('ixia', 'trex'):
             return "Not implemented yet"
         else:
             return self.send_expect("ping -w 5 -c 5 -A -I %s %s" % (self.ports_info[localPort]['intf'], ipv4), "# ", 10)
@@ -403,8 +403,10 @@ class Tester(Crb):
         """
         Send ping6 packet from local port with destination ipv6 address.
         """
-        if self.ports_info[localPort]['type'] == 'ixia':
+        if self.ports_info[localPort]['type'].lower() == 'ixia':
             return self.ixia_packet_gen.send_ping6(self.ports_info[localPort]['pci'], mac, ipv6)
+        elif self.ports_info[localPort]['type'].lower() == 'trex':
+            return "Not implemented yet"
         else:
             return self.send_expect("ping6 -w 5 -c 5 -A %s%%%s" % (ipv6, self.ports_info[localPort]['intf']), "# ", 10)
 
diff --git a/framework/virt_dut.py b/framework/virt_dut.py
index 62688dc83b9e..d4cd805d2a03 100644
--- a/framework/virt_dut.py
+++ b/framework/virt_dut.py
@@ -385,7 +385,7 @@ class VirtDut(DPDKdut):
                 remotepci = self.tester.ports_info[remotePort]['pci']
                 port_type = self.tester.ports_info[remotePort]['type']
                 # IXIA port should not check whether has vfs
-                if port_type != 'ixia':
+                if port_type.lower() not in ('ixia', 'trex'):
                     remoteport = self.tester.ports_info[remotePort]['port']
                     vfs = []
                     # vm_dut and tester in same dut
-- 
2.14.1

[dts] [PATCH for-next v3 3/7] tests/TestSuite_nic_single_core_perf: Fix config parsing

[Patrick MacArthur]

The configuration values in test suite configuration are already
eval()'d as of commit 081fbf69fe20 ("framework/config: utilize eval to
parse configurations") so we don't need to do any special parsing here.

This also changes the expected_throughput dictionary keys and values
from strings to floats, which is more convenient.

Tested-by: Ali Alnubani <alialnu@mellanox.com>
Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 tests/TestSuite_nic_single_core_perf.py | 27 ++++-----------------------
 1 file changed, 4 insertions(+), 23 deletions(-)

diff --git a/tests/TestSuite_nic_single_core_perf.py b/tests/TestSuite_nic_single_core_perf.py
index 3411a31b7532..21de93624cb4 100644
--- a/tests/TestSuite_nic_single_core_perf.py
+++ b/tests/TestSuite_nic_single_core_perf.py
@@ -59,8 +59,8 @@ class TestNicSingleCorePerf(TestCase):
         self.trafficDuration = 60
 
         #load the expected throughput for required nic
-        self.expected_throughput_nnt = self.parse_string(self.get_suite_cfg()["throughput_nnt"])
-        self.expected_throughput_fvl25g = self.parse_string(self.get_suite_cfg()["throughput_fvl25g"])
+        self.expected_throughput_nnt = self.get_suite_cfg()["throughput_nnt"]
+        self.expected_throughput_fvl25g = self.get_suite_cfg()["throughput_fvl25g"]
 
         # The acdepted gap between expected throughput and actual throughput, 1 Mpps
         self.gap = 1
@@ -169,9 +169,9 @@ class TestNicSingleCorePerf(TestCase):
                 ret_data[header[2]] = str(float("%.3f" % throughput)) + " Mpps"
                 ret_data[header[3]] = str(float("%.3f" % (throughput * 100 / wirespeed))) + "%"
                 if self.nic == "niantic":
-                    ret_data[header[4]] = self.expected_throughput_nnt[str(frame_size)][str(descriptor)] + " Mpps"
+                    ret_data[header[4]] = str(self.expected_throughput_nnt[frame_size][descriptor]) + " Mpps"
                 elif self.nic == "fortville_25g":
-                    ret_data[header[4]] = self.expected_throughput_fvl25g[str(frame_size)][str(descriptor)] + " Mpps"
+                    ret_data[header[4]] = str(self.expected_throughput_fvl25g[frame_size][descriptor]) + " Mpps"
                 ret_datas[descriptor] = deepcopy(ret_data)
                 self.test_result[frame_size] = deepcopy(ret_datas)
         
@@ -248,25 +248,6 @@ class TestNicSingleCorePerf(TestCase):
         file_to_save.write(str(table))
         file_to_save.close()
 
-    def parse_string(self, string):
-        '''
-        Parse a string in the formate of a dictionary and convert it into a real dictionary. 
-        '''
-        element_pattern = re.compile(".\d+:.*?}")
-        string_elements = element_pattern.findall(string)
-        ret = {}
-        for element in string_elements:
-            ex_pattern = re.compile("(\d+): *{(.*)}")
-            ex_ret = ex_pattern.search(element)
-            ret[ex_ret.groups()[0]] = ex_ret.groups()[1]
-            inner_datas = ex_ret.groups()[1].split(",")
-            ret_inner = {}
-            for data in inner_datas:
-                match_inner = data.split(":")
-                ret_inner[match_inner[0].strip()] = match_inner[1].strip()
-            ret[ex_ret.groups()[0]] = deepcopy(ret_inner)
-        return ret
-
     def tear_down(self):
         """
         Run after each test case.
-- 
2.14.1

[dts] [PATCH for-next v3 4/7] tests/TestSuite_nic_single_core_perf: Use user-specified output dir

[Patrick MacArthur]

The user can override the default output directory via a command line
argument, but this script does not account for that. Fix the script to
look up the output directory that the user requested and place the
custom output file in that directory.

Tested-by: Ali Alnubani <alialnu@mellanox.com>
Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 tests/TestSuite_nic_single_core_perf.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tests/TestSuite_nic_single_core_perf.py b/tests/TestSuite_nic_single_core_perf.py
index 21de93624cb4..860fc29f607f 100644
--- a/tests/TestSuite_nic_single_core_perf.py
+++ b/tests/TestSuite_nic_single_core_perf.py
@@ -33,6 +33,7 @@ DPDK Test suite.
 """
 
 import utils
+import os
 import re
 import time
 from test_case import TestCase
@@ -41,6 +42,7 @@ from settings import HEADER_SIZE
 from pmd_output import PmdOutput
 from copy import deepcopy
 from prettytable import PrettyTable
+import rst
 
 class TestNicSingleCorePerf(TestCase):
 
@@ -244,7 +246,8 @@ class TestNicSingleCorePerf(TestCase):
                 table_row.append(self.test_result[frame_size][descriptor][header[3]])
                 table_row.append(self.test_result[frame_size][descriptor][header[4]])
                 table.add_row(table_row)
-        file_to_save = open("output/%s_single_core_perf.txt" % self.nic, 'w')
+        file_to_save = open(os.path.join(
+            rst.path2Result, "%s_single_core_perf.txt" % self.nic), 'w')
         file_to_save.write(str(table))
         file_to_save.close()
 
-- 
2.14.1

[dts] [PATCH for-next v3 5/7] framework/pktgen: Start T-Rex during prepare_generator()

[Patrick MacArthur]

The currently merged patchset for T-Rex does not actually start T-Rex.
Add a configuration option "start_trex" and start T-Rex during the
prepare_generator() step if this is set to a non-empty string.  We need
T-Rex running during this step so we can determine which ports the T-Rex
API is making available to the DUT before we set up the DUT's network
interfaces accordingly.

This configuration option assumes that T-Rex will always be run on the
tester node.

Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 conf/pktgen.cfg     |  2 ++
 framework/pktgen.py | 34 +++++++++++++++++++++++-----------
 framework/tester.py |  1 +
 3 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg
index 1f3c7952f7f8..cc3fe61356b1 100644
--- a/conf/pktgen.cfg
+++ b/conf/pktgen.cfg
@@ -9,6 +9,7 @@
 # socket_memory --socket-mem: Memory to allocate on specific sockets
 # mapping_ports -m: Matrix for mapping ports to logical cores.
 # pcap_port_num -s P:file: The PCAP packet file to stream. P is the port number.
+# start_trex: Set to a nonempty value to start trex ourselves.
 [TREX]
 trex_root_path=/opt/trex-core-2.26
 config_file=/etc/trex_cfg.yaml
@@ -20,3 +21,4 @@ ip_src=16.0.0.1
 ip_dst=10.0.0.1
 warmup=15
 duration=-1
+#start_trex=yes
diff --git a/framework/pktgen.py b/framework/pktgen.py
index 9e052c832da9..86110382e9ab 100644
--- a/framework/pktgen.py
+++ b/framework/pktgen.py
@@ -204,6 +204,7 @@ class TrexPacketGenerator(PacketGenerator):
     def __init__(self, tester):
         self.pktgen_type = "trex"
         self._conn = None
+        self.control_session = None
         self._ports = []
         self._traffic_ports = []
         self._transmit_streams = {}
@@ -218,7 +219,6 @@ class TrexPacketGenerator(PacketGenerator):
 
     def connect(self):
         self._conn = self.trex_client(server=self.conf["server"])
-        time.sleep(30)
         self._conn.connect()
         for p in self._conn.get_all_ports():
             self._ports.append(p)
@@ -287,15 +287,20 @@ class TrexPacketGenerator(PacketGenerator):
         return vm
 
     def _prepare_generator(self):
-        app_param_temp = "-i"
-
-        for key in self.conf:
-            #key, value = pktgen_conf
-            if key == 'config_file':
-                app_param_temp = app_param_temp + " --cfg " + self.conf[key]
-            elif key == 'core_num':
-                app_param_temp = app_param_temp + " -c " + self.conf[key]
-
+        if self.conf.has_key('start_trex') and self.conf['start_trex']:
+            app_param_temp = "-i"
+
+            for key in self.conf:
+                #key, value = pktgen_conf
+                if key == 'config_file':
+                    app_param_temp = app_param_temp + " --cfg " + self.conf[key]
+                elif key == 'core_num':
+                    app_param_temp = app_param_temp + " -c " + self.conf[key]
+            app = self.conf['trex_root_path'] + os.sep + self.trex_app
+            cmd = app + " " + app_param_temp
+            self.control_session = self.tester.create_session('trex_control_session')
+            self.control_session.send_expect('cd ' + self.conf['trex_root_path'] + os.sep + 'scripts', '# ')
+            self.control_session.send_expect(app + " " + app_param_temp, '-Per port stats table', 30)
 
         # Insert Trex api library
         sys.path.insert(0, "{0}/scripts/automation/trex_control_plane/stl".format(self.conf['trex_root_path']))
@@ -377,6 +382,7 @@ class TrexPacketGenerator(PacketGenerator):
         if self.conf.has_key("warmup"):
             warmup = int(self.conf["warmup"])
 
+        self._traffic_ports = []
         for stream_id in stream_ids:
             stream = self._get_stream(stream_id)
             # tester port to Trex port
@@ -421,7 +427,13 @@ class TrexPacketGenerator(PacketGenerator):
         return rate_rx_bits, rate_rx_pkts
 
     def quit_generator(self):
-        self.disconnect()
+        if self._conn is not None:
+            self.disconnect()
+        if self.control_session is not None:
+            self.tester.send_expect('pkill -f _t-rex-64', '# ')
+            time.sleep(5)
+            self.tester.destroy_session(self.control_session)
+            self.control_session = None
 
 def getPacketGenerator(tester, pktgen_type="trex"):
     """
diff --git a/framework/tester.py b/framework/tester.py
index be5b0e13897c..e20c4f8ffcf7 100755
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -743,5 +743,6 @@ class Tester(Crb):
         """
         Close all resource before crb exit
         """
+        self.pktgen.quit_generator()
         self.logger.logger_exit()
         self.close()
-- 
2.14.1

[dts] [PATCH for-next v3 6/7] doc: Add T-Rex section to documentation

[Patrick MacArthur]

Explain how DTS expects T-Rex to be set up; in particular, explain that
T-Rex should be started in Layer 2 mode as opposed to Layer 3 mode, to
avoid the need to resolve the destination MAC address.

Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 doc/dts_gsg/index.rst                         |  1 +
 doc/dts_gsg/trex.rst                          | 27 +++++++++++++++++++++++++++
 test_plans/nic_single_core_perf_test_plan.rst |  2 ++
 3 files changed, 30 insertions(+)
 create mode 100644 doc/dts_gsg/trex.rst

diff --git a/doc/dts_gsg/index.rst b/doc/dts_gsg/index.rst
index fbd16dc053b0..d21e3343fa14 100644
--- a/doc/dts_gsg/index.rst
+++ b/doc/dts_gsg/index.rst
@@ -42,3 +42,4 @@ Getting Started Guide
     virtualization
     scenario
     multiple_vm
+    trex
diff --git a/doc/dts_gsg/trex.rst b/doc/dts_gsg/trex.rst
new file mode 100644
index 000000000000..ab6dfa520feb
--- /dev/null
+++ b/doc/dts_gsg/trex.rst
@@ -0,0 +1,27 @@
+Configuring T-Rex for DPDK Test Suite
+=====================================
+
+DPDK can utilize T-Rex as a traffic generator in stateless Layer 2 mode.
+This is supported for the ``nic_single_core_perf`` test but may be usable
+by other tests in the future.
+
+An example T-Rex configuration that accomplishes this is shown as follows:
+
+.. code-block:: console
+
+  - port_limit: 2
+    version: 2
+    interfaces: ["03:00.0", "03:00.1"]
+    port_info:
+            - src_mac: "aa:bb:cc:dd:ee:ff"
+              dest_mac: "ff:ee:dd:cc:bb:aa"
+            - src_mac: "ff:ee:dd:cc:bb:aa"
+              dest_mac: "aa:bb:cc:dd:ee:ff"
+
+DTS may use a standalone T-Rex instance, or can be configured to start T-Rex
+itself using the settings in ``pktgen.conf``.
+
+To read more about T-Rex stateless mode, read the
+`T-Rex stateless support guide`_.
+
+:: _T-Rex stateless support guide: https://trex-tgn.cisco.com/trex/doc/trex_stateless.html
diff --git a/test_plans/nic_single_core_perf_test_plan.rst b/test_plans/nic_single_core_perf_test_plan.rst
index 912c60d57097..428d5db29fbf 100644
--- a/test_plans/nic_single_core_perf_test_plan.rst
+++ b/test_plans/nic_single_core_perf_test_plan.rst
@@ -49,6 +49,8 @@ Prerequisites
     dts (next branch): git clone http://dpdk.org/git/tools/dts, 
                        then "git checkout next" 
     Trex code: http://trex-tgn.cisco.com/trex/release/v2.26.tar.gz 
+               (to be run in stateless Layer 2 mode, see section in
+                Getting Started Guide for more details)
     python-prettytable: 
         apt install python-prettytable (for ubuntu os) 
         or dnf install python-prettytable (for fedora os). 
-- 
2.14.1

[dts] [PATCH for-next v3 7/7] tests/TestSuite_nic_single_core_perf: Report the results in JSON format

[Patrick MacArthur]

Add a JSON report file in addition to the texttable report file, for
easier consumption by other automated tools. The JSON output produced
will look like:

    {
      "results": [
        {
          "parameters": { "frame_size": 64, "txd/rxd": 1024 },
          "throughput": { "result": "PASS", "delta": -0.452, "unit": "Mpps" }
        },
        /* ... */
      ]
    }

Each entry in the results list is essentially a table row. The
parameters "frame_size" and "txd/rxd" are the input parameters for
each given measurement. Each other element in the result dictionary
is an output measurement for the given measurement.

Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu>
---
 tests/TestSuite_nic_single_core_perf.py | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/tests/TestSuite_nic_single_core_perf.py b/tests/TestSuite_nic_single_core_perf.py
index 860fc29f607f..d1b88c3d0157 100644
--- a/tests/TestSuite_nic_single_core_perf.py
+++ b/tests/TestSuite_nic_single_core_perf.py
@@ -33,6 +33,7 @@ DPDK Test suite.
 """
 
 import utils
+import json
 import os
 import re
 import time
@@ -251,6 +252,28 @@ class TestNicSingleCorePerf(TestCase):
         file_to_save.write(str(table))
         file_to_save.close()
 
+        json_obj = dict()
+        json_obj['nic_type'] = self.nic
+        json_obj['results'] = list()
+        for frame_size in self.frame_sizes:
+            for descriptor in self.descriptors:
+                row_in = self.test_result[frame_size][descriptor]
+                row_dict = dict()
+                row_dict['parameters'] = dict()
+                row_dict['parameters']['frame_size'] = dict(
+                    value=row_in['Frame Size'], unit='bytes')
+                row_dict['parameters']['txd/rxd'] = dict(
+                    value=row_in['TXD/RXD'], unit='descriptors')
+                delta = (float(row_in['Throughput'].split()[0]) -
+                         float(row_in['Expected Throughput'].split()[0]))
+                row_dict['throughput'] = dict(
+                    delta=delta, unit=row_in['Throughput'].split()[1])
+                json_obj['results'].append(row_dict)
+        with open(os.path.join(rst.path2Result,
+                               '{0:s}_single_core_perf.json'.format(
+                                   self.nic)), 'w') as fp:
+            json.dump(json_obj, fp)
+
     def tear_down(self):
         """
         Run after each test case.
-- 
2.14.1

Re: [dts] [PATCH for-next v3 0/7] Fixes and modifications for T-Rex integration and nic_single_core_perf

[Liu, Yong]

Thanks Patrick, applied whole patch set into next branch.

Regards,
Marvin

On 03/30/2018 03:15 AM, Patrick MacArthur wrote:
> The first four patches of this series are bug fixes necessary to run the
> nic_single_core_perf test in its current state in the UNH-IOL
> environment.
>
> Patch 5 automates starting T-Rex as part of the prepare_generator()
> setup step, if the "start_trex" configuration option is enabled.
>
> Patch 6 adds a section to the DTS getting started guide which describes
> how T-Rex should be configured for relevant tests.
>
> Patch 7 adds JSON output to the nic_single_core_perf test, which can be
> more easily consumed by automated tools.
>
> This patch set targets the next branch, and I have uploaded these
> patches to our local open-source git server:
>
> The following changes since commit 41685e25bedcbb1bf895c1094913aab3bf42de1e:
>
>    framework/dut: adjust memory allocation for arm64 (2018-02-08 21:27:50 +0800)