* [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf @ 2018-03-20 22:20 Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 1/5] framework/texttable: Update to latest upstream version Patrick MacArthur ` (4 more replies) 0 siblings, 5 replies; 8+ messages in thread From: Patrick MacArthur @ 2018-03-20 22:20 UTC (permalink / raw) To: dts; +Cc: dpdklab The first four patches of this series are bug fixes necessary to run the nic_single_core_perf test in its current state in the UNH-IOL environment. Patch 5 automates starting T-Rex as part of the prepare_generator() setup step, if the "start_trex" configuration option is enabled. This patch set targets the next branch, and I have uploaded these patches to our local open-source git server: The following changes since commit 41685e25bedcbb1bf895c1094913aab3bf42de1e: framework/dut: adjust memory allocation for arm64 (2018-02-08 21:27:50 +0800) are available in the git repository at: https://oss.iol.unh.edu/dpdk/dts.git tags/for-next-v2-trex-single-core-perf for you to fetch changes up to 0d1c06d11777c7266677989a1200f264748322d1: framework/pktgen: Start T-Rex during prepare_generator() (2018-03-20 18:05:08 -0400) Patrick MacArthur (5): framework/texttable: Update to latest upstream version framework: Do not attempt ping6 on T-Rex ports tests/TestSuite_nic_single_core_perf: Fix config parsing tests/TestSuite_nic_single_core_perf: Use user-specified output dir framework/pktgen: Start T-Rex during prepare_generator() Changes from v1: - Add a configuration option to pktgen.cfg; trex is only started by DTS if the option is present and non-empty. - Drop patch 6 which renamed the test case. conf/pktgen.cfg | 2 + framework/dut.py | 4 +- framework/pktgen.py | 47 +++++++---- framework/tester.py | 11 ++- framework/texttable.py | 145 +++++++++++++++++++++----------- framework/virt_dut.py | 2 +- tests/TestSuite_nic_single_core_perf.py | 31 ++----- 7 files changed, 147 insertions(+), 95 deletions(-) -- 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [dts] [PATCH for-next v2 1/5] framework/texttable: Update to latest upstream version 2018-03-20 22:20 [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf Patrick MacArthur @ 2018-03-20 22:20 ` Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 2/5] framework: Do not attempt ping6 on T-Rex ports Patrick MacArthur ` (3 subsequent siblings) 4 siblings, 0 replies; 8+ messages in thread From: Patrick MacArthur @ 2018-03-20 22:20 UTC (permalink / raw) To: dts; +Cc: dpdklab Both DTS and the T-Rex Python API use the Python texttable module. The texttable module that is currently bundled with DTS is old and does not support methods required by t-rex. Fix this by replacing it with an updated version of the module [1]. Both the old and new version of this module are made available by the author under the LGPL-2.1+ license. [1]: https://github.com/foutaise/texttable/blob/master/texttable.py Tested-by: Ali Alnubani <alialnu@mellanox.com> Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> --- framework/texttable.py | 145 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 96 insertions(+), 49 deletions(-) diff --git a/framework/texttable.py b/framework/texttable.py index 2f37a0c7ba58..9f049035cf3e 100644 --- a/framework/texttable.py +++ b/framework/texttable.py @@ -1,7 +1,7 @@ #!/usr/bin/env python # # texttable - module for creating simple ASCII tables -# Copyright (C) 2003-2011 Gerome Fournier <jef(at)foutaise.org> +# Copyright (C) 2003-2015 Gerome Fournier <jef(at)foutaise.org> # # This library is free software; you can redistribute it and/or # modify it under the terms of the GNU Lesser General Public @@ -15,7 +15,7 @@ # # You should have received a copy of the GNU Lesser General Public # License along with this library; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA """module for creating simple ASCII tables @@ -25,9 +25,9 @@ Example: table = Texttable() table.set_cols_align(["l", "r", "c"]) table.set_cols_valign(["t", "m", "b"]) - table.add_rows([ ["Name", "Age", "Nickname"], - ["Mr\\nXavier\\nHuon", 32, "Xav'"], - ["Mr\\nBaptiste\\nClement", 1, "Baby"] ]) + table.add_rows([["Name", "Age", "Nickname"], + ["Mr\\nXavier\\nHuon", 32, "Xav'"], + ["Mr\\nBaptiste\\nClement", 1, "Baby"]]) print table.draw() + "\\n" table = Texttable() @@ -70,8 +70,8 @@ Result: __all__ = ["Texttable", "ArraySizeError"] __author__ = 'Gerome Fournier <jef(at)foutaise.org>' -__license__ = 'GPL' -__version__ = '0.8.1' +__license__ = 'LGPL' +__version__ = '0.8.4' __credits__ = """\ Jeff Kowalczyk: - textwrap improved import @@ -88,11 +88,17 @@ Roger Lew: Brian Peterson: - better handling of unicode errors + +Frank Sachsenheim: + - add Python 2/3-compatibility + +Maximilian Hils: + - fix minor bug for Python 3 compatibility """ import sys import string -from functools import reduce +import re try: if sys.version >= '2.3': @@ -105,11 +111,8 @@ except ImportError: sys.stderr.write("Can't import textwrap module!\n") raise -try: - True, False -except NameError: - (True, False) = (1, 0) - +if sys.version >= '2.7': + from functools import reduce def len(iterable): """Redefining len here so it will be able to work with non-ASCII characters @@ -118,13 +121,45 @@ def len(iterable): return iterable.__len__() try: - return len(unicode(iterable, 'utf')) + if sys.version >= '3.0': + return len(str) + else: + return len(unicode(iterable, 'utf')) except: return iterable.__len__() -class ArraySizeError(Exception): +TEXT_CODES = {'bold': {'start': '\x1b[1m', + 'end': '\x1b[22m'}, + 'cyan': {'start': '\x1b[36m', + 'end': '\x1b[39m'}, + 'blue': {'start': '\x1b[34m', + 'end': '\x1b[39m'}, + 'red': {'start': '\x1b[31m', + 'end': '\x1b[39m'}, + 'magenta': {'start': '\x1b[35m', + 'end': '\x1b[39m'}, + 'green': {'start': '\x1b[32m', + 'end': '\x1b[39m'}, + 'yellow': {'start': '\x1b[33m', + 'end': '\x1b[39m'}, + 'underline': {'start': '\x1b[4m', + 'end': '\x1b[24m'}} + +class TextCodesStripper: + keys = [re.escape(v['start']) for k,v in TEXT_CODES.items()] + keys += [re.escape(v['end']) for k,v in TEXT_CODES.items()] + pattern = re.compile("|".join(keys)) + + @staticmethod + def strip (s): + return re.sub(TextCodesStripper.pattern, '', s) + +def ansi_len (iterable): + return len(TextCodesStripper.strip(iterable)) + +class ArraySizeError(Exception): """Exception raised when specified rows don't fit the required size """ @@ -185,7 +220,7 @@ class Texttable: if len(array) != 4: raise ArraySizeError("array should contain 4 characters") - array = [x[:1] for x in [str(s) for s in array]] + array = [ x[:1] for x in [ str(s) for s in array ] ] (self._char_horiz, self._char_vert, self._char_corner, self._char_header) = array @@ -262,7 +297,7 @@ class Texttable: self._check_row_size(array) try: - array = map(int, array) + array = list(map(int, array)) if reduce(min, array) <= 0: raise ValueError except ValueError: @@ -278,7 +313,7 @@ class Texttable: - default value is set to 3 """ - if not isinstance(width, int) or width < 0: + if not type(width) is int or width < 0: raise ValueError('width must be an integer greater then 0') self._precision = width @@ -287,7 +322,7 @@ class Texttable: """ self._check_row_size(array) - self._header = map(str, array) + self._header = list(map(str, array)) def add_row(self, array): """Add a row in the rows stack @@ -361,7 +396,11 @@ class Texttable: try: f = float(x) except: - return str(x) + try: + return str(x) + except: + return x.encode('utf-8') + n = self._precision dtype = self._dtype[i] @@ -393,8 +432,8 @@ class Texttable: if not self._row_size: self._row_size = len(array) elif self._row_size != len(array): - raise ArraySizeError("array should contain %d elements" - % self._row_size) + raise ArraySizeError("array should contain %d elements" \ + % self._row_size) def _has_vlines(self): """Return a boolean, if vlines are required or not @@ -443,13 +482,13 @@ class Texttable: horiz = self._char_header # compute cell separator s = "%s%s%s" % (horiz, [horiz, self._char_corner][self._has_vlines()], - horiz) + horiz) # build the line - l = string.join([horiz * n for n in self._width], s) + l = s.join([horiz * n for n in self._width]) # add border if needed if self._has_border(): l = "%s%s%s%s%s\n" % (self._char_corner, horiz, l, horiz, - self._char_corner) + self._char_corner) else: l += "\n" return l @@ -466,10 +505,10 @@ class Texttable: for line in cell_lines: length = 0 parts = line.split('\t') - for part, i in zip(parts, range(1, len(parts) + 1)): + for part, i in zip(parts, list(range(1, len(parts) + 1))): length = length + len(part) if i < len(parts): - length = (length / 8 + 1) * 8 + length = (length//8 + 1) * 8 maxi = max(maxi, length) return maxi @@ -485,17 +524,17 @@ class Texttable: return maxi = [] if self._header: - maxi = [self._len_cell(x) for x in self._header] + maxi = [ self._len_cell(x) for x in self._header ] for row in self._rows: - for cell, i in zip(row, range(len(row))): + for cell,i in zip(row, list(range(len(row)))): try: maxi[i] = max(maxi[i], self._len_cell(cell)) except (TypeError, IndexError): maxi.append(self._len_cell(cell)) items = len(maxi) - length = reduce(lambda x, y: x + y, maxi) + length = reduce(lambda x, y: x+y, maxi) if self._max_width and length + items * 3 + 1 > self._max_width: - maxi = [(self._max_width - items * 3 - 1) / items + maxi = [(self._max_width - items * 3 -1) // items \ for n in range(items)] self._width = maxi @@ -524,14 +563,14 @@ class Texttable: for cell, width, align in zip(line, self._width, self._align): length += 1 cell_line = cell[i] - fill = width - len(cell_line) + fill = width - ansi_len(cell_line) if isheader: - align = "l" + align = "c" if align == "r": out += "%s " % (fill * space + cell_line) elif align == "c": - out += "%s " % (fill / 2 * space + cell_line - + (fill / 2 + fill % 2) * space) + out += "%s " % (int(fill/2) * space + cell_line \ + + int(fill/2 + fill%2) * space) else: out += "%s " % (cell_line + fill * space) if length < len(line): @@ -551,26 +590,34 @@ class Texttable: array = [] for c in cell.split('\n'): try: - c = unicode(c, 'utf') + c = str(c) except UnicodeDecodeError as strerror: sys.stderr.write("UnicodeDecodeError exception for string '%s': %s\n" % (c, strerror)) - c = unicode(c, 'utf', 'replace') - array.extend(textwrap.wrap(c, width)) + if sys.version >= '3.0': + c = str(c, 'utf', 'replace') + else: + c = unicode(c, 'utf', 'replace') + + # imarom - no wrap for now + #array.extend(textwrap.wrap(c, width)) + array.extend([c]) + line_wrapped.append(array) - max_cell_lines = reduce(max, map(len, line_wrapped)) + max_cell_lines = reduce(max, list(map(len, line_wrapped))) for cell, valign in zip(line_wrapped, self._valign): if isheader: valign = "t" if valign == "m": missing = max_cell_lines - len(cell) - cell[:0] = [""] * (missing / 2) - cell.extend([""] * (missing / 2 + missing % 2)) + cell[:0] = [""] * int(missing / 2) + cell.extend([""] * int(missing / 2 + missing % 2)) elif valign == "b": cell[:0] = [""] * (max_cell_lines - len(cell)) else: cell.extend([""] * (max_cell_lines - len(cell))) return line_wrapped + if __name__ == '__main__': table = Texttable() table.set_cols_align(["l", "r", "c"]) @@ -578,7 +625,7 @@ if __name__ == '__main__': table.add_rows([["Name", "Age", "Nickname"], ["Mr\nXavier\nHuon", 32, "Xav'"], ["Mr\nBaptiste\nClement", 1, "Baby"]]) - print table.draw() + "\n" + print(table.draw() + "\n") table = Texttable() table.set_deco(Texttable.HEADER) @@ -586,11 +633,11 @@ if __name__ == '__main__': 'f', # float (decimal) 'e', # float (exponent) 'i', # integer - 'a']) # automatic + 'a']) # automatic table.set_cols_align(["l", "r", "r", "r", "l"]) - table.add_rows([["text", "float", "exp", "int", "auto"], - ["abcd", "67", 654, 89, 128.001], - ["efghijk", 67.5434, .654, 89.6, 12800000000000000000000.00023], - ["lmn", 5e-78, 5e-78, 89.4, .000000000000128], - ["opqrstu", .023, 5e+78, 92., 12800000000000000000000]]) - print table.draw() + table.add_rows([["text", "float", "exp", "int", "auto"], + ["abcd", "67", 654, 89, 128.001], + ["efghijk", 67.5434, .654, 89.6, 12800000000000000000000.00023], + ["lmn", 5e-78, 5e-78, 89.4, .000000000000128], + ["opqrstu", .023, 5e+78, 92., 12800000000000000000000]]) + print(table.draw()) -- 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [dts] [PATCH for-next v2 2/5] framework: Do not attempt ping6 on T-Rex ports 2018-03-20 22:20 [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 1/5] framework/texttable: Update to latest upstream version Patrick MacArthur @ 2018-03-20 22:20 ` Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 3/5] tests/TestSuite_nic_single_core_perf: Fix config parsing Patrick MacArthur ` (2 subsequent siblings) 4 siblings, 0 replies; 8+ messages in thread From: Patrick MacArthur @ 2018-03-20 22:20 UTC (permalink / raw) To: dts; +Cc: dpdklab DTS uses ping6 to determine whether the DUT and tester node ports are connected. Currently the IXIA ports are excluded from this test because they are matched explicitly with DUT ports in the ports.cfg file and the traffic generator will not respond to pings in the usual fashion. For the same reason, exclude T-Rex ports from ping6 matching and connectivity testing. This avoids a traceback due to the T-Rex ports not having enough available information to complete the ping6 check. Tested-by: Ali Alnubani <alialnu@mellanox.com> Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> --- framework/dut.py | 4 ++-- framework/tester.py | 10 ++++++---- framework/virt_dut.py | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/framework/dut.py b/framework/dut.py index 31b3daaaebcb..5b72c7a523e3 100644 --- a/framework/dut.py +++ b/framework/dut.py @@ -980,13 +980,13 @@ class Dut(Crb): def disable_tester_ipv6(self): for tester_port in self.ports_map: - if self.tester.ports_info[tester_port]['type'] != 'ixia': + if self.tester.ports_info[tester_port]['type'].lower() not in ('ixia', 'trex'): port = self.tester.ports_info[tester_port]['port'] port.disable_ipv6() def enable_tester_ipv6(self): for tester_port in range(len(self.tester.ports_info)): - if self.tester.ports_info[tester_port]['type'] != 'ixia': + if self.tester.ports_info[tester_port]['type'].lower() not in ('ixia', 'trex'): port = self.tester.ports_info[tester_port]['port'] port.enable_ipv6() diff --git a/framework/tester.py b/framework/tester.py index 151200872338..be5b0e13897c 100755 --- a/framework/tester.py +++ b/framework/tester.py @@ -200,7 +200,7 @@ class Tester(Crb): if localPort == -1: raise ParameterInvalidException("local port should not be -1") - if self.ports_info[localPort]['type'] == 'ixia': + if self.ports_info[localPort]['type'] in ('ixia', 'trex'): return "00:00:00:00:00:01" else: return self.ports_info[localPort]['mac'] @@ -305,7 +305,7 @@ class Tester(Crb): return for port_info in self.ports_info: - if port_info['type'] == 'ixia': + if port_info['type'].lower() in ('ixia', 'trex'): continue addr_array = port_info['pci'].split(':') @@ -394,7 +394,7 @@ class Tester(Crb): """ Send ping6 packet from local port with destination ipv4 address. """ - if self.ports_info[localPort]['type'] == 'ixia': + if self.ports_info[localPort]['type'].lower() in ('ixia', 'trex'): return "Not implemented yet" else: return self.send_expect("ping -w 5 -c 5 -A -I %s %s" % (self.ports_info[localPort]['intf'], ipv4), "# ", 10) @@ -403,8 +403,10 @@ class Tester(Crb): """ Send ping6 packet from local port with destination ipv6 address. """ - if self.ports_info[localPort]['type'] == 'ixia': + if self.ports_info[localPort]['type'].lower() == 'ixia': return self.ixia_packet_gen.send_ping6(self.ports_info[localPort]['pci'], mac, ipv6) + elif self.ports_info[localPort]['type'].lower() == 'trex': + return "Not implemented yet" else: return self.send_expect("ping6 -w 5 -c 5 -A %s%%%s" % (ipv6, self.ports_info[localPort]['intf']), "# ", 10) diff --git a/framework/virt_dut.py b/framework/virt_dut.py index 62688dc83b9e..d4cd805d2a03 100644 --- a/framework/virt_dut.py +++ b/framework/virt_dut.py @@ -385,7 +385,7 @@ class VirtDut(DPDKdut): remotepci = self.tester.ports_info[remotePort]['pci'] port_type = self.tester.ports_info[remotePort]['type'] # IXIA port should not check whether has vfs - if port_type != 'ixia': + if port_type.lower() not in ('ixia', 'trex'): remoteport = self.tester.ports_info[remotePort]['port'] vfs = [] # vm_dut and tester in same dut -- 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [dts] [PATCH for-next v2 3/5] tests/TestSuite_nic_single_core_perf: Fix config parsing 2018-03-20 22:20 [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 1/5] framework/texttable: Update to latest upstream version Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 2/5] framework: Do not attempt ping6 on T-Rex ports Patrick MacArthur @ 2018-03-20 22:20 ` Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 5/5] framework/pktgen: Start T-Rex during prepare_generator() Patrick MacArthur 4 siblings, 0 replies; 8+ messages in thread From: Patrick MacArthur @ 2018-03-20 22:20 UTC (permalink / raw) To: dts; +Cc: dpdklab The configuration values in test suite configuration are already eval()'d as of commit 081fbf69fe20 ("framework/config: utilize eval to parse configurations") so we don't need to do any special parsing here. This also changes the expected_throughput dictionary keys and values from strings to floats, which is more convenient. Tested-by: Ali Alnubani <alialnu@mellanox.com> Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> --- tests/TestSuite_nic_single_core_perf.py | 27 ++++----------------------- 1 file changed, 4 insertions(+), 23 deletions(-) diff --git a/tests/TestSuite_nic_single_core_perf.py b/tests/TestSuite_nic_single_core_perf.py index 3411a31b7532..21de93624cb4 100644 --- a/tests/TestSuite_nic_single_core_perf.py +++ b/tests/TestSuite_nic_single_core_perf.py @@ -59,8 +59,8 @@ class TestNicSingleCorePerf(TestCase): self.trafficDuration = 60 #load the expected throughput for required nic - self.expected_throughput_nnt = self.parse_string(self.get_suite_cfg()["throughput_nnt"]) - self.expected_throughput_fvl25g = self.parse_string(self.get_suite_cfg()["throughput_fvl25g"]) + self.expected_throughput_nnt = self.get_suite_cfg()["throughput_nnt"] + self.expected_throughput_fvl25g = self.get_suite_cfg()["throughput_fvl25g"] # The acdepted gap between expected throughput and actual throughput, 1 Mpps self.gap = 1 @@ -169,9 +169,9 @@ class TestNicSingleCorePerf(TestCase): ret_data[header[2]] = str(float("%.3f" % throughput)) + " Mpps" ret_data[header[3]] = str(float("%.3f" % (throughput * 100 / wirespeed))) + "%" if self.nic == "niantic": - ret_data[header[4]] = self.expected_throughput_nnt[str(frame_size)][str(descriptor)] + " Mpps" + ret_data[header[4]] = str(self.expected_throughput_nnt[frame_size][descriptor]) + " Mpps" elif self.nic == "fortville_25g": - ret_data[header[4]] = self.expected_throughput_fvl25g[str(frame_size)][str(descriptor)] + " Mpps" + ret_data[header[4]] = str(self.expected_throughput_fvl25g[frame_size][descriptor]) + " Mpps" ret_datas[descriptor] = deepcopy(ret_data) self.test_result[frame_size] = deepcopy(ret_datas) @@ -248,25 +248,6 @@ class TestNicSingleCorePerf(TestCase): file_to_save.write(str(table)) file_to_save.close() - def parse_string(self, string): - ''' - Parse a string in the formate of a dictionary and convert it into a real dictionary. - ''' - element_pattern = re.compile(".\d+:.*?}") - string_elements = element_pattern.findall(string) - ret = {} - for element in string_elements: - ex_pattern = re.compile("(\d+): *{(.*)}") - ex_ret = ex_pattern.search(element) - ret[ex_ret.groups()[0]] = ex_ret.groups()[1] - inner_datas = ex_ret.groups()[1].split(",") - ret_inner = {} - for data in inner_datas: - match_inner = data.split(":") - ret_inner[match_inner[0].strip()] = match_inner[1].strip() - ret[ex_ret.groups()[0]] = deepcopy(ret_inner) - return ret - def tear_down(self): """ Run after each test case. -- 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir 2018-03-20 22:20 [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf Patrick MacArthur ` (2 preceding siblings ...) 2018-03-20 22:20 ` [dts] [PATCH for-next v2 3/5] tests/TestSuite_nic_single_core_perf: Fix config parsing Patrick MacArthur @ 2018-03-20 22:20 ` Patrick MacArthur 2018-03-28 16:19 ` Ali Alnubani 2018-03-20 22:20 ` [dts] [PATCH for-next v2 5/5] framework/pktgen: Start T-Rex during prepare_generator() Patrick MacArthur 4 siblings, 1 reply; 8+ messages in thread From: Patrick MacArthur @ 2018-03-20 22:20 UTC (permalink / raw) To: dts; +Cc: dpdklab The user can override the default output directory via a command line argument, but this script does not account for that. Fix the script to look up the output directory that the user requested and place the custom output file in that directory. Tested-by: Ali Alnubani <alialnu@mellanox.com> Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> --- tests/TestSuite_nic_single_core_perf.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/TestSuite_nic_single_core_perf.py b/tests/TestSuite_nic_single_core_perf.py index 21de93624cb4..bff763c129fa 100644 --- a/tests/TestSuite_nic_single_core_perf.py +++ b/tests/TestSuite_nic_single_core_perf.py @@ -41,6 +41,7 @@ from settings import HEADER_SIZE from pmd_output import PmdOutput from copy import deepcopy from prettytable import PrettyTable +import rst class TestNicSingleCorePerf(TestCase): @@ -244,7 +245,8 @@ class TestNicSingleCorePerf(TestCase): table_row.append(self.test_result[frame_size][descriptor][header[3]]) table_row.append(self.test_result[frame_size][descriptor][header[4]]) table.add_row(table_row) - file_to_save = open("output/%s_single_core_perf.txt" % self.nic, 'w') + file_to_save = open(os.path.join( + rst.path2Result, "%s_single_core_perf.txt" % self.nic), 'w') file_to_save.write(str(table)) file_to_save.close() -- 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir 2018-03-20 22:20 ` [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir Patrick MacArthur @ 2018-03-28 16:19 ` Ali Alnubani 2018-03-29 18:55 ` Patrick MacArthur 0 siblings, 1 reply; 8+ messages in thread From: Ali Alnubani @ 2018-03-28 16:19 UTC (permalink / raw) To: 'Patrick MacArthur', 'dts@dpdk.org' Cc: 'dpdklab@iol.unh.edu' Hi Patrick, You are using os module w/o it being imported in this patch. Thanks, Ali > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Patrick MacArthur > Sent: Wednesday, March 21, 2018 12:21 AM > To: dts@dpdk.org > Cc: dpdklab@iol.unh.edu > Subject: [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: > Use user-specified output dir > > The user can override the default output directory via a command line > argument, but this script does not account for that. Fix the script to look up > the output directory that the user requested and place the custom output > file in that directory. > > Tested-by: Ali Alnubani <alialnu@mellanox.com> > Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> > --- > tests/TestSuite_nic_single_core_perf.py | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/tests/TestSuite_nic_single_core_perf.py > b/tests/TestSuite_nic_single_core_perf.py > index 21de93624cb4..bff763c129fa 100644 > --- a/tests/TestSuite_nic_single_core_perf.py > +++ b/tests/TestSuite_nic_single_core_perf.py > @@ -41,6 +41,7 @@ from settings import HEADER_SIZE from pmd_output > import PmdOutput from copy import deepcopy from prettytable import > PrettyTable > +import rst > > class TestNicSingleCorePerf(TestCase): > > @@ -244,7 +245,8 @@ class TestNicSingleCorePerf(TestCase): > > table_row.append(self.test_result[frame_size][descriptor][header[3]]) > > table_row.append(self.test_result[frame_size][descriptor][header[4]]) > table.add_row(table_row) > - file_to_save = open("output/%s_single_core_perf.txt" % self.nic, 'w') > + file_to_save = open(os.path.join( > + rst.path2Result, "%s_single_core_perf.txt" % self.nic), > + 'w') > file_to_save.write(str(table)) > file_to_save.close() > > -- > 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir 2018-03-28 16:19 ` Ali Alnubani @ 2018-03-29 18:55 ` Patrick MacArthur 0 siblings, 0 replies; 8+ messages in thread From: Patrick MacArthur @ 2018-03-29 18:55 UTC (permalink / raw) To: Ali Alnubani; +Cc: dts, dpdklab Hi, Ali, That was a rebase error which will be fixed in v3. Thanks, Patrick On Wed, Mar 28, 2018 at 12:19 PM, Ali Alnubani <alialnu@mellanox.com> wrote: > Hi Patrick, > You are using os module w/o it being imported in this patch. > > Thanks, > Ali > >> -----Original Message----- >> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Patrick MacArthur >> Sent: Wednesday, March 21, 2018 12:21 AM >> To: dts@dpdk.org >> Cc: dpdklab@iol.unh.edu >> Subject: [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: >> Use user-specified output dir >> >> The user can override the default output directory via a command line >> argument, but this script does not account for that. Fix the script to look up >> the output directory that the user requested and place the custom output >> file in that directory. >> >> Tested-by: Ali Alnubani <alialnu@mellanox.com> >> Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> >> --- >> tests/TestSuite_nic_single_core_perf.py | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/tests/TestSuite_nic_single_core_perf.py >> b/tests/TestSuite_nic_single_core_perf.py >> index 21de93624cb4..bff763c129fa 100644 >> --- a/tests/TestSuite_nic_single_core_perf.py >> +++ b/tests/TestSuite_nic_single_core_perf.py >> @@ -41,6 +41,7 @@ from settings import HEADER_SIZE from pmd_output >> import PmdOutput from copy import deepcopy from prettytable import >> PrettyTable >> +import rst >> >> class TestNicSingleCorePerf(TestCase): >> >> @@ -244,7 +245,8 @@ class TestNicSingleCorePerf(TestCase): >> >> table_row.append(self.test_result[frame_size][descriptor][header[3]]) >> >> table_row.append(self.test_result[frame_size][descriptor][header[4]]) >> table.add_row(table_row) >> - file_to_save = open("output/%s_single_core_perf.txt" % self.nic, 'w') >> + file_to_save = open(os.path.join( >> + rst.path2Result, "%s_single_core_perf.txt" % self.nic), >> + 'w') >> file_to_save.write(str(table)) >> file_to_save.close() >> >> -- >> 2.14.1 > -- Patrick MacArthur Research and Development, High Performance Networking and Storage UNH InterOperability Laboratory ^ permalink raw reply [flat|nested] 8+ messages in thread
* [dts] [PATCH for-next v2 5/5] framework/pktgen: Start T-Rex during prepare_generator() 2018-03-20 22:20 [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf Patrick MacArthur ` (3 preceding siblings ...) 2018-03-20 22:20 ` [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir Patrick MacArthur @ 2018-03-20 22:20 ` Patrick MacArthur 4 siblings, 0 replies; 8+ messages in thread From: Patrick MacArthur @ 2018-03-20 22:20 UTC (permalink / raw) To: dts; +Cc: dpdklab The currently merged patchset for T-Rex does not actually start T-Rex. Add a configuration option "start_trex" and start T-Rex during the prepare_generator() step if this is set to a non-empty string. We need T-Rex running during this step so we can determine which ports the T-Rex API is making available to the DUT before we set up the DUT's network interfaces accordingly. This configuration option assumes that T-Rex will always be run on the tester node. Signed-off-by: Patrick MacArthur <pmacarth@iol.unh.edu> --- conf/pktgen.cfg | 2 ++ framework/pktgen.py | 47 ++++++++++++++++++++++++++++++++--------------- framework/tester.py | 1 + 3 files changed, 35 insertions(+), 15 deletions(-) diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg index 1f3c7952f7f8..cc3fe61356b1 100644 --- a/conf/pktgen.cfg +++ b/conf/pktgen.cfg @@ -9,6 +9,7 @@ # socket_memory --socket-mem: Memory to allocate on specific sockets # mapping_ports -m: Matrix for mapping ports to logical cores. # pcap_port_num -s P:file: The PCAP packet file to stream. P is the port number. +# start_trex: Set to a nonempty value to start trex ourselves. [TREX] trex_root_path=/opt/trex-core-2.26 config_file=/etc/trex_cfg.yaml @@ -20,3 +21,4 @@ ip_src=16.0.0.1 ip_dst=10.0.0.1 warmup=15 duration=-1 +#start_trex=yes diff --git a/framework/pktgen.py b/framework/pktgen.py index 9e052c832da9..cffa7191da6d 100644 --- a/framework/pktgen.py +++ b/framework/pktgen.py @@ -204,6 +204,7 @@ class TrexPacketGenerator(PacketGenerator): def __init__(self, tester): self.pktgen_type = "trex" self._conn = None + self.control_session = None self._ports = [] self._traffic_ports = [] self._transmit_streams = {} @@ -218,7 +219,6 @@ class TrexPacketGenerator(PacketGenerator): def connect(self): self._conn = self.trex_client(server=self.conf["server"]) - time.sleep(30) self._conn.connect() for p in self._conn.get_all_ports(): self._ports.append(p) @@ -287,15 +287,20 @@ class TrexPacketGenerator(PacketGenerator): return vm def _prepare_generator(self): - app_param_temp = "-i" - - for key in self.conf: - #key, value = pktgen_conf - if key == 'config_file': - app_param_temp = app_param_temp + " --cfg " + self.conf[key] - elif key == 'core_num': - app_param_temp = app_param_temp + " -c " + self.conf[key] - + if self.conf.has_key('start_trex') and self.conf['start_trex']: + app_param_temp = "-i" + + for key in self.conf: + #key, value = pktgen_conf + if key == 'config_file': + app_param_temp = app_param_temp + " --cfg " + self.conf[key] + elif key == 'core_num': + app_param_temp = app_param_temp + " -c " + self.conf[key] + app = self.conf['trex_root_path'] + os.sep + self.trex_app + cmd = app + " " + app_param_temp + self.control_session = self.tester.create_session('trex_control_session') + self.control_session.send_expect('cd ' + self.conf['trex_root_path'] + os.sep + 'scripts', '# ') + self.control_session.send_expect(app + " " + app_param_temp, '-Per port stats table', 30) # Insert Trex api library sys.path.insert(0, "{0}/scripts/automation/trex_control_plane/stl".format(self.conf['trex_root_path'])) @@ -377,16 +382,22 @@ class TrexPacketGenerator(PacketGenerator): if self.conf.has_key("warmup"): warmup = int(self.conf["warmup"]) + self._traffic_ports = list() + for stream_id in stream_ids: + self._traffic_ports.append(self._ports[self._get_stream(stream_id)['tx_port']]) + + print self._traffic_ports + self._conn.set_service_mode(ports=self._traffic_ports, enabled=True) + self._conn.resolve(ports=self._traffic_ports) + self._conn.set_service_mode(ports=self._traffic_ports, enabled=False) + for stream_id in stream_ids: stream = self._get_stream(stream_id) # tester port to Trex port tx_port = stream["tx_port"] p = self._ports[tx_port] - self._conn.add_streams(self._transmit_streams[stream_id], ports=[p]) + sid = self._conn.add_streams(self._transmit_streams[stream_id], ports=[p]) rate = stream["options"]["rate"] - self._traffic_ports.append(p) - - print self._traffic_ports if self.conf.has_key("core_mask"): self._conn.start(ports=self._traffic_ports, mult=rate, duration=warmup, core_mask=self.conf["core_mask"]) @@ -421,7 +432,13 @@ class TrexPacketGenerator(PacketGenerator): return rate_rx_bits, rate_rx_pkts def quit_generator(self): - self.disconnect() + if self._conn is not None: + self.disconnect() + self.tester.send_expect('pkill -f _t-rex-64', '# ') + time.sleep(5) + if self.control_session is not None: + self.tester.destroy_session(self.control_session) + self.control_session = None def getPacketGenerator(tester, pktgen_type="trex"): """ diff --git a/framework/tester.py b/framework/tester.py index be5b0e13897c..e20c4f8ffcf7 100755 --- a/framework/tester.py +++ b/framework/tester.py @@ -743,5 +743,6 @@ class Tester(Crb): """ Close all resource before crb exit """ + self.pktgen.quit_generator() self.logger.logger_exit() self.close() -- 2.14.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-03-29 18:55 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-03-20 22:20 [dts] [PATCH for-next v2 0/5] Fixes and modifications for T-Rex integration and nic_single_core_perf Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 1/5] framework/texttable: Update to latest upstream version Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 2/5] framework: Do not attempt ping6 on T-Rex ports Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 3/5] tests/TestSuite_nic_single_core_perf: Fix config parsing Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 4/5] tests/TestSuite_nic_single_core_perf: Use user-specified output dir Patrick MacArthur 2018-03-28 16:19 ` Ali Alnubani 2018-03-29 18:55 ` Patrick MacArthur 2018-03-20 22:20 ` [dts] [PATCH for-next v2 5/5] framework/pktgen: Start T-Rex during prepare_generator() Patrick MacArthur
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).