v3 Add 2544 dichotomy algorithm as ma,lihong&han,yingya required. *. add measure_rfc2544_dichotomy method *. add rfc2544 dichotomy usage document v2 Add new demand of pktgen as wang,yinan required. *. add a callback to deal with executing other tools query actions in throughput traffic lasting status. *. rebase source code. v1 As chen zhaoyan new demand of pktgen, add these changes *. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to realize measure_throughput support return several throughput statistic data in a duration. *. add test method(latency/loss/throughput/rfc2544) options parameter usage comment in pktgen_base module. *. add test method(latency/loss/throughput) delay/duration options new usage definition and relevant process source code in pktgen_base module. *. set delay option to the warm up time before start main traffic. *. set pktgen duration default value to 10 second. *. remove duration option used in trex/ixia module, move duration option in testing scenario methods(latency/loss/throughput) in pktgen_base module. *. remove un-used optons in pktgen.cfg. *. remove un-used parameter in trex/ixia _start_transmission. *. rebase coremask new setting options from next branch. yufengmx (5): conf/pktgen: remove un-used optons doc/dts_gsg/pktgen_prog_guide: update description framework/pktgen_base: add new features framework/pktgen_ixia: add new features framework/pktgen_trex: add new features conf/pktgen.cfg | 18 ++- doc/dts_gsg/pktgen_prog_guide.rst | 86 +++++++++-- framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++-- framework/pktgen_ixia.py | 31 +--- framework/pktgen_trex.py | 63 +++----- 5 files changed, 340 insertions(+), 102 deletions(-) -- 2.21.0
*. remove core_num, it is not used by pktgen. *. remove warmup key, set it in suite script. *. remove duration key, set it in suite script. *. remove ip_src key, set it in suite script. *. remove ip_dst key, set it in suite script. *. add more core_mask setting option. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- conf/pktgen.cfg | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg index 667e1e8..e111459 100644 --- a/conf/pktgen.cfg +++ b/conf/pktgen.cfg @@ -4,7 +4,14 @@ # trex server binary file is under this directory. # trex_lib_path(optional): trex stateless client libs directory, it is optional. # If it is not set, use a default relative directory. -# coremask -c: A hexadecimal bitmask of cores to run on +# coremask: +# 1. a list of masks (one core mask per port), sort sequence as +# <port 0 core mask>,<port 1 core mask> +# example: core_mask=0x3,0x5 +# 2. CORE_MASK_PIN +# for each dual ports (a group that shares the same cores), the cores will +# be divided half pinned for each port. +# example: core_mask=CORE_MASK_PIN # num -n: Number of memory channels # proc_type --proc-type: Type of this process # pci_blacklist --pci-blacklist, -b: Add a PCI device in black list. @@ -20,13 +27,8 @@ trex_lib_path=/opt/trex/vx.xxx/automation/trex_control_plane/interactive config_file=/etc/trex_cfg.yaml server=10.67.111.143 pcap_file=/opt/trex-core-2.26/scripts/stl/sample.pcap -core_num=4 -#core_mask=0x3 -ip_src=16.0.0.1 -ip_dst=10.0.0.1 -warmup=15 -duration=-1 -#start_trex=yes +core_mask=0x3,0x5 +start_trex=yes # IXIA port Configuration # IxiaGroup: Group name for IXIA ports -- 2.21.0
*. update test methods(latency/loss/throughput/rfc2544) input parameters description. *. add rfc2544 dichotomy usage document. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- doc/dts_gsg/pktgen_prog_guide.rst | 86 ++++++++++++++++++++++++++----- 1 file changed, 73 insertions(+), 13 deletions(-) diff --git a/doc/dts_gsg/pktgen_prog_guide.rst b/doc/dts_gsg/pktgen_prog_guide.rst index 32d2082..e0fdcf4 100644 --- a/doc/dts_gsg/pktgen_prog_guide.rst +++ b/doc/dts_gsg/pktgen_prog_guide.rst @@ -145,7 +145,7 @@ define a stream transmit behavior. basic content including:: - 'rate': 0 ~ 100 int type + 'rate': 0 ~ 100 int type, port line rate should set it. 'transmit_mode': TRANSMIT_CONT/TRANSMIT_S_BURST TRANSMIT_CONT define a continuous transmit. TRANSMIT_S_BURST define a burst transmit with custom number of packets. @@ -332,9 +332,22 @@ option traffic_option = { # test method name, if use `measure_throughput`, ignore this key 'method': 'throughput', - # port rate percent + # port rate percent, float(0--100), default value is 100.(reserved) 'rate': 100, - # transmit lasting time second + # warm up time before start main transmission. If it is set, it will start + # a custom time transmission to make sure packet generator under good + # status. It is an optional key. + 'delay': 5, + # the interval time of get throughput statistic (second). + # If set this key value, pktgen will return several throughput statistic + # data in a duration. If not set this key value, only return one statistic + # data. It is used coupled with `duration` option. + 'interval': 1, + # this key works with ``interval`` key. If it is set, the callback + # of suite level will be executed after getting throughput statistic. + # callback method should define as below, don't add sleep in this method. + 'callback' : callback_method, + # transmission lasting time(second), default value is 10 second. 'duration': 5} return value @@ -358,9 +371,13 @@ option traffic_option = { # test method name, if use `measure_loss`, ignore this key 'method': 'loss', - # port rate percent + # port rate percent, float(0--100), default value is 100.(reserved) 'rate': 100, - # transmit lasting time second + # warm up time before start main transmission. If it is set, it will start + # a custom time transmission to make sure packet generator under good + # status. It is an optional key. + 'delay': 5, + # transmission lasting time(second), default value is 10 second. 'duration': 5} return value @@ -382,9 +399,13 @@ option traffic_option = { # test method name, if use `measure_latency`, ignore this key 'method': 'latency', - # port rate percent + # port rate percent, float(0--100), default value is 100.(reserved) 'rate': 100, - # transmit lasting time second + # warm up time before start main transmission. If it is set, it will start + # a custom time transmission to make sure packet generator under ready + # status. It is an optional key. + 'delay': 5, + # transmission lasting time(second), default value is 10 second. 'duration': 5} return value @@ -398,7 +419,7 @@ return value rfc2544 option -------------- -single stream & rfc2544 +rfc2544 testing scenario by decreasing step. option ++++++ @@ -406,15 +427,19 @@ option .. code-block:: python traffic_option = { - # test method name, if use `measure_rfc2544`, ignore this key + # test method name, if use `measure_rfc2544`, ignore this key. 'method': 'rfc2544', - # port rate percent at first round testing, 0 ~ 100, default is 100 + # port rate percent at first round testing, 0 ~ 100, default is 100. 'rate': 100, - # permit packet drop rate + # permit packet drop rate, default is 0.001. 'pdr': 0.001, - # port rate percent drop step, 0 ~ 100 , default is 1 + # port rate percent drop step, 0 ~ 100 , default is 1. 'drop_step': 1, - # transmit lasting time second + # warm up time before start main transmission. If it is set, it will start + # a custom time transmission to make sure packet generator under ready + # status. It is an optional key. + 'delay': 5, + # transmission lasting time(second), default value is 10 second. 'duration': 5} return value @@ -424,6 +449,41 @@ return value loss_stats = (loss_rate, tx_pkts, rx_pkts) +rfc2544_dichotomy option +------------------------ +rfc2544 testing scenario using dichotomy algorithm. + +option +++++++ + +.. code-block:: python + + traffic_option = { + # test method name, if use `measure_rfc2544_dichotomy` method, ignore this key. + 'method': 'rfc2544_dichotomy', + # dichotomy algorithm lower bound rate percent, default is 0. + 'min_rate': 0, + # dichotomy algorithm upper bound rate percent, default is 100. + 'max_rate': 100, + # dichotomy algorithm accuracy, default 0.001. + 'accuracy': 0.001, + # permit packet drop rate, default is 0.001. + 'pdr': 0.001, + # warm up time before start main transmission. If it is set, it will start + # a custom time transmission to make sure packet generator under ready + # status. It is an optional key. + 'delay': 5, + # transmission lasting time(second), default value is 10 second. + 'duration': 10} + +return value +++++++++++++ + +.. code-block:: python + + loss_stats = (loss_rate, tx_pkts, rx_pkts) + + reference example ================= This example show how to use pktgen in suite script. In fact, most scenario are -- 2.21.0
*. add test method(latency/loss/throughput) delay/duration options new usage definition and relevant process source code. *. set delay option to be the warm up time after start transmission. *. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to realize measure_throughput support return several throughput numbers in a duration. *. set duration option default value to 10 second. *. add test method(latency/loss/throughput/rfc2544) options parameter usage comment, which is the same as doc pktgen_prog_guide.rst. *. add a callback to deal with executing other tools query actions in throughput traffic lasting status. *. add measure_rfc2544_dichotomy method. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 228 insertions(+), 16 deletions(-) diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py index 7855f6f..75c3036 100644 --- a/framework/pktgen_base.py +++ b/framework/pktgen_base.py @@ -81,7 +81,7 @@ class PacketGenerator(object): return port_idx else: port = -1 - except: + except Exception as e: port = -1 return port @@ -100,7 +100,7 @@ class PacketGenerator(object): port = self._get_gen_port(tester_pci) msg = "test port {0} map gen port {1}".format(port_id, port) self.logger.debug(msg) - except: + except Exception as e: port = -1 return port @@ -157,39 +157,120 @@ class PacketGenerator(object): def reset_streams(self): self.__streams = [] - def measure_throughput(self, stream_ids=[], options={}): - """ - Measure throughput on each tx ports - """ - bps_rx = [] - pps_rx = [] - self._prepare_transmission(stream_ids=stream_ids) + def __warm_up_pktgen(self, stream_ids, options, delay): + ''' run warm up traffic before start main traffic ''' + if not delay: + return + msg = '{1} packet generator: run traffic {0}s to warm up ... '.format( + delay, self.pktgen_type) + self.logger.info(msg) self._start_transmission(stream_ids, options) - - delay = options.get('delay') or 5 time.sleep(delay) + self._stop_transmission(stream_ids) + self._clear_streams() + + def __get_single_throughput_statistic(self, stream_ids): + bps_rx = [] + pps_rx = [] used_rx_port = [] + msg = 'begin get port statistic ...' + self.logger.info(msg) for stream_id in stream_ids: if self.__streams[stream_id]['rx_port'] not in used_rx_port: rxbps_rates, rxpps_rates = self._retrieve_port_statistic( - stream_id, 'throughput') + stream_id, 'throughput') used_rx_port.append(self.__streams[stream_id]['rx_port']) bps_rx.append(rxbps_rates) pps_rx.append(rxpps_rates) - self._stop_transmission(stream_id) - bps_rx_total = self._summary_statistic(bps_rx) pps_rx_total = self._summary_statistic(pps_rx) - self.logger.info("throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total)) + self.logger.info( + "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total)) return bps_rx_total, pps_rx_total + def __get_multi_throughput_statistic( + self, stream_ids, duration, interval, callback=None): + """ + duration: traffic duration (second) + interval: interval of get throughput statistics (second) + callback: a callback method of suite, which is used to do some actions + during traffic lasting. + + Return: a list of throughput instead of a single tuple of pps/bps rate + """ + time_elapsed = 0 + stats = [] + while time_elapsed < duration: + time.sleep(interval) + stats.append(self.__get_single_throughput_statistic(stream_ids)) + if callback and callable(callback): + callback() + time_elapsed += interval + return stats + + def measure_throughput(self, stream_ids=[], options={}): + """ + Measure throughput on each tx ports + + options usage: + rate: + port rate percent, float(0--100). Default value is 100. + + delay: + warm up time before start main traffic. If it is set, it will start + a delay time traffic to make sure packet generator under good status. + Warm up flow is ignored by default. + + interval: + a interval time of get throughput statistic (second) + If set this key value, pktgen will return several throughput statistic + data within a duration traffic. If not set this key value, only + return one statistic data. It is ignored by default. + + callback: + this key works with ``interval`` key. If it is set, the callback + of suite level will be executed after getting throughput statistic. + callback method should define as below, don't add sleep in this method. + + def callback(self): + xxxx() + + duration: + traffic lasting time(second). Default value is 10 second. + """ + interval = options.get('interval') + callback = options.get('callback') + duration = options.get('duration') or 10 + delay = options.get('delay') + self._prepare_transmission(stream_ids=stream_ids) + # start warm up traffic + self.__warm_up_pktgen(stream_ids, options, delay) + # main traffic + self._start_transmission(stream_ids) + # keep traffic within a duration time and get throughput statistic + if interval and duration: + stats = self.__get_multi_throughput_statistic( + stream_ids, duration, interval, callback) + else: + time.sleep(duration) + stats = self.__get_single_throughput_statistic(stream_ids) + self._stop_transmission(stream_ids) + return stats + def _measure_loss(self, stream_ids=[], options={}): """ Measure lost rate on each tx/rx ports """ + delay = options.get('delay') + duration = options.get('duration') or 10 self._prepare_transmission(stream_ids=stream_ids) + # start warm up traffic + self.__warm_up_pktgen(stream_ids, options, delay) + # main traffic self._start_transmission(stream_ids, options) + # keep traffic within a duration time + time.sleep(duration) self._stop_transmission(None) result = {} used_rx_port = [] @@ -210,6 +291,19 @@ class PacketGenerator(object): return result def measure_loss(self, stream_ids=[], options={}): + ''' + options usage: + rate: + port rate percent, float(0--100). Default value is 100. + + delay: + warm up time before start main traffic. If it is set, it will + start a delay time traffic to make sure packet generator + under good status. Warm up flow is ignored by default. + + duration: + traffic lasting time(second). Default value is 10 second. + ''' result = self._measure_loss(stream_ids, options) # here only to make sure that return value is the same as dts/etgen format # In real testing scenario, this method can offer more data than it @@ -218,9 +312,28 @@ class PacketGenerator(object): def measure_latency(self, stream_ids=[], options={}): """ Measure latency on each tx/rx ports + + options usage: + rate: + port rate percent, float(0--100). Default value is 100. + + delay: + warm up time before start main traffic. If it is set, it will + start a delay time transmission to make sure packet generator + under correct status. Warm up flow is ignored by default. + + duration: + traffic lasting time(second). Default value is 10 second. """ + delay = options.get('delay') + duration = options.get('duration') or 10 self._prepare_transmission(stream_ids=stream_ids, latency=True) + # start warm up traffic + self.__warm_up_pktgen(stream_ids, options, delay) + # main traffic self._start_transmission(stream_ids, options) + # keep traffic within a duration time + time.sleep(duration) self._stop_transmission(None) result = {} @@ -248,7 +361,26 @@ class PacketGenerator(object): return True def measure_rfc2544(self, stream_ids=[], options={}): - """ check loss rate with rate percent dropping """ + """ check loss rate with rate percent dropping + + options usage: + rate: + port rate percent at first round testing(0 ~ 100), default is 100. + + pdr: + permit packet drop rate, , default is 0. + + drop_step: + port rate percent drop step(0 ~ 100), default is 1. + + delay: + warm up time before start main traffic. If it is set, it will + start a delay time traffic to make sure packet generator + under good status. Warm up flow is ignored by default. + + duration: + traffic lasting time(second). Default value is 10 second. + """ loss_rate_table = [] rate_percent = options.get('rate') or float(100) permit_loss_rate = options.get('pdr') or 0 @@ -265,6 +397,9 @@ class PacketGenerator(object): tx_num, rx_num = result.values()[0][1:] return rate_percent, tx_num, rx_num _options = deepcopy(options) + # if warm up option 'delay' is set, ignore it in next work flow + if 'delay' in _options: + _options.pop('delay') if 'rate' in _options: _options.pop('rate') while not status and rate_percent > 0: @@ -328,6 +463,81 @@ class PacketGenerator(object): # here only pick one return loss_pps_table[-1][1].values()[0] + def measure_rfc2544_dichotomy(self, stream_ids=[], options={}): + """ check loss rate using dichotomy algorithm + + options usage: + delay: + warm up time before start main traffic. If it is set, it will + start a delay time traffic to make sure packet generator + under good status. Warm up flow is ignored by default. + + duration: + traffic lasting time(second). Default value is 10 second. + + min_rate: + lower bound rate percent , default is 0. + + max_rate: + upper bound rate percent , default is 100. + + pdr: + permit packet drop rate(<1.0), default is 0. + + accuracy : + dichotomy algorithm accuracy, default 0.001. + """ + max_rate = options.get('max_rate') or 100.0 + min_rate = options.get('min_rate') or 0.0 + accuracy = options.get('accuracy') or 0.001 + permit_loss_rate = options.get('pdr') or 0.0 + duration = options.get('duration') or 10.0 + # start warm up traffic + delay = options.get('delay') + _options = {'duration': duration} + self.__warm_up_pktgen(stream_ids, _options, delay) + # traffic parameters for dichotomy algorithm + loss_rate_table = [] + hit_result = None + rate = traffic_rate_max = max_rate + traffic_rate_min = min_rate + while True: + # run loss rate testing + _options = {'duration': duration} + result = self._measure_loss(stream_ids, _options) + loss_rate_table.append([rate, result]) + status = self._check_loss_rate(result, permit_loss_rate) + # if upper bound rate percent hit, quit the left flow + if rate == max_rate and status: + hit_result = result + break + # if lower bound rate percent not hit, quit the left flow + if rate == min_rate and not status: + break + if status: + traffic_rate_min = rate + hit_result = result + else: + traffic_rate_max = rate + if traffic_rate_max - traffic_rate_min < accuracy: + break + rate = (traffic_rate_max - traffic_rate_min)/2 + traffic_rate_min + self._clear_streams() + # set stream rate percent to custom value + self._set_stream_rate_percent(rate) + + if not hit_result: + msg = ('expected permit loss rate <{0}>' + 'not between rate {1} and rate {2}').format( + permit_loss_rate, max_rate, min_rate) + self.logger.error(msg) + self.logger.info(pformat(loss_rate_table)) + else: + self.logger.debug(pformat(loss_rate_table)) + self.logger.info("zero loss rate is %f" % rate) + + return hit_result + def measure(self, stream_ids, traffic_opt): ''' use as an unify interface method for packet generator @@ -343,6 +553,8 @@ class PacketGenerator(object): result = self.measure_rfc2544(stream_ids, traffic_opt) elif method == 'rfc2544_with_pps': result = self.measure_rfc2544_with_pps(stream_ids, traffic_opt) + elif method == 'rfc2544_dichotomy': + result = self.measure_rfc2544_dichotomy(stream_ids, traffic_opt) else: result = None -- 2.21.0
*. remove duration option used in ixia module, move duration option in testing scenario methods(latency/loss/throughput) in pktgen_base module. *. remove un-used parameters in _start_transmission method. *. add a warning message for rate percent input parameter. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- framework/pktgen_ixia.py | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/framework/pktgen_ixia.py b/framework/pktgen_ixia.py index 69f858e..14b5d5b 100644 --- a/framework/pktgen_ixia.py +++ b/framework/pktgen_ixia.py @@ -1247,7 +1247,6 @@ class Ixia(SSHConnection): ''' start ixia ports ''' self.configure_transmission(run_opt) self.start_transmission() - time.sleep(run_opt.get('duration') or 5) def remove_all_streams(self): ''' delete all streams on all ixia ports ''' @@ -1652,28 +1651,13 @@ class IxiaPacketGenerator(PacketGenerator): self._preset_ixia_port() def _start_transmission(self, stream_ids, options={}): - ''' - :param sample_delay: - After traffic start ``sample_delay`` seconds, start get runtime statistics - ''' # get rate percentage - rate_percent = options.get('rate') or '100' - # get duration - duration = options.get("duration") or 5 - duration = int(duration) if isinstance(duration, (str, unicode)) \ - else duration - # get sample interval - _sample_delay = options.get("sample_delay") or duration/2 - sample_delay = int(_sample_delay) \ - if isinstance(_sample_delay, (str, unicode)) \ - else _sample_delay - # get configuration from pktgen config file - warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \ - else 25 - wait_interval, core_mask = (warmup+30, self.conf["core_mask"]) \ - if self.conf.has_key("core_mask") \ - else (warmup+5, None) - #------------------------------------------------------------------- + rate_percent = options.get('rate') + if rate_percent: + msg = ('{0} only support set rate percent in streams, ' + 'current run traffic with stream rate percent').format( + self.pktgen_type) + self.logger.warning(msg) # run ixia server try: ########################################### @@ -1682,8 +1666,6 @@ class IxiaPacketGenerator(PacketGenerator): run_opt = { 'ports': self._traffic_ports, 'mult': rate_percent, - 'duration': duration, - 'core_mask':core_mask, 'force': True,} self._conn.start(**run_opt) except Exception as e: @@ -1693,6 +1675,7 @@ class IxiaPacketGenerator(PacketGenerator): # using ixia server command if self._traffic_ports: self._conn.stop_transmit() + self.logger.info("traffic completed. ") def _retrieve_port_statistic(self, stream_id, mode): ''' ixia traffic statistics ''' -- 2.21.0
*. remove duration option in trex module, move duration option in testing scenario methods(latency/loss/throughput) in pktgen_base module. *. remove sample_delay option in trex module. *. remove runtim_stat in trex module. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- framework/pktgen_trex.py | 63 ++++++++++++++-------------------------- 1 file changed, 22 insertions(+), 41 deletions(-) diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py index 690d070..6a1b36a 100644 --- a/framework/pktgen_trex.py +++ b/framework/pktgen_trex.py @@ -411,7 +411,6 @@ class TrexPacketGenerator(PacketGenerator): self._ports = [] self._traffic_ports = [] self._rx_ports = [] - self.runtime_stats = {} conf_inst = self._get_generator_conf_instance() self.conf = conf_inst.load_pktgen_config() @@ -450,6 +449,23 @@ class TrexPacketGenerator(PacketGenerator): from trex_stl_lib.api import STLClient # set trex class self.STLClient = STLClient + # get configuration from pktgen config file + self._get_traffic_option() + + def _get_traffic_option(self): + ''' get configuration from pktgen config file ''' + # set trex coremask + _core_mask = self.conf.get("core_mask") + if _core_mask: + if '0x' in _core_mask: + self.core_mask = \ + [int(item[2:], 16) for item in _core_mask.split(',')] + else: + self.core_mask = self.STLClient.CORE_MASK_PIN \ + if _core_mask.upper() == 'CORE_MASK_PIN' else \ + None + else: + self.core_mask = None def _connect(self): self._conn = self.STLClient(server=self.conf["server"]) @@ -652,7 +668,7 @@ class TrexPacketGenerator(PacketGenerator): def _throughput_stats(self, stream, stats): # tx packet tx_port_id = stream["tx_port"] - port_stats = self.runtime_stats.get(tx_port_id) + port_stats = stats.get(tx_port_id) if not port_stats: msg = "failed to get tx_port {0} statistics".format(tx_port_id) raise Exception(msg) @@ -666,7 +682,7 @@ class TrexPacketGenerator(PacketGenerator): self.logger.debug(os.linesep.join(msg)) # rx bps/pps rx_port_id = stream["rx_port"] - port_stats = self.runtime_stats.get(rx_port_id) + port_stats = stats.get(rx_port_id) if not port_stats: msg = "failed to get rx_port {0} statistics".format(rx_port_id) raise Exception(msg) @@ -768,65 +784,30 @@ class TrexPacketGenerator(PacketGenerator): self._preset_trex_port() def _start_transmission(self, stream_ids, options={}): - ''' - :param sample_delay: - After traffic start ``sample_delay`` seconds, start get runtime statistics - ''' + test_mode = options.get('method') # get rate percentage rate_percent = "{0}%".format(options.get('rate') or self._traffic_opt.get('rate') or '100') - # get duration - duration = options.get("duration") or 20 - duration = int(duration) if isinstance(duration, (str, unicode)) \ - else duration - # get sample interval - _sample_delay = options.get("sample_delay") or duration/2 - sample_delay = int(_sample_delay) \ - if isinstance(_sample_delay, (str, unicode)) \ - else _sample_delay - # get configuration from pktgen config file - warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \ - else 25 - # set trex coremask - wait_interval, core_mask = ( - warmup+30, int(self.conf["core_mask"], 16)) \ - if self.conf.has_key("core_mask") \ - else (warmup+5, 0x3) - try: - ########################################### # clear the stats before injecting self._conn.clear_stats() # Start traffic on port(s) run_opt = { 'ports': self._traffic_ports, 'mult': rate_percent, - 'duration': duration, - 'core_mask':core_mask, + 'core_mask': self.core_mask, 'force': True,} self.logger.info("begin traffic ......") self.logger.debug(run_opt) self._conn.start(**run_opt) - ########################################### - if sample_delay: - time.sleep(sample_delay) # wait - # get ports runtime statistics - self.runtime_stats = self._conn.get_stats() - self.logger.debug(pformat(self.runtime_stats)) - ########################################### - # Block until traffic on specified port(s) has ended - wait_opt = {'ports': self._traffic_ports} - if duration: - time.sleep(wait_interval + 10) - wait_opt['timeout'] = wait_interval + duration - self._conn.wait_on_traffic(**wait_opt) except Exception as e: self.logger.error(e) def _stop_transmission(self, stream_id): if self._traffic_ports: self._conn.stop(ports=self._traffic_ports, rx_delay_ms=5000) + self.logger.info("traffic completed. ") def _retrieve_port_statistic(self, stream_id, mode): ''' -- 2.21.0
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 0/5] dts/pktgen: add new features
>
> v3
> Add 2544 dichotomy algorithm as ma,lihong&han,yingya required.
> *. add measure_rfc2544_dichotomy method *. add rfc2544 dichotomy usage
> document
>
> v2
> Add new demand of pktgen as wang,yinan required.
> *. add a callback to deal with executing other tools query actions in throughput
> traffic lasting status.
> *. rebase source code.
>
> v1
> As chen zhaoyan new demand of pktgen, add these changes
>
> *. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to
> realize measure_throughput support return several throughput statistic data in a
> duration.
> *. add test method(latency/loss/throughput/rfc2544) options parameter usage
> comment in pktgen_base module.
> *. add test method(latency/loss/throughput) delay/duration options new usage
> definition and relevant process source code in pktgen_base module.
> *. set delay option to the warm up time before start main traffic.
> *. set pktgen duration default value to 10 second.
> *. remove duration option used in trex/ixia module, move duration option in testing
> scenario methods(latency/loss/throughput) in pktgen_base module.
> *. remove un-used optons in pktgen.cfg.
> *. remove un-used parameter in trex/ixia _start_transmission.
> *. rebase coremask new setting options from next branch.
>
> yufengmx (5):
> conf/pktgen: remove un-used optons
> doc/dts_gsg/pktgen_prog_guide: update description
> framework/pktgen_base: add new features
> framework/pktgen_ixia: add new features
> framework/pktgen_trex: add new features
>
> conf/pktgen.cfg | 18 ++-
> doc/dts_gsg/pktgen_prog_guide.rst | 86 +++++++++--
> framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++--
> framework/pktgen_ixia.py | 31 +---
> framework/pktgen_trex.py | 63 +++-----
> 5 files changed, 340 insertions(+), 102 deletions(-)
>
> --
> 2.21.0
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons
>
>
> *. remove core_num, it is not used by pktgen.
> *. remove warmup key, set it in suite script.
> *. remove duration key, set it in suite script.
> *. remove ip_src key, set it in suite script.
> *. remove ip_dst key, set it in suite script.
> *. add more core_mask setting option.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> conf/pktgen.cfg | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg index 667e1e8..e111459 100644
> --- a/conf/pktgen.cfg
> +++ b/conf/pktgen.cfg
> @@ -4,7 +4,14 @@
> # trex server binary file is under this directory.
> # trex_lib_path(optional): trex stateless client libs directory, it is optional.
> # If it is not set, use a default relative directory.
> -# coremask -c: A hexadecimal bitmask of cores to run on
> +# coremask:
> +# 1. a list of masks (one core mask per port), sort sequence as
> +# <port 0 core mask>,<port 1 core mask>
> +# example: core_mask=0x3,0x5
> +# 2. CORE_MASK_PIN
> +# for each dual ports (a group that shares the same cores), the cores will
> +# be divided half pinned for each port.
> +# example: core_mask=CORE_MASK_PIN
> # num -n: Number of memory channels
> # proc_type --proc-type: Type of this process # pci_blacklist --pci-blacklist, -b: Add a
> PCI device in black list.
> @@ -20,13 +27,8 @@
> trex_lib_path=/opt/trex/vx.xxx/automation/trex_control_plane/interactive
> config_file=/etc/trex_cfg.yaml
> server=10.67.111.143
> pcap_file=/opt/trex-core-2.26/scripts/stl/sample.pcap
> -core_num=4
> -#core_mask=0x3
> -ip_src=16.0.0.1
> -ip_dst=10.0.0.1
> -warmup=15
> -duration=-1
> -#start_trex=yes
> +core_mask=0x3,0x5
> +start_trex=yes
>
> # IXIA port Configuration
> # IxiaGroup: Group name for IXIA ports
> --
> 2.21.0
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description
>
>
> *. update test methods(latency/loss/throughput/rfc2544) input parameters
> description.
> *. add rfc2544 dichotomy usage document.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> doc/dts_gsg/pktgen_prog_guide.rst | 86 ++++++++++++++++++++++++++-----
> 1 file changed, 73 insertions(+), 13 deletions(-)
>
> diff --git a/doc/dts_gsg/pktgen_prog_guide.rst b/doc/dts_gsg/pktgen_prog_guide.rst
> index 32d2082..e0fdcf4 100644
> --- a/doc/dts_gsg/pktgen_prog_guide.rst
> +++ b/doc/dts_gsg/pktgen_prog_guide.rst
> @@ -145,7 +145,7 @@ define a stream transmit behavior.
>
> basic content including::
>
> - 'rate': 0 ~ 100 int type
> + 'rate': 0 ~ 100 int type, port line rate should set it.
> 'transmit_mode': TRANSMIT_CONT/TRANSMIT_S_BURST
> TRANSMIT_CONT define a continuous transmit.
> TRANSMIT_S_BURST define a burst transmit with custom number of packets.
> @@ -332,9 +332,22 @@ option
> traffic_option = {
> # test method name, if use `measure_throughput`, ignore this key
> 'method': 'throughput',
> - # port rate percent
> + # port rate percent, float(0--100), default value is 100.(reserved)
> 'rate': 100,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under good
> + # status. It is an optional key.
> + 'delay': 5,
> + # the interval time of get throughput statistic (second).
> + # If set this key value, pktgen will return several throughput statistic
> + # data in a duration. If not set this key value, only return one statistic
> + # data. It is used coupled with `duration` option.
> + 'interval': 1,
> + # this key works with ``interval`` key. If it is set, the callback
> + # of suite level will be executed after getting throughput statistic.
> + # callback method should define as below, don't add sleep in this method.
> + 'callback' : callback_method,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -358,9 +371,13 @@ option
> traffic_option = {
> # test method name, if use `measure_loss`, ignore this key
> 'method': 'loss',
> - # port rate percent
> + # port rate percent, float(0--100), default value is 100.(reserved)
> 'rate': 100,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under good
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -382,9 +399,13 @@ option
> traffic_option = {
> # test method name, if use `measure_latency`, ignore this key
> 'method': 'latency',
> - # port rate percent
> + # port rate percent, float(0--100), default value is 100.(reserved)
> 'rate': 100,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under ready
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -398,7 +419,7 @@ return value
>
> rfc2544 option
> --------------
> -single stream & rfc2544
> +rfc2544 testing scenario by decreasing step.
>
> option
> ++++++
> @@ -406,15 +427,19 @@ option
> .. code-block:: python
>
> traffic_option = {
> - # test method name, if use `measure_rfc2544`, ignore this key
> + # test method name, if use `measure_rfc2544`, ignore this key.
> 'method': 'rfc2544',
> - # port rate percent at first round testing, 0 ~ 100, default is 100
> + # port rate percent at first round testing, 0 ~ 100, default is 100.
> 'rate': 100,
> - # permit packet drop rate
> + # permit packet drop rate, default is 0.001.
> 'pdr': 0.001,
> - # port rate percent drop step, 0 ~ 100 , default is 1
> + # port rate percent drop step, 0 ~ 100 , default is 1.
> 'drop_step': 1,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under ready
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -424,6 +449,41 @@ return value
>
> loss_stats = (loss_rate, tx_pkts, rx_pkts)
>
> +rfc2544_dichotomy option
> +------------------------
> +rfc2544 testing scenario using dichotomy algorithm.
> +
> +option
> +++++++
> +
> +.. code-block:: python
> +
> + traffic_option = {
> + # test method name, if use `measure_rfc2544_dichotomy` method, ignore this
> key.
> + 'method': 'rfc2544_dichotomy',
> + # dichotomy algorithm lower bound rate percent, default is 0.
> + 'min_rate': 0,
> + # dichotomy algorithm upper bound rate percent, default is 100.
> + 'max_rate': 100,
> + # dichotomy algorithm accuracy, default 0.001.
> + 'accuracy': 0.001,
> + # permit packet drop rate, default is 0.001.
> + 'pdr': 0.001,
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under ready
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> + 'duration': 10}
> +
> +return value
> +++++++++++++
> +
> +.. code-block:: python
> +
> + loss_stats = (loss_rate, tx_pkts, rx_pkts)
> +
> +
> reference example
> =================
> This example show how to use pktgen in suite script. In fact, most scenario are
> --
> 2.21.0
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 3/5] framework/pktgen_base: add new features
>
>
> *. add test method(latency/loss/throughput) delay/duration options new usage
> definition and relevant process source code.
> *. set delay option to be the warm up time after start transmission.
> *. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to
> realize
> measure_throughput support return several throughput numbers in a duration.
> *. set duration option default value to 10 second.
> *. add test method(latency/loss/throughput/rfc2544) options parameter usage
> comment,
> which is the same as doc pktgen_prog_guide.rst.
> *. add a callback to deal with executing other tools query actions in throughput traffic
> lasting status.
> *. add measure_rfc2544_dichotomy method.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++++++++++---
> 1 file changed, 228 insertions(+), 16 deletions(-)
>
> diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py index
> 7855f6f..75c3036 100644
> --- a/framework/pktgen_base.py
> +++ b/framework/pktgen_base.py
> @@ -81,7 +81,7 @@ class PacketGenerator(object):
> return port_idx
> else:
> port = -1
> - except:
> + except Exception as e:
> port = -1
>
> return port
> @@ -100,7 +100,7 @@ class PacketGenerator(object):
> port = self._get_gen_port(tester_pci)
> msg = "test port {0} map gen port {1}".format(port_id, port)
> self.logger.debug(msg)
> - except:
> + except Exception as e:
> port = -1
>
> return port
> @@ -157,39 +157,120 @@ class PacketGenerator(object):
> def reset_streams(self):
> self.__streams = []
>
> - def measure_throughput(self, stream_ids=[], options={}):
> - """
> - Measure throughput on each tx ports
> - """
> - bps_rx = []
> - pps_rx = []
> - self._prepare_transmission(stream_ids=stream_ids)
> + def __warm_up_pktgen(self, stream_ids, options, delay):
> + ''' run warm up traffic before start main traffic '''
> + if not delay:
> + return
> + msg = '{1} packet generator: run traffic {0}s to warm up ... '.format(
> + delay, self.pktgen_type)
> + self.logger.info(msg)
> self._start_transmission(stream_ids, options)
> -
> - delay = options.get('delay') or 5
> time.sleep(delay)
> + self._stop_transmission(stream_ids)
> + self._clear_streams()
> +
> + def __get_single_throughput_statistic(self, stream_ids):
> + bps_rx = []
> + pps_rx = []
> used_rx_port = []
> + msg = 'begin get port statistic ...'
> + self.logger.info(msg)
> for stream_id in stream_ids:
> if self.__streams[stream_id]['rx_port'] not in used_rx_port:
> rxbps_rates, rxpps_rates = self._retrieve_port_statistic(
> - stream_id, 'throughput')
> + stream_id, 'throughput')
> used_rx_port.append(self.__streams[stream_id]['rx_port'])
> bps_rx.append(rxbps_rates)
> pps_rx.append(rxpps_rates)
> - self._stop_transmission(stream_id)
> -
> bps_rx_total = self._summary_statistic(bps_rx)
> pps_rx_total = self._summary_statistic(pps_rx)
> - self.logger.info("throughput: pps_rx %f, bps_rx %f" % (pps_rx_total,
> bps_rx_total))
> + self.logger.info(
> + "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total,
> + bps_rx_total))
>
> return bps_rx_total, pps_rx_total
>
> + def __get_multi_throughput_statistic(
> + self, stream_ids, duration, interval, callback=None):
> + """
> + duration: traffic duration (second)
> + interval: interval of get throughput statistics (second)
> + callback: a callback method of suite, which is used to do some actions
> + during traffic lasting.
> +
> + Return: a list of throughput instead of a single tuple of pps/bps rate
> + """
> + time_elapsed = 0
> + stats = []
> + while time_elapsed < duration:
> + time.sleep(interval)
> + stats.append(self.__get_single_throughput_statistic(stream_ids))
> + if callback and callable(callback):
> + callback()
> + time_elapsed += interval
> + return stats
> +
> + def measure_throughput(self, stream_ids=[], options={}):
> + """
> + Measure throughput on each tx ports
> +
> + options usage:
> + rate:
> + port rate percent, float(0--100). Default value is 100.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will start
> + a delay time traffic to make sure packet generator under good status.
> + Warm up flow is ignored by default.
> +
> + interval:
> + a interval time of get throughput statistic (second)
> + If set this key value, pktgen will return several throughput statistic
> + data within a duration traffic. If not set this key value, only
> + return one statistic data. It is ignored by default.
> +
> + callback:
> + this key works with ``interval`` key. If it is set, the callback
> + of suite level will be executed after getting throughput statistic.
> + callback method should define as below, don't add sleep in this method.
> +
> + def callback(self):
> + xxxx()
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> + """
> + interval = options.get('interval')
> + callback = options.get('callback')
> + duration = options.get('duration') or 10
> + delay = options.get('delay')
> + self._prepare_transmission(stream_ids=stream_ids)
> + # start warm up traffic
> + self.__warm_up_pktgen(stream_ids, options, delay)
> + # main traffic
> + self._start_transmission(stream_ids)
> + # keep traffic within a duration time and get throughput statistic
> + if interval and duration:
> + stats = self.__get_multi_throughput_statistic(
> + stream_ids, duration, interval, callback)
> + else:
> + time.sleep(duration)
> + stats = self.__get_single_throughput_statistic(stream_ids)
> + self._stop_transmission(stream_ids)
> + return stats
> +
> def _measure_loss(self, stream_ids=[], options={}):
> """
> Measure lost rate on each tx/rx ports
> """
> + delay = options.get('delay')
> + duration = options.get('duration') or 10
> self._prepare_transmission(stream_ids=stream_ids)
> + # start warm up traffic
> + self.__warm_up_pktgen(stream_ids, options, delay)
> + # main traffic
> self._start_transmission(stream_ids, options)
> + # keep traffic within a duration time
> + time.sleep(duration)
> self._stop_transmission(None)
> result = {}
> used_rx_port = []
> @@ -210,6 +291,19 @@ class PacketGenerator(object):
> return result
>
> def measure_loss(self, stream_ids=[], options={}):
> + '''
> + options usage:
> + rate:
> + port rate percent, float(0--100). Default value is 100.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time traffic to make sure packet generator
> + under good status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> + '''
> result = self._measure_loss(stream_ids, options)
> # here only to make sure that return value is the same as dts/etgen format
> # In real testing scenario, this method can offer more data than it @@ -218,9
> +312,28 @@ class PacketGenerator(object):
> def measure_latency(self, stream_ids=[], options={}):
> """
> Measure latency on each tx/rx ports
> +
> + options usage:
> + rate:
> + port rate percent, float(0--100). Default value is 100.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time transmission to make sure packet generator
> + under correct status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> """
> + delay = options.get('delay')
> + duration = options.get('duration') or 10
> self._prepare_transmission(stream_ids=stream_ids, latency=True)
> + # start warm up traffic
> + self.__warm_up_pktgen(stream_ids, options, delay)
> + # main traffic
> self._start_transmission(stream_ids, options)
> + # keep traffic within a duration time
> + time.sleep(duration)
> self._stop_transmission(None)
>
> result = {}
> @@ -248,7 +361,26 @@ class PacketGenerator(object):
> return True
>
> def measure_rfc2544(self, stream_ids=[], options={}):
> - """ check loss rate with rate percent dropping """
> + """ check loss rate with rate percent dropping
> +
> + options usage:
> + rate:
> + port rate percent at first round testing(0 ~ 100), default is 100.
> +
> + pdr:
> + permit packet drop rate, , default is 0.
> +
> + drop_step:
> + port rate percent drop step(0 ~ 100), default is 1.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time traffic to make sure packet generator
> + under good status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> + """
> loss_rate_table = []
> rate_percent = options.get('rate') or float(100)
> permit_loss_rate = options.get('pdr') or 0 @@ -265,6 +397,9 @@ class
> PacketGenerator(object):
> tx_num, rx_num = result.values()[0][1:]
> return rate_percent, tx_num, rx_num
> _options = deepcopy(options)
> + # if warm up option 'delay' is set, ignore it in next work flow
> + if 'delay' in _options:
> + _options.pop('delay')
> if 'rate' in _options:
> _options.pop('rate')
> while not status and rate_percent > 0:
> @@ -328,6 +463,81 @@ class PacketGenerator(object):
> # here only pick one
> return loss_pps_table[-1][1].values()[0]
>
> + def measure_rfc2544_dichotomy(self, stream_ids=[], options={}):
> + """ check loss rate using dichotomy algorithm
> +
> + options usage:
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time traffic to make sure packet generator
> + under good status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> +
> + min_rate:
> + lower bound rate percent , default is 0.
> +
> + max_rate:
> + upper bound rate percent , default is 100.
> +
> + pdr:
> + permit packet drop rate(<1.0), default is 0.
> +
> + accuracy :
> + dichotomy algorithm accuracy, default 0.001.
> + """
> + max_rate = options.get('max_rate') or 100.0
> + min_rate = options.get('min_rate') or 0.0
> + accuracy = options.get('accuracy') or 0.001
> + permit_loss_rate = options.get('pdr') or 0.0
> + duration = options.get('duration') or 10.0
> + # start warm up traffic
> + delay = options.get('delay')
> + _options = {'duration': duration}
> + self.__warm_up_pktgen(stream_ids, _options, delay)
> + # traffic parameters for dichotomy algorithm
> + loss_rate_table = []
> + hit_result = None
> + rate = traffic_rate_max = max_rate
> + traffic_rate_min = min_rate
> + while True:
> + # run loss rate testing
> + _options = {'duration': duration}
> + result = self._measure_loss(stream_ids, _options)
> + loss_rate_table.append([rate, result])
> + status = self._check_loss_rate(result, permit_loss_rate)
> + # if upper bound rate percent hit, quit the left flow
> + if rate == max_rate and status:
> + hit_result = result
> + break
> + # if lower bound rate percent not hit, quit the left flow
> + if rate == min_rate and not status:
> + break
> + if status:
> + traffic_rate_min = rate
> + hit_result = result
> + else:
> + traffic_rate_max = rate
> + if traffic_rate_max - traffic_rate_min < accuracy:
> + break
> + rate = (traffic_rate_max - traffic_rate_min)/2 + traffic_rate_min
> + self._clear_streams()
> + # set stream rate percent to custom value
> + self._set_stream_rate_percent(rate)
> +
> + if not hit_result:
> + msg = ('expected permit loss rate <{0}>'
> + 'not between rate {1} and rate {2}').format(
> + permit_loss_rate, max_rate, min_rate)
> + self.logger.error(msg)
> + self.logger.info(pformat(loss_rate_table))
> + else:
> + self.logger.debug(pformat(loss_rate_table))
> + self.logger.info("zero loss rate is %f" % rate)
> +
> + return hit_result
> +
> def measure(self, stream_ids, traffic_opt):
> '''
> use as an unify interface method for packet generator @@ -343,6 +553,8 @@
> class PacketGenerator(object):
> result = self.measure_rfc2544(stream_ids, traffic_opt)
> elif method == 'rfc2544_with_pps':
> result = self.measure_rfc2544_with_pps(stream_ids, traffic_opt)
> + elif method == 'rfc2544_dichotomy':
> + result = self.measure_rfc2544_dichotomy(stream_ids,
> + traffic_opt)
> else:
> result = None
>
> --
> 2.21.0
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 5/5] framework/pktgen_trex: add new features
>
>
> *. remove duration option in trex module, move duration option in
> testing scenario methods(latency/loss/throughput) in pktgen_base module.
> *. remove sample_delay option in trex module.
> *. remove runtim_stat in trex module.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> framework/pktgen_trex.py | 63 ++++++++++++++--------------------------
> 1 file changed, 22 insertions(+), 41 deletions(-)
>
> diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py index
> 690d070..6a1b36a 100644
> --- a/framework/pktgen_trex.py
> +++ b/framework/pktgen_trex.py
> @@ -411,7 +411,6 @@ class TrexPacketGenerator(PacketGenerator):
> self._ports = []
> self._traffic_ports = []
> self._rx_ports = []
> - self.runtime_stats = {}
>
> conf_inst = self._get_generator_conf_instance()
> self.conf = conf_inst.load_pktgen_config() @@ -450,6 +449,23 @@ class
> TrexPacketGenerator(PacketGenerator):
> from trex_stl_lib.api import STLClient
> # set trex class
> self.STLClient = STLClient
> + # get configuration from pktgen config file
> + self._get_traffic_option()
> +
> + def _get_traffic_option(self):
> + ''' get configuration from pktgen config file '''
> + # set trex coremask
> + _core_mask = self.conf.get("core_mask")
> + if _core_mask:
> + if '0x' in _core_mask:
> + self.core_mask = \
> + [int(item[2:], 16) for item in _core_mask.split(',')]
> + else:
> + self.core_mask = self.STLClient.CORE_MASK_PIN \
> + if _core_mask.upper() == 'CORE_MASK_PIN' else \
> + None
> + else:
> + self.core_mask = None
>
> def _connect(self):
> self._conn = self.STLClient(server=self.conf["server"])
> @@ -652,7 +668,7 @@ class TrexPacketGenerator(PacketGenerator):
> def _throughput_stats(self, stream, stats):
> # tx packet
> tx_port_id = stream["tx_port"]
> - port_stats = self.runtime_stats.get(tx_port_id)
> + port_stats = stats.get(tx_port_id)
> if not port_stats:
> msg = "failed to get tx_port {0} statistics".format(tx_port_id)
> raise Exception(msg)
> @@ -666,7 +682,7 @@ class TrexPacketGenerator(PacketGenerator):
> self.logger.debug(os.linesep.join(msg))
> # rx bps/pps
> rx_port_id = stream["rx_port"]
> - port_stats = self.runtime_stats.get(rx_port_id)
> + port_stats = stats.get(rx_port_id)
> if not port_stats:
> msg = "failed to get rx_port {0} statistics".format(rx_port_id)
> raise Exception(msg)
> @@ -768,65 +784,30 @@ class TrexPacketGenerator(PacketGenerator):
> self._preset_trex_port()
>
> def _start_transmission(self, stream_ids, options={}):
> - '''
> - :param sample_delay:
> - After traffic start ``sample_delay`` seconds, start get runtime statistics
> - '''
> + test_mode = options.get('method')
> # get rate percentage
> rate_percent = "{0}%".format(options.get('rate') or
> self._traffic_opt.get('rate') or
> '100')
> - # get duration
> - duration = options.get("duration") or 20
> - duration = int(duration) if isinstance(duration, (str, unicode)) \
> - else duration
> - # get sample interval
> - _sample_delay = options.get("sample_delay") or duration/2
> - sample_delay = int(_sample_delay) \
> - if isinstance(_sample_delay, (str, unicode)) \
> - else _sample_delay
> - # get configuration from pktgen config file
> - warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \
> - else 25
> - # set trex coremask
> - wait_interval, core_mask = (
> - warmup+30, int(self.conf["core_mask"], 16)) \
> - if self.conf.has_key("core_mask") \
> - else (warmup+5, 0x3)
> -
> try:
> - ###########################################
> # clear the stats before injecting
> self._conn.clear_stats()
> # Start traffic on port(s)
> run_opt = {
> 'ports': self._traffic_ports,
> 'mult': rate_percent,
> - 'duration': duration,
> - 'core_mask':core_mask,
> + 'core_mask': self.core_mask,
> 'force': True,}
> self.logger.info("begin traffic ......")
> self.logger.debug(run_opt)
> self._conn.start(**run_opt)
> - ###########################################
> - if sample_delay:
> - time.sleep(sample_delay) # wait
> - # get ports runtime statistics
> - self.runtime_stats = self._conn.get_stats()
> - self.logger.debug(pformat(self.runtime_stats))
> - ###########################################
> - # Block until traffic on specified port(s) has ended
> - wait_opt = {'ports': self._traffic_ports}
> - if duration:
> - time.sleep(wait_interval + 10)
> - wait_opt['timeout'] = wait_interval + duration
> - self._conn.wait_on_traffic(**wait_opt)
> except Exception as e:
> self.logger.error(e)
>
> def _stop_transmission(self, stream_id):
> if self._traffic_ports:
> self._conn.stop(ports=self._traffic_ports, rx_delay_ms=5000)
> + self.logger.info("traffic completed. ")
>
> def _retrieve_port_statistic(self, stream_id, mode):
> '''
> --
> 2.21.0
Applied the series, thanks
> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 0/5] dts/pktgen: add new features
>
> v3
> Add 2544 dichotomy algorithm as ma,lihong&han,yingya required.
> *. add measure_rfc2544_dichotomy method *. add rfc2544 dichotomy
> usage document
>
> v2
> Add new demand of pktgen as wang,yinan required.
> *. add a callback to deal with executing other tools query actions in
> throughput traffic lasting status.
> *. rebase source code.
>
> v1
> As chen zhaoyan new demand of pktgen, add these changes
>
> *. add __get_single_throughput_statistic/__get_multi_throughput_statistic
> methods to realize measure_throughput support return several throughput
> statistic data in a duration.
> *. add test method(latency/loss/throughput/rfc2544) options parameter
> usage comment in pktgen_base module.
> *. add test method(latency/loss/throughput) delay/duration options new
> usage definition and relevant process source code in pktgen_base module.
> *. set delay option to the warm up time before start main traffic.
> *. set pktgen duration default value to 10 second.
> *. remove duration option used in trex/ixia module, move duration option
> in testing scenario methods(latency/loss/throughput) in pktgen_base
> module.
> *. remove un-used optons in pktgen.cfg.
> *. remove un-used parameter in trex/ixia _start_transmission.
> *. rebase coremask new setting options from next branch.
>
> yufengmx (5):
> conf/pktgen: remove un-used optons
> doc/dts_gsg/pktgen_prog_guide: update description
> framework/pktgen_base: add new features
> framework/pktgen_ixia: add new features
> framework/pktgen_trex: add new features
>
> conf/pktgen.cfg | 18 ++-
> doc/dts_gsg/pktgen_prog_guide.rst | 86 +++++++++--
> framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++--
> framework/pktgen_ixia.py | 31 +---
> framework/pktgen_trex.py | 63 +++-----
> 5 files changed, 340 insertions(+), 102 deletions(-)
>
> --
> 2.21.0