* [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
@ 2019-11-29 9:03 ` yufengmx
2020-01-02 3:29 ` Chen, Zhaoyan
2019-11-29 9:03 ` [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description yufengmx
` (5 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: yufengmx @ 2019-11-29 9:03 UTC (permalink / raw)
To: dts, yinan.wang, lihongx.ma; +Cc: yufengmx
*. remove core_num, it is not used by pktgen.
*. remove warmup key, set it in suite script.
*. remove duration key, set it in suite script.
*. remove ip_src key, set it in suite script.
*. remove ip_dst key, set it in suite script.
*. add more core_mask setting option.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
conf/pktgen.cfg | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg
index 667e1e8..e111459 100644
--- a/conf/pktgen.cfg
+++ b/conf/pktgen.cfg
@@ -4,7 +4,14 @@
# trex server binary file is under this directory.
# trex_lib_path(optional): trex stateless client libs directory, it is optional.
# If it is not set, use a default relative directory.
-# coremask -c: A hexadecimal bitmask of cores to run on
+# coremask:
+# 1. a list of masks (one core mask per port), sort sequence as
+# <port 0 core mask>,<port 1 core mask>
+# example: core_mask=0x3,0x5
+# 2. CORE_MASK_PIN
+# for each dual ports (a group that shares the same cores), the cores will
+# be divided half pinned for each port.
+# example: core_mask=CORE_MASK_PIN
# num -n: Number of memory channels
# proc_type --proc-type: Type of this process
# pci_blacklist --pci-blacklist, -b: Add a PCI device in black list.
@@ -20,13 +27,8 @@ trex_lib_path=/opt/trex/vx.xxx/automation/trex_control_plane/interactive
config_file=/etc/trex_cfg.yaml
server=10.67.111.143
pcap_file=/opt/trex-core-2.26/scripts/stl/sample.pcap
-core_num=4
-#core_mask=0x3
-ip_src=16.0.0.1
-ip_dst=10.0.0.1
-warmup=15
-duration=-1
-#start_trex=yes
+core_mask=0x3,0x5
+start_trex=yes
# IXIA port Configuration
# IxiaGroup: Group name for IXIA ports
--
2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons
2019-11-29 9:03 ` [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons yufengmx
@ 2020-01-02 3:29 ` Chen, Zhaoyan
0 siblings, 0 replies; 12+ messages in thread
From: Chen, Zhaoyan @ 2020-01-02 3:29 UTC (permalink / raw)
To: Mo, YufengX, dts, Wang, Yinan, Ma, LihongX; +Cc: Mo, YufengX, Chen, Zhaoyan
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons
>
>
> *. remove core_num, it is not used by pktgen.
> *. remove warmup key, set it in suite script.
> *. remove duration key, set it in suite script.
> *. remove ip_src key, set it in suite script.
> *. remove ip_dst key, set it in suite script.
> *. add more core_mask setting option.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> conf/pktgen.cfg | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg index 667e1e8..e111459 100644
> --- a/conf/pktgen.cfg
> +++ b/conf/pktgen.cfg
> @@ -4,7 +4,14 @@
> # trex server binary file is under this directory.
> # trex_lib_path(optional): trex stateless client libs directory, it is optional.
> # If it is not set, use a default relative directory.
> -# coremask -c: A hexadecimal bitmask of cores to run on
> +# coremask:
> +# 1. a list of masks (one core mask per port), sort sequence as
> +# <port 0 core mask>,<port 1 core mask>
> +# example: core_mask=0x3,0x5
> +# 2. CORE_MASK_PIN
> +# for each dual ports (a group that shares the same cores), the cores will
> +# be divided half pinned for each port.
> +# example: core_mask=CORE_MASK_PIN
> # num -n: Number of memory channels
> # proc_type --proc-type: Type of this process # pci_blacklist --pci-blacklist, -b: Add a
> PCI device in black list.
> @@ -20,13 +27,8 @@
> trex_lib_path=/opt/trex/vx.xxx/automation/trex_control_plane/interactive
> config_file=/etc/trex_cfg.yaml
> server=10.67.111.143
> pcap_file=/opt/trex-core-2.26/scripts/stl/sample.pcap
> -core_num=4
> -#core_mask=0x3
> -ip_src=16.0.0.1
> -ip_dst=10.0.0.1
> -warmup=15
> -duration=-1
> -#start_trex=yes
> +core_mask=0x3,0x5
> +start_trex=yes
>
> # IXIA port Configuration
> # IxiaGroup: Group name for IXIA ports
> --
> 2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
2019-11-29 9:03 ` [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons yufengmx
@ 2019-11-29 9:03 ` yufengmx
2020-01-02 3:29 ` Chen, Zhaoyan
2019-11-29 9:03 ` [dts] [PATCH V3 3/5] framework/pktgen_base: add new features yufengmx
` (4 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: yufengmx @ 2019-11-29 9:03 UTC (permalink / raw)
To: dts, yinan.wang, lihongx.ma; +Cc: yufengmx
*. update test methods(latency/loss/throughput/rfc2544) input parameters description.
*. add rfc2544 dichotomy usage document.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
doc/dts_gsg/pktgen_prog_guide.rst | 86 ++++++++++++++++++++++++++-----
1 file changed, 73 insertions(+), 13 deletions(-)
diff --git a/doc/dts_gsg/pktgen_prog_guide.rst b/doc/dts_gsg/pktgen_prog_guide.rst
index 32d2082..e0fdcf4 100644
--- a/doc/dts_gsg/pktgen_prog_guide.rst
+++ b/doc/dts_gsg/pktgen_prog_guide.rst
@@ -145,7 +145,7 @@ define a stream transmit behavior.
basic content including::
- 'rate': 0 ~ 100 int type
+ 'rate': 0 ~ 100 int type, port line rate should set it.
'transmit_mode': TRANSMIT_CONT/TRANSMIT_S_BURST
TRANSMIT_CONT define a continuous transmit.
TRANSMIT_S_BURST define a burst transmit with custom number of packets.
@@ -332,9 +332,22 @@ option
traffic_option = {
# test method name, if use `measure_throughput`, ignore this key
'method': 'throughput',
- # port rate percent
+ # port rate percent, float(0--100), default value is 100.(reserved)
'rate': 100,
- # transmit lasting time second
+ # warm up time before start main transmission. If it is set, it will start
+ # a custom time transmission to make sure packet generator under good
+ # status. It is an optional key.
+ 'delay': 5,
+ # the interval time of get throughput statistic (second).
+ # If set this key value, pktgen will return several throughput statistic
+ # data in a duration. If not set this key value, only return one statistic
+ # data. It is used coupled with `duration` option.
+ 'interval': 1,
+ # this key works with ``interval`` key. If it is set, the callback
+ # of suite level will be executed after getting throughput statistic.
+ # callback method should define as below, don't add sleep in this method.
+ 'callback' : callback_method,
+ # transmission lasting time(second), default value is 10 second.
'duration': 5}
return value
@@ -358,9 +371,13 @@ option
traffic_option = {
# test method name, if use `measure_loss`, ignore this key
'method': 'loss',
- # port rate percent
+ # port rate percent, float(0--100), default value is 100.(reserved)
'rate': 100,
- # transmit lasting time second
+ # warm up time before start main transmission. If it is set, it will start
+ # a custom time transmission to make sure packet generator under good
+ # status. It is an optional key.
+ 'delay': 5,
+ # transmission lasting time(second), default value is 10 second.
'duration': 5}
return value
@@ -382,9 +399,13 @@ option
traffic_option = {
# test method name, if use `measure_latency`, ignore this key
'method': 'latency',
- # port rate percent
+ # port rate percent, float(0--100), default value is 100.(reserved)
'rate': 100,
- # transmit lasting time second
+ # warm up time before start main transmission. If it is set, it will start
+ # a custom time transmission to make sure packet generator under ready
+ # status. It is an optional key.
+ 'delay': 5,
+ # transmission lasting time(second), default value is 10 second.
'duration': 5}
return value
@@ -398,7 +419,7 @@ return value
rfc2544 option
--------------
-single stream & rfc2544
+rfc2544 testing scenario by decreasing step.
option
++++++
@@ -406,15 +427,19 @@ option
.. code-block:: python
traffic_option = {
- # test method name, if use `measure_rfc2544`, ignore this key
+ # test method name, if use `measure_rfc2544`, ignore this key.
'method': 'rfc2544',
- # port rate percent at first round testing, 0 ~ 100, default is 100
+ # port rate percent at first round testing, 0 ~ 100, default is 100.
'rate': 100,
- # permit packet drop rate
+ # permit packet drop rate, default is 0.001.
'pdr': 0.001,
- # port rate percent drop step, 0 ~ 100 , default is 1
+ # port rate percent drop step, 0 ~ 100 , default is 1.
'drop_step': 1,
- # transmit lasting time second
+ # warm up time before start main transmission. If it is set, it will start
+ # a custom time transmission to make sure packet generator under ready
+ # status. It is an optional key.
+ 'delay': 5,
+ # transmission lasting time(second), default value is 10 second.
'duration': 5}
return value
@@ -424,6 +449,41 @@ return value
loss_stats = (loss_rate, tx_pkts, rx_pkts)
+rfc2544_dichotomy option
+------------------------
+rfc2544 testing scenario using dichotomy algorithm.
+
+option
+++++++
+
+.. code-block:: python
+
+ traffic_option = {
+ # test method name, if use `measure_rfc2544_dichotomy` method, ignore this key.
+ 'method': 'rfc2544_dichotomy',
+ # dichotomy algorithm lower bound rate percent, default is 0.
+ 'min_rate': 0,
+ # dichotomy algorithm upper bound rate percent, default is 100.
+ 'max_rate': 100,
+ # dichotomy algorithm accuracy, default 0.001.
+ 'accuracy': 0.001,
+ # permit packet drop rate, default is 0.001.
+ 'pdr': 0.001,
+ # warm up time before start main transmission. If it is set, it will start
+ # a custom time transmission to make sure packet generator under ready
+ # status. It is an optional key.
+ 'delay': 5,
+ # transmission lasting time(second), default value is 10 second.
+ 'duration': 10}
+
+return value
+++++++++++++
+
+.. code-block:: python
+
+ loss_stats = (loss_rate, tx_pkts, rx_pkts)
+
+
reference example
=================
This example show how to use pktgen in suite script. In fact, most scenario are
--
2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description
2019-11-29 9:03 ` [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description yufengmx
@ 2020-01-02 3:29 ` Chen, Zhaoyan
0 siblings, 0 replies; 12+ messages in thread
From: Chen, Zhaoyan @ 2020-01-02 3:29 UTC (permalink / raw)
To: Mo, YufengX, dts, Wang, Yinan, Ma, LihongX; +Cc: Mo, YufengX, Chen, Zhaoyan
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description
>
>
> *. update test methods(latency/loss/throughput/rfc2544) input parameters
> description.
> *. add rfc2544 dichotomy usage document.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> doc/dts_gsg/pktgen_prog_guide.rst | 86 ++++++++++++++++++++++++++-----
> 1 file changed, 73 insertions(+), 13 deletions(-)
>
> diff --git a/doc/dts_gsg/pktgen_prog_guide.rst b/doc/dts_gsg/pktgen_prog_guide.rst
> index 32d2082..e0fdcf4 100644
> --- a/doc/dts_gsg/pktgen_prog_guide.rst
> +++ b/doc/dts_gsg/pktgen_prog_guide.rst
> @@ -145,7 +145,7 @@ define a stream transmit behavior.
>
> basic content including::
>
> - 'rate': 0 ~ 100 int type
> + 'rate': 0 ~ 100 int type, port line rate should set it.
> 'transmit_mode': TRANSMIT_CONT/TRANSMIT_S_BURST
> TRANSMIT_CONT define a continuous transmit.
> TRANSMIT_S_BURST define a burst transmit with custom number of packets.
> @@ -332,9 +332,22 @@ option
> traffic_option = {
> # test method name, if use `measure_throughput`, ignore this key
> 'method': 'throughput',
> - # port rate percent
> + # port rate percent, float(0--100), default value is 100.(reserved)
> 'rate': 100,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under good
> + # status. It is an optional key.
> + 'delay': 5,
> + # the interval time of get throughput statistic (second).
> + # If set this key value, pktgen will return several throughput statistic
> + # data in a duration. If not set this key value, only return one statistic
> + # data. It is used coupled with `duration` option.
> + 'interval': 1,
> + # this key works with ``interval`` key. If it is set, the callback
> + # of suite level will be executed after getting throughput statistic.
> + # callback method should define as below, don't add sleep in this method.
> + 'callback' : callback_method,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -358,9 +371,13 @@ option
> traffic_option = {
> # test method name, if use `measure_loss`, ignore this key
> 'method': 'loss',
> - # port rate percent
> + # port rate percent, float(0--100), default value is 100.(reserved)
> 'rate': 100,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under good
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -382,9 +399,13 @@ option
> traffic_option = {
> # test method name, if use `measure_latency`, ignore this key
> 'method': 'latency',
> - # port rate percent
> + # port rate percent, float(0--100), default value is 100.(reserved)
> 'rate': 100,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under ready
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -398,7 +419,7 @@ return value
>
> rfc2544 option
> --------------
> -single stream & rfc2544
> +rfc2544 testing scenario by decreasing step.
>
> option
> ++++++
> @@ -406,15 +427,19 @@ option
> .. code-block:: python
>
> traffic_option = {
> - # test method name, if use `measure_rfc2544`, ignore this key
> + # test method name, if use `measure_rfc2544`, ignore this key.
> 'method': 'rfc2544',
> - # port rate percent at first round testing, 0 ~ 100, default is 100
> + # port rate percent at first round testing, 0 ~ 100, default is 100.
> 'rate': 100,
> - # permit packet drop rate
> + # permit packet drop rate, default is 0.001.
> 'pdr': 0.001,
> - # port rate percent drop step, 0 ~ 100 , default is 1
> + # port rate percent drop step, 0 ~ 100 , default is 1.
> 'drop_step': 1,
> - # transmit lasting time second
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under ready
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> 'duration': 5}
>
> return value
> @@ -424,6 +449,41 @@ return value
>
> loss_stats = (loss_rate, tx_pkts, rx_pkts)
>
> +rfc2544_dichotomy option
> +------------------------
> +rfc2544 testing scenario using dichotomy algorithm.
> +
> +option
> +++++++
> +
> +.. code-block:: python
> +
> + traffic_option = {
> + # test method name, if use `measure_rfc2544_dichotomy` method, ignore this
> key.
> + 'method': 'rfc2544_dichotomy',
> + # dichotomy algorithm lower bound rate percent, default is 0.
> + 'min_rate': 0,
> + # dichotomy algorithm upper bound rate percent, default is 100.
> + 'max_rate': 100,
> + # dichotomy algorithm accuracy, default 0.001.
> + 'accuracy': 0.001,
> + # permit packet drop rate, default is 0.001.
> + 'pdr': 0.001,
> + # warm up time before start main transmission. If it is set, it will start
> + # a custom time transmission to make sure packet generator under ready
> + # status. It is an optional key.
> + 'delay': 5,
> + # transmission lasting time(second), default value is 10 second.
> + 'duration': 10}
> +
> +return value
> +++++++++++++
> +
> +.. code-block:: python
> +
> + loss_stats = (loss_rate, tx_pkts, rx_pkts)
> +
> +
> reference example
> =================
> This example show how to use pktgen in suite script. In fact, most scenario are
> --
> 2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [dts] [PATCH V3 3/5] framework/pktgen_base: add new features
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
2019-11-29 9:03 ` [dts] [PATCH V3 1/5] conf/pktgen: remove un-used optons yufengmx
2019-11-29 9:03 ` [dts] [PATCH V3 2/5] doc/dts_gsg/pktgen_prog_guide: update description yufengmx
@ 2019-11-29 9:03 ` yufengmx
2020-01-02 3:29 ` Chen, Zhaoyan
2019-11-29 9:03 ` [dts] [PATCH V3 4/5] framework/pktgen_ixia: " yufengmx
` (3 subsequent siblings)
6 siblings, 1 reply; 12+ messages in thread
From: yufengmx @ 2019-11-29 9:03 UTC (permalink / raw)
To: dts, yinan.wang, lihongx.ma; +Cc: yufengmx
*. add test method(latency/loss/throughput) delay/duration options new usage
definition and relevant process source code.
*. set delay option to be the warm up time after start transmission.
*. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to realize
measure_throughput support return several throughput numbers in a duration.
*. set duration option default value to 10 second.
*. add test method(latency/loss/throughput/rfc2544) options parameter usage comment,
which is the same as doc pktgen_prog_guide.rst.
*. add a callback to deal with executing other tools query actions in throughput traffic lasting status.
*. add measure_rfc2544_dichotomy method.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++++++++++---
1 file changed, 228 insertions(+), 16 deletions(-)
diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py
index 7855f6f..75c3036 100644
--- a/framework/pktgen_base.py
+++ b/framework/pktgen_base.py
@@ -81,7 +81,7 @@ class PacketGenerator(object):
return port_idx
else:
port = -1
- except:
+ except Exception as e:
port = -1
return port
@@ -100,7 +100,7 @@ class PacketGenerator(object):
port = self._get_gen_port(tester_pci)
msg = "test port {0} map gen port {1}".format(port_id, port)
self.logger.debug(msg)
- except:
+ except Exception as e:
port = -1
return port
@@ -157,39 +157,120 @@ class PacketGenerator(object):
def reset_streams(self):
self.__streams = []
- def measure_throughput(self, stream_ids=[], options={}):
- """
- Measure throughput on each tx ports
- """
- bps_rx = []
- pps_rx = []
- self._prepare_transmission(stream_ids=stream_ids)
+ def __warm_up_pktgen(self, stream_ids, options, delay):
+ ''' run warm up traffic before start main traffic '''
+ if not delay:
+ return
+ msg = '{1} packet generator: run traffic {0}s to warm up ... '.format(
+ delay, self.pktgen_type)
+ self.logger.info(msg)
self._start_transmission(stream_ids, options)
-
- delay = options.get('delay') or 5
time.sleep(delay)
+ self._stop_transmission(stream_ids)
+ self._clear_streams()
+
+ def __get_single_throughput_statistic(self, stream_ids):
+ bps_rx = []
+ pps_rx = []
used_rx_port = []
+ msg = 'begin get port statistic ...'
+ self.logger.info(msg)
for stream_id in stream_ids:
if self.__streams[stream_id]['rx_port'] not in used_rx_port:
rxbps_rates, rxpps_rates = self._retrieve_port_statistic(
- stream_id, 'throughput')
+ stream_id, 'throughput')
used_rx_port.append(self.__streams[stream_id]['rx_port'])
bps_rx.append(rxbps_rates)
pps_rx.append(rxpps_rates)
- self._stop_transmission(stream_id)
-
bps_rx_total = self._summary_statistic(bps_rx)
pps_rx_total = self._summary_statistic(pps_rx)
- self.logger.info("throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
+ self.logger.info(
+ "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
return bps_rx_total, pps_rx_total
+ def __get_multi_throughput_statistic(
+ self, stream_ids, duration, interval, callback=None):
+ """
+ duration: traffic duration (second)
+ interval: interval of get throughput statistics (second)
+ callback: a callback method of suite, which is used to do some actions
+ during traffic lasting.
+
+ Return: a list of throughput instead of a single tuple of pps/bps rate
+ """
+ time_elapsed = 0
+ stats = []
+ while time_elapsed < duration:
+ time.sleep(interval)
+ stats.append(self.__get_single_throughput_statistic(stream_ids))
+ if callback and callable(callback):
+ callback()
+ time_elapsed += interval
+ return stats
+
+ def measure_throughput(self, stream_ids=[], options={}):
+ """
+ Measure throughput on each tx ports
+
+ options usage:
+ rate:
+ port rate percent, float(0--100). Default value is 100.
+
+ delay:
+ warm up time before start main traffic. If it is set, it will start
+ a delay time traffic to make sure packet generator under good status.
+ Warm up flow is ignored by default.
+
+ interval:
+ a interval time of get throughput statistic (second)
+ If set this key value, pktgen will return several throughput statistic
+ data within a duration traffic. If not set this key value, only
+ return one statistic data. It is ignored by default.
+
+ callback:
+ this key works with ``interval`` key. If it is set, the callback
+ of suite level will be executed after getting throughput statistic.
+ callback method should define as below, don't add sleep in this method.
+
+ def callback(self):
+ xxxx()
+
+ duration:
+ traffic lasting time(second). Default value is 10 second.
+ """
+ interval = options.get('interval')
+ callback = options.get('callback')
+ duration = options.get('duration') or 10
+ delay = options.get('delay')
+ self._prepare_transmission(stream_ids=stream_ids)
+ # start warm up traffic
+ self.__warm_up_pktgen(stream_ids, options, delay)
+ # main traffic
+ self._start_transmission(stream_ids)
+ # keep traffic within a duration time and get throughput statistic
+ if interval and duration:
+ stats = self.__get_multi_throughput_statistic(
+ stream_ids, duration, interval, callback)
+ else:
+ time.sleep(duration)
+ stats = self.__get_single_throughput_statistic(stream_ids)
+ self._stop_transmission(stream_ids)
+ return stats
+
def _measure_loss(self, stream_ids=[], options={}):
"""
Measure lost rate on each tx/rx ports
"""
+ delay = options.get('delay')
+ duration = options.get('duration') or 10
self._prepare_transmission(stream_ids=stream_ids)
+ # start warm up traffic
+ self.__warm_up_pktgen(stream_ids, options, delay)
+ # main traffic
self._start_transmission(stream_ids, options)
+ # keep traffic within a duration time
+ time.sleep(duration)
self._stop_transmission(None)
result = {}
used_rx_port = []
@@ -210,6 +291,19 @@ class PacketGenerator(object):
return result
def measure_loss(self, stream_ids=[], options={}):
+ '''
+ options usage:
+ rate:
+ port rate percent, float(0--100). Default value is 100.
+
+ delay:
+ warm up time before start main traffic. If it is set, it will
+ start a delay time traffic to make sure packet generator
+ under good status. Warm up flow is ignored by default.
+
+ duration:
+ traffic lasting time(second). Default value is 10 second.
+ '''
result = self._measure_loss(stream_ids, options)
# here only to make sure that return value is the same as dts/etgen format
# In real testing scenario, this method can offer more data than it
@@ -218,9 +312,28 @@ class PacketGenerator(object):
def measure_latency(self, stream_ids=[], options={}):
"""
Measure latency on each tx/rx ports
+
+ options usage:
+ rate:
+ port rate percent, float(0--100). Default value is 100.
+
+ delay:
+ warm up time before start main traffic. If it is set, it will
+ start a delay time transmission to make sure packet generator
+ under correct status. Warm up flow is ignored by default.
+
+ duration:
+ traffic lasting time(second). Default value is 10 second.
"""
+ delay = options.get('delay')
+ duration = options.get('duration') or 10
self._prepare_transmission(stream_ids=stream_ids, latency=True)
+ # start warm up traffic
+ self.__warm_up_pktgen(stream_ids, options, delay)
+ # main traffic
self._start_transmission(stream_ids, options)
+ # keep traffic within a duration time
+ time.sleep(duration)
self._stop_transmission(None)
result = {}
@@ -248,7 +361,26 @@ class PacketGenerator(object):
return True
def measure_rfc2544(self, stream_ids=[], options={}):
- """ check loss rate with rate percent dropping """
+ """ check loss rate with rate percent dropping
+
+ options usage:
+ rate:
+ port rate percent at first round testing(0 ~ 100), default is 100.
+
+ pdr:
+ permit packet drop rate, , default is 0.
+
+ drop_step:
+ port rate percent drop step(0 ~ 100), default is 1.
+
+ delay:
+ warm up time before start main traffic. If it is set, it will
+ start a delay time traffic to make sure packet generator
+ under good status. Warm up flow is ignored by default.
+
+ duration:
+ traffic lasting time(second). Default value is 10 second.
+ """
loss_rate_table = []
rate_percent = options.get('rate') or float(100)
permit_loss_rate = options.get('pdr') or 0
@@ -265,6 +397,9 @@ class PacketGenerator(object):
tx_num, rx_num = result.values()[0][1:]
return rate_percent, tx_num, rx_num
_options = deepcopy(options)
+ # if warm up option 'delay' is set, ignore it in next work flow
+ if 'delay' in _options:
+ _options.pop('delay')
if 'rate' in _options:
_options.pop('rate')
while not status and rate_percent > 0:
@@ -328,6 +463,81 @@ class PacketGenerator(object):
# here only pick one
return loss_pps_table[-1][1].values()[0]
+ def measure_rfc2544_dichotomy(self, stream_ids=[], options={}):
+ """ check loss rate using dichotomy algorithm
+
+ options usage:
+ delay:
+ warm up time before start main traffic. If it is set, it will
+ start a delay time traffic to make sure packet generator
+ under good status. Warm up flow is ignored by default.
+
+ duration:
+ traffic lasting time(second). Default value is 10 second.
+
+ min_rate:
+ lower bound rate percent , default is 0.
+
+ max_rate:
+ upper bound rate percent , default is 100.
+
+ pdr:
+ permit packet drop rate(<1.0), default is 0.
+
+ accuracy :
+ dichotomy algorithm accuracy, default 0.001.
+ """
+ max_rate = options.get('max_rate') or 100.0
+ min_rate = options.get('min_rate') or 0.0
+ accuracy = options.get('accuracy') or 0.001
+ permit_loss_rate = options.get('pdr') or 0.0
+ duration = options.get('duration') or 10.0
+ # start warm up traffic
+ delay = options.get('delay')
+ _options = {'duration': duration}
+ self.__warm_up_pktgen(stream_ids, _options, delay)
+ # traffic parameters for dichotomy algorithm
+ loss_rate_table = []
+ hit_result = None
+ rate = traffic_rate_max = max_rate
+ traffic_rate_min = min_rate
+ while True:
+ # run loss rate testing
+ _options = {'duration': duration}
+ result = self._measure_loss(stream_ids, _options)
+ loss_rate_table.append([rate, result])
+ status = self._check_loss_rate(result, permit_loss_rate)
+ # if upper bound rate percent hit, quit the left flow
+ if rate == max_rate and status:
+ hit_result = result
+ break
+ # if lower bound rate percent not hit, quit the left flow
+ if rate == min_rate and not status:
+ break
+ if status:
+ traffic_rate_min = rate
+ hit_result = result
+ else:
+ traffic_rate_max = rate
+ if traffic_rate_max - traffic_rate_min < accuracy:
+ break
+ rate = (traffic_rate_max - traffic_rate_min)/2 + traffic_rate_min
+ self._clear_streams()
+ # set stream rate percent to custom value
+ self._set_stream_rate_percent(rate)
+
+ if not hit_result:
+ msg = ('expected permit loss rate <{0}>'
+ 'not between rate {1} and rate {2}').format(
+ permit_loss_rate, max_rate, min_rate)
+ self.logger.error(msg)
+ self.logger.info(pformat(loss_rate_table))
+ else:
+ self.logger.debug(pformat(loss_rate_table))
+ self.logger.info("zero loss rate is %f" % rate)
+
+ return hit_result
+
def measure(self, stream_ids, traffic_opt):
'''
use as an unify interface method for packet generator
@@ -343,6 +553,8 @@ class PacketGenerator(object):
result = self.measure_rfc2544(stream_ids, traffic_opt)
elif method == 'rfc2544_with_pps':
result = self.measure_rfc2544_with_pps(stream_ids, traffic_opt)
+ elif method == 'rfc2544_dichotomy':
+ result = self.measure_rfc2544_dichotomy(stream_ids, traffic_opt)
else:
result = None
--
2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [dts] [PATCH V3 3/5] framework/pktgen_base: add new features
2019-11-29 9:03 ` [dts] [PATCH V3 3/5] framework/pktgen_base: add new features yufengmx
@ 2020-01-02 3:29 ` Chen, Zhaoyan
0 siblings, 0 replies; 12+ messages in thread
From: Chen, Zhaoyan @ 2020-01-02 3:29 UTC (permalink / raw)
To: Mo, YufengX, dts, Wang, Yinan, Ma, LihongX; +Cc: Mo, YufengX, Chen, Zhaoyan
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 3/5] framework/pktgen_base: add new features
>
>
> *. add test method(latency/loss/throughput) delay/duration options new usage
> definition and relevant process source code.
> *. set delay option to be the warm up time after start transmission.
> *. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to
> realize
> measure_throughput support return several throughput numbers in a duration.
> *. set duration option default value to 10 second.
> *. add test method(latency/loss/throughput/rfc2544) options parameter usage
> comment,
> which is the same as doc pktgen_prog_guide.rst.
> *. add a callback to deal with executing other tools query actions in throughput traffic
> lasting status.
> *. add measure_rfc2544_dichotomy method.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++++++++++---
> 1 file changed, 228 insertions(+), 16 deletions(-)
>
> diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py index
> 7855f6f..75c3036 100644
> --- a/framework/pktgen_base.py
> +++ b/framework/pktgen_base.py
> @@ -81,7 +81,7 @@ class PacketGenerator(object):
> return port_idx
> else:
> port = -1
> - except:
> + except Exception as e:
> port = -1
>
> return port
> @@ -100,7 +100,7 @@ class PacketGenerator(object):
> port = self._get_gen_port(tester_pci)
> msg = "test port {0} map gen port {1}".format(port_id, port)
> self.logger.debug(msg)
> - except:
> + except Exception as e:
> port = -1
>
> return port
> @@ -157,39 +157,120 @@ class PacketGenerator(object):
> def reset_streams(self):
> self.__streams = []
>
> - def measure_throughput(self, stream_ids=[], options={}):
> - """
> - Measure throughput on each tx ports
> - """
> - bps_rx = []
> - pps_rx = []
> - self._prepare_transmission(stream_ids=stream_ids)
> + def __warm_up_pktgen(self, stream_ids, options, delay):
> + ''' run warm up traffic before start main traffic '''
> + if not delay:
> + return
> + msg = '{1} packet generator: run traffic {0}s to warm up ... '.format(
> + delay, self.pktgen_type)
> + self.logger.info(msg)
> self._start_transmission(stream_ids, options)
> -
> - delay = options.get('delay') or 5
> time.sleep(delay)
> + self._stop_transmission(stream_ids)
> + self._clear_streams()
> +
> + def __get_single_throughput_statistic(self, stream_ids):
> + bps_rx = []
> + pps_rx = []
> used_rx_port = []
> + msg = 'begin get port statistic ...'
> + self.logger.info(msg)
> for stream_id in stream_ids:
> if self.__streams[stream_id]['rx_port'] not in used_rx_port:
> rxbps_rates, rxpps_rates = self._retrieve_port_statistic(
> - stream_id, 'throughput')
> + stream_id, 'throughput')
> used_rx_port.append(self.__streams[stream_id]['rx_port'])
> bps_rx.append(rxbps_rates)
> pps_rx.append(rxpps_rates)
> - self._stop_transmission(stream_id)
> -
> bps_rx_total = self._summary_statistic(bps_rx)
> pps_rx_total = self._summary_statistic(pps_rx)
> - self.logger.info("throughput: pps_rx %f, bps_rx %f" % (pps_rx_total,
> bps_rx_total))
> + self.logger.info(
> + "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total,
> + bps_rx_total))
>
> return bps_rx_total, pps_rx_total
>
> + def __get_multi_throughput_statistic(
> + self, stream_ids, duration, interval, callback=None):
> + """
> + duration: traffic duration (second)
> + interval: interval of get throughput statistics (second)
> + callback: a callback method of suite, which is used to do some actions
> + during traffic lasting.
> +
> + Return: a list of throughput instead of a single tuple of pps/bps rate
> + """
> + time_elapsed = 0
> + stats = []
> + while time_elapsed < duration:
> + time.sleep(interval)
> + stats.append(self.__get_single_throughput_statistic(stream_ids))
> + if callback and callable(callback):
> + callback()
> + time_elapsed += interval
> + return stats
> +
> + def measure_throughput(self, stream_ids=[], options={}):
> + """
> + Measure throughput on each tx ports
> +
> + options usage:
> + rate:
> + port rate percent, float(0--100). Default value is 100.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will start
> + a delay time traffic to make sure packet generator under good status.
> + Warm up flow is ignored by default.
> +
> + interval:
> + a interval time of get throughput statistic (second)
> + If set this key value, pktgen will return several throughput statistic
> + data within a duration traffic. If not set this key value, only
> + return one statistic data. It is ignored by default.
> +
> + callback:
> + this key works with ``interval`` key. If it is set, the callback
> + of suite level will be executed after getting throughput statistic.
> + callback method should define as below, don't add sleep in this method.
> +
> + def callback(self):
> + xxxx()
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> + """
> + interval = options.get('interval')
> + callback = options.get('callback')
> + duration = options.get('duration') or 10
> + delay = options.get('delay')
> + self._prepare_transmission(stream_ids=stream_ids)
> + # start warm up traffic
> + self.__warm_up_pktgen(stream_ids, options, delay)
> + # main traffic
> + self._start_transmission(stream_ids)
> + # keep traffic within a duration time and get throughput statistic
> + if interval and duration:
> + stats = self.__get_multi_throughput_statistic(
> + stream_ids, duration, interval, callback)
> + else:
> + time.sleep(duration)
> + stats = self.__get_single_throughput_statistic(stream_ids)
> + self._stop_transmission(stream_ids)
> + return stats
> +
> def _measure_loss(self, stream_ids=[], options={}):
> """
> Measure lost rate on each tx/rx ports
> """
> + delay = options.get('delay')
> + duration = options.get('duration') or 10
> self._prepare_transmission(stream_ids=stream_ids)
> + # start warm up traffic
> + self.__warm_up_pktgen(stream_ids, options, delay)
> + # main traffic
> self._start_transmission(stream_ids, options)
> + # keep traffic within a duration time
> + time.sleep(duration)
> self._stop_transmission(None)
> result = {}
> used_rx_port = []
> @@ -210,6 +291,19 @@ class PacketGenerator(object):
> return result
>
> def measure_loss(self, stream_ids=[], options={}):
> + '''
> + options usage:
> + rate:
> + port rate percent, float(0--100). Default value is 100.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time traffic to make sure packet generator
> + under good status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> + '''
> result = self._measure_loss(stream_ids, options)
> # here only to make sure that return value is the same as dts/etgen format
> # In real testing scenario, this method can offer more data than it @@ -218,9
> +312,28 @@ class PacketGenerator(object):
> def measure_latency(self, stream_ids=[], options={}):
> """
> Measure latency on each tx/rx ports
> +
> + options usage:
> + rate:
> + port rate percent, float(0--100). Default value is 100.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time transmission to make sure packet generator
> + under correct status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> """
> + delay = options.get('delay')
> + duration = options.get('duration') or 10
> self._prepare_transmission(stream_ids=stream_ids, latency=True)
> + # start warm up traffic
> + self.__warm_up_pktgen(stream_ids, options, delay)
> + # main traffic
> self._start_transmission(stream_ids, options)
> + # keep traffic within a duration time
> + time.sleep(duration)
> self._stop_transmission(None)
>
> result = {}
> @@ -248,7 +361,26 @@ class PacketGenerator(object):
> return True
>
> def measure_rfc2544(self, stream_ids=[], options={}):
> - """ check loss rate with rate percent dropping """
> + """ check loss rate with rate percent dropping
> +
> + options usage:
> + rate:
> + port rate percent at first round testing(0 ~ 100), default is 100.
> +
> + pdr:
> + permit packet drop rate, , default is 0.
> +
> + drop_step:
> + port rate percent drop step(0 ~ 100), default is 1.
> +
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time traffic to make sure packet generator
> + under good status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> + """
> loss_rate_table = []
> rate_percent = options.get('rate') or float(100)
> permit_loss_rate = options.get('pdr') or 0 @@ -265,6 +397,9 @@ class
> PacketGenerator(object):
> tx_num, rx_num = result.values()[0][1:]
> return rate_percent, tx_num, rx_num
> _options = deepcopy(options)
> + # if warm up option 'delay' is set, ignore it in next work flow
> + if 'delay' in _options:
> + _options.pop('delay')
> if 'rate' in _options:
> _options.pop('rate')
> while not status and rate_percent > 0:
> @@ -328,6 +463,81 @@ class PacketGenerator(object):
> # here only pick one
> return loss_pps_table[-1][1].values()[0]
>
> + def measure_rfc2544_dichotomy(self, stream_ids=[], options={}):
> + """ check loss rate using dichotomy algorithm
> +
> + options usage:
> + delay:
> + warm up time before start main traffic. If it is set, it will
> + start a delay time traffic to make sure packet generator
> + under good status. Warm up flow is ignored by default.
> +
> + duration:
> + traffic lasting time(second). Default value is 10 second.
> +
> + min_rate:
> + lower bound rate percent , default is 0.
> +
> + max_rate:
> + upper bound rate percent , default is 100.
> +
> + pdr:
> + permit packet drop rate(<1.0), default is 0.
> +
> + accuracy :
> + dichotomy algorithm accuracy, default 0.001.
> + """
> + max_rate = options.get('max_rate') or 100.0
> + min_rate = options.get('min_rate') or 0.0
> + accuracy = options.get('accuracy') or 0.001
> + permit_loss_rate = options.get('pdr') or 0.0
> + duration = options.get('duration') or 10.0
> + # start warm up traffic
> + delay = options.get('delay')
> + _options = {'duration': duration}
> + self.__warm_up_pktgen(stream_ids, _options, delay)
> + # traffic parameters for dichotomy algorithm
> + loss_rate_table = []
> + hit_result = None
> + rate = traffic_rate_max = max_rate
> + traffic_rate_min = min_rate
> + while True:
> + # run loss rate testing
> + _options = {'duration': duration}
> + result = self._measure_loss(stream_ids, _options)
> + loss_rate_table.append([rate, result])
> + status = self._check_loss_rate(result, permit_loss_rate)
> + # if upper bound rate percent hit, quit the left flow
> + if rate == max_rate and status:
> + hit_result = result
> + break
> + # if lower bound rate percent not hit, quit the left flow
> + if rate == min_rate and not status:
> + break
> + if status:
> + traffic_rate_min = rate
> + hit_result = result
> + else:
> + traffic_rate_max = rate
> + if traffic_rate_max - traffic_rate_min < accuracy:
> + break
> + rate = (traffic_rate_max - traffic_rate_min)/2 + traffic_rate_min
> + self._clear_streams()
> + # set stream rate percent to custom value
> + self._set_stream_rate_percent(rate)
> +
> + if not hit_result:
> + msg = ('expected permit loss rate <{0}>'
> + 'not between rate {1} and rate {2}').format(
> + permit_loss_rate, max_rate, min_rate)
> + self.logger.error(msg)
> + self.logger.info(pformat(loss_rate_table))
> + else:
> + self.logger.debug(pformat(loss_rate_table))
> + self.logger.info("zero loss rate is %f" % rate)
> +
> + return hit_result
> +
> def measure(self, stream_ids, traffic_opt):
> '''
> use as an unify interface method for packet generator @@ -343,6 +553,8 @@
> class PacketGenerator(object):
> result = self.measure_rfc2544(stream_ids, traffic_opt)
> elif method == 'rfc2544_with_pps':
> result = self.measure_rfc2544_with_pps(stream_ids, traffic_opt)
> + elif method == 'rfc2544_dichotomy':
> + result = self.measure_rfc2544_dichotomy(stream_ids,
> + traffic_opt)
> else:
> result = None
>
> --
> 2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* [dts] [PATCH V3 4/5] framework/pktgen_ixia: add new features
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
` (2 preceding siblings ...)
2019-11-29 9:03 ` [dts] [PATCH V3 3/5] framework/pktgen_base: add new features yufengmx
@ 2019-11-29 9:03 ` yufengmx
2019-11-29 9:03 ` [dts] [PATCH V3 5/5] framework/pktgen_trex: " yufengmx
` (2 subsequent siblings)
6 siblings, 0 replies; 12+ messages in thread
From: yufengmx @ 2019-11-29 9:03 UTC (permalink / raw)
To: dts, yinan.wang, lihongx.ma; +Cc: yufengmx
*. remove duration option used in ixia module, move duration option in
testing scenario methods(latency/loss/throughput) in pktgen_base module.
*. remove un-used parameters in _start_transmission method.
*. add a warning message for rate percent input parameter.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen_ixia.py | 31 +++++++------------------------
1 file changed, 7 insertions(+), 24 deletions(-)
diff --git a/framework/pktgen_ixia.py b/framework/pktgen_ixia.py
index 69f858e..14b5d5b 100644
--- a/framework/pktgen_ixia.py
+++ b/framework/pktgen_ixia.py
@@ -1247,7 +1247,6 @@ class Ixia(SSHConnection):
''' start ixia ports '''
self.configure_transmission(run_opt)
self.start_transmission()
- time.sleep(run_opt.get('duration') or 5)
def remove_all_streams(self):
''' delete all streams on all ixia ports '''
@@ -1652,28 +1651,13 @@ class IxiaPacketGenerator(PacketGenerator):
self._preset_ixia_port()
def _start_transmission(self, stream_ids, options={}):
- '''
- :param sample_delay:
- After traffic start ``sample_delay`` seconds, start get runtime statistics
- '''
# get rate percentage
- rate_percent = options.get('rate') or '100'
- # get duration
- duration = options.get("duration") or 5
- duration = int(duration) if isinstance(duration, (str, unicode)) \
- else duration
- # get sample interval
- _sample_delay = options.get("sample_delay") or duration/2
- sample_delay = int(_sample_delay) \
- if isinstance(_sample_delay, (str, unicode)) \
- else _sample_delay
- # get configuration from pktgen config file
- warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \
- else 25
- wait_interval, core_mask = (warmup+30, self.conf["core_mask"]) \
- if self.conf.has_key("core_mask") \
- else (warmup+5, None)
- #-------------------------------------------------------------------
+ rate_percent = options.get('rate')
+ if rate_percent:
+ msg = ('{0} only support set rate percent in streams, '
+ 'current run traffic with stream rate percent').format(
+ self.pktgen_type)
+ self.logger.warning(msg)
# run ixia server
try:
###########################################
@@ -1682,8 +1666,6 @@ class IxiaPacketGenerator(PacketGenerator):
run_opt = {
'ports': self._traffic_ports,
'mult': rate_percent,
- 'duration': duration,
- 'core_mask':core_mask,
'force': True,}
self._conn.start(**run_opt)
except Exception as e:
@@ -1693,6 +1675,7 @@ class IxiaPacketGenerator(PacketGenerator):
# using ixia server command
if self._traffic_ports:
self._conn.stop_transmit()
+ self.logger.info("traffic completed. ")
def _retrieve_port_statistic(self, stream_id, mode):
''' ixia traffic statistics '''
--
2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread* [dts] [PATCH V3 5/5] framework/pktgen_trex: add new features
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
` (3 preceding siblings ...)
2019-11-29 9:03 ` [dts] [PATCH V3 4/5] framework/pktgen_ixia: " yufengmx
@ 2019-11-29 9:03 ` yufengmx
2020-01-02 3:29 ` Chen, Zhaoyan
2020-01-02 3:28 ` [dts] [PATCH V3 0/5] dts/pktgen: " Chen, Zhaoyan
2020-01-02 5:04 ` Tu, Lijuan
6 siblings, 1 reply; 12+ messages in thread
From: yufengmx @ 2019-11-29 9:03 UTC (permalink / raw)
To: dts, yinan.wang, lihongx.ma; +Cc: yufengmx
*. remove duration option in trex module, move duration option in
testing scenario methods(latency/loss/throughput) in pktgen_base module.
*. remove sample_delay option in trex module.
*. remove runtim_stat in trex module.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen_trex.py | 63 ++++++++++++++--------------------------
1 file changed, 22 insertions(+), 41 deletions(-)
diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py
index 690d070..6a1b36a 100644
--- a/framework/pktgen_trex.py
+++ b/framework/pktgen_trex.py
@@ -411,7 +411,6 @@ class TrexPacketGenerator(PacketGenerator):
self._ports = []
self._traffic_ports = []
self._rx_ports = []
- self.runtime_stats = {}
conf_inst = self._get_generator_conf_instance()
self.conf = conf_inst.load_pktgen_config()
@@ -450,6 +449,23 @@ class TrexPacketGenerator(PacketGenerator):
from trex_stl_lib.api import STLClient
# set trex class
self.STLClient = STLClient
+ # get configuration from pktgen config file
+ self._get_traffic_option()
+
+ def _get_traffic_option(self):
+ ''' get configuration from pktgen config file '''
+ # set trex coremask
+ _core_mask = self.conf.get("core_mask")
+ if _core_mask:
+ if '0x' in _core_mask:
+ self.core_mask = \
+ [int(item[2:], 16) for item in _core_mask.split(',')]
+ else:
+ self.core_mask = self.STLClient.CORE_MASK_PIN \
+ if _core_mask.upper() == 'CORE_MASK_PIN' else \
+ None
+ else:
+ self.core_mask = None
def _connect(self):
self._conn = self.STLClient(server=self.conf["server"])
@@ -652,7 +668,7 @@ class TrexPacketGenerator(PacketGenerator):
def _throughput_stats(self, stream, stats):
# tx packet
tx_port_id = stream["tx_port"]
- port_stats = self.runtime_stats.get(tx_port_id)
+ port_stats = stats.get(tx_port_id)
if not port_stats:
msg = "failed to get tx_port {0} statistics".format(tx_port_id)
raise Exception(msg)
@@ -666,7 +682,7 @@ class TrexPacketGenerator(PacketGenerator):
self.logger.debug(os.linesep.join(msg))
# rx bps/pps
rx_port_id = stream["rx_port"]
- port_stats = self.runtime_stats.get(rx_port_id)
+ port_stats = stats.get(rx_port_id)
if not port_stats:
msg = "failed to get rx_port {0} statistics".format(rx_port_id)
raise Exception(msg)
@@ -768,65 +784,30 @@ class TrexPacketGenerator(PacketGenerator):
self._preset_trex_port()
def _start_transmission(self, stream_ids, options={}):
- '''
- :param sample_delay:
- After traffic start ``sample_delay`` seconds, start get runtime statistics
- '''
+ test_mode = options.get('method')
# get rate percentage
rate_percent = "{0}%".format(options.get('rate') or
self._traffic_opt.get('rate') or
'100')
- # get duration
- duration = options.get("duration") or 20
- duration = int(duration) if isinstance(duration, (str, unicode)) \
- else duration
- # get sample interval
- _sample_delay = options.get("sample_delay") or duration/2
- sample_delay = int(_sample_delay) \
- if isinstance(_sample_delay, (str, unicode)) \
- else _sample_delay
- # get configuration from pktgen config file
- warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \
- else 25
- # set trex coremask
- wait_interval, core_mask = (
- warmup+30, int(self.conf["core_mask"], 16)) \
- if self.conf.has_key("core_mask") \
- else (warmup+5, 0x3)
-
try:
- ###########################################
# clear the stats before injecting
self._conn.clear_stats()
# Start traffic on port(s)
run_opt = {
'ports': self._traffic_ports,
'mult': rate_percent,
- 'duration': duration,
- 'core_mask':core_mask,
+ 'core_mask': self.core_mask,
'force': True,}
self.logger.info("begin traffic ......")
self.logger.debug(run_opt)
self._conn.start(**run_opt)
- ###########################################
- if sample_delay:
- time.sleep(sample_delay) # wait
- # get ports runtime statistics
- self.runtime_stats = self._conn.get_stats()
- self.logger.debug(pformat(self.runtime_stats))
- ###########################################
- # Block until traffic on specified port(s) has ended
- wait_opt = {'ports': self._traffic_ports}
- if duration:
- time.sleep(wait_interval + 10)
- wait_opt['timeout'] = wait_interval + duration
- self._conn.wait_on_traffic(**wait_opt)
except Exception as e:
self.logger.error(e)
def _stop_transmission(self, stream_id):
if self._traffic_ports:
self._conn.stop(ports=self._traffic_ports, rx_delay_ms=5000)
+ self.logger.info("traffic completed. ")
def _retrieve_port_statistic(self, stream_id, mode):
'''
--
2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [dts] [PATCH V3 5/5] framework/pktgen_trex: add new features
2019-11-29 9:03 ` [dts] [PATCH V3 5/5] framework/pktgen_trex: " yufengmx
@ 2020-01-02 3:29 ` Chen, Zhaoyan
0 siblings, 0 replies; 12+ messages in thread
From: Chen, Zhaoyan @ 2020-01-02 3:29 UTC (permalink / raw)
To: Mo, YufengX, dts, Wang, Yinan, Ma, LihongX; +Cc: Mo, YufengX, Chen, Zhaoyan
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 5/5] framework/pktgen_trex: add new features
>
>
> *. remove duration option in trex module, move duration option in
> testing scenario methods(latency/loss/throughput) in pktgen_base module.
> *. remove sample_delay option in trex module.
> *. remove runtim_stat in trex module.
>
> Signed-off-by: yufengmx <yufengx.mo@intel.com>
> ---
> framework/pktgen_trex.py | 63 ++++++++++++++--------------------------
> 1 file changed, 22 insertions(+), 41 deletions(-)
>
> diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py index
> 690d070..6a1b36a 100644
> --- a/framework/pktgen_trex.py
> +++ b/framework/pktgen_trex.py
> @@ -411,7 +411,6 @@ class TrexPacketGenerator(PacketGenerator):
> self._ports = []
> self._traffic_ports = []
> self._rx_ports = []
> - self.runtime_stats = {}
>
> conf_inst = self._get_generator_conf_instance()
> self.conf = conf_inst.load_pktgen_config() @@ -450,6 +449,23 @@ class
> TrexPacketGenerator(PacketGenerator):
> from trex_stl_lib.api import STLClient
> # set trex class
> self.STLClient = STLClient
> + # get configuration from pktgen config file
> + self._get_traffic_option()
> +
> + def _get_traffic_option(self):
> + ''' get configuration from pktgen config file '''
> + # set trex coremask
> + _core_mask = self.conf.get("core_mask")
> + if _core_mask:
> + if '0x' in _core_mask:
> + self.core_mask = \
> + [int(item[2:], 16) for item in _core_mask.split(',')]
> + else:
> + self.core_mask = self.STLClient.CORE_MASK_PIN \
> + if _core_mask.upper() == 'CORE_MASK_PIN' else \
> + None
> + else:
> + self.core_mask = None
>
> def _connect(self):
> self._conn = self.STLClient(server=self.conf["server"])
> @@ -652,7 +668,7 @@ class TrexPacketGenerator(PacketGenerator):
> def _throughput_stats(self, stream, stats):
> # tx packet
> tx_port_id = stream["tx_port"]
> - port_stats = self.runtime_stats.get(tx_port_id)
> + port_stats = stats.get(tx_port_id)
> if not port_stats:
> msg = "failed to get tx_port {0} statistics".format(tx_port_id)
> raise Exception(msg)
> @@ -666,7 +682,7 @@ class TrexPacketGenerator(PacketGenerator):
> self.logger.debug(os.linesep.join(msg))
> # rx bps/pps
> rx_port_id = stream["rx_port"]
> - port_stats = self.runtime_stats.get(rx_port_id)
> + port_stats = stats.get(rx_port_id)
> if not port_stats:
> msg = "failed to get rx_port {0} statistics".format(rx_port_id)
> raise Exception(msg)
> @@ -768,65 +784,30 @@ class TrexPacketGenerator(PacketGenerator):
> self._preset_trex_port()
>
> def _start_transmission(self, stream_ids, options={}):
> - '''
> - :param sample_delay:
> - After traffic start ``sample_delay`` seconds, start get runtime statistics
> - '''
> + test_mode = options.get('method')
> # get rate percentage
> rate_percent = "{0}%".format(options.get('rate') or
> self._traffic_opt.get('rate') or
> '100')
> - # get duration
> - duration = options.get("duration") or 20
> - duration = int(duration) if isinstance(duration, (str, unicode)) \
> - else duration
> - # get sample interval
> - _sample_delay = options.get("sample_delay") or duration/2
> - sample_delay = int(_sample_delay) \
> - if isinstance(_sample_delay, (str, unicode)) \
> - else _sample_delay
> - # get configuration from pktgen config file
> - warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \
> - else 25
> - # set trex coremask
> - wait_interval, core_mask = (
> - warmup+30, int(self.conf["core_mask"], 16)) \
> - if self.conf.has_key("core_mask") \
> - else (warmup+5, 0x3)
> -
> try:
> - ###########################################
> # clear the stats before injecting
> self._conn.clear_stats()
> # Start traffic on port(s)
> run_opt = {
> 'ports': self._traffic_ports,
> 'mult': rate_percent,
> - 'duration': duration,
> - 'core_mask':core_mask,
> + 'core_mask': self.core_mask,
> 'force': True,}
> self.logger.info("begin traffic ......")
> self.logger.debug(run_opt)
> self._conn.start(**run_opt)
> - ###########################################
> - if sample_delay:
> - time.sleep(sample_delay) # wait
> - # get ports runtime statistics
> - self.runtime_stats = self._conn.get_stats()
> - self.logger.debug(pformat(self.runtime_stats))
> - ###########################################
> - # Block until traffic on specified port(s) has ended
> - wait_opt = {'ports': self._traffic_ports}
> - if duration:
> - time.sleep(wait_interval + 10)
> - wait_opt['timeout'] = wait_interval + duration
> - self._conn.wait_on_traffic(**wait_opt)
> except Exception as e:
> self.logger.error(e)
>
> def _stop_transmission(self, stream_id):
> if self._traffic_ports:
> self._conn.stop(ports=self._traffic_ports, rx_delay_ms=5000)
> + self.logger.info("traffic completed. ")
>
> def _retrieve_port_statistic(self, stream_id, mode):
> '''
> --
> 2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [dts] [PATCH V3 0/5] dts/pktgen: add new features
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
` (4 preceding siblings ...)
2019-11-29 9:03 ` [dts] [PATCH V3 5/5] framework/pktgen_trex: " yufengmx
@ 2020-01-02 3:28 ` Chen, Zhaoyan
2020-01-02 5:04 ` Tu, Lijuan
6 siblings, 0 replies; 12+ messages in thread
From: Chen, Zhaoyan @ 2020-01-02 3:28 UTC (permalink / raw)
To: Mo, YufengX, dts, Wang, Yinan, Ma, LihongX; +Cc: Mo, YufengX, Chen, Zhaoyan
Acked-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Regards,
Zhaoyan Chen
> -----Original Message-----
> From: dts <dts-bounces@dpdk.org> On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 0/5] dts/pktgen: add new features
>
> v3
> Add 2544 dichotomy algorithm as ma,lihong&han,yingya required.
> *. add measure_rfc2544_dichotomy method *. add rfc2544 dichotomy usage
> document
>
> v2
> Add new demand of pktgen as wang,yinan required.
> *. add a callback to deal with executing other tools query actions in throughput
> traffic lasting status.
> *. rebase source code.
>
> v1
> As chen zhaoyan new demand of pktgen, add these changes
>
> *. add __get_single_throughput_statistic/__get_multi_throughput_statistic methods to
> realize measure_throughput support return several throughput statistic data in a
> duration.
> *. add test method(latency/loss/throughput/rfc2544) options parameter usage
> comment in pktgen_base module.
> *. add test method(latency/loss/throughput) delay/duration options new usage
> definition and relevant process source code in pktgen_base module.
> *. set delay option to the warm up time before start main traffic.
> *. set pktgen duration default value to 10 second.
> *. remove duration option used in trex/ixia module, move duration option in testing
> scenario methods(latency/loss/throughput) in pktgen_base module.
> *. remove un-used optons in pktgen.cfg.
> *. remove un-used parameter in trex/ixia _start_transmission.
> *. rebase coremask new setting options from next branch.
>
> yufengmx (5):
> conf/pktgen: remove un-used optons
> doc/dts_gsg/pktgen_prog_guide: update description
> framework/pktgen_base: add new features
> framework/pktgen_ixia: add new features
> framework/pktgen_trex: add new features
>
> conf/pktgen.cfg | 18 ++-
> doc/dts_gsg/pktgen_prog_guide.rst | 86 +++++++++--
> framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++--
> framework/pktgen_ixia.py | 31 +---
> framework/pktgen_trex.py | 63 +++-----
> 5 files changed, 340 insertions(+), 102 deletions(-)
>
> --
> 2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [dts] [PATCH V3 0/5] dts/pktgen: add new features
2019-11-29 9:03 [dts] [PATCH V3 0/5] dts/pktgen: add new features yufengmx
` (5 preceding siblings ...)
2020-01-02 3:28 ` [dts] [PATCH V3 0/5] dts/pktgen: " Chen, Zhaoyan
@ 2020-01-02 5:04 ` Tu, Lijuan
6 siblings, 0 replies; 12+ messages in thread
From: Tu, Lijuan @ 2020-01-02 5:04 UTC (permalink / raw)
To: Mo, YufengX, dts, Wang, Yinan, Ma, LihongX; +Cc: Mo, YufengX
Applied the series, thanks
> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of yufengmx
> Sent: Friday, November 29, 2019 5:03 PM
> To: dts@dpdk.org; Wang, Yinan <yinan.wang@intel.com>; Ma, LihongX
> <lihongx.ma@intel.com>
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [PATCH V3 0/5] dts/pktgen: add new features
>
> v3
> Add 2544 dichotomy algorithm as ma,lihong&han,yingya required.
> *. add measure_rfc2544_dichotomy method *. add rfc2544 dichotomy
> usage document
>
> v2
> Add new demand of pktgen as wang,yinan required.
> *. add a callback to deal with executing other tools query actions in
> throughput traffic lasting status.
> *. rebase source code.
>
> v1
> As chen zhaoyan new demand of pktgen, add these changes
>
> *. add __get_single_throughput_statistic/__get_multi_throughput_statistic
> methods to realize measure_throughput support return several throughput
> statistic data in a duration.
> *. add test method(latency/loss/throughput/rfc2544) options parameter
> usage comment in pktgen_base module.
> *. add test method(latency/loss/throughput) delay/duration options new
> usage definition and relevant process source code in pktgen_base module.
> *. set delay option to the warm up time before start main traffic.
> *. set pktgen duration default value to 10 second.
> *. remove duration option used in trex/ixia module, move duration option
> in testing scenario methods(latency/loss/throughput) in pktgen_base
> module.
> *. remove un-used optons in pktgen.cfg.
> *. remove un-used parameter in trex/ixia _start_transmission.
> *. rebase coremask new setting options from next branch.
>
> yufengmx (5):
> conf/pktgen: remove un-used optons
> doc/dts_gsg/pktgen_prog_guide: update description
> framework/pktgen_base: add new features
> framework/pktgen_ixia: add new features
> framework/pktgen_trex: add new features
>
> conf/pktgen.cfg | 18 ++-
> doc/dts_gsg/pktgen_prog_guide.rst | 86 +++++++++--
> framework/pktgen_base.py | 244 ++++++++++++++++++++++++++++--
> framework/pktgen_ixia.py | 31 +---
> framework/pktgen_trex.py | 63 +++-----
> 5 files changed, 340 insertions(+), 102 deletions(-)
>
> --
> 2.21.0
^ permalink raw reply [flat|nested] 12+ messages in thread