* [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api
@ 2019-04-28 2:48 yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 1/14] conf/pktgen: add pktgen key word in crbs.cfg yufengmx
` (14 more replies)
0 siblings, 15 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:48 UTC (permalink / raw)
To: dts; +Cc: yufengmx
dts packet generator(pktgen) source code, including:
*. packet generator configure file definition.
*. packet generator configure file parse class.
*. packet generator base class.
*. ixia packet generator relevant classes.
*. trex packet generator relevant classes.
*. packet generator helper class.
*. trex packet generator port mapping process.
*. new utils methods used by pktgen.
*. pktgen api program guide document.
*. migrate suite source code from etgen api to pktgen api guide document.
*. trex tool known issue document.
*. change copyright year.
yufengmx (14):
conf/pktgen: add pktgen key word in crbs.cfg
conf/pktgen: packet generator configure file definition
doc/pktgen: trex tool known issues
doc/pktgen: migrate from etgen api to pktgen api
doc/pktgen: pktgen api program guide document
framework/pktgen: packet generator configure file parse class
framework/pktgen: initialize pktgen logger
framework/pktgen: packet generator base class
framework/pktgen: ixia packet generator relevant classes
framework/pktgen: trex packet generator relevant classes
framework/pktgen: pktgen instance creation and helper class
framework/pktgen: packet generator types definition
framework/pktgen: packet generator init and trex port mapping
framework/pktgen: utils methods
conf/crbs.cfg | 3 +
conf/pktgen.cfg | 23 +-
doc/dts_gsg/migrate_from_etgen_to_pktgen.rst | 214 ++++
doc/dts_gsg/pktgen_prog_guide.rst | 518 ++++++++
doc/dts_gsg/trex_known_issue.rst | 112 ++
framework/config.py | 107 +-
framework/dts.py | 21 +-
framework/pktgen.py | 610 +++------
framework/pktgen_base.py | 400 ++++++
framework/pktgen_ixia.py | 1750 ++++++++++++++++++++++++++
framework/pktgen_trex.py | 857 +++++++++++++
framework/settings.py | 8 +-
framework/tester.py | 136 +-
framework/utils.py | 32 +-
14 files changed, 4284 insertions(+), 507 deletions(-)
create mode 100644 doc/dts_gsg/migrate_from_etgen_to_pktgen.rst
create mode 100644 doc/dts_gsg/pktgen_prog_guide.rst
create mode 100644 doc/dts_gsg/trex_known_issue.rst
create mode 100644 framework/pktgen_base.py
create mode 100644 framework/pktgen_ixia.py
create mode 100644 framework/pktgen_trex.py
mode change 100755 => 100644 framework/tester.py
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 1/14] conf/pktgen: add pktgen key word in crbs.cfg
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
@ 2019-04-28 2:48 ` yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 2/14] conf/pktgen: packet generator configure file definition yufengmx
` (13 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:48 UTC (permalink / raw)
To: dts; +Cc: yufengmx
add pktgen key word in crbs.cfg
Add pktgen_group key word to decide which section in packet generator configure file
is used to initialize packet generator.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
conf/crbs.cfg | 3 +++
1 file changed, 3 insertions(+)
diff --git a/conf/crbs.cfg b/conf/crbs.cfg
index f8676e6..5c28ab2 100644
--- a/conf/crbs.cfg
+++ b/conf/crbs.cfg
@@ -7,6 +7,7 @@
# tester_ip: Tester ip address
# tester_passwd: Tester password
# ixia_group: IXIA group name
+# pktgen_group: packet generator group name
# channels: Board channel number
# bypass_core0: Whether by pass core0
[DUT IP1]
@@ -18,6 +19,7 @@ dut_arch=
tester_ip=xxx.xxx.xxx.xxx
tester_passwd=
ixia_group=
+pktgen_group=
channels=4
bypass_core0=True
[DUT IP2]
@@ -29,5 +31,6 @@ dut_arch=
tester_ip=yyy.yyy.yyy.yyy
tester_passwd=
ixia_group=
+pktgen_group=
channels=4
bypass_core0=True
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 2/14] conf/pktgen: packet generator configure file definition
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 1/14] conf/pktgen: add pktgen key word in crbs.cfg yufengmx
@ 2019-04-28 2:48 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 3/14] doc/pktgen: trex tool known issues yufengmx
` (12 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:48 UTC (permalink / raw)
To: dts; +Cc: yufengmx
packet generator configure file definition
Create a file named pktgen.cfg under conf folder to save ixia/trex configuration content.
Ixia and trex configuration content can be set in different section of pktgen.cfg.
Section title should be ixia/trex and it is case insensitive.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
conf/pktgen.cfg | 23 ++++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/conf/pktgen.cfg b/conf/pktgen.cfg
index cc3fe61..667e1e8 100644
--- a/conf/pktgen.cfg
+++ b/conf/pktgen.cfg
@@ -1,9 +1,13 @@
#PkTGEN configuration
#[PKTGEN DPDK]
+# trex_root_path: trex tar package decompression directory
+# trex server binary file is under this directory.
+# trex_lib_path(optional): trex stateless client libs directory, it is optional.
+# If it is not set, use a default relative directory.
# coremask -c: A hexadecimal bitmask of cores to run on
# num -n: Number of memory channels
# proc_type --proc-type: Type of this process
-# pci_blacklist --pci-blacklist, -b: Add a PCI device in black list.
+# pci_blacklist --pci-blacklist, -b: Add a PCI device in black list.
# pci_whitelist --pci-whitelist, -w: Add a PCI device in white list.
# file_prefix --file-prefix: Prefix for hugepage filenames
# socket_memory --socket-mem: Memory to allocate on specific sockets
@@ -12,6 +16,7 @@
# start_trex: Set to a nonempty value to start trex ourselves.
[TREX]
trex_root_path=/opt/trex-core-2.26
+trex_lib_path=/opt/trex/vx.xxx/automation/trex_control_plane/interactive
config_file=/etc/trex_cfg.yaml
server=10.67.111.143
pcap_file=/opt/trex-core-2.26/scripts/stl/sample.pcap
@@ -22,3 +27,19 @@ ip_dst=10.0.0.1
warmup=15
duration=-1
#start_trex=yes
+
+# IXIA port Configuration
+# IxiaGroup: Group name for IXIA ports
+# Version : IXIA TCL server version
+# IP : IXIA server IP address
+# Ports : [IXIA port list]
+# ixia_force100g: We need to set this to enable if we need IXIA port work in 100G mode.
+[IXIA]
+ixia_version=6.62
+ixia_ip=xxx.xxx.xxx.xxx
+ixia_ports=
+ card=1,port=1;
+ card=1,port=2;
+ card=1,port=3;
+ card=1,port=4;
+ixia_force100g=disable
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 3/14] doc/pktgen: trex tool known issues
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 1/14] conf/pktgen: add pktgen key word in crbs.cfg yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 2/14] conf/pktgen: packet generator configure file definition yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 4/14] doc/pktgen: migrate from etgen api to pktgen api yufengmx
` (11 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
trex tool known issues
The document describes trex tool common issues and resolved methods. When use
trex tool as packet generator, these issues should be taken care.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
doc/dts_gsg/trex_known_issue.rst | 112 +++++++++++++++++++++++++++++++++++++++
1 file changed, 112 insertions(+)
create mode 100644 doc/dts_gsg/trex_known_issue.rst
diff --git a/doc/dts_gsg/trex_known_issue.rst b/doc/dts_gsg/trex_known_issue.rst
new file mode 100644
index 0000000..53cbc1a
--- /dev/null
+++ b/doc/dts_gsg/trex_known_issue.rst
@@ -0,0 +1,112 @@
+======================
+How dts work with trex
+======================
+
+dpdk hugepage management conflict issue
+=======================================
+trex use older dpdk version than we release cycle source code. When dpdk change
+the memory management merchanism, trex will meet the following issue.
+
+Trex should run on an independent platform. DUT/Trex should run on two platforms
+*. one is used as TESTER and trex server, another one is used as DUT.(dts/pktgen)
+*. one is used as trex server, another one is used as DUT/TESTER.(recommended scheme)
+ This scheme can make sure that trex run on its full status capability.
+
+When trex run with dts on the same platform, trex server sometimes boot up
+failed for hugepage error.
+
+.. code-block:: console
+
+ ./t-rex-64 -i --stl -k 4
+
+ Starting Scapy server..... Scapy server is started
+ Trying to bind to igb_uio ...
+ /usr/bin/python3 dpdk_nic_bind.py --bind=igb_uio 0000:85:00.0 0000:8a:00.1
+ The ports are bound/configured.
+ Starting TRex v2.41 please wait ...
+ EAL: Can only reserve 1766 pages from 4096 requested
+ Current CONFIG_RTE_MAX_MEMSEG=256 is not enough
+ Please either increase it or request less amount of memory.
+ EAL: FATAL: Cannot init memory
+
+ EAL: Cannot init memory
+
+ You might need to run ./trex-cfg once
+ EAL: Error - exiting with code: 1
+ Cause: Invalid EAL arguments
+
+trex quit when using NNT
+========================
+when bind dut NNT port to igb_uio, peer port will get a link down status, then
+trex server using NNT nic will quit.
+
+.. code-block:: console
+
+ WATCHDOG: task 'master' has not responded for more than 2.00044 seconds - timeout is 2 seconds
+
+ *** traceback follows ***
+
+ 1 0x55a7c779561a ./_t-rex-64(+0x12761a) [0x55a7c779561a]
+ 2 0x7f23da4be1b0 /lib64/libpthread.so.0(+0x121b0) [0x7f23da4be1b0]
+ 3 0x55a7c7942d40 rte_delay_us_block + 128
+ 4 0x55a7c798d731 ixgbe_setup_mac_link_multispeed_fiber + 337
+ 5 0x55a7c79a8f14 ./_t-rex-64(+0x33af14) [0x55a7c79a8f14]
+ 6 0x55a7c7954c72 rte_eth_link_get_nowait + 114
+ 7 0x55a7c776a988 DpdkTRexPortAttr::update_link_status_nowait() + 24
+ 8 0x55a7c77856a6 CGlobalTRex::handle_slow_path() + 118
+ 9 0x55a7c7785ad7 CGlobalTRex::run_in_master() + 759
+ 10 0x55a7c7785e3c ./_t-rex-64(+0x117e3c) [0x55a7c7785e3c]
+ 11 0x55a7c793efba rte_eal_mp_remote_launch + 346
+ 12 0x55a7c7789e1e main_test(int, char**) + 1038
+ 13 0x7f23d9417f2a __libc_start_main + 234
+ 14 0x55a7c7719b9d ./_t-rex-64(+0xabb9d) [0x55a7c7719b9d]
+
+
+ *** addr2line information follows ***
+
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+ ??:0
+
+
+ ./t-rex-64: line 80: 25870 Aborted (core dumped) ./_$(
+
+scapy name space conflict
+=========================
+trex scapy lib will be conflict with
+
+resolved scheme
+---------------
+
+#. backup your scapy::
+ cp -fr /usr/lib/python2.7/site-packages/scapy /usr/lib/python2.7/site-packages/scapy_backup
+
+#. unify scapy version with trex::
+ cp -fr /opt/trex/v2.41/trex_client/external_libs/scapy-2.3.1/python2/scapy /usr/lib/python2.7/site-packages/scapy
+
+other issues
+============
+
+#. linux kernel verion should not be too low.
+
+#. Trex only works with even number link peers.
+
+#. Trex only works with nics, which are using the same driver.
+
+#. Before boot up trex, please make sure the peer ports are on up status.
+
+#. If you have ran dpdk on the platform which you want to deploy trex-server,
+ reboot the platform to make sure that trex-server can work fine.
+
+#. If using i40e driver, Trex v2.41 version need i40e nic firmware version newer than 5.02.
\ No newline at end of file
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 4/14] doc/pktgen: migrate from etgen api to pktgen api
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (2 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 3/14] doc/pktgen: trex tool known issues yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 5/14] doc/pktgen: pktgen api program guide document yufengmx
` (10 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
migrate from etgen api to pktgen api.
The document describes a convenient way to meet with the simple packet generator
usage scenario. Pktgen offer a helper class to be compatible with old coding
style in suite source code.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
doc/dts_gsg/migrate_from_etgen_to_pktgen.rst | 214 +++++++++++++++++++++++++++
1 file changed, 214 insertions(+)
create mode 100644 doc/dts_gsg/migrate_from_etgen_to_pktgen.rst
diff --git a/doc/dts_gsg/migrate_from_etgen_to_pktgen.rst b/doc/dts_gsg/migrate_from_etgen_to_pktgen.rst
new file mode 100644
index 0000000..bee7190
--- /dev/null
+++ b/doc/dts_gsg/migrate_from_etgen_to_pktgen.rst
@@ -0,0 +1,214 @@
+
+=================
+etgen replacement
+=================
+pktgen usage please refer to doc **pktgen_prog_guide.rst**.
+
+import new class
+----------------
+
+#. import a new module::
+
+.. code-block:: python
+
+ from pktgen import PacketGeneratorHelper
+
+initialize an instance in `def set_up_all(self)`
+------------------------------------------------
+
+.. code-block:: python
+
+ def set_up_all(self):
+ ...
+ self.pktgen_helper = PacketGeneratorHelper()
+
+create streams for pktgen instance
+----------------------------------
+each pcap file should only contain one packet.
+
+.. code-block:: python
+
+ pcap1 = os.sep.join([self.pktgen.output_path, "{0}.pcap".format(port)])
+ flow1 = "Ether()/IP()/UDP()/("X")"
+ self.tester.scapy_append('wrpcap("%s", [flow])' % (pcap1, flow1))
+ self.tester.scapy_execute()
+
+ pcap2 = os.sep.join([self.pktgen.output_path, "{0}.pcap".format(port)])
+ flow2 = "Ether()/IP()/UDP()/("X")"
+ self.tester.scapy_append('wrpcap("%s", [flow])' % (pcap2, flow2))
+ self.tester.scapy_execute()
+
+ tgen_input = []
+ tgen_input.append([tx_port, rx_port, pcap1])
+ tgen_input.append([tx_port, rx_port, pcap2])
+
+pcap field variable(optional)
+-----------------------------
+If no protocol layer field vary requirement, ignore this content.
+
+field key definition
+~~~~~~~~~~~~~~~~~~~~
+
+#. ip protocol layer::
+ # protocol layer name
+ 'mac': {
+ # field name
+ 'src': {
+ # field value vary range
+ 'range': 64,
+ # field value vary step
+ 'step': 1,
+ # action: inc/dec/random
+ 'action': 'inc'},
+ 'dst': {'range': 64, 'step': 1, 'action': 'inc'},
+ }
+
+#. mac protocol layer::
+ # protocol layer name
+ 'mac': {
+ # field name
+ 'src': {
+ # field value vary range
+ 'range': 64,
+ # field value vary step
+ 'step': 1,
+ # action: inc/dec/random
+ 'action': 'inc'},
+ 'dst': {'range': 64, 'step': 1, 'action': 'inc'},
+ }
+
+#. vlan protocol layer::
+ # protocol layer name
+ 'vlan': {
+ '0': {
+ # field value vary range
+ 'range': 64,
+ # field value vary step
+ 'step': 1,
+ # action: inc/dec/random
+ 'action': 'inc'},}
+
+usage example
+~~~~~~~~~~~~~
+
+.. code-block:: python
+
+ def set_up_all(self):
+ ...
+ self.pktgen_helper = PacketGeneratorHelper()
+ ...
+
+ def set_fields(self):
+ fields_config = {
+ 'ip': {
+ 'src': {'range': 64, 'action': 'inc'},
+ 'dst': {'action': 'random'}},}
+ return fields_config
+
+ def test_perf_xxxx(self):
+ ...
+ vm_config= self.set_fields() # optional
+ # clear streams before add new streams
+ self.tester.pktgen.clear_streams()
+ # run packet generator
+ ratePercent = 100
+ streams = self.pktgen_helper.prepare_stream_from_tginput(
+ tgenInput, ratePercent, vm_config, self.tester.pktgen)
+ _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
+ ...
+
+change etgen interface to pktgen interface
+------------------------------------------
+pktgen interface use the same input parameter/return value definition as
+etgen interface.
+
+throughput
+~~~~~~~~~~
+
+etgen::
+
+.. code-block:: python
+
+ self.tester.traffic_generator_throughput(tgen_input)
+
+pktgen::
+
+.. code-block:: python
+
+ vm_config= self.set_fields() # optional
+ # clear streams before add new streams
+ self.tester.pktgen.clear_streams()
+ # run packet generator
+ ratePercent = 100
+ streams = self.pktgen_helper.prepare_stream_from_tginput(
+ tgenInput, ratePercent, vm_config, self.tester.pktgen)
+ _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
+
+loss
+~~~~
+
+etgen::
+
+.. code-block:: python
+
+ self.tester.traffic_generator_loss(tgen_input)
+
+pktgen::
+
+.. code-block:: python
+
+ vm_config= self.set_fields() # optional
+ # clear streams before add new streams
+ self.tester.pktgen.clear_streams()
+ # run packet generator
+ ratePercent = 100
+ streams = self.pktgen_helper.prepare_stream_from_tginput(
+ tgenInput, ratePercent, vm_config, self.tester.pktgen)
+ result = self.tester.pktgen.measure_loss(stream_ids=streams)
+
+latency
+~~~~~~~
+
+etgen::
+
+.. code-block:: python
+
+ self.tester.traffic_generator_latency(tgen_input)
+
+pktgen::
+
+.. code-block:: python
+
+ vm_config= self.set_fields() # optional
+ # clear streams before add new streams
+ self.tester.pktgen.clear_streams()
+ # run packet generator
+ ratePercent = 100
+ streams = self.pktgen_helper.prepare_stream_from_tginput(
+ tgenInput, ratePercent, vm_config, self.tester.pktgen)
+ latencys = self.tester.pktgen.measure_latency(stream_ids=streams)
+
+rfc2544
+~~~~~~~
+
+etgen::
+
+.. code-block:: python
+
+ self.tester.run_rfc2544(tgen_input)
+
+pktgen::
+
+.. code-block:: python
+
+ vm_config= self.set_fields() # optional
+ # clear streams before add new streams
+ self.tester.pktgen.clear_streams()
+ # run packet generator
+ ratePercent = 100
+ streams = self.pktgen_helper.prepare_stream_from_tginput(
+ tgenInput, ratePercent, vm_config, self.tester.pktgen)
+ # set traffic option
+ traffic_opt = {'pdr': 0.01, 'duration': 5}
+ zero_loss_rate, tx_pkts, rx_pkts = \
+ self.tester.pktgen.measure_rfc2544(stream_ids=streams, options=traffic_opt)
\ No newline at end of file
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 5/14] doc/pktgen: pktgen api program guide document
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (3 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 4/14] doc/pktgen: migrate from etgen api to pktgen api yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 6/14] framework/pktgen: packet generator configure file parse yufengmx
` (9 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
pktgen api program guide document
The document describes input parameter and return value definition of pktgen api.
These api includes add stream --> config stream --> measure. The document
has an example source code of a complex application scenario for program reference.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
doc/dts_gsg/pktgen_prog_guide.rst | 518 ++++++++++++++++++++++++++++++++++++++
1 file changed, 518 insertions(+)
create mode 100644 doc/dts_gsg/pktgen_prog_guide.rst
diff --git a/doc/dts_gsg/pktgen_prog_guide.rst b/doc/dts_gsg/pktgen_prog_guide.rst
new file mode 100644
index 0000000..32d2082
--- /dev/null
+++ b/doc/dts_gsg/pktgen_prog_guide.rst
@@ -0,0 +1,518 @@
+
+=================
+how to use pktgen
+=================
+these definition and usage pattern come from doc `pktgen-API-1.1.docx` and etgen
+usage in dts. For trex(CISCO) rapid iterative development speed, we lack of
+adequate manpower to keep up with it. Here we recommend to use trex v2.41/v2.42/v2.43
+to run pktgen/trex.
+
+add stream
+==========
+add stream in pktgen streams table.
+
+one stream content including::
+
+ tx_port: transmit port idx in tester.ports_info.
+ rx_port: receive port idx in tester.ports_info.
+ pcap: pcap file or Packet instance, Only support one packet in it.
+
+.. code-block:: python
+
+ tx_port = self.tester.get_local_port(dut_port_index1)
+ rx_port = self.tester.get_local_port(dut_port_index2)
+
+ stream_id = hPktgen.add_stream(tx_port, rx_port, pcap)
+
+config stream
+=============
+configure a stream option.
+
+definition
+----------
+Currently pktgen support ethernet/ipv4/vlan protocol layer some field vary with
+increase/decrease/random value.
+
+stream option contain::
+
+ 'pcap': network packet format
+ 'fields_config': protocol layer field behavior(optional)
+ 'stream_config': stream transmit behavior
+ 'flow_control': port flow control(optional)
+
+pcap
+++++
+It is a network packet format. It can be a absolute path of pcap file or
+an instance of scapy Packet. It should only contain one packet format.
+
+.. code-block:: python
+
+ Example 1:
+
+ pcap = <Ether dst=FF:FF:FF:FF:FF:FF src=00:00:00:00:00:00 type=IPv4 |<IP frag=0 proto=udp src=0.0.0.1 dst=0.0.0.255 |<UDP sport=22 dport=50 |<Raw load='xxxxxxxxxxxxxxxxxx' |>>>>
+
+ Example 2:
+
+ pcap = "/root/xxx.pcap"
+
+field option
+++++++++++++
+define every layer's field behavior.
+
+'mac'
+`````
+'mac' is ethernet protocol layer name.
+
+.. code-block:: python
+
+ # field name
+ 'src': {
+ # action: inc/dec/random
+ 'action':'inc',
+ # field end value should be bigger than field start value
+ 'end': '00:00:00:00:00:FF',
+ # field start value
+ 'start': '00:00:00:00:00:02',
+ # field value vary step
+ 'step': 1},
+ 'dst': {
+ # action: inc/dec/random
+ 'action':'inc',
+ # field end value should be bigger than field start value
+ 'end': 'ff:00:00:00:00:FF',
+ # field start value
+ 'start': 'ff:00:00:00:00:02',
+ # field value vary step
+ 'step': 1},
+
+'ip'
+````
+'ip' is ip protocol layer name.
+
+.. code-block:: python
+
+ # field name
+ 'src': {
+ # action: inc/dec/random
+ 'action': 'inc',
+ # field end value should be bigger than field start value
+ 'end': '16.0.0.16',
+ # field start value
+ 'start': '16.0.0.1',
+ # field value vary step
+ 'step': 1},
+ # field name
+ 'dst': {
+ # action: inc/dec/random
+ 'action': 'inc',
+ # field end value should be bigger than field start value
+ 'end': '48.0.0.255',
+ # field start value
+ 'start': '48.0.0.1',
+ # field value vary step
+ 'step': 1},
+
+'vlan'
+``````
+'vlan' is vlan protocol layer name.
+
+.. code-block:: python
+
+ # internal vlan
+ 0: {
+ # action: inc/dec/random
+ 'action': 'inc',
+ # field end value should be bigger than field start value
+ 'end': 52,
+ # field start value
+ 'start': 50,
+ # field value vary step
+ 'step': 1},
+ # external vlan
+ 1: {
+ # action: inc/dec/random
+ 'action': 'inc',
+ # field end value should be bigger than field start value
+ 'end': 52,
+ # field start value
+ 'start': 50,
+ # field value vary step
+ 'step': 1},
+
+'stream_config'
++++++++++++++++
+define a stream transmit behavior.
+
+basic content including::
+
+ 'rate': 0 ~ 100 int type
+ 'transmit_mode': TRANSMIT_CONT/TRANSMIT_S_BURST
+ TRANSMIT_CONT define a continuous transmit.
+ TRANSMIT_S_BURST define a burst transmit with custom number of packets.
+
+.. code-block:: python
+
+ from pktgen_base import TRANSMIT_CONT, TRANSMIT_S_BURST
+
+ stream_config = {
+ 'rate': 100,
+ # TRANSMIT_CONT define a continuous transmit.
+ # TRANSMIT_S_BURST define a burst transmit with custom number of packets.
+ 'transmit_mode': TRANSMIT_CONT
+ }
+
+stream option examples
+----------------------
+
+normal stream option
+++++++++++++++++++++
+normal stream ignore `fields_config` configuration option.
+
+.. code-block:: python
+
+ Example 1:
+ option = {
+ 'pcap': "/root/xxx.pcap",
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT}}
+
+ Example 2:
+ option = {
+ 'pcap': <Ether dst=00:00:00:00:20:00 src=00:00:00:00:00:FF type=IPv4 |<IP frag=0 proto=udp src=0.0.0.1 dst=0.0.0.255 |<UDP sport=22 dport=50 |<Raw load='xxxxxxxxxxxxxxxxxx' |>>>>,
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT}}
+
+stream option with mac increase/decrease/random
++++++++++++++++++++++++++++++++++++++++++++++++
+
+.. code-block:: python
+
+ action = 'inc' or 'dec' or 'random'
+ option = {
+ 'pcap': "/root/xxx.pcap",
+ 'fields_config': {
+ 'mac': {
+ 'dst': {
+ 'action': action,
+ 'end': '00:00:00:00:20:00',
+ 'start': '00:00:00:00:00:FF',
+ 'step': 1},
+ 'src': {
+ 'action': action,
+ 'end': '00:00:00:00:00:FF',
+ 'start': '00:00:00:00:00:02',
+ 'step': 1}}},
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT
+ }
+ }
+
+stream option with ip increase/decrease/random
+++++++++++++++++++++++++++++++++++++++++++++++
+
+.. code-block:: python
+
+ action = 'inc' or 'dec' or 'random'
+ option = {
+ 'pcap': "/root/xxx.pcap",
+ 'fields_config': {
+ 'ip': {
+ 'dst': {
+ 'action': action,
+ 'end': '48.0.0.255',
+ 'start': '48.0.0.1',
+ 'step': 1},
+ 'src': {
+ 'action': action,
+ 'end': '16.0.0.16',
+ 'start': '16.0.0.1',
+ 'step': 1}}},
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT,
+ }
+ }
+
+stream option with vlan increase/decrease/random
+++++++++++++++++++++++++++++++++++++++++++++++++
+
+.. code-block:: python
+
+ action = 'inc' or 'dec' or 'random'
+ option = {
+ 'pcap': "/root/xxx.pcap",
+ 'fields_config': {
+ 'ip': {
+ 0: {
+ 'action': action,
+ 'end': 55,
+ 'start': 50,
+ 'step': 1},
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT,
+ }
+ }
+
+burst stream option
++++++++++++++++++++
+
+.. code-block:: python
+
+ option = {
+ 'pcap': "/root/xxx.pcap",
+ 'stream_config': {
+ 'rate': 100,
+ # set stream transmit mode
+ 'transmit_mode': TRANSMIT_S_BURST,
+ 'txmode' : {
+ # total packets
+ 'total_pkts': 1000},
+ }
+ }
+
+stream option with flow control
++++++++++++++++++++++++++++++++
+flow control open (trex not supported)
+
+.. code-block:: python
+
+ option = {
+ 'flow_control': {
+ # 0: disable flow control
+ # 1: enable flow control
+ 'flag': 1},
+ 'pcap': "/root/xxx.pcap",
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT}}
+
+measure
+=======
+pktgen measure_xxxx return value is the same as etgen, `measure_xxxx` and
+`measure` are both supported. If traffic option is not set, use default values.
+
+two usage examples of pktgen measure method
+-------------------------------------------
+
+.. code-block:: python
+
+ Example 1:
+
+ from pktgen import getPacketGenerator, PKTGEN_TREX
+
+ hPktgen = getPacketGenerator(tester, PKTGEN_TREX)
+
+ traffic_option = {'rate': 100}
+ hPktgen.measure_throughput(stream_ids, traffic_opt)
+
+ Example 2:
+
+ from pktgen import getPacketGenerator, PKTGEN_TREX
+
+ hPktgen = getPacketGenerator(tester, PKTGEN_TREX)
+
+ traffic_option = {
+ 'method': 'throughput',
+ 'rate': 100
+ }
+ hPktgen.measure(stream_ids, traffic_opt)
+
+throughput
+----------
+throughput testing scenario.
+
+option
+++++++
+.. code-block:: python
+
+ traffic_option = {
+ # test method name, if use `measure_throughput`, ignore this key
+ 'method': 'throughput',
+ # port rate percent
+ 'rate': 100,
+ # transmit lasting time second
+ 'duration': 5}
+
+return value
+++++++++++++
+bps_rx_total: Received bits per second
+pps_rx_total: Received packets per second
+
+.. code-block:: python
+
+ return_value = (bps_rx_total, pps_rx_total)
+
+loss
+----
+loss rate testing scenario.
+
+option
+++++++
+
+.. code-block:: python
+
+ traffic_option = {
+ # test method name, if use `measure_loss`, ignore this key
+ 'method': 'loss',
+ # port rate percent
+ 'rate': 100,
+ # transmit lasting time second
+ 'duration': 5}
+
+return value
+++++++++++++
+
+.. code-block:: python
+
+ loss_stats = (loss_rate, tx_pkts, rx_pkts)
+
+latency
+-------
+latency testing scenario.
+
+option
+++++++
+
+.. code-block:: python
+
+ traffic_option = {
+ # test method name, if use `measure_latency`, ignore this key
+ 'method': 'latency',
+ # port rate percent
+ 'rate': 100,
+ # transmit lasting time second
+ 'duration': 5}
+
+return value
+++++++++++++
+
+.. code-block:: python
+
+ latency_stats = { 'min': 15,
+ 'max': 15,
+ 'average': 15,}
+
+rfc2544 option
+--------------
+single stream & rfc2544
+
+option
+++++++
+
+.. code-block:: python
+
+ traffic_option = {
+ # test method name, if use `measure_rfc2544`, ignore this key
+ 'method': 'rfc2544',
+ # port rate percent at first round testing, 0 ~ 100, default is 100
+ 'rate': 100,
+ # permit packet drop rate
+ 'pdr': 0.001,
+ # port rate percent drop step, 0 ~ 100 , default is 1
+ 'drop_step': 1,
+ # transmit lasting time second
+ 'duration': 5}
+
+return value
+++++++++++++
+
+.. code-block:: python
+
+ loss_stats = (loss_rate, tx_pkts, rx_pkts)
+
+reference example
+=================
+This example show how to use pktgen in suite script. In fact, most scenario are
+more simpler than this. Part of code is pseudo code and it can't be ran directly.
+
+testing scenario::
+
+ create four streams on two links, each link attach two streams. On one link,
+ one stream set mac src increase and packet format is a pcap file, the other
+ stream set ip src random / dst decrease and packet format is a scapy Packet
+ instance. All streams use continuous transmit and run rfc2544 scenario using
+ trex packet generator.
+
+.. code-block:: python
+
+ # import pktgen lib
+ from pktgen import getPacketGenerator, PKTGEN_TREX, TRANSMIT_CONT
+
+ # create a pktgen instance
+ hPktgen = getPacketGenerator(tester, PKTGEN_TREX)
+
+ # create packet
+ pcap1 = <Ether dst=FF:FF:FF:FF:FF:FF src=00:00:00:00:00:00 type=IPv4 |<IP frag=0 proto=udp src=0.0.0.1 dst=0.0.0.255 |<UDP sport=22 dport=50 |<Raw load='xxxxxxxxxxxxxxxxxx' |>>>>
+ pcap2 = "/root/xxx.pcap"
+
+ # attach stream to pktgen
+ stream_ids = []
+ tx_port1 = self.tester.get_local_port(dut_port_index1)
+ rx_port1 = self.tester.get_local_port(dut_port_index2)
+ stream_id_1 = hPktgen.add_stream(tx_port1, rx_port1, pcap1)
+ stream_id_2 = hPktgen.add_stream(tx_port1, rx_port1, pcap2)
+ stream_ids.append(stream_id_1)
+ stream_ids.append(stream_id_2)
+
+ tx_port2 = self.tester.get_local_port(dut_port_index2)
+ rx_port2 = self.tester.get_local_port(dut_port_index1)
+ stream_id_3 = hPktgen.add_stream(tx_port2, rx_port2, pcap1)
+ stream_id_4 = hPktgen.add_stream(tx_port2, rx_port2, pcap2)
+ stream_ids.append(stream_id_3)
+ stream_ids.append(stream_id_4)
+
+ # set pcap1 with mac protocol layer field vary configuration
+ stream_option1 = {
+ 'pcap': pcap1,
+ 'fields_config': {
+ 'mac': {
+ 'src': {
+ 'action': 'inc',
+ 'end': '00:00:00:00:00:FF',
+ 'start': '00:00:00:00:00:00',
+ 'step': 1}}},
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT
+ }
+ }
+ # set stream option
+ hPktgen.config_stream(stream_id_1, stream_option1)
+ hPktgen.config_stream(stream_id_3, stream_option1)
+
+ # set pcap2 with ip protocol layer field vary configuration
+ stream_option2 = {
+ 'pcap': pcap2,
+ 'fields_config': {
+ 'ip': {
+ 'dst': {
+ 'action': 'dec',
+ 'end': '0.0.0.255',
+ 'start': '0.0.0.1',
+ 'step': 1},
+ 'src': {
+ 'action': 'random',
+ 'end': '0.0.0.64',
+ 'start': '0.0.0.1',
+ 'step': 1}}},
+ 'stream_config': {
+ 'rate': 100,
+ 'transmit_mode': TRANSMIT_CONT
+ }
+ }
+ # set stream option
+ hPktgen.config_stream(stream_id_2, stream_option2)
+ hPktgen.config_stream(stream_id_4, stream_option2)
+
+ # run testing scenario
+ traffic_option = {
+ 'method': 'rfc2544',
+ 'rate': 100,
+ 'pdr': 0.001,
+ 'drop_step': 1}
+
+ hPktgen.measure(stream_ids, traffic_opt)
\ No newline at end of file
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 6/14] framework/pktgen: packet generator configure file parse
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (4 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 5/14] doc/pktgen: pktgen api program guide document yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 7/14] framework/pktgen: initialize pktgen logger yufengmx
` (8 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
class
packet generator configure file parse class
Class PktgenConf is used to parse config file pktgen.cfg. Ixia and trex
configuration content can be set in different section. Section name is case insensitive.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/config.py | 107 +++++++++++++++++++++++++++++++++++-----------------
1 file changed, 72 insertions(+), 35 deletions(-)
diff --git a/framework/config.py b/framework/config.py
index fb23c6e..abb1469 100644
--- a/framework/config.py
+++ b/framework/config.py
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -35,8 +35,9 @@ Generic port and crbs configuration file load function
import os
import re
import ConfigParser # config parse module
-import argparse # prase arguments module
-from settings import IXIA, CONFIG_ROOT_PATH, SUITE_SECTION_NAME
+import argparse # parse arguments module
+from settings import (IXIA, PKTGEN, PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA,
+ CONFIG_ROOT_PATH, SUITE_SECTION_NAME)
from settings import load_global_setting, DTS_CFG_FOLDER
from exception import ConfigParseException, VirtConfigParseException, PortConfigParseException
@@ -89,6 +90,7 @@ class UserConf():
paramDict[key] = value
return paramDict
+
class GlobalConf(UserConf):
def __init__(self):
self.global_cfg = {}
@@ -115,6 +117,7 @@ class GlobalConf(UserConf):
return global_cfg
+
class SuiteConf(UserConf):
def __init__(self, suite_name=""):
self.suite_cfg = GlobalConf().load_global_config()
@@ -262,6 +265,7 @@ class CrbsConf(UserConf):
DEF_CRB = {'IP': '', 'board': 'default', 'user': '',
'pass': '', 'tester IP': '', 'tester pass': '',
IXIA: None, 'memory channels': 4,
+ PKTGEN: None,
'bypass core0': True}
def __init__(self, crbs_conf=CRBCONF):
@@ -305,6 +309,8 @@ class CrbsConf(UserConf):
if value.lower() == 'none':
value = None
crb[IXIA] = value
+ elif key == 'pktgen_group':
+ crb[PKTGEN] = value.lower()
elif key == 'channels':
crb['memory channels'] = int(value)
elif key == 'bypass_core0':
@@ -381,9 +387,9 @@ class IxiaConf(UserConf):
class PktgenConf(UserConf):
- def __init__(self, pktgen_type='dpdk', pktgen_conf=PKTGENCONF):
+ def __init__(self, pktgen_type='ixia', pktgen_conf=PKTGENCONF):
self.config_file = pktgen_conf
- self.pktgen_type = pktgen_type
+ self.pktgen_type = pktgen_type.lower()
self.pktgen_cfg = {}
try:
self.pktgen_conf = UserConf(self.config_file)
@@ -391,45 +397,76 @@ class PktgenConf(UserConf):
self.pktgen_conf = None
raise ConfigParseException
+ def load_pktgen_ixia_config(self, section):
+ port_reg = r'card=(\d+),port=(\d+)'
+ pktgen_confs = self.pktgen_conf.load_section(section)
+ if not pktgen_confs:
+ return
+ # convert file configuration to dts ixiacfg
+ ixia_group = {}
+ for conf in pktgen_confs:
+ key, value = conf
+ if key == 'ixia_version':
+ ixia_group['Version'] = value
+ elif key == 'ixia_ip':
+ ixia_group['IP'] = value
+ elif key == 'ixia_ports':
+ ports = self.pktgen_conf.load_config(value)
+ ixia_ports = []
+ for port in ports:
+ m = re.match(port_reg, port)
+ if m:
+ ixia_port = {}
+ ixia_port["card"] = int(m.group(1))
+ ixia_port["port"] = int(m.group(2))
+ ixia_ports.append(ixia_port)
+ ixia_group['Ports'] = ixia_ports
+ elif key == 'ixia_enable_rsfec':
+ ixia_group['enable_rsfec'] = value
+
+ if 'Version' not in ixia_group:
+ print 'ixia configuration file request ixia_version option!!!'
+ return
+ if 'IP' not in ixia_group:
+ print 'ixia configuration file request ixia_ip option!!!'
+ return
+ if 'Ports' not in ixia_group:
+ print 'ixia configuration file request ixia_ports option!!!'
+ return
+
+ self.pktgen_cfg[section.lower()] = ixia_group
+
def load_pktgen_config(self):
sections = self.pktgen_conf.get_sections()
if not sections:
return self.pktgen_cfg
for section in sections:
- if self.pktgen_type=='dpdk':
- if section == 'PKTGEN DPDK':
- pktgen_confs = self.pktgen_conf.load_section(section)
- if not pktgen_confs:
- continue
-
- # covert file configuration to dts pktgen cfg
- for conf in pktgen_confs:
- key, value = conf
- self.pktgen_cfg[key] = value
- elif self.pktgen_type=='trex':
- if section == 'TREX':
- pktgen_confs = self.pktgen_conf.load_section(section)
- if not pktgen_confs:
- continue
-
- # covert file configuration to dts pktgen cfg
- for conf in pktgen_confs:
- key, value = conf
- self.pktgen_cfg[key] = value
- elif self.pktgen_type=='ixia':
- if section == 'IXIA':
- pktgen_confs = self.pktgen_conf.load_section(section)
- if not pktgen_confs:
- continue
-
- # covert file configuration to dts pktgen cfg
- for conf in pktgen_confs:
- key, value = conf
- self.pktgen_cfg[key] = value
+ if self.pktgen_type == PKTGEN_DPDK and section.lower() == PKTGEN_DPDK:
+ pktgen_confs = self.pktgen_conf.load_section(section)
+ if not pktgen_confs:
+ continue
+
+ # covert file configuration to dts pktgen cfg
+ for conf in pktgen_confs:
+ key, value = conf
+ self.pktgen_cfg[key] = value
+ elif self.pktgen_type == PKTGEN_TREX and section.lower() == PKTGEN_TREX:
+ pktgen_confs = self.pktgen_conf.load_section(section)
+ if not pktgen_confs:
+ continue
+
+ # covert file configuration to dts pktgen cfg
+ for conf in pktgen_confs:
+ key, value = conf
+ self.pktgen_cfg[key] = value
+ elif self.pktgen_type == PKTGEN_IXIA and section.lower() == PKTGEN_IXIA:
+ # covert file configuration to dts pktgen cfg
+ self.load_pktgen_ixia_config(section)
return self.pktgen_cfg
+
if __name__ == '__main__':
parser = argparse.ArgumentParser(
description="Load DTS configuration files")
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 7/14] framework/pktgen: initialize pktgen logger
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (5 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 6/14] framework/pktgen: packet generator configure file parse yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 8/14] framework/pktgen: packet generator base class yufengmx
` (7 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
initialize pktgen logger
Set packet generator logger name and add check condition for ixia/etgen logger.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/dts.py | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/framework/dts.py b/framework/dts.py
index b6f91b7..7a1200b 100644
--- a/framework/dts.py
+++ b/framework/dts.py
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -63,7 +63,6 @@ import sys
reload(sys)
sys.setdefaultencoding('UTF8')
-
requested_tests = None
result = None
excel_report = None
@@ -244,8 +243,13 @@ def dts_log_testsuite(duts, tester, suite_obj, log_handler, test_classname):
try:
if tester.it_uses_external_generator():
- getattr(tester, 'ixia_packet_gen')
- tester.ixia_packet_gen.logger.config_suite(test_classname, 'ixia')
+ if tester.is_pktgen and \
+ hasattr(tester, 'pktgen') and \
+ getattr(tester, 'pktgen'):
+ tester.pktgen.logger.config_suite(test_classname, 'pktgen')
+ elif hasattr(tester, 'ixia_packet_gen') and \
+ getattr(tester, 'ixia_packet_gen'):
+ tester.ixia_packet_gen.logger.config_suite(test_classname, 'ixia')
except Exception as ex:
pass
@@ -262,8 +266,13 @@ def dts_log_execution(duts, tester, log_handler):
try:
if tester.it_uses_external_generator():
- getattr(tester, 'ixia_packet_gen')
- tester.ixia_packet_gen.logger.config_execution('ixia')
+ if tester.is_pktgen and \
+ hasattr(tester, 'pktgen') and \
+ getattr(tester, 'pktgen'):
+ tester.pktgen.logger.config_execution('pktgen')
+ elif hasattr(tester, 'ixia_packet_gen') and \
+ getattr(tester, 'ixia_packet_gen'):
+ tester.ixia_packet_gen.logger.config_execution('ixia')
except Exception as ex:
pass
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 8/14] framework/pktgen: packet generator base class
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (6 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 7/14] framework/pktgen: initialize pktgen logger yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 9/14] framework/pktgen: ixia packet generator relevant classes yufengmx
` (6 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
packet generator base class
Move class PacketGenerator and stream macro definition from pktgen.py to here.
PacketGenerator is the base class of IxiaPacketGenerator/TrexPacketGenerator.
Add rfc2544/latency/loss measure methods to support more testing scenario. Use
measure() as an unify interface api for all measure methods. Add clear_streams()
to clear streams data managed by pktgen. Use dts port mapping process to take the
place of pktgen port mapping process to reduce redundancy source code. Add
_get_stream() to make subclass possible to get streams option.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen_base.py | 400 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 400 insertions(+)
create mode 100644 framework/pktgen_base.py
diff --git a/framework/pktgen_base.py b/framework/pktgen_base.py
new file mode 100644
index 0000000..e9d3fcb
--- /dev/null
+++ b/framework/pktgen_base.py
@@ -0,0 +1,400 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import os
+import time
+import logging
+from abc import abstractmethod
+from copy import deepcopy
+from logger import getLogger
+from pprint import pformat
+
+from config import PktgenConf
+# packet generator name
+from settings import PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA, PKTGEN
+
+# macro definition
+TRANSMIT_CONT = 'continuous'
+TRANSMIT_M_BURST = 'multi_burst'
+TRANSMIT_S_BURST = 'single_burst'
+# set logger
+FORMAT = '%(message)s'
+logging.basicConfig(format=FORMAT)
+logger = logging.getLogger(os.path.basename(__file__)[:-3].upper())
+logger.setLevel(logging.INFO)
+
+
+class PacketGenerator(object):
+ """
+ Basic class for packet generator, define basic function for each kinds of
+ generators
+ """
+ def __init__(self, tester):
+ self.logger = getLogger(PKTGEN)
+ self.tester = tester
+ self.__streams = []
+ self._ports_map = []
+
+ def prepare_generator(self):
+ self._prepare_generator()
+
+ def _convert_pktgen_port(self, port_id):
+ '''
+ :param port_id:
+ index of a port in packet generator tool
+ '''
+ try:
+ gen_pci = self._get_port_pci(port_id)
+ if not gen_pci:
+ msg = "can't get port {0} pci address".format(port_id)
+ raise Exception(msg)
+ for port_idx, info in enumerate(self.tester.ports_info):
+ if 'pci' not in info or info['pci'] == 'N/A':
+ return -1
+ tester_pci = info['pci']
+ if tester_pci == gen_pci:
+ msg = "gen port {0} map test port {1}".format(
+ port_id, port_idx)
+ self.logger.info(msg)
+ return port_idx
+ else:
+ port = -1
+ except:
+ port = -1
+
+ return port
+
+ def _convert_tester_port(self, port_id):
+ '''
+ :param port_id:
+ index of a port in dts tester ports info
+ '''
+ try:
+ info = self.tester.ports_info[port_id]
+ # limit to nic port, not including ixia port
+ if 'pci' not in info or info['pci'] == 'N/A':
+ return -1
+ tester_pci = info['pci']
+ port = self._get_gen_port(tester_pci)
+ msg = "test port {0} map gen port {1}".format(port_id, port)
+ self.logger.info(msg)
+ except:
+ port = -1
+
+ return port
+
+ def add_stream(self, tx_port, rx_port, pcap_file):
+ pktgen_tx_port = self._convert_tester_port(tx_port)
+ pktgen_rx_port = self._convert_tester_port(rx_port)
+
+ stream_id = len(self.__streams)
+ stream = {'tx_port': pktgen_tx_port,
+ 'rx_port': pktgen_rx_port,
+ 'pcap_file': pcap_file}
+ self.__streams.append(stream)
+
+ return stream_id
+
+ def add_streams(self, streams):
+ '''' a group of streams '''
+ raise NotImplementedError
+
+ def config_stream(self, stream_id=0, opts={}):
+ if self._check_options(opts) is not True:
+ self.logger.error("Failed to configure stream[%d]" % stream_id)
+ return
+ stream = self.__streams[stream_id]
+ stream['options'] = opts
+
+ def config_streams(self, stream_ids, nic, frame_size, port_num):
+ ''' all streams using the default option '''
+ raise NotImplementedError
+
+ def get_streams(self):
+ return self.__streams
+
+ def clear_streams(self):
+ ''' clear streams '''
+ self._clear_streams()
+ self.__streams = []
+
+ def _set_stream_rate_percent(self, rate_percent):
+ ''' set all streams' rate percent '''
+ if not self.__streams:
+ return
+ for stream in self.__streams:
+ stream['rate'] = rate_percent
+
+ def _set_stream_pps(self, pps):
+ ''' set all streams' pps '''
+ if not self.__streams:
+ return
+ for stream in self.__streams:
+ stream['pps'] = pps
+
+ def reset_streams(self):
+ self.__streams = []
+
+ def measure_throughput(self, stream_ids=[], options={}):
+ """
+ Measure throughput on each tx ports
+ """
+ bps_rx = []
+ pps_rx = []
+ self._prepare_transmission(stream_ids=stream_ids)
+ self._start_transmission(stream_ids)
+
+ delay = options.get('delay') or 5
+ time.sleep(delay)
+ used_rx_port = []
+ for stream_id in stream_ids:
+ if self.__streams[stream_id]['rx_port'] not in used_rx_port:
+ rxbps_rates, rxpps_rates = self._retrieve_port_statistic(
+ stream_id, 'throughput')
+ used_rx_port.append(self.__streams[stream_id]['rx_port'])
+ bps_rx.append(rxbps_rates)
+ pps_rx.append(rxpps_rates)
+ self._stop_transmission(stream_id)
+
+ bps_rx_total = self._summary_statistic(bps_rx)
+ pps_rx_total = self._summary_statistic(pps_rx)
+ self.logger.info("throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total))
+
+ return bps_rx_total, pps_rx_total
+
+ def _measure_loss(self, stream_ids=[], options={}):
+ """
+ Measure lost rate on each tx/rx ports
+ """
+ self._prepare_transmission(stream_ids=stream_ids)
+ self._start_transmission(stream_ids, options)
+ self._stop_transmission(None)
+ result = {}
+ used_rx_port = []
+ for stream_id in stream_ids:
+ port_id = self.__streams[stream_id]['rx_port']
+ if port_id in used_rx_port:
+ continue
+ stats = self._retrieve_port_statistic(stream_id, 'loss')
+ tx_pkts, rx_pkts = stats
+ lost_p = tx_pkts - rx_pkts
+ if tx_pkts <= 0:
+ loss_rate = 0
+ else:
+ loss_rate = float(lost_p) / float(tx_pkts)
+ if loss_rate < 0:
+ loss_rate = 0
+ result[port_id] = (loss_rate, tx_pkts, rx_pkts)
+ return result
+
+ def measure_loss(self, stream_ids=[], options={}):
+ result = self._measure_loss(stream_ids, options)
+ # here only to make sure that return value is the same as dts/etgen format
+ # In real testing scenario, this method can offer more data than it
+ return result.values()[0]
+
+ def measure_latency(self, stream_ids=[], options={}):
+ """
+ Measure latency on each tx/rx ports
+ """
+ self._prepare_transmission(stream_ids=stream_ids, latency=True)
+ self._start_transmission(stream_ids, options)
+ self._stop_transmission(None)
+
+ result = {}
+ used_rx_port = []
+ for stream_id in stream_ids:
+ port_id = self.__streams[stream_id]['rx_port']
+ if port_id in used_rx_port:
+ continue
+ stats = self._retrieve_port_statistic(stream_id, 'latency')
+ result[port_id] = stats
+ self.logger.info(result)
+
+ return result
+
+ def _check_loss_rate(self, result, permit_loss_rate):
+ '''
+ support multiple link peer, if any link peer loss rate happen set
+ return value to False
+ '''
+ for port_id, _result in result.iteritems():
+ loss_rate, _, _ = _result
+ if loss_rate > permit_loss_rate:
+ return False
+ else:
+ return True
+
+ def measure_rfc2544(self, stream_ids=[], options={}):
+ """ check loss rate with rate percent dropping """
+ loss_rate_table = []
+ rate_percent = float(100)
+ permit_loss_rate = options.get('pdr') or 0
+ self.logger.info("allow loss rate: %f " % permit_loss_rate)
+ rate_step = options.get('drop_step') or 1
+ result = self._measure_loss(stream_ids, options)
+ status = self._check_loss_rate(result, permit_loss_rate)
+ loss_rate_table.append(
+ [options.get('rate') or rate_percent, result])
+ # if first time loss rate is ok, ignore left flow
+ if status:
+ # return data is the same with dts/etgen format
+ # In fact, multiple link peer have multiple loss rate value,
+ # here only pick one
+ return result.values()[0]
+ _options = deepcopy(options)
+ while not status and rate_percent > 0:
+ rate_percent = rate_percent - rate_step
+ if rate_percent <= 0:
+ msg = "rfc2544 run under zero rate"
+ self.logger.warning(msg)
+ break
+ # set stream rate percent to custom value
+ self._set_stream_rate_percent(rate_percent)
+ # run loss rate testing
+ result = self._measure_loss(stream_ids, _options)
+ loss_rate_table.append([rate_percent, result])
+ status = self._check_loss_rate(result, permit_loss_rate)
+ self.logger.info(pformat(loss_rate_table))
+ self.logger.info("zero loss rate percent is %f" % rate_percent)
+ # use last result as return data to keep the same with dts/etgen format
+ # In fact, multiple link peer have multiple loss rate value,
+ # here only pick one
+ return loss_rate_table[-1][1].values()[0]
+
+ def measure_rfc2544_with_pps(self, stream_ids=[], options={}):
+ """
+ check loss rate with pps bisecting.(not implemented)
+
+ Currently, ixia/trex use rate percent to control port flow rate,
+ pps not supported.
+ """
+ max_pps = options.get('max_pps')
+ min_pps = options.get('min_pps')
+ step = options.get('step') or 10000
+ permit_loss_rate = options.get('permit_loss_rate') or 0.0001
+ # traffic parameters
+ loss_pps_table = []
+ pps = traffic_pps_max = max_pps
+ traffic_pps_min = min_pps
+
+ while True:
+ # set stream rate percent to custom value
+ self._set_stream_pps(pps)
+ # run loss rate testing
+ _options = deepcopy(options)
+ result = self._measure_loss(stream_ids, _options)
+ loss_pps_table.append([pps, result])
+ status = self._check_loss_rate(result, permit_loss_rate)
+ if status:
+ traffic_pps_max = pps
+ else:
+ traffic_pps_min = pps
+ if traffic_pps_max - traffic_pps_min < step:
+ break
+ pps = (traffic_pps_max - traffic_pps_min)/2 + traffic_pps_min
+
+ self.logger.info("zero loss pps is %f" % last_no_lost_mult)
+ # use last result as return data to keep the same with dts/etgen format
+ # In fact, multiple link peer have multiple loss rate value,
+ # here only pick one
+ return loss_pps_table[-1][1].values()[0]
+
+ def measure(self, stream_ids, traffic_opt):
+ '''
+ use as an unify interface method for packet generator
+ '''
+ method = traffic_opt.get('method')
+ if method == 'throughput':
+ result = self.measure_throughput(stream_ids, traffic_opt)
+ elif method == 'latency':
+ result = self.measure_latency(stream_ids, traffic_opt)
+ elif method == 'loss':
+ result = self.measure_loss(stream_ids, traffic_opt)
+ elif method == 'rfc2544':
+ result = self.measure_rfc2544(stream_ids, traffic_opt)
+ elif method == 'rfc2544_with_pps':
+ result = self.measure_rfc2544_with_pps(stream_ids, traffic_opt)
+ else:
+ result = None
+
+ self.logger.info(result)
+
+ return result
+
+ def _summary_statistic(self, array=[]):
+ """
+ Summary all values in statistic array
+ """
+ summary = 0.000
+ for value in array:
+ summary += value
+
+ return summary
+
+ def _get_stream(self, stream_id):
+ return self.__streams[stream_id]
+
+ def _get_generator_conf_instance(self):
+ conf_inst = PktgenConf(self.pktgen_type)
+ pktgen_inst_type = conf_inst.pktgen_conf.get_sections()
+ if len(pktgen_inst_type) < 1:
+ msg = ("packet generator <{0}> has no configuration "
+ "in pktgen.cfg").format(self.pktgen_type)
+ raise Exception(msg)
+ return conf_inst
+
+ @abstractmethod
+ def _prepare_transmission(self, stream_ids=[], latency=False):
+ pass
+
+ @abstractmethod
+ def _start_transmission(self, stream_ids, options={}):
+ pass
+
+ @abstractmethod
+ def _stop_transmission(self, stream_id):
+ pass
+
+ @abstractmethod
+ def _retrieve_port_statistic(self, stream_id, mode):
+ pass
+
+ @abstractmethod
+ def _check_options(self, opts={}):
+ pass
+
+ @abstractmethod
+ def quit_generator(self):
+ pass
+
+
+class DpdkPacketGenerator(PacketGenerator): pass # not implemented
\ No newline at end of file
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 9/14] framework/pktgen: ixia packet generator relevant classes
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (7 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 8/14] framework/pktgen: packet generator base class yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 0/14] framework/pktgen: trex " yufengmx
` (5 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
ixia packet generator relevant classes
Use etgen class IxiaPacketGenerator as ixia tool class and rename it to Ixia.
Use IxiaPacketGenerator as PacketGenerator subclass to wrap ixia tool behavior to
a series of unified interfaces as suite script API. Class Ixia add protocol layer fields
behavior setting process, unify statistics return value format and add some methods
to be compatible with pktgen design.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen_ixia.py | 1750 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 1750 insertions(+)
create mode 100644 framework/pktgen_ixia.py
diff --git a/framework/pktgen_ixia.py b/framework/pktgen_ixia.py
new file mode 100644
index 0000000..80481fe
--- /dev/null
+++ b/framework/pktgen_ixia.py
@@ -0,0 +1,1750 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+import os
+import re
+import string
+import time
+from pprint import pformat
+
+from ssh_connection import SSHConnection
+from settings import SCAPY2IXIA
+from logger import getLogger
+from utils import (convert_int2ip, convert_ip2int,
+ convert_mac2long, convert_mac2str)
+
+from pktgen_base import (PacketGenerator, PKTGEN_IXIA,
+ TRANSMIT_CONT, TRANSMIT_M_BURST, TRANSMIT_S_BURST)
+
+from scapy.packet import Packet
+from scapy.utils import wrpcap
+
+
+class Ixia(SSHConnection):
+ """
+ IXIA performance measurement class.
+ """
+
+ def __init__(self, tester, ixiaPorts):
+ self.tester = tester
+ self.NAME = PKTGEN_IXIA
+ self.logger = getLogger(self.NAME)
+ super(Ixia, self).__init__(
+ self.get_ip_address(),
+ self.NAME,
+ self.tester.get_username(),
+ self.get_password())
+ super(Ixia, self).init_log(self.logger)
+
+ self.tcl_cmds = []
+ self.chasId = None
+ self.conRelation = {}
+
+ ixiaRef = self.NAME
+ if ixiaRef is None or ixiaRef not in ixiaPorts:
+ return
+
+ self.ixiaVersion = ixiaPorts[ixiaRef]["Version"]
+ self.ports = ixiaPorts[ixiaRef]["Ports"]
+
+ if ixiaPorts[ixiaRef].has_key('force100g'):
+ self.enable100g = ixiaPorts[ixiaRef]['force100g']
+ else:
+ self.enable100g = 'disable'
+
+ self.logger.info(self.ixiaVersion)
+ self.logger.info(self.ports)
+
+ self.tclServerIP = ixiaPorts[ixiaRef]["IP"]
+
+ # prepare tcl shell and ixia library
+ self.send_expect("tclsh", "% ")
+ self.send_expect("source ./IxiaWish.tcl", "% ")
+ self.send_expect("set ::env(IXIA_VERSION) %s" % self.ixiaVersion, "% ")
+ out = self.send_expect("package req IxTclHal", "% ")
+ self.logger.debug("package req IxTclHal return:" + out)
+ if self.ixiaVersion in out:
+ if not self.tcl_server_login():
+ self.close()
+ self.session = None
+ for port in self.ports:
+ port['speed'] = self.get_line_rate(self.chasId, port)
+ # ixia port stream management table
+ self.stream_index = {}
+ self.stream_total = {}
+
+ def get_line_rate(self, chasid, port):
+ ixia_port = "%d %d %d" % (self.chasId, port['card'], port['port'])
+ return self.send_expect("stat getLineSpeed %s" % ixia_port, '%')
+
+ def get_ip_address(self):
+ return self.tester.get_ip_address()
+
+ def get_password(self):
+ return self.tester.get_password()
+
+ def add_tcl_cmd(self, cmd):
+ """
+ Add one tcl command into command list.
+ """
+ self.tcl_cmds.append(cmd)
+
+ def add_tcl_cmds(self, cmds):
+ """
+ Add one tcl command list into command list.
+ """
+ self.tcl_cmds += cmds
+
+ def clean(self):
+ """
+ Clean ownership of IXIA devices and logout tcl session.
+ """
+ self.send_expect("clearOwnershipAndLogout", "% ")
+ self.close()
+
+ def parse_pcap(self, fpcap):
+ # save Packet instance to pcap file
+ if isinstance(fpcap, Packet):
+ pcap_path = '/root/temp.pcap'
+ if os.path.exists(pcap_path):
+ os.remove(pcap_path)
+ wrpcap(pcap_path, fpcap)
+ else:
+ pcap_path = fpcap
+
+ dump_str1 = "cmds = []\n"
+ dump_str2 = "for i in rdpcap('%s', -1):\n" % pcap_path
+ dump_str3 = " if 'Vxlan' in i.command():\n" + \
+ " vxlan_str = ''\n" + \
+ " l = len(i[Vxlan])\n" + \
+ " vxlan = str(i[Vxlan])\n" + \
+ " first = True\n" + \
+ " for j in range(l):\n" + \
+ " if first:\n" + \
+ " vxlan_str += \"Vxlan(hexval='%02X\" %ord(vxlan[j])\n" + \
+ " first = False\n" + \
+ " else:\n" + \
+ " vxlan_str += \" %02X\" %ord(vxlan[j])\n" + \
+ " vxlan_str += \"\')\"\n" + \
+ " command = re.sub(r\"Vxlan(.*)\", vxlan_str, i.command())\n" + \
+ " else:\n" + \
+ " command = i.command()\n" + \
+ " cmds.append(command)\n" + \
+ "print(cmds)\n" + \
+ "exit()"
+
+ f = open("dumppcap.py", "w")
+ f.write(dump_str1)
+ f.write(dump_str2)
+ f.write(dump_str3)
+ f.close()
+
+ self.session.copy_file_to("dumppcap.py")
+ out = self.send_expect("scapy -c dumppcap.py 2>/dev/null", "% ", 120)
+ flows = eval(out)
+ return flows
+
+ def macToTclFormat(self, macAddr):
+ """
+ Convert normal mac address format into IXIA's format.
+ """
+ macAddr = macAddr.upper()
+ return "%s %s %s %s %s %s" % (
+ macAddr[:2], macAddr[3:5], macAddr[6:8],
+ macAddr[9:11], macAddr[12:14], macAddr[15:17])
+
+ def set_ether_fields(self, fields, default_fields):
+ """
+ Configure Ether protocol field value.
+ """
+ addr_mode = {
+ # decrement the MAC address for as many numSA/numDA specified
+ 'dec': 'decrement',
+ # increment the MAC address for as many numSA/numDA specified
+ 'inc': 'increment',
+ # Generate random destination MAC address for each frame
+ 'random': 'ctrRandom',
+ # set RepeatCounter mode to be idle as default
+ 'default': 'idle'}
+
+ cmds = []
+ for name, config in fields.iteritems():
+ default_config = default_fields.get(name)
+ mac_start = config.get('start') or default_config.get('start')
+ mac_end = config.get('end')
+ step = config.get('step') or 1
+ action = config.get('action') or default_config.get('action')
+ prefix = 'sa' if name == 'src' else 'da'
+ if action == 'dec' and mac_end:
+ cmds.append('stream config -{0} "{1}"'.format(
+ prefix, mac_end))
+ else:
+ cmds.append('stream config -{0} "{1}"'.format(
+ prefix, mac_start))
+ if step:
+ cmds.append('stream config -{0}Step {1}'.format(prefix, step))
+ if action:
+ cmds.append('stream config -{0}RepeatCounter {1}'.format(
+ prefix, addr_mode.get(action)))
+ if mac_end:
+ mac_start_int = convert_mac2long(mac_start)
+ mac_end_int = convert_mac2long(mac_end)
+ flow_num = mac_end_int - mac_start_int + 1
+ if flow_num <= 0:
+ msg = "end mac should not be bigger than start mac"
+ raise Exception(msg)
+ else:
+ flow_num = None
+
+ if flow_num:
+ cmds.append('stream config -num{0} {1}'.format(
+ prefix.upper(), flow_num))
+ # clear default field after it has been set
+ default_fields.pop(name)
+ # if some filed not set, set it here
+ if default_fields:
+ for name, config in default_fields.iteritems():
+ ip_start = config.get('start')
+ prefix = 'sa' if name == 'src' else 'da'
+ cmds.append('stream config -{0} "{1}"'.format(prefix, ip_start))
+
+ return cmds
+
+ def ether(self, port, vm, src, dst, type):
+ """
+ Configure Ether protocol.
+ """
+ fields = vm.get('mac')
+ srcMac = self.macToTclFormat(src)
+ dstMac = self.macToTclFormat(dst)
+ # common command setting
+ self.add_tcl_cmd("protocol config -ethernetType ethernetII")
+ cmds = []
+ # if vm has been set, pick pcap fields' as default value
+ if fields:
+ default_fields = {
+ 'src': { 'action': 'default', 'start': src,},
+ 'dst': { 'action': 'default', 'start': dst,},}
+ # set custom setting for field actions
+ cmds = self.set_ether_fields(fields, default_fields)
+ # set them in tcl commands group
+ self.add_tcl_cmds(cmds)
+ else:
+ self.add_tcl_cmd('stream config -sa "%s"' % srcMac)
+ self.add_tcl_cmd('stream config -da "%s"' % dstMac)
+
+ def set_ip_fields(self, fields, default_fields):
+ addr_mode = {
+ # increment the host portion of the IP address for as many
+ # IpAddrRepeatCount specified
+ 'dec': 'ipDecrHost',
+ # increment the host portion of the IP address for as many
+ # IpAddrRepeatCount specified
+ 'inc': 'ipIncrHost',
+ # Generate random IP addresses
+ 'random': 'ipRandom',
+ # no change to IP address regardless of IpAddrRepeatCount
+ 'idle': 'ipIdle',
+ # set default
+ 'default': 'ipIdle',}
+ cmds = []
+ for name, config in fields.iteritems():
+ default_config = default_fields.get(name)
+ fv_name = 'IP.{0}'.format(name)
+ ip_start = config.get('start') or default_config.get('start')
+ ip_end = config.get('end')
+ if ip_end:
+ ip_start_int = convert_ip2int(ip_start)
+ ip_end_int = convert_ip2int(ip_end)
+ flow_num = ip_end_int - ip_start_int + 1
+ if flow_num <= 0:
+ msg = "end ip address parameter is wrong"
+ raise Exception(msg)
+ else:
+ flow_num = None
+
+ mask = config.get('mask')
+ _step = config.get('step')
+ step = int(_step) if _step and isinstance(_step, (str, unicode)) else \
+ _step or 1
+ action = config.get('action')
+ # get ixia command prefix
+ prefix = 'source' if name == 'src' else 'dest'
+ # set command
+ if action == 'dec' and ip_end:
+ cmds.append('ip config -{0}IpAddr "{1}"'.format(
+ prefix, ip_end))
+ else:
+ cmds.append('ip config -{0}IpAddr "{1}"'.format(
+ prefix, ip_start))
+ if flow_num:
+ cmds.append('ip config -{0}IpAddrRepeatCount {1}'.format(
+ prefix, flow_num))
+
+ cmds.append('ip config -{0}IpAddrMode {1}'.format(
+ prefix, addr_mode.get(action or 'default')))
+
+ if mask:
+ cmds.append("ip config -{0}IpMask '{1}'".format(prefix, mask))
+ # clear default field after it has been set
+ default_fields.pop(name)
+ # if all fields are set
+ if not default_fields:
+ return cmds
+ # if some filed not set, set it here
+ for name, config in default_fields.iteritems():
+ ip_start = config.get('start')
+ prefix = 'source' if name == 'src' else 'dest'
+ cmds.append('ip config -{0}IpAddr "{1}"'.format(prefix, ip_start))
+ cmds.append('ip config -{0}IpAddrMode {1}'.format(
+ prefix, addr_mode.get('default')))
+
+ return cmds
+
+ def ip(self, port, vm, frag, src, proto, tos, dst, chksum, len, version,
+ flags, ihl, ttl, id, options=None):
+ """
+ Configure IP protocol.
+ """
+ fields = vm.get('ip')
+ # common command setting
+ self.add_tcl_cmd("protocol config -name ip")
+ # if fields has been set
+ if fields:
+ # pick pcap fields' as default value
+ default_fields = {
+ 'src': { 'action': 'default', 'start': src,},
+ 'dst': { 'action': 'default', 'start': dst,},}
+ # set custom setting for field actions
+ cmds = self.set_ip_fields(fields, default_fields)
+ # append custom setting
+ self.add_tcl_cmds(cmds)
+ else:
+ self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src)
+ self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst)
+ # common command setting
+ self.add_tcl_cmd("ip config -ttl %d" % ttl)
+ self.add_tcl_cmd("ip config -totalLength %d" % len)
+ self.add_tcl_cmd("ip config -fragment %d" % frag)
+ self.add_tcl_cmd("ip config -ipProtocol {0}".format(proto))
+ self.add_tcl_cmd("ip config -identifier %d" % id)
+ self.add_tcl_cmd("stream config -framesize %d" % (len + 18))
+ # set stream setting in port
+ self.add_tcl_cmd("ip set %s" % port)
+
+ def ipv6(self, port, vm, version, tc, fl, plen, nh, hlim, src, dst):
+ """
+ Configure IPv6 protocol.
+ """
+ self.add_tcl_cmd("protocol config -name ipV6")
+ self.add_tcl_cmd('ipV6 setDefault')
+ self.add_tcl_cmd('ipV6 config -destAddr "%s"' %
+ self.ipv6_to_tcl_format(dst))
+ self.add_tcl_cmd('ipV6 config -sourceAddr "%s"' %
+ self.ipv6_to_tcl_format(src))
+ self.add_tcl_cmd('ipV6 config -flowLabel %d' % fl)
+ self.add_tcl_cmd('ipV6 config -nextHeader %d' % nh)
+ self.add_tcl_cmd('ipV6 config -hopLimit %d' % hlim)
+ self.add_tcl_cmd('ipV6 config -trafficClass %d' % tc)
+ self.add_tcl_cmd("ipV6 clearAllExtensionHeaders")
+ self.add_tcl_cmd("ipV6 addExtensionHeader %d" % nh)
+
+ self.add_tcl_cmd("stream config -framesize %d" % (plen + 40 + 18))
+ self.add_tcl_cmd("ipV6 set %s" % port)
+
+ def udp(self, port, vm, dport, sport, len, chksum):
+ """
+ Configure UDP protocol.
+ """
+ self.add_tcl_cmd("udp setDefault")
+ self.add_tcl_cmd("udp config -sourcePort %d" % sport)
+ self.add_tcl_cmd("udp config -destPort %d" % dport)
+ self.add_tcl_cmd("udp config -length %d" % len)
+ self.add_tcl_cmd("udp set %s" % port)
+
+ def vxlan(self, port, vm, hexval):
+ self.add_tcl_cmd("protocolPad setDefault")
+ self.add_tcl_cmd("protocol config -enableProtocolPad true")
+ self.add_tcl_cmd("protocolPad config -dataBytes \"%s\"" % hexval)
+ self.add_tcl_cmd("protocolPad set %s" % port)
+
+ def tcp(self, port, vm, sport, dport, seq, ack, dataofs, reserved, flags,
+ window, chksum, urgptr, options=None):
+ """
+ Configure TCP protocol.
+ """
+ self.add_tcl_cmd("tcp setDefault")
+ self.add_tcl_cmd("tcp config -sourcePort %d" % sport)
+ self.add_tcl_cmd("tcp config -destPort %d" % dport)
+ self.add_tcl_cmd("tcp set %s" % port)
+
+ def sctp(self, port, vm, sport, dport, tag, chksum):
+ """
+ Configure SCTP protocol.
+ """
+ self.add_tcl_cmd("tcp config -sourcePort %d" % sport)
+ self.add_tcl_cmd("tcp config -destPort %d" % dport)
+ self.add_tcl_cmd("tcp set %s" % port)
+
+ def set_dot1q_fields(self, fields):
+ '''
+ Configure 8021Q protocol field name.
+ '''
+ addr_mode = {
+ # The VlanID tag is decremented by step for repeat number of times
+ 'dec': 'vDecrement',
+ # The VlanID tag is incremented by step for repeat number of times
+ 'inc': 'vIncrement',
+ # Generate random VlanID tag for each frame
+ 'random': 'vCtrRandom',
+ # No change to VlanID tag regardless of repeat
+ 'idle': 'vIdle',}
+ cmds = []
+ for name, config in fields.iteritems():
+ fv_name = '8021Q.{0}'.format(name)
+ vlan_start = config.get('start') or 0
+ vlan_end = config.get('end') or 256
+ if vlan_end:
+ flow_num = vlan_end - vlan_start + 1
+ if flow_num <= 0:
+ msg = "end vlan id parameter is wrong"
+ raise Exception(msg)
+ else:
+ flow_num = None
+ step = config.get('step') or 1
+ action = config.get('action')
+ #------------------------------------------------
+ # set command
+ if step:
+ cmds.append('vlan config -step {0}'.format(step))
+ if flow_num:
+ cmds.append('vlan config -repeat {0}'.format(flow_num))
+ if action:
+ cmds.append('vlan config -mode {0}'.format(
+ addr_mode.get(action)))
+ return cmds
+
+ def dot1q(self, port, vm, prio, id, vlan, type):
+ """
+ Configure 8021Q protocol.
+ """
+ fields = vm.get('vlan')
+ # common command setting
+ self.add_tcl_cmd("protocol config -enable802dot1qTag true")
+ # if fields has been set
+ if fields:
+ # set custom setting for field actions
+ cmds = self.set_dot1q_fields(fields)
+ self.add_tcl_cmds(cmds)
+ self.add_tcl_cmd("vlan config -vlanID %d" % vlan)
+ self.add_tcl_cmd("vlan config -userPriority %d" % prio)
+ # set stream in port
+ self.add_tcl_cmd("vlan set %s" % port)
+
+ def config_stream(self, fpcap, vm, port_index, rate_percent, stream_id=1,
+ latency=False):
+ """
+ Configure IXIA stream and enable multiple flows.
+ """
+ ixia_port = self.get_ixia_port(port_index)
+ flows = self.parse_pcap(fpcap)
+ if not flows:
+ msg = "flow has no format, it should be one."
+ raise Exception(msg)
+ if len(flows) >= 2:
+ msg = "flow contain more than one format, it should be one."
+ raise Exception(msg)
+
+ # set commands at first stream
+ if stream_id == 1:
+ self.add_tcl_cmd("ixGlobalSetDefault")
+ # set burst stream if burst stream is required
+ stream_config = vm.get('stream_config')
+ transmit_mode = stream_config.get('transmit_mode') or TRANSMIT_CONT
+ if transmit_mode == TRANSMIT_S_BURST:
+ cmds = self.config_single_burst_stream(
+ stream_config.get('txmode'), rate_percent)
+ self.add_tcl_cmds(cmds)
+ else:
+ self.config_ixia_stream(
+ rate_percent, self.stream_total.get(port_index), latency)
+
+ pat = re.compile(r"(\w+)\((.*)\)")
+ for flow in flows:
+ for header in flow.split('/'):
+ match = pat.match(header)
+ params = eval('dict(%s)' % match.group(2))
+ method_name = match.group(1)
+ if method_name == 'Vxlan':
+ method = getattr(self, method_name.lower())
+ method(ixia_port, vm.get('fields_config', {}), **params)
+ break
+ if method_name in SCAPY2IXIA:
+ method = getattr(self, method_name.lower())
+ method(ixia_port, vm.get('fields_config', {}), **params)
+ self.add_tcl_cmd("stream set %s %d" % (ixia_port, stream_id))
+ # only use one packet format in pktgen
+ break
+
+ # set commands at last stream
+ if stream_id > 1:
+ self.add_tcl_cmd("stream config -dma gotoFirst")
+ self.add_tcl_cmd("stream set %s %d" % (ixia_port, stream_id))
+
+ def config_single_burst_stream(self, txmode, rate_percent):
+ """ configure burst stream. """
+ gapUnits = {
+ # (default) Sets units of time for gap to nanoseconds
+ 'ns': 'gapNanoSeconds',
+ # Sets units of time for gap to microseconds
+ 'us': 'gapMicroSeconds',
+ # Sets units of time for gap to milliseconds
+ 'm': 'gapMilliSeconds',
+ # Sets units of time for gap to seconds
+ 's': 'gapSeconds',}
+ pkt_count = 1
+ burst_count = txmode.get('total_pkts', 32)
+ frameType = txmode.get('frameType') or {}
+ time_unit = frameType.get('type', 'ns')
+ gapUnit = gapUnits.get(time_unit) \
+ if time_unit in gapUnits.keys() else gapUnits.get('ns')
+ # The inter-stream gap is the delay in clock ticks between stream.
+ # This delay comes after the receive trigger is enabled. Setting this
+ # option to 0 means no delay. (default = 960.0)
+ isg = frameType.get('isg', 100)
+ # The inter-frame gap specified in clock ticks (default = 960.0).
+ ifg = frameType.get('ifg', 100)
+ # Inter-Burst Gap is the delay between bursts of frames in clock ticks
+ # (see ifg option for definition of clock ticks). If the IBG is set to
+ # 0 then the IBG is equal to the ISG and the IBG becomes disabled.
+ # (default = 960.0)
+ ibg = frameType.get('ibg', 100)
+ frame_cmds = [
+ "stream config -rateMode usePercentRate",
+ "stream config -percentPacketRate %s" % rate_percent,
+ "stream config -dma stopStream",
+ "stream config -rateMode useGap",
+ "stream config -gapUnit {0}".format(gapUnit),
+ "stream config -numFrames {0}".format(pkt_count),
+ "stream config -numBursts {0}".format(burst_count),
+ "stream config -ifg {0}".format(ifg),
+ "stream config -ifgType gapFixed",
+# "stream config -enableIbg true", # reserve
+# "stream config -ibg {0}".format(ibg), # reserve
+# "stream config -enableIsg true", # reserve
+# "stream config -isg {0}".format(isg), # reserve
+ "stream config -frameSizeType sizeFixed",]
+
+ return frame_cmds
+
+ def config_ixia_stream(self, rate_percent, total_flows, latency):
+ """
+ Configure IXIA stream with rate and latency.
+ Override this method if you want to add custom stream configuration.
+ """
+ self.add_tcl_cmd("stream config -rateMode usePercentRate")
+ self.add_tcl_cmd("stream config -percentPacketRate %s" % rate_percent)
+ self.add_tcl_cmd("stream config -numFrames 1")
+ if total_flows == 1:
+ self.add_tcl_cmd("stream config -dma contPacket")
+ else:
+ self.add_tcl_cmd("stream config -dma advance")
+ # request by packet Group
+ if latency is not False:
+ self.add_tcl_cmd("stream config -fir true")
+
+ def tcl_server_login(self):
+ """
+ Connect to tcl server and take ownership of all the ports needed.
+ """
+ out = self.send_expect("ixConnectToTclServer %s" % self.tclServerIP,
+ "% ", 30)
+ self.logger.debug("ixConnectToTclServer return:" + out)
+ if out.strip()[-1] != '0':
+ return False
+
+ self.send_expect("ixLogin IxiaTclUser", "% ")
+
+ out = self.send_expect("ixConnectToChassis %s" % self.tclServerIP,
+ "% ", 30)
+ if out.strip()[-1] != '0':
+ return False
+
+ out = self.send_expect(
+ "set chasId [ixGetChassisID %s]" % self.tclServerIP, "% ")
+ self.chasId = int(out.strip())
+
+ self.send_expect("ixClearOwnership [list %s]" % string.join(
+ ['[list %d %d %d]' % (self.chasId, item['card'], item['port'])
+ for item in self.ports], ' '),
+ "% ", 10)
+ self.send_expect("ixTakeOwnership [list %s] force" % string.join(
+ ['[list %d %d %d]' % (self.chasId, item['card'], item['port'])
+ for item in self.ports], ' '),
+ "% ", 10)
+
+ return True
+
+ def tcl_server_logout(self):
+ """
+ Disconnect to tcl server and make sure has been logged out.
+ """
+ self.send_expect("ixDisconnectFromChassis %s" % self.tclServerIP, "%")
+ self.send_expect("ixLogout", "%")
+ self.send_expect("ixDisconnectTclServer %s" % self.tclServerIP, "%")
+
+ def config_port(self, pList):
+ """
+ Configure ports and make them ready for performance validation.
+ """
+ pl = list()
+ for item in pList:
+ ixia_port = "%d %d %d" % (self.chasId, item['card'], item['port'])
+ self.add_tcl_cmd("port setFactoryDefaults %s" % ixia_port)
+ # if the line rate is 100G and we need this port work in 100G mode,
+ # we need to add some configure to make it so.
+ if int(self.get_line_rate(self.chasId, item).strip()) == 100000 and \
+ self.enable100g == 'enable':
+ self.add_tcl_cmd("port config -ieeeL1Defaults 0")
+ self.add_tcl_cmd("port config -autonegotiate false")
+ self.add_tcl_cmd("port config -enableRsFec true")
+ self.add_tcl_cmd("port set %d %d %d" % (
+ self.chasId,item['card'], item['port']))
+
+ pl.append('[list %d %d %d]' % (
+ self.chasId, item['card'], item['port']))
+
+ self.add_tcl_cmd("set portList [list %s]" % string.join(pl, ' '))
+
+ self.add_tcl_cmd("ixClearTimeStamp portList")
+ self.add_tcl_cmd("ixWritePortsToHardware portList")
+ self.add_tcl_cmd("ixCheckLinkState portList")
+
+ def set_ixia_port_list(self, pList):
+ """
+ Implement ports/streams configuration on specified ports.
+ """
+ self.add_tcl_cmd("set portList [list %s]" %
+ string.join(['[list %s]' % ixia_port for ixia_port in pList], ' '))
+
+ def send_ping6(self, pci, mac, ipv6):
+ """
+ Send ping6 packet from IXIA ports.
+ """
+ port = self.pci_to_port(pci)['card']
+ ixia_port = "%d %d %d" % (self.chasId, port['card'], port['port'])
+ self.send_expect("source ./ixTcl1.0/ixiaPing6.tcl", "% ")
+ cmd = 'ping6 "%s" "%s" %s' % (self.ipv6_to_tcl_format(ipv6),
+ self.macToTclFormat(mac), ixia_port)
+ out = self.send_expect(cmd, "% ", 90)
+ return out
+
+ def ipv6_to_tcl_format(self, ipv6):
+ """
+ Convert normal IPv6 address to IXIA format.
+ """
+ ipv6 = ipv6.upper()
+ singleAddr = ipv6.split(":")
+ if '' == singleAddr[0]:
+ singleAddr = singleAddr[1:]
+ if '' in singleAddr:
+ tclFormatAddr = ''
+ addStr = '0:' * (8 - len(singleAddr)) + '0'
+ for i in range(len(singleAddr)):
+ if singleAddr[i] == '':
+ tclFormatAddr += addStr + ":"
+ else:
+ tclFormatAddr += singleAddr[i] + ":"
+ tclFormatAddr = tclFormatAddr[0:len(tclFormatAddr) - 1]
+ return tclFormatAddr
+ else:
+ return ipv6
+
+ def get_ports(self):
+ """
+ API to get ixia ports for dts `ports_info`
+ """
+ plist = list()
+ if self.session is None:
+ return plist
+
+ for p in self.ports:
+ plist.append({'type': 'ixia',
+ 'pci': 'IXIA:%d.%d' % (p['card'], p['port'])})
+ return plist
+
+ def get_ixia_port_pci(self, port_id):
+ ports_info = self.get_ports()
+ pci = ports_info[port_id]['pci']
+ return pci
+
+ def pci_to_port(self, pci):
+ """
+ Convert IXIA fake pci to IXIA port.
+ """
+ ixia_pci_regex = "IXIA:(\d*).(\d*)"
+ m = re.match(ixia_pci_regex, pci)
+ if m is None:
+ msg = "ixia port not found"
+ self.logger.warning(msg)
+ return {'card': -1, 'port': -1}
+
+ return {'card': int(m.group(1)), 'port': int(m.group(2))}
+
+ def get_ixia_port_info(self, port):
+ if port == None or port >= len(self.ports):
+ msg = "<{0}> exceed maximum ixia ports".format(port)
+ raise Exception(msg)
+ pci_addr = self.get_ixia_port_pci(port)
+ port_info = self.pci_to_port(pci_addr)
+ return port_info
+
+ def get_ixia_port(self, port):
+ port_info= self.get_ixia_port_info(port)
+ ixia_port = "%d %d %d" % (self.chasId,
+ port_info['card'], port_info['port'])
+ return ixia_port
+
+ def loss(self, portList, ratePercent, delay=5):
+ """
+ Run loss performance test and return loss rate.
+ """
+ rxPortlist, txPortlist = self._configure_everything(portList,
+ ratePercent)
+ return self.get_loss_packet_rate(rxPortlist, txPortlist, delay)
+
+ def get_loss_packet_rate(self, rxPortlist, txPortlist, delay=5):
+ """
+ Get RX/TX packet statistics and calculate loss rate.
+ """
+ time.sleep(delay)
+
+ self.send_expect("ixStopTransmit portList", "%", 10)
+ time.sleep(2)
+ sendNumber = 0
+ for port in txPortlist:
+ self.stat_get_stat_all_stats(port)
+ sendNumber += self.get_frames_sent()
+ time.sleep(0.5)
+
+ self.logger.info("send :%f" % sendNumber)
+
+ assert sendNumber != 0
+
+ revNumber = 0
+ for port in rxPortlist:
+ self.stat_get_stat_all_stats(port)
+ revNumber += self.get_frames_received()
+ self.logger.info("rev :%f" % revNumber)
+
+ return float(sendNumber - revNumber) / sendNumber, sendNumber, revNumber
+
+ def latency(self, portList, ratePercent, delay=5):
+ """
+ Run latency performance test and return latency statistics.
+ """
+ rxPortlist, txPortlist = self._configure_everything(
+ portList, ratePercent, True)
+ return self.get_packet_latency(rxPortlist)
+
+ def get_packet_latency(self, rxPortlist):
+ """
+ Stop IXIA transmit and return latency statistics.
+ """
+ latencyList = []
+ time.sleep(10)
+ self.send_expect("ixStopTransmit portList", "%", 10)
+ for rx_port in rxPortlist:
+ self.pktGroup_get_stat_all_stats(rx_port)
+ latency = {"port": rx_port,
+ "min": self.get_min_latency(),
+ "max": self.get_max_latency(),
+ "average": self.get_average_latency()}
+ latencyList.append(latency)
+ return latencyList
+
+ def throughput(self, port_list, rate_percent=100, delay=5):
+ """
+ Run throughput performance test and return throughput statistics.
+ """
+ rxPortlist, txPortlist = self._configure_everything(
+ port_list, rate_percent)
+ return self.get_transmission_results(rxPortlist, txPortlist, delay)
+
+ def is_packet_ordered(self, port_list, delay):
+ """
+ This function could be used to check the packets' order whether same as
+ the receive sequence.
+
+ Please notice that this function only support single-stream mode.
+ """
+ port = self.ports[0]
+ ixia_port = "%d %d %d" % (self.chasId, port['card'], port['port'])
+ rxPortlist, txPortlist = self.prepare_port_list(port_list)
+ self.prepare_ixia_for_transmission(txPortlist, rxPortlist)
+ self.send_expect('port config -receiveMode [expr $::portCapture|$::portRxSequenceChecking|$::portRxModeWidePacketGroup]', '%')
+ self.send_expect('port config -autonegotiate true', '%')
+ self.send_expect('ixWritePortsToHardware portList', '%')
+ self.send_expect('set streamId 1', '%')
+ self.send_expect('stream setDefault', '%')
+ self.send_expect('ixStartPortPacketGroups %s' % ixia_port, '%')
+ self.send_expect('ixStartTransmit portList', '%')
+ # wait `delay` seconds to make sure link is up
+ self.send_expect('after 1000 * %d' % delay, '%')
+ self.send_expect('ixStopTransmit portList', '%')
+ self.send_expect('ixStopPortPacketGroups %s' % ixia_port, '%')
+ self.send_expect('packetGroupStats get %s 1 1' % ixia_port, '%')
+ self.send_expect('packetGroupStats getGroup 1', '%')
+ self.send_expect('set reverseSequenceError [packetGroupStats cget -reverseSequenceError]]', '%')
+ output = self.send_expect('puts $reverseSequenceError', '%')
+ return int(output[:-2])
+
+ def _configure_everything(self, port_list, rate_percent, latency=False):
+ """
+ Prepare and configure IXIA ports for performance test.
+ """
+ rxPortlist, txPortlist = self.prepare_port_list(
+ port_list, rate_percent, latency)
+ self.prepare_ixia_for_transmission(txPortlist, rxPortlist)
+ self.configure_transmission()
+ self.start_transmission()
+ self.clear_tcl_commands()
+ return rxPortlist, txPortlist
+
+ def clear_tcl_commands(self):
+ """
+ Clear all commands in command list.
+ """
+ del self.tcl_cmds[:]
+
+ def start_transmission(self):
+ """
+ Run commands in command list.
+ """
+ fileContent = "\n".join(self.tcl_cmds) + "\n"
+ self.tester.create_file(fileContent, 'ixiaConfig.tcl')
+ self.send_expect("source ixiaConfig.tcl", "% ", 75)
+
+ def configure_transmission(self, option=None):
+ """
+ Start IXIA ports transmission.
+ """
+ self.add_tcl_cmd("ixStartTransmit portList")
+
+ def prepare_port_list(self, portList, rate_percent=100, latency=False):
+ """
+ Configure stream and flow on every IXIA ports.
+ """
+ txPortlist = set()
+ rxPortlist = set()
+
+ for subPortList in portList:
+ txPort, rxPort = subPortList[:2]
+ txPortlist.add(txPort)
+ rxPortlist.add(rxPort)
+
+ # port init
+ self.config_port([self.get_ixia_port_info(port)
+ for port in txPortlist.union(rxPortlist)])
+
+ # calculate total streams of ports
+ for (txPort, rxPort, pcapFile, option) in portList:
+ if txPort not in self.stream_total.keys():
+ self.stream_total[txPort] = 1
+ else:
+ self.stream_total[txPort] += 1
+
+ # stream/flow setting
+ for (txPort, rxPort, pcapFile, option) in portList:
+ if txPort not in self.stream_index.keys():
+ self.stream_index[txPort] = 1
+ frame_index = self.stream_index[txPort]
+ self.config_stream(pcapFile, option, txPort,
+ rate_percent, frame_index, latency)
+ self.stream_index[txPort] += 1
+ # clear stream ids table
+ self.stream_index.clear()
+ self.stream_total.clear()
+
+ # config stream before packetGroup
+ if latency is not False:
+ for subPortList in portList:
+ txPort, rxPort = subPortList[:2]
+ flow_num = len(self.parse_pcap(pcapFile))
+ self.config_pktGroup_rx(self.get_ixia_port(rxPort))
+ self.config_pktGroup_tx(self.get_ixia_port(txPort))
+ return rxPortlist, txPortlist
+
+ def prepare_ixia_for_transmission(self, txPortlist, rxPortlist):
+ """
+ Clear all statistics and implement configuration to IXIA hardware.
+ """
+ self.add_tcl_cmd("ixClearStats portList")
+ self.set_ixia_port_list([self.get_ixia_port(port)
+ for port in txPortlist])
+ self.add_tcl_cmd("ixWriteConfigToHardware portList")
+ # Wait for changes to take affect and make sure links are up
+ self.add_tcl_cmd("after 1000")
+ for port in txPortlist:
+ self.start_pktGroup(self.get_ixia_port(port))
+ for port in rxPortlist:
+ self.start_pktGroup(self.get_ixia_port(port))
+
+ def hook_transmission_func(self):
+ pass
+
+ def get_transmission_results(self, rx_port_list, tx_port_list, delay=5):
+ """
+ Override this method if you want to change the way of getting results
+ back from IXIA.
+ """
+ time.sleep(delay)
+ bpsRate = 0
+ rate = 0
+ oversize = 0
+ for port in rx_port_list:
+ self.stat_get_rate_stat_all_stats(port)
+ out = self.send_expect("stat cget -framesReceived", '%', 10)
+ rate += int(out.strip())
+ out = self.send_expect("stat cget -bitsReceived", '% ', 10)
+ self.logger.debug("port %d bits rate:" % (port) + out)
+ bpsRate += int(out.strip())
+ out = self.send_expect("stat cget -oversize", '%', 10)
+ oversize += int(out.strip())
+
+ self.logger.info("Rate: %f Mpps" % (rate * 1.0 / 1000000))
+ self.logger.info("Mbps rate: %f Mbps" % (bpsRate * 1.0 / 1000000))
+
+ self.hook_transmission_func()
+
+ self.send_expect("ixStopTransmit portList", "%", 30)
+
+ if rate == 0 and oversize > 0:
+ return (bpsRate, oversize)
+ else:
+ return (bpsRate, rate)
+
+ def config_ixia_dcb_init(self, rxPort, txPort):
+ """
+ Configure Ixia for DCB.
+ """
+ self.send_expect("source ./ixTcl1.0/ixiaDCB.tcl", "% ")
+ self.send_expect("configIxia %d %s" % (
+ self.chasId,
+ string.join(["%s" % (
+ repr(self.conRelation[port][n]))
+ for port in [rxPort, txPort]
+ for n in range(3)])),
+ "% ", 100)
+
+ def config_port_dcb(self, direction, tc):
+ """
+ Configure Port for DCB.
+ """
+ self.send_expect("configPort %s %s" % (direction, tc), "% ", 100)
+
+ def config_port_flow_control(self, ports, option):
+ ''' configure the type of flow control on a port '''
+ if not ports:
+ return
+ # mac address, default is "01 80 C2 00 00 01"
+ dst_mac = option.get('dst_mac') or "\"01 80 C2 00 00 01\""
+ if not dst_mac:
+ return
+ pause_time = option.get('pause_time') or 255
+ flow_ctrl_cmds = [
+ "protocol setDefault",
+ "port config -flowControl true",
+ "port config -flowControlType ieee8023x"]
+ for port in ports:
+ ixia_port = self.get_ixia_port(port)
+ flow_ctrl_cmds = [
+ # configure a pause control packet.
+ "port set {0}".format(ixia_port),
+ "protocol config -name pauseControl",
+ "pauseControl setDefault",
+ "pauseControl config -pauseControlType ieee8023x",
+ 'pauseControl config -da "{0}"'.format(dst_mac),
+ "pauseControl config -pauseTime {0}".format(pause_time),
+ "pauseControl set {0}".format(ixia_port),]
+ self.add_tcl_cmds(flow_ctrl_cmds)
+
+ def cfgStreamDcb(self, stream, rate, prio, types):
+ """
+ Configure Stream for DCB.
+ """
+ self.send_expect("configStream %s %s %s %s" % (stream, rate, prio, types), "% ", 100)
+
+ def get_connection_relation(self, dutPorts):
+ """
+ Get the connect relations between DUT and Ixia.
+ """
+ for port in dutPorts:
+ info = self.tester.get_pci(self.tester.get_local_port(port)).split('.')
+ self.conRelation[port] = [
+ int(info[0]), int(info[1]),
+ repr(self.tester.dut.get_mac_address(port).replace(':', ' ').upper())]
+ return self.conRelation
+
+ def config_pktGroup_rx(self, ixia_port):
+ """
+ Sets the transmit Packet Group configuration of the stream
+ Default streamID is 1
+ """
+ self.add_tcl_cmd("port config -receiveMode $::portRxModeWidePacketGroup")
+ self.add_tcl_cmd("port set %s" % ixia_port)
+ self.add_tcl_cmd("packetGroup setDefault")
+ self.add_tcl_cmd("packetGroup config -latencyControl cutThrough")
+ self.add_tcl_cmd("packetGroup setRx %s" % ixia_port)
+ self.add_tcl_cmd("packetGroup setTx %s 1" % ixia_port)
+
+ def config_pktGroup_tx(self, ixia_port):
+ """
+ Configure tx port pktGroup for latency.
+ """
+ self.add_tcl_cmd("packetGroup setDefault")
+ self.add_tcl_cmd("packetGroup config -insertSignature true")
+ self.add_tcl_cmd("packetGroup setTx %s 1" % ixia_port)
+
+ def start_pktGroup(self, ixia_port):
+ """
+ Start tx port pktGroup for latency.
+ """
+ self.add_tcl_cmd("ixStartPortPacketGroups %s" % ixia_port)
+
+ def pktGroup_get_stat_all_stats(self, port_number):
+ """
+ Stop Packet Group operation on port and get current Packet Group
+ statistics on port.
+ """
+ ixia_port = self.get_ixia_port(port_number)
+ self.send_expect("ixStopPortPacketGroups %s" % ixia_port, "%", 100)
+ self.send_expect("packetGroupStats get %s 0 16384" % ixia_port, "%", 100)
+ self.send_expect("packetGroupStats getGroup 0", "%", 100)
+
+ def close(self):
+ """
+ We first close the tclsh session opened at the beginning,
+ then the SSH session.
+ """
+ if self.isalive():
+ self.send_expect('exit', '# ')
+ super(Ixia, self).close()
+
+ def stat_get_stat_all_stats(self, port_number):
+ """
+ Sends a IXIA TCL command to obtain all the stat values on a given port.
+ """
+ ixia_port = self.get_ixia_port(port_number)
+ command = 'stat get statAllStats {0}'.format(ixia_port)
+ self.send_expect(command, '% ', 10)
+
+ def prepare_ixia_internal_buffers(self, port_number):
+ """
+ Tells IXIA to prepare the internal buffers were the frames were captured.
+ """
+ ixia_port = self.get_ixia_port(port_number)
+ command = 'capture get {0} {1} {2}'.format(ixia_port)
+ self.send_expect(command, '% ', 30)
+
+ def stat_get_rate_stat_all_stats(self, port_number):
+ """
+ All statistics of specified IXIA port.
+ """
+ ixia_port = self.get_ixia_port(port_number)
+ command = 'stat getRate statAllStats {0}'.format(ixia_port)
+ out = self.send_expect(command, '% ', 30)
+ return out
+
+ def ixia_capture_buffer(self, port_number, first_frame, last_frame):
+ """
+ Tells IXIA to load the captured frames into the internal buffers.
+ """
+ ixia_port = self.get_ixia_port(port_number)
+ command = 'captureBuffer get {0} {1} {2}'.format(
+ ixia_port, first_frame, last_frame)
+ self.send_expect(command, '%', 60)
+
+ def ixia_export_buffer_to_file(self, frames_filename):
+ """
+ Tells IXIA to dump the frames it has loaded in its internal buffer to a
+ text file.
+ """
+ command = 'captureBuffer export %s' % frames_filename
+ self.send_expect(command, '%', 30)
+
+ def _stat_cget_value(self, requested_value):
+ """
+ Sends a IXIA TCL command to obtain a given stat value.
+ """
+ command = "stat cget -" + requested_value
+ result = self.send_expect(command, '%', 10)
+ return int(result.strip())
+
+ def _capture_cget_value(self, requested_value):
+ """
+ Sends a IXIA TCL command to capture certain number of packets.
+ """
+ command = "capture cget -" + requested_value
+ result = self.send_expect(command, '%', 10)
+ return int(result.strip())
+
+ def _packetgroup_cget_value(self, requested_value):
+ """
+ Sends a IXIA TCL command to get pktGroup stat value.
+ """
+ command = "packetGroupStats cget -" + requested_value
+ result = self.send_expect(command, '%', 10)
+ return int(result.strip())
+
+ def number_of_captured_packets(self):
+ """
+ Returns the number of packets captured by IXIA on a previously set
+ port. Call self.stat_get_stat_all_stats(port) before.
+ """
+ return self._capture_cget_value('nPackets')
+
+ def get_frames_received(self):
+ """
+ Returns the number of packets captured by IXIA on a previously set
+ port. Call self.stat_get_stat_all_stats(port) before.
+ """
+ if self._stat_cget_value('framesReceived') != 0:
+ return self._stat_cget_value('framesReceived')
+ else:
+ # if the packet size is large than 1518, this line will avoid return
+ # a wrong number
+ return self._stat_cget_value('oversize')
+
+ def get_flow_control_frames(self):
+ """
+ Returns the number of control frames captured by IXIA on a
+ previously set port. Call self.stat_get_stat_all_stats(port) before.
+ """
+ return self._stat_cget_value('flowControlFrames')
+
+ def get_frames_sent(self):
+ """
+ Returns the number of packets sent by IXIA on a previously set
+ port. Call self.stat_get_stat_all_stats(port) before.
+ """
+ return self._stat_cget_value('framesSent')
+
+ def get_transmit_duration(self):
+ """
+ Returns the duration in nanosecs of the last transmission on a
+ previously set port. Call self.stat_get_stat_all_stats(port) before.
+ """
+ return self._stat_cget_value('transmitDuration')
+
+ def get_min_latency(self):
+ """
+ Returns the minimum latency in nanoseconds of the frames in the
+ retrieved capture buffer. Call packetGroupStats get before.
+ """
+ return self._packetgroup_cget_value('minLatency')
+
+ def get_max_latency(self):
+ """
+ Returns the maximum latency in nanoseconds of the frames in the
+ retrieved capture buffer. Call packetGroupStats get before.
+ """
+ return self._packetgroup_cget_value('maxLatency')
+
+ def get_average_latency(self):
+ """
+ Returns the average latency in nanoseconds of the frames in the
+ retrieved capture buffer. Call packetGroupStats get before.
+ """
+ return self._packetgroup_cget_value('averageLatency')
+
+ def _transmission_pre_config(self, port_list, rate_percent, latency=False):
+ """
+ Prepare and configure IXIA ports for performance test. And remove the
+ transmission step in this config sequence.
+
+ This function is set only for function send_number_packets for
+ nic_single_core_perf test case use
+ """
+ rxPortlist, txPortlist = self.prepare_port_list(
+ port_list, rate_percent, latency)
+ self.prepare_ixia_for_transmission(txPortlist, rxPortlist)
+ self.start_transmission()
+ self.clear_tcl_commands()
+ return rxPortlist, txPortlist
+
+ def send_number_packets(self, portList, ratePercent, packetNum):
+ """
+ Configure ixia to send fixed number of packets
+ Note that this function is only set for test_suite nic_single_core_perf,
+ Not for common use
+ """
+ rxPortlist, txPortlist = self._transmission_pre_config(portList,
+ ratePercent)
+
+ self.send_expect("stream config -numFrames %s" % packetNum, "%", 5)
+ self.send_expect("stream config -dma stopStream", "%", 5)
+ for txPort in txPortlist:
+ ixia_port = self.get_ixia_port(txPort)
+ self.send_expect("stream set %s 1" % ixia_port, "%", 5)
+
+ self.send_expect("ixWritePortsToHardware portList", "%", 5)
+ self.send_expect("ixClearStats portList", "%", 5)
+ self.send_expect("ixStartTransmit portList", "%", 5)
+ time.sleep(10)
+
+ rxPackets = 0
+ for port in txPortlist:
+ self.stat_get_stat_all_stats(port)
+ txPackets = self.get_frames_sent()
+ while txPackets != packetNum:
+ time.sleep(10)
+ self.stat_get_stat_all_stats(port)
+ txPackets = self.get_frames_sent()
+ rxPackets += self.get_frames_received()
+ self.logger.info("Received packets :%s" % rxPackets)
+
+ return rxPackets
+
+ #---------------------------------------------------------
+ # extend methods for pktgen subclass `IxiaPacketGenerator
+ #---------------------------------------------------------
+ def disconnect(self):
+ ''' quit from ixia server '''
+ pass
+
+ def start(self, **run_opt):
+ ''' start ixia ports '''
+ self.configure_transmission(run_opt)
+ self.start_transmission()
+ time.sleep(run_opt.get('duration') or 5)
+
+ def remove_all_streams(self):
+ ''' delete all streams on all ixia ports '''
+ if not self.ports:
+ return
+ for item in self.ports:
+ cmd = 'port reset {0} {1} {2}'.format(
+ self.chasId, item['card'], item['port'])
+ self.send_expect(cmd, "%", 10)
+
+ def reset(self, ports=None):
+ ''' reset ixia configuration for ports '''
+ pass
+
+ def clear_tcl_buffer(self):
+ ''' clear tcl commands buffer '''
+ self.tcl_cmds = []
+
+ def clear_stats(self):
+ pass
+
+ def stop_transmit(self):
+ '''
+ Stop IXIA transmit
+ '''
+ time.sleep(2)
+ self.send_expect("ixStopTransmit portList", "%", 40)
+
+ def get_latency_stat(self, port_list):
+ """
+ get latency statistics.
+ """
+ stats = {}
+ for port in port_list:
+ self.pktGroup_get_stat_all_stats(port)
+ stats[port] = {
+ 'average': self.get_average_latency(),
+ 'total_max': self.get_max_latency(),
+ 'total_min': self.get_min_latency()}
+ return stats
+
+ def get_loss_stat(self, port_list):
+ """
+ Get RX/TX packet statistics.
+ """
+ stats = {}
+ for port in port_list:
+ self.stat_get_stat_all_stats(port)
+ stats[port] = {
+ 'ibytes': 0,
+ 'ierrors': 0,
+ 'ipackets': self.get_frames_received(),
+ 'obytes': 0,
+ 'oerrors': 0,
+ 'opackets': self.get_frames_sent(),
+ 'rx_bps': 0,
+ 'rx_pps': 0,
+ 'tx_bps': 0,
+ 'tx_pps': 0,}
+ time.sleep(0.5)
+ return stats
+
+ def get_throughput_stat(self, port_list):
+ """
+ Get RX transmit rate.
+ """
+ stats = {}
+ for port in port_list:
+ self.stat_get_rate_stat_all_stats(port)
+ out = self.send_expect("stat cget -framesReceived", '%', 10)
+ rate = int(out.strip())
+ out = self.send_expect("stat cget -bitsReceived", '% ', 10)
+ bpsRate = int(out.strip())
+ out = self.send_expect("stat cget -oversize", '%', 10)
+ oversize = int(out.strip())
+ rate = oversize if rate == 0 and oversize > 0 else rate
+
+ stats[port] = {
+ 'ibytes': 0,
+ 'ierrors': 0,
+ 'ipackets': 0,
+ 'obytes': 0,
+ 'oerrors': 0,
+ 'opackets': 0,
+ 'rx_bps': bpsRate,
+ 'rx_pps': rate,
+ 'tx_bps': 0,
+ 'tx_pps': 0,}
+
+ return stats
+
+ def get_stats(self, ports, mode):
+ '''
+ get statistics of custom mode
+ '''
+ methods = {
+ 'throughput': self.get_throughput_stat,
+ 'loss': self.get_loss_stat,
+ 'latency': self.get_latency_stat,}
+ if mode not in methods.keys():
+ msg = "not support mode <{0}>".format(mode)
+ raise Exception(msg)
+ # get custom mode stat
+ func = methods.get(mode)
+ stats = func(ports)
+
+ return stats
+
+
+class IxiaPacketGenerator(PacketGenerator):
+ """
+ Ixia packet generator
+ """
+ def __init__(self, tester):
+ # ixia management
+ self.pktgen_type = PKTGEN_IXIA
+ self._conn = None
+ # ixia configuration information of dts
+ conf_inst = self._get_generator_conf_instance()
+ self.conf = conf_inst.load_pktgen_config()
+ # ixia port configuration
+ self._traffic_opt = {}
+ self._traffic_ports = []
+ self._ports = []
+ self._rx_ports = []
+ # statistics management
+ self.runtime_stats = {}
+ # check configuration options
+ self.options_keys = [
+ 'txmode', 'ip', 'vlan', 'transmit_mode', 'rate']
+ self.ip_keys = ['start', 'end','action', 'step', 'mask',]
+ self.vlan_keys = ['start', 'end', 'action', 'step', 'count',]
+
+ super(IxiaPacketGenerator, self).__init__(tester)
+ self.tester = tester
+
+ def get_ports(self):
+ ''' only used for ixia packet generator '''
+ return self._conn.get_ports()
+
+ def _prepare_generator(self):
+ ''' start ixia server '''
+ try:
+ self._connect(self.tester, self.conf)
+ except Exception as e:
+ msg = "failed to connect to ixia server"
+ raise Exception(msg)
+
+ def _connect(self, tester, conf):
+ # initialize ixia class
+ self._conn = Ixia(tester, conf)
+ for p in self._conn.get_ports():
+ self._ports.append(p)
+
+ self.logger.debug(self._ports)
+
+ def _disconnect(self):
+ '''
+ disconnect with ixia server
+ '''
+ try:
+ self._remove_all_streams()
+ self._conn.disconnect()
+ except Exception as e:
+ msg = 'Error disconnecting: %s' % e
+ self.logger.error(msg)
+ self._conn = None
+
+ def _get_port_pci(self, port_id):
+ '''
+ get ixia port pci address
+ '''
+ for pktgen_port_id, info in enumerate(self._ports):
+ if pktgen_port_id == port_id:
+ _pci = info.get('pci')
+ return _pci
+ else:
+ return None
+
+ def _get_gen_port(self, pci):
+ '''
+ get port management id of the packet generator
+ '''
+ for pktgen_port_id, info in enumerate(self._ports):
+ _pci = info.get('pci')
+ if _pci == pci:
+ return pktgen_port_id
+ else:
+ return -1
+
+ def _is_gen_port(self, pci):
+ '''
+ check if a pci address is managed by the packet generator
+ '''
+ for name, _port_obj in self._conn.ports.iteritems():
+ _pci = _port_obj.info['pci_addr']
+ self.logger.info((_pci, pci))
+ if _pci == pci:
+ return True
+ else:
+ return False
+
+ def _get_ports(self):
+ """
+ Return self ports information
+ """
+ ports = []
+ for idx in range(len(self._ports)):
+ ports.append('IXIA:%d' % idx)
+ return ports
+
+ @property
+ def _vm_conf(self):
+ # close it and wait for more discussion about pktgen framework
+ return None
+ conf = {}
+ #get the subnet range of src and dst ip
+ if self.conf.has_key("ip_src"):
+ conf['src'] = {}
+ ip_src = self.conf['ip_src']
+ ip_src_range = ip_src.split('-')
+ conf['src']['start'] = ip_src_range[0]
+ conf['src']['end'] = ip_src_range[1]
+
+ if self.conf.has_key("ip_dst"):
+ conf['dst'] = {}
+ ip_dst = self.conf['ip_dst']
+ ip_dst_range = ip_dst.split('-')
+ conf['dst']['start'] = ip_dst_range[0]
+ conf['dst']['end'] = ip_dst_range[1]
+
+ return conf if conf else None
+
+ def _clear_streams(self):
+ ''' clear streams in `PacketGenerator` '''
+ # if streams has been attached, remove them from trex server.
+ self._remove_all_streams()
+
+ def _remove_all_streams(self):
+ '''
+ remove all stream deployed on the packet generator
+ '''
+ if not self.get_streams():
+ return
+ self._conn.remove_all_streams()
+
+ def _get_port_features(self, port_id):
+ ''' get ports features '''
+ ports = self._conn.ports
+ if port_id not in ports:
+ return None
+ features = self._conn.ports[port_id].get_formatted_info()
+
+ return features
+
+ def _is_support_flow_control(self, port_id):
+ ''' check if a port support flow control '''
+ features = self._get_port_features(port_id)
+ if not features or features.get('fc_supported') == 'no':
+ return False
+ else:
+ return True
+
+ def _preset_ixia_port(self):
+ ''' set ports flow_ctrl attribute '''
+ rx_ports = self._rx_ports
+ flow_ctrl_opt = self._traffic_opt.get('flow_control')
+ if not flow_ctrl_opt:
+ return
+ # flow control of port running trex traffic
+ self._conn.config_port_flow_control(rx_ports, flow_ctrl_opt)
+
+ def _throughput_stats(self, stream, stats):
+ ''' convert ixia throughput statistics format to dts PacketGenerator format '''
+ # tx packet
+ tx_port_id = stream["tx_port"]
+ port_stats = stats.get(tx_port_id)
+ if not port_stats:
+ msg = "failed to get tx_port {0} statistics".format(tx_port_id)
+ raise Exception(msg)
+ tx_bps = port_stats.get("tx_bps")
+ tx_pps = port_stats.get("tx_pps")
+ msg = [
+ "Tx Port %d stats: " % (tx_port_id),
+ "tx_port: %d, tx_bps: %f, tx_pps: %f " % (
+ tx_port_id, tx_bps, tx_pps)]
+ self.logger.info(pformat(port_stats))
+ self.logger.info(os.linesep.join(msg))
+ # rx bps/pps
+ rx_port_id = stream["rx_port"]
+ port_stats = stats.get(rx_port_id)
+ if not port_stats:
+ msg = "failed to get rx_port {0} statistics".format(rx_port_id)
+ raise Exception(msg)
+ rx_bps = port_stats.get("rx_bps")
+ rx_pps = port_stats.get("rx_pps")
+ msg = [
+ "Rx Port %d stats: " % (rx_port_id),
+ "rx_port: %d, rx_bps: %f, rx_pps: %f" % (
+ rx_port_id, rx_bps, rx_pps)]
+
+ self.logger.info(pformat(port_stats))
+ self.logger.info(os.linesep.join(msg))
+
+ return rx_bps, rx_pps
+
+ def _loss_rate_stats(self, stream, stats):
+ ''' convert ixia loss rate statistics format to dts PacketGenerator format '''
+ # tx packet
+ port_id = stream.get("tx_port")
+ if port_id in stats.keys():
+ port_stats = stats[port_id]
+ else:
+ msg = "port {0} statistics is not found".format(port_id)
+ self.logger.error(msg)
+ return None
+ msg = "Tx Port %d stats: " % (port_id)
+ self.logger.info(msg)
+ opackets = port_stats["opackets"]
+ # rx packet
+ port_id = stream.get("rx_port")
+ port_stats = stats[port_id]
+ msg = "Rx Port %d stats: " % (port_id)
+ self.logger.info(msg)
+ ipackets = port_stats["ipackets"]
+
+ return opackets, ipackets
+
+ def _latency_stats(self, stream, stats):
+ ''' convert ixia latency statistics format to dts PacketGenerator format '''
+ port_id = stream.get("tx_port")
+ if port_id in stats.keys():
+ port_stats = stats[port_id]
+ else:
+ msg = "port {0} latency stats is not found".format(port_id)
+ self.logger.error(msg)
+ return None
+
+ latency_stats = {
+ 'min': port_stats.get('total_min'),
+ 'max': port_stats.get('total_max'),
+ 'average':port_stats.get('average'),}
+
+ return latency_stats
+
+ def send_ping6(self, pci, mac, ipv6):
+ ''' Send ping6 packet from IXIA ports. '''
+ return self._conn.send_ping6(pci, mac, ipv6)
+
+ ##########################################################################
+ #
+ # class ``PacketGenerator`` abstract methods should be implemented here
+ #
+ ##########################################################################
+ def _prepare_transmission(self, stream_ids=[], latency=False):
+ ''' add one/multiple streams in one/multiple ports '''
+ port_config = {}
+
+ for stream_id in stream_ids:
+ stream = self._get_stream(stream_id)
+ tx_port = stream.get('tx_port')
+ rx_port = stream.get('rx_port')
+ pcap_file = stream.get('pcap_file')
+ rate_percent = stream.get('rate')
+ # save port id list
+ if tx_port not in self._traffic_ports:
+ self._traffic_ports.append(tx_port)
+ if rx_port not in self._traffic_ports:
+ self._traffic_ports.append(rx_port)
+ if rx_port not in self._rx_ports:
+ self._rx_ports.append(rx_port)
+ # set all streams in one port to do batch configuration
+ options = stream['options']
+ if tx_port not in port_config.keys():
+ port_config[tx_port] = []
+ config = {}
+ config.update(options)
+ # In pktgen, all streams flow control option are the same by design.
+ self._traffic_opt['flow_control'] = options.get('flow_control') or {}
+ # if vm config by pktgen config file, set it here to take the place
+ # of setting on suite
+ if self._vm_conf: # TBD, remove this process later
+ config['fields_config'] = self._vm_conf
+ # get stream rate percent
+ stream_config = options.get('stream_config')
+ # set port list input parameter of ixia class
+ ixia_option = [tx_port, rx_port, pcap_file, options]
+ port_config[tx_port].append(ixia_option)
+
+ if not port_config:
+ msg = 'no stream options for ixia packet generator'
+ raise Exception(msg)
+ #-------------------------------------------------------------------
+ port_lists = []
+ for port_id, option in port_config.iteritems():
+ port_lists += option
+ self._conn.clear_tcl_buffer()
+ rxPortlist, txPortlist = self._conn.prepare_port_list(
+ port_lists, rate_percent or 100, latency)
+ self._conn.prepare_ixia_for_transmission(txPortlist, rxPortlist)
+ # preset port status before running traffic
+ self._preset_ixia_port()
+
+ def _start_transmission(self, stream_ids, options={}):
+ '''
+ :param sample_delay:
+ After traffic start ``sample_delay`` seconds, start get runtime statistics
+ '''
+ # get rate percentage
+ rate_percent = options.get('rate') or '100'
+ # get duration
+ duration = options.get("duration") or 5
+ duration = int(duration) if isinstance(duration, (str, unicode)) \
+ else duration
+ # get sample interval
+ _sample_delay = options.get("sample_delay") or duration/2
+ sample_delay = int(_sample_delay) \
+ if isinstance(_sample_delay, (str, unicode)) \
+ else _sample_delay
+ # get configuration from pktgen config file
+ warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \
+ else 25
+ wait_interval, core_mask = (warmup+30, self.conf["core_mask"]) \
+ if self.conf.has_key("core_mask") \
+ else (warmup+5, None)
+ #-------------------------------------------------------------------
+ # run ixia server
+ try:
+ ###########################################
+ # Start traffic on port(s)
+ self.logger.info("begin traffic ......")
+ run_opt = {
+ 'ports': self._traffic_ports,
+ 'mult': rate_percent,
+ 'duration': duration,
+ 'core_mask':core_mask,
+ 'force': True,}
+ self._conn.start(**run_opt)
+ except Exception as e:
+ self.logger.error(e)
+
+ def _stop_transmission(self, stream_id):
+ # using ixia server command
+ if self._traffic_ports:
+ self._conn.stop_transmit()
+
+ def _retrieve_port_statistic(self, stream_id, mode):
+ ''' ixia traffic statistics '''
+ stats = self._conn.get_stats(self._traffic_ports, mode)
+ stream = self._get_stream(stream_id)
+ self.logger.info(pformat(stream))
+ self.logger.info(pformat(stats))
+ if mode == 'throughput':
+ return self._throughput_stats(stream, stats)
+ elif mode == 'loss':
+ return self._loss_rate_stats(stream, stats)
+ elif mode == 'latency':
+ return self._latency_stats(stream, stats)
+ else:
+ msg = "not support mode <{0}>".format(mode)
+ raise Exception(msg)
+
+ def _check_options(self, opts={}):
+ # remove it to upper level class and wait for more discussion about
+ # pktgen framework
+ return True
+ for key in opts:
+ if key in self.options_keys:
+ if key == 'ip':
+ ip = opts['ip']
+ for ip_key in ip:
+ if not ip_key in self.ip_keys:
+ msg = " %s is invalid ip option" % ip_key
+ self.logger.info(msg)
+ return False
+ if key == 'action':
+ if not ip[key] == 'inc' or not ip[key] == 'dec':
+ msg = " %s is invalid ip action" % ip[key]
+ self.logger.info(msg)
+ return False
+ elif key == 'vlan':
+ vlan = opts['vlan']
+ for vlan_key in vlan:
+ if not vlan_key in self.vlan_keys:
+ msg = " %s is invalid vlan option" % vlan_key
+ self.logger.info(msg)
+ return False
+ if key == 'action':
+ if not vlan[key] == 'inc' or not ip[key] == 'dec':
+ msg = " %s is invalid vlan action" % vlan[key]
+ self.logger.info(msg)
+ return False
+ else:
+ msg = " %s is invalid option" % key
+ self.logger.info(msg)
+ return False
+ return True
+
+ def quit_generator(self):
+ ''' close ixia session '''
+ if self._conn is not None:
+ self._disconnect()
+ return
\ No newline at end of file
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 0/14] framework/pktgen: trex packet generator relevant classes
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (8 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 9/14] framework/pktgen: ixia packet generator relevant classes yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 1/14] framework/pktgen: pktgen instance creation and helper yufengmx
` (4 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
trex packet generator relevant classes
Move class TrexPacketGenerator from pktgen.py to here and put all trex relevant
source code in this module. Add _latency_stats()/_loss_rate_stats() to support
latency/loss testing scenario. Add _remove_all_streams() to clear streams managed by
trex tool. Add _is_support_flow_control() to check trex flow control setting. Rename
trex classes back to trex original names. Create TrexConfigVm class to convert pktgen
unified protocol layer fields config format to trex format. Create TrexConfigStream
class to set streams option into trex tool.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen_trex.py | 857 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 857 insertions(+)
create mode 100644 framework/pktgen_trex.py
diff --git a/framework/pktgen_trex.py b/framework/pktgen_trex.py
new file mode 100644
index 0000000..159750e
--- /dev/null
+++ b/framework/pktgen_trex.py
@@ -0,0 +1,857 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import os
+import sys
+import time
+import logging
+from pprint import pformat
+
+from pktgen_base import (PacketGenerator, PKTGEN_TREX,
+ TRANSMIT_CONT, TRANSMIT_M_BURST, TRANSMIT_S_BURST)
+
+FORMAT = '%(message)s'
+logging.basicConfig(format=FORMAT)
+logger = logging.getLogger(os.path.basename(__file__)[:-3].upper())
+logger.setLevel(logging.INFO)
+
+
+class TrexConfigVm(object):
+ '''
+ config one stream vm format of trex
+ '''
+ def __init__(self):
+ from trex_stl_lib.api import (ipv4_str_to_num, mac2str, is_valid_ipv4_ret)
+ self.ipv4_str_to_num = ipv4_str_to_num
+ self.is_valid_ipv4_ret = is_valid_ipv4_ret
+ self.mac2str = mac2str
+
+ def _mac_var(self, fv_name, mac_start, mac_end, step, mode):
+ '''
+ create mac address vm format of trex
+ '''
+ _mac_start = self.ipv4_str_to_num(self.mac2str(mac_start)[2:])
+ _mac_end = self.ipv4_str_to_num(self.mac2str(mac_end)[2:])
+ if mode == 'inc' or mode == 'dec':
+ min_value = _mac_start
+ max_value = _mac_end
+ elif mode == 'random':
+ max_value = 0xffffffff
+ min_value = 0
+ add_val = 0
+
+ var = [{
+ 'name': fv_name,
+ 'min_value': min_value,
+ 'max_value': max_value,
+ 'size': 4,
+ 'step': step,
+ 'op': mode,},
+ {'write': {'add_val': add_val, 'offset_fixup': 2}}]
+
+ return var
+
+ def _ip_vm_var(self, fv_name, ip_start, ip_end, step, mode):
+ '''
+ create ip address vm format of trex
+ '''
+ _ip_start = self.ipv4_str_to_num(self.is_valid_ipv4_ret(ip_start))
+ _ip_end = self.ipv4_str_to_num(self.is_valid_ipv4_ret(ip_end))
+ _step = self.ipv4_str_to_num(self.is_valid_ipv4_ret(step)) \
+ if isinstance(step, (str, unicode)) else step
+ if mode == 'inc' or mode == 'dec':
+ min_value = _ip_start
+ max_value = _ip_end
+ elif mode == 'random':
+ max_value = 0xffffffff
+ min_value = 0
+ add_val = 0
+
+ var = [{
+ 'name': fv_name,
+ 'min_value': min_value,
+ 'max_value': max_value,
+ 'size': 4,
+ 'step': _step,
+ 'op': mode,},
+ {'write': {'add_val': add_val},
+ 'fix_chksum': {}}]
+
+ return var
+
+ def config_trex_vm(self, option):
+ '''
+ config one stream vm
+ '''
+ vm_var = {}
+ ###################################################################
+ # mac inc/dec/random
+ if 'mac' in option:
+ for name, config in option['mac'].iteritems():
+ mac_start = config.get('start') or '00:00:00:00:00:00'
+ mac_end = config.get('end') or 'FF:FF:FF:FF:FF:FF'
+ step = config.get('step') or 1
+ mode = config.get('action') or 'inc'
+ #-----------------
+ fv_name = 'Ethernet.{0}'.format(name)
+ # layer/field name
+ vm_var[fv_name] = self._mac_var(fv_name,
+ mac_start, mac_end,
+ step, mode)
+ ###################################################################
+ # src ip mask inc/dec/random
+ if 'ip' in option:
+ for name, config in option['ip'].iteritems():
+ ip_start = config.get('start') or '0.0.0.1'
+ ip_end = config.get('end') or '0.0.0.255'
+ step = config.get('step') or 1
+ mode = config.get('action') or 'inc'
+ #-----------------
+ fv_name = 'IP.{0}'.format(name)
+ # layer/field name
+ vm_var[fv_name] = self._ip_vm_var(fv_name, ip_start, ip_end,
+ step, mode)
+ ###################################################################
+ # merge var1/var2/random/cache into one method
+ ###################################################################
+ # src ip mask inc/dec/random
+ if 'port' in option:
+ for name, config in option['port'].iteritems():
+ protocol = config.get('protocol') or 'UDP'
+ port_start = config.get('start') or 1
+ port_end = config.get('end') or 255
+ step = config.get('step') or 1
+ mode = config.get('action') or 'inc'
+ #-----------------
+ fv_name = '{0}.{1}'.format(protocol.upper(), name)
+ # layer/field name
+ vm_var[fv_name] = {
+ 'name': fv_name,
+ 'min_value': port_start,
+ 'max_value': port_end,
+ 'size': 2,
+ 'step': step,
+ 'op': mode,}
+ ###################################################################
+ # vlan field inc/dec/random
+ if 'vlan' in option:
+ for name, config in option['vlan'].iteritems():
+ vlan_start = config.get('start') \
+ if config.get('start') != None else 0
+ vlan_end = config.get('end') or 256
+ step = config.get('step') or 1
+ mode = config.get('action') or 'inc'
+ #-----------------
+ fv_name = '802|1Q:{0}.vlan'.format(name)
+ # vlan layer/field name
+ vm_var[fv_name] = {
+ 'name': fv_name,
+ 'min_value': vlan_start,
+ 'max_value': vlan_end,
+ 'size': 2,
+ 'step': step,
+ 'op': mode,}
+ ###################################################################
+ # payload change with custom sizes
+ if 'pkt_size' in option:
+ # note:
+ # when using mixed stream, which have different sizes
+ # this will be forbidden
+ step = 1
+ mode = 'random'
+ min_pkt_size = option['pkt_size']['start']
+ max_pkt_size = option['pkt_size']['end']
+ #-----------------
+ l3_len_fix = -len(Ether())
+ l4_len_fix = l3_len_fix - len(IP())
+
+ var = {
+ 'name': 'fv_rand',
+ # src ip increase with a range
+ 'min_value': min_pkt_size - 4,
+ 'max_value': max_pkt_size - 4,
+ 'size': 2,
+ 'step': step,
+ 'op': mode,}
+
+ vm_var = {
+ 'IP.len': [
+ var, {'write': {'add_val': l3_len_fix},
+ 'trim': {},
+ 'fix_chksum': {}}],
+ 'UDP.len': [
+ var, {'write': {'add_val': l4_len_fix},
+ 'trim': {},
+ 'fix_chksum': {}}]}
+
+ return vm_var
+
+
+class TrexConfigStream(object):
+
+ def __init__(self):
+ from trex_stl_lib.api import (
+ STLTXCont, STLTXSingleBurst, STLTXMultiBurst,
+ STLPktBuilder, STLProfile, STLVM,
+ STLStream, STLFlowLatencyStats)
+
+ # set trex class
+ self.STLStream = STLStream
+ self.STLPktBuilder = STLPktBuilder
+ self.STLProfile = STLProfile
+ self.STLVM = STLVM
+ self.STLTXCont = STLTXCont
+ self.STLTXSingleBurst = STLTXSingleBurst
+ self.STLTXMultiBurst = STLTXMultiBurst
+ self.STLFlowLatencyStats = STLFlowLatencyStats
+
+ def _set_var_default_value(self, config):
+ default = {
+ 'init_value': None,
+ 'min_value': 0,
+ 'max_value': 255,
+ 'size': 4,
+ 'step': 1}
+ for name, value in default.iteritems():
+ if name not in config:
+ config[name] = value
+
+ def _preset_layers(self, vm_var, configs):
+ '''
+ configure stream behavior on pcap format
+ '''
+ msg = "layer <{0}> field name <{1}> is not defined".format
+ fv_names = []
+ fix_chksum = False
+ for layer, _config in configs.iteritems():
+ # set default value
+ if isinstance(_config, (tuple, list)):
+ config = _config[0]
+ op_config = _config[1]
+ else:
+ config = _config
+ op_config = None
+
+ name = config.get('name')
+ if not name:
+ error = msg(layer, name)
+ raise Exception(error)
+
+ self._set_var_default_value(config)
+ # different fields with a range (relevance variables)
+ if isinstance(layer, (tuple, list)):
+ vm_var.tuple_var(**config)
+ for offset in layer:
+ fv_name = name+'.ip' if offset.startswith('IP') else \
+ name+'.port'
+ _vars = {'fv_name': fv_name, 'pkt_offset': offset}
+ if op_config and 'write' in op_config:
+ _vars.update(op_config['write'])
+
+ if fv_name not in fv_names:
+ fv_names.append(fv_name)
+ vm_var.write(**_vars)
+ # different fields with a range (independent variable)
+ else:
+ if name not in fv_names:
+ fv_names.append(name)
+ vm_var.var(**config)
+ # write behavior in field
+ _vars = {'fv_name': name, 'pkt_offset': layer}
+ if op_config and 'write' in op_config:
+ _vars.update(op_config['write'])
+ vm_var.write(**_vars)
+
+ # Trim the packet size by the stream variable size
+ if op_config and 'trim' in op_config:
+ vm_var.trim(name)
+ # set VM as cached with a cache size
+ if op_config and 'set_cached' in op_config:
+ vm_var.set_cached(op_config['set_cached'])
+ # Fix IPv4 header checksum
+ if op_config and 'fix_chksum' in op_config:
+ fix_chksum = True
+
+ # protocol type
+ if fix_chksum:
+ vm_var.fix_chksum()
+
+ def _create_stream(self, _pkt, stream_opt, vm=None, flow_stats=None):
+ '''
+ create trex stream
+ '''
+ isg = stream_opt.get('isg') or 0.5
+ mode = stream_opt.get('transmit_mode') or TRANSMIT_CONT
+ txmode_opt = stream_opt.get('txmode') or {}
+ pps = txmode_opt.get('pps')
+ # Continuous mode
+ if mode == TRANSMIT_CONT:
+ mode_inst = self.STLTXCont(pps=pps)
+ # Single burst mode
+ elif mode == TRANSMIT_S_BURST:
+ total_pkts = txmode_opt.get('total_pkts') or 32
+ mode_inst = self.STLTXSingleBurst(pps=pps, total_pkts=total_pkts)
+ # Multi-burst mode
+ elif mode == TRANSMIT_M_BURST:
+ burst_pkts = txmode_opt.get('burst_pkts') or 32
+ bursts_count = txmode_opt.get('bursts_count') or 2
+ ibg = txmode_opt.get('ibg') or 10
+ mode_inst = self.STLTXMultiBurst(pkts_per_burst = burst_pkts,
+ count = bursts_count,
+ ibg = ibg)
+ else:
+ msg = 'not support format {0}'.format(mode)
+ raise Exception(msg)
+
+ pkt = self.STLPktBuilder(pkt=_pkt, vm=vm)
+ _stream = self.STLStream(packet=pkt, mode=mode_inst, isg=isg,
+ flow_stats=flow_stats)
+
+ return _stream
+
+ def _generate_vm(self, vm_conf):
+ '''
+ create packet fields trex vm instance
+ '''
+ if not vm_conf:
+ return None
+ # config packet vm format for trex
+ hVmConfig = TrexConfigVm()
+ _vm_var = hVmConfig.config_trex_vm(vm_conf)
+ if not isinstance(_vm_var, self.STLVM):
+ vm_var = self.STLVM()
+ self._preset_layers(vm_var, _vm_var)
+ else:
+ vm_var = _vm_var
+
+ return vm_var
+
+ def _get_streams(self, streams_config):
+ '''
+ create a group of streams
+ '''
+ # vm_var is the instance to config pcap fields
+ # create a group of streams, which are using different size payload
+ streams = []
+
+ for config in streams_config:
+ _pkt = config.get('pcap')
+ vm_conf = config.get('fields_config')
+ _stream_op = config.get('stream_config')
+ # configure trex vm
+ vm_var = self._generate_vm(vm_conf)
+ # create
+ streams.append(self._create_stream( _pkt, _stream_op, vm_var))
+ _streams = self.STLProfile(streams).get_streams()
+
+ return _streams
+
+ def add_streams(self, conn, streams_config, ports=None, latency=False):
+ '''
+ create one/multiple of streams on one port of trex server
+ '''
+ # normal streams configuration
+ _streams = self._get_streams(streams_config)
+ # create latency statistics stream
+ # use first one of main stream config as latency statistics stream
+ if latency:
+ streams = list(_streams)
+ flow_stats = self.STLFlowLatencyStats(pg_id=ports[0])
+ latency_opt = streams_config[0]
+ _pkt = latency_opt.get('pcap')
+ _stream_op = latency_opt.get('stream_config')
+ _stream = self._create_stream( _pkt, _stream_op,
+ flow_stats=flow_stats)
+ streams.append(_stream)
+ else:
+ streams = _streams
+
+ conn.add_streams(streams, ports=ports)
+
+
+class TrexPacketGenerator(PacketGenerator):
+ """
+ Trex packet generator, detail usage can be seen at
+ https://trex-tgn.cisco.com/trex/doc/trex_manual.html
+ """
+ def __init__(self, tester):
+ self.pktgen_type = PKTGEN_TREX
+ self.trex_app = "t-rex-64"
+ self._conn = None
+ self.control_session = None
+ # trex management
+ self._traffic_opt = {}
+ self._ports = []
+ self._traffic_ports = []
+ self._rx_ports = []
+ self.runtime_stats = {}
+
+ conf_inst = self._get_generator_conf_instance()
+ self.conf = conf_inst.load_pktgen_config()
+
+ self.options_keys = [
+ 'txmode', 'ip', 'vlan', 'transmit_mode', 'rate']
+ self.ip_keys = ['start', 'end','action', 'mask', 'step']
+ self.vlan_keys = ['start', 'end', 'action', 'step', 'count']
+
+ super(TrexPacketGenerator, self).__init__(tester)
+
+ # check trex binary file
+ trex_bin = os.sep.join([self.conf.get('trex_root_path'), self.trex_app])
+ if not os.path.exists(trex_bin):
+ msg = "{0} is not existed, please check {1} content".format(
+ trex_bin, conf_inst.config_file)
+ raise Exception(msg)
+ # if `trex_lib_path` is not set, use a default relative directory.
+ trex_lib_dir = \
+ self.conf.get('trex_lib_path') \
+ if self.conf.get('trex_lib_path') else \
+ "{0}/automation/trex_control_plane/stl".format(
+ self.conf.get('trex_root_path'))
+ # check trex lib root directory
+ if not os.path.exists(trex_lib_dir):
+ msg = ("{0} is not existed, please check {1} content and "
+ "set `trex_lib_path`").format(trex_lib_dir, conf_inst.config_file)
+ raise Exception(msg)
+ # check if trex lib is existed
+ trex_lib = os.sep.join([trex_lib_dir, 'trex_stl_lib'])
+ if not os.path.exists(trex_lib):
+ msg = "no 'trex_stl_lib' package under {0}".format(trex_lib_dir)
+ raise Exception(msg)
+ # import t-rex libs
+ sys.path.insert(0, trex_lib_dir)
+ from trex_stl_lib.api import STLClient
+ # set trex class
+ self.STLClient = STLClient
+
+ def _connect(self):
+ self._conn = self.STLClient(server=self.conf["server"])
+ self._conn.connect()
+ for p in self._conn.get_all_ports():
+ self._ports.append(p)
+
+ self.logger.debug(self._ports)
+
+ def _get_port_pci(self, port_id):
+ '''
+ get port pci address
+ '''
+ for name, _port_obj in self._conn.ports.iteritems():
+ if name == port_id:
+ _pci = _port_obj.info['pci_addr']
+ return _pci
+ else:
+ return None
+
+ def _get_gen_port(self, pci):
+ '''
+ get port management id of the packet generator
+ '''
+ for name, _port_obj in self._conn.ports.iteritems():
+ _pci = _port_obj.info['pci_addr']
+ if _pci == pci:
+ return name
+ else:
+ return -1
+
+ def _is_gen_port(self, pci):
+ '''
+ check if a pci address is managed by the packet generator
+ '''
+ for name, _port_obj in self._conn.ports.iteritems():
+ _pci = _port_obj.info['pci_addr']
+ self.logger.info((_pci, pci))
+ if _pci == pci:
+ return True
+ else:
+ return False
+
+ def get_ports(self):
+ """
+ Return self ports information
+ """
+ ports = []
+ for idx in range(len(self._ports)):
+ port_info = self._conn.ports[idx]
+ pci = port_info.info['pci_addr']
+ mac = port_info.info['hw_mac']
+ ports.append({
+ 'intf': 'TREX:%d' % idx,
+ 'mac': mac,
+ 'pci': pci,
+ 'type': 'trex',})
+ return ports
+
+ def _clear_streams(self):
+ ''' clear streams in trex and `PacketGenerator` '''
+ # if streams has been attached, remove them from trex server.
+ self._remove_all_streams()
+
+ def _remove_all_streams(self):
+ ''' remove all stream deployed on trex port(s) '''
+ if not self.get_streams():
+ return
+ if not self._conn.get_acquired_ports():
+ return
+ self._conn.remove_all_streams()
+
+ def _disconnect(self):
+ ''' disconnect with trex server '''
+ try:
+ self._remove_all_streams()
+ self._conn.disconnect()
+ except Exception as e:
+ msg = 'Error disconnecting: %s' % e
+ self.logger.error(msg)
+ self._conn = None
+
+ def _check_options(self, opts={}):
+ return True # close it and wait for more discussion about pktgen framework
+ for key in opts:
+ if key in self.options_keys:
+ if key == 'ip':
+ ip = opts['ip']
+ for ip_key in ip:
+ if not ip_key in self.ip_keys:
+ msg = " %s is invalid ip option" % ip_key
+ self.logger.info(msg)
+ return False
+ if key == 'action':
+ if not ip[key] == 'inc' or not ip[key] == 'dec':
+ msg = " %s is invalid ip action" % ip[key]
+ self.logger.info(msg)
+ return False
+ elif key == 'vlan':
+ vlan = opts['vlan']
+ for vlan_key in vlan:
+ if not vlan_key in self.vlan_keys:
+ msg = " %s is invalid vlan option" % vlan_key
+ self.logger.info(msg)
+ return False
+ if key == 'action':
+ if not vlan[key] == 'inc' or not ip[key] == 'dec':
+ msg = " %s is invalid vlan action" % vlan[key]
+ self.logger.info(msg)
+ return False
+ else:
+ msg = " %s is invalid option" % key
+ self.logger.info(msg)
+ return False
+ return True
+
+ def _prepare_generator(self):
+ ''' start trex server '''
+ if self.conf.has_key('start_trex') and self.conf['start_trex']:
+ app_param_temp = "-i"
+ # flow control
+ flow_control = self.conf.get('flow_control')
+ flow_control_opt = '--no-flow-control-change' if flow_control else ''
+
+ for key in self.conf:
+ #key, value = pktgen_conf
+ if key == 'config_file':
+ app_param_temp = app_param_temp + " --cfg " + self.conf[key]
+ elif key == 'core_num':
+ app_param_temp = app_param_temp + " -c " + self.conf[key]
+ self.control_session = \
+ self.tester.create_session('trex_control_session')
+
+ self.control_session.send_expect(
+ ';'.join(['cd ' + self.conf['trex_root_path'],
+ './' + self.trex_app + " " + app_param_temp]),
+ '-Per port stats table', 30)
+ try:
+ self._connect()
+ except Exception as e:
+ msg = "failed to connect to t-rex server"
+ raise Exception(msg)
+
+ @property
+ def _vm_conf(self):
+ return None # close it and wait for more discussion about pktgen framework
+ conf = {}
+ #get the subnet range of src and dst ip
+ if self.conf.has_key("ip_src"):
+ conf['src'] = {}
+ ip_src = self.conf['ip_src']
+ ip_src_range = ip_src.split('-')
+ conf['src']['start'] = ip_src_range[0]
+ conf['src']['end'] = ip_src_range[1]
+
+ if self.conf.has_key("ip_dst"):
+ conf['dst'] = {}
+ ip_dst = self.conf['ip_dst']
+ ip_dst_range = ip_dst.split('-')
+ conf['dst']['start'] = ip_dst_range[0]
+ conf['dst']['end'] = ip_dst_range[1]
+
+ if conf:
+ return conf
+ else:
+ return None
+
+ def _get_port_features(self, port_id):
+ ''' get ports' features '''
+ ports = self._conn.ports
+ if port_id not in ports:
+ return None
+ features = self._conn.ports[port_id].get_formatted_info()
+ self.logger.info(pformat(features))
+
+ return features
+
+ def _is_support_flow_control(self, port_id):
+ ''' check if a port support flow control '''
+ features = self._get_port_features(port_id)
+ if not features or features.get('fc_supported') == 'no':
+ msg = "trex port <{0}> not support flow control".format(port_id)
+ self.logger.warning(msg)
+ return False
+ else:
+ return True
+
+ def _preset_trex_port(self):
+ ''' set ports promiscuous/flow_ctrl attribute '''
+ rx_ports = self._rx_ports
+ # for trex design requirement, all ports of trex should be the same type
+ # nic, here use first port to check flow control attribute
+ flow_ctrl = self._traffic_opt.get('flow_control') \
+ if self._is_support_flow_control(rx_ports[0]) else None
+ flow_ctrl_flag = flow_ctrl.get('flag') or 1 if flow_ctrl else None
+ # flow control of port running trex traffic
+ self._conn.set_port_attr( rx_ports,
+ promiscuous=True,
+ link_up=True,
+ flow_ctrl = flow_ctrl_flag)
+
+ def _throughput_stats(self, stream, stats):
+ # tx packet
+ tx_port_id = stream["tx_port"]
+ port_stats = self.runtime_stats.get(tx_port_id)
+ if not port_stats:
+ msg = "failed to get tx_port {0} statistics".format(tx_port_id)
+ raise Exception(msg)
+ tx_bps = port_stats.get("tx_bps")
+ tx_pps = port_stats.get("tx_pps")
+ msg = [
+ "Tx Port %d stats: " % (tx_port_id),
+ "tx_port: %d, tx_bps: %f, tx_pps: %f " % (
+ tx_port_id, tx_bps, tx_pps)]
+ self.logger.info(pformat(port_stats))
+ self.logger.info(os.linesep.join(msg))
+ # rx bps/pps
+ rx_port_id = stream["rx_port"]
+ port_stats = self.runtime_stats.get(rx_port_id)
+ if not port_stats:
+ msg = "failed to get rx_port {0} statistics".format(rx_port_id)
+ raise Exception(msg)
+ rx_bps = port_stats.get("rx_bps")
+ rx_pps = port_stats.get("rx_pps")
+ msg = [
+ "Rx Port %d stats: " % (rx_port_id),
+ "rx_port: %d, rx_bps: %f, rx_pps: %f" % (
+ rx_port_id, rx_bps, rx_pps)]
+
+ self.logger.info(pformat(port_stats))
+ self.logger.info(os.linesep.join(msg))
+
+ return rx_bps, rx_pps
+
+ def _loss_rate_stats(self, stream, stats):
+ # tx packet
+ port_id = stream.get("tx_port")
+ if port_id in stats.keys():
+ port_stats = stats[port_id]
+ else:
+ msg = "port {0} statistics is not found".format(port_id)
+ self.logger.error(msg)
+ return None
+ msg = "Tx Port %d stats: " % (port_id)
+ self.logger.info(msg)
+ opackets = port_stats["opackets"]
+ # rx packet
+ port_id = stream.get("rx_port")
+ port_stats = stats[port_id]
+ msg = "Rx Port %d stats: " % (port_id)
+ self.logger.info(msg)
+ ipackets = port_stats["ipackets"]
+
+ return opackets, ipackets
+
+ def _latency_stats(self, stream, stats):
+ _stats = stats.get('latency')
+ port_id = stream.get("tx_port")
+ if port_id in _stats.keys():
+ port_stats = _stats[port_id]['latency']
+ else:
+ msg = "port {0} latency stats is not found".format(port_id)
+ self.logger.error(msg)
+ return None
+
+ latency_stats = {
+ 'min': port_stats.get('total_min'),
+ 'max': port_stats.get('total_max'),
+ 'average':port_stats.get('average'),}
+
+ return latency_stats
+
+ def _prepare_transmission(self, stream_ids=[], latency=False):
+ ''' add one/multiple streams in one/multiple ports '''
+ port_config = {}
+ for stream_id in stream_ids:
+ stream = self._get_stream(stream_id)
+ tx_port = stream['tx_port']
+ rx_port = stream['rx_port']
+ # save port id list
+ if tx_port not in self._traffic_ports:
+ self._traffic_ports.append(tx_port)
+ if rx_port not in self._rx_ports:
+ self._rx_ports.append(rx_port)
+ # set all streams in one port to do batch configuration
+ options = stream['options']
+ if tx_port not in port_config.keys():
+ port_config[tx_port] = []
+ config = {}
+ config.update(options)
+ # since trex stream rate percent haven't taken effect, here use one
+ # stream rate percent as port rate percent. In pktgen, all streams
+ # rate percent are the same value by design. flow control option is
+ # the same.
+ stream_config = options.get('stream_config') or {}
+ self._traffic_opt['rate'] = stream_config.get('rate') or 100
+ if stream_config.get('pps'): # reserve feature
+ self._traffic_opt['pps'] = stream_config.get('pps')
+ # flow control option is deployed on all ports by design
+ self._traffic_opt['flow_control'] = options.get('flow_control') or {}
+ # if vm config by pktgen config file, set it here to take the place
+ # of user setting
+ if self._vm_conf:
+ config['fields_config'] = self._vm_conf
+ port_config[tx_port].append(config)
+
+ if not port_config:
+ msg = 'no stream options for trex packet generator'
+ raise Exception(msg)
+
+ self._conn.reset(ports=self._ports)
+ config_inst = TrexConfigStream()
+ for port_id, config in port_config.iteritems():
+ # add a group of streams in one port
+ config_inst.add_streams(self._conn, config, ports=[port_id],
+ latency=latency)
+ # preset port status before running traffic
+ self._preset_trex_port()
+
+ def _start_transmission(self, stream_ids, options={}):
+ '''
+ :param sample_delay:
+ After traffic start ``sample_delay`` seconds, start get runtime statistics
+ '''
+ # get rate percentage
+ rate_percent = "{0}%".format(options.get('rate') or
+ self._traffic_opt.get('rate') or
+ '100')
+ # get duration
+ duration = options.get("duration") or 20
+ duration = int(duration) if isinstance(duration, (str, unicode)) \
+ else duration
+ # get sample interval
+ _sample_delay = options.get("sample_delay") or duration/2
+ sample_delay = int(_sample_delay) \
+ if isinstance(_sample_delay, (str, unicode)) \
+ else _sample_delay
+ # get configuration from pktgen config file
+ warmup = int(self.conf["warmup"]) if self.conf.has_key("warmup") \
+ else 25
+ # set trex coremask
+ wait_interval, core_mask = (
+ warmup+30, int(self.conf["core_mask"], 16)) \
+ if self.conf.has_key("core_mask") \
+ else (warmup+5, 0x3)
+
+ try:
+ ###########################################
+ # clear the stats before injecting
+ self._conn.clear_stats()
+ # Start traffic on port(s)
+ run_opt = {
+ 'ports': self._traffic_ports,
+ 'mult': rate_percent,
+ 'duration': duration,
+ 'core_mask':core_mask,
+ 'force': True,}
+ self.logger.info("begin traffic ......")
+ self._conn.start(**run_opt)
+ ###########################################
+ if sample_delay:
+ time.sleep(sample_delay) # wait
+ # get ports runtime statistics
+ self.runtime_stats = self._conn.get_stats()
+ self.logger.info(pformat(self.runtime_stats))
+ ###########################################
+ # Block until traffic on specified port(s) has ended
+ wait_opt = {'ports': self._traffic_ports}
+ if duration:
+ time.sleep(wait_interval + 10)
+ wait_opt['timeout'] = wait_interval + duration
+ self._conn.wait_on_traffic(**wait_opt)
+ except Exception as e:
+ self.logger.error(e)
+
+ def _stop_transmission(self, stream_id):
+ if self._traffic_ports:
+ self._conn.stop(ports=self._traffic_ports, rx_delay_ms=5000)
+
+ def _retrieve_port_statistic(self, stream_id, mode):
+ '''
+ trex traffic statistics
+ '''
+ stats = self._conn.get_stats()
+ stream = self._get_stream(stream_id)
+ self.logger.info(pformat(stream))
+ self.logger.info(pformat(stats))
+ if mode == 'throughput':
+ return self._throughput_stats(stream, stats)
+ elif mode == 'loss':
+ return self._loss_rate_stats(stream, stats)
+ elif mode == 'latency':
+ return self._latency_stats(stream, stats)
+ else:
+ return None
+
+ def quit_generator(self):
+ if self._conn is not None:
+ self._disconnect()
+ if self.control_session is not None:
+ self.tester.send_expect('pkill -f _t-rex-64', '# ')
+ time.sleep(5)
+ self.tester.destroy_session(self.control_session)
+ self.control_session = None
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 1/14] framework/pktgen: pktgen instance creation and helper
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (9 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 0/14] framework/pktgen: trex " yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 2/14] framework/pktgen: packet generator types definition yufengmx
` (3 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
class
pktgen instance creation and helper class
Initialize packet generator instance with custom generator name. Create PacketGeneratorHelper
class to meet with the simple packet generator usage scenario and it is compatible with
old coding style in suite source code. All suite source code should import pktgen objects
from this module.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/pktgen.py | 610 +++++++++++++++-------------------------------------
1 file changed, 172 insertions(+), 438 deletions(-)
diff --git a/framework/pktgen.py b/framework/pktgen.py
index 5294ff8..1c8acac 100644
--- a/framework/pktgen.py
+++ b/framework/pktgen.py
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -29,459 +29,193 @@
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-import os
-import sys
-import re
-import string
-import time
-import json
-import argparse
-import IPy
-import logging
+import os, sys
+from copy import deepcopy
-from abc import abstractmethod
-from config import IxiaConf
-from ssh_connection import SSHConnection
-from settings import SCAPY2IXIA
-from logger import getLogger
-from exception import VerifyFailure
-from utils import create_mask
-from uuid import uuid4
-from pickletools import optimize
-#from serializer import Serializer
+from scapy.all import conf
+from scapy.packet import NoPayload
+from scapy.packet import Packet as scapyPacket
+from scapy.fields import ConditionalField
+from scapy.utils import rdpcap
-FORMAT = '%(message)s'
-logging.basicConfig(format=FORMAT)
-logger = logging.getLogger('TrexA')
-logger.setLevel(logging.INFO)
-# change operation directory
+# import dts libs
cwd = os.getcwd()
sys.path.append(cwd + '/nics')
sys.path.append(cwd + '/framework')
sys.path.append(cwd + '/tests')
sys.path.append(cwd + '/dep')
-from crb import Crb
-from config import PktgenConf, CrbsConf, PortConf
-
-
-class PacketGenerator(object):
-#class PacketGenerator(Crb):
- """
- Basic class for packet generator, define basic function for each kinds of
- generators
- """
- def __init__(self, tester):
- self.__streams = []
- self._ports_map = []
- self.tester = tester
-
- @abstractmethod
- def _check_options(self, opts={}):
- pass
-
- def prepare_generator(self):
- self._prepare_generator()
-
- # extened tester port map and self port map
- ports = self._get_ports()
- print ports
- tester_portnum = len(self.tester.ports_info)
- for port_idx in range(len(ports)):
- port_info = {'type': '%s' % self.pktgen_type, 'pci': '%s' % ports[port_idx]}
- self._ports_map.append(tester_portnum + port_idx)
- self.tester.ports_info.append(port_info)
- print self._ports_map
- # update dut port map
- portconf = PortConf()
- for dut in self.tester.duts:
- dut.map_available_ports()
-
- def _convert_pktgen_port(self, port_id):
- try:
- port = self._ports_map[port_id]
- except:
- port = -1
-
- return port
-
- def _convert_tester_port(self, port_id):
- try:
- port = self._ports_map.index(port_id)
- except:
- port = -1
-
- return port
-
- @abstractmethod
- def _prepare_transmission(self, stream_ids=[]):
- pass
-
- @abstractmethod
- def _start_transmission(self, stream_ids, delay=50):
- pass
-
- @abstractmethod
- def _stop_transmission(self, stream_id):
- pass
-
- @abstractmethod
- def _retrieve_port_statistic(self, stream_id):
- pass
-
- def add_stream(self, tx_port, rx_port, pcap_file):
- stream_id = None
-
- pktgen_tx_port = self._convert_tester_port(tx_port)
- pktgen_rx_port = self._convert_tester_port(rx_port)
-
- stream_id = len(self.__streams)
- stream = {'tx_port': pktgen_tx_port,
- 'rx_port': pktgen_rx_port,
- 'pcap_file': pcap_file}
- self.__streams.append(stream)
-
- return stream_id
-
- def config_stream(self, stream_id=0, opts={}):
- if self._check_options(opts) is not True:
- self.logger.error("Failed to configure stream[%d]" % stream_id)
+# dts libs
+from utils import (convert_int2ip, convert_ip2int,
+ convert_mac2long, convert_mac2str)
+
+from pktgen_base import (PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA,
+ TRANSMIT_CONT, TRANSMIT_M_BURST, TRANSMIT_S_BURST)
+from pktgen_base import DpdkPacketGenerator
+from pktgen_ixia import IxiaPacketGenerator
+from pktgen_trex import TrexPacketGenerator
+
+
+class PacketGeneratorHelper(object):
+ ''' default packet generator stream option for all streams '''
+ default_opt = {
+ 'stream_config':{
+ 'txmode' : {},
+ 'transmit_mode': TRANSMIT_CONT,
+ # for temporary usage because current pktgen design don't support
+ # port level configuration, here using stream configuration to pass
+ # rate percent
+ 'rate': 100,}}
+
+ def __init__(self):
+ self.packetLayers = dict()
+
+ def _parse_packet_layer(self, pkt_object):
+ ''' parse one packet every layers' fields and value '''
+ if pkt_object == None:
return
- stream = self.__streams[stream_id]
- stream['options'] = opts
-
- def measure_throughput(self, stream_ids=[], delay=50):
- """
- Measure throughput on each tx ports
- """
-
- bps_rx = []
- pps_rx = []
- self._prepare_transmission(stream_ids=stream_ids)
- self._start_transmission(stream_ids)
-
- time.sleep(delay)
- used_rx_port = []
- for stream_id in stream_ids:
- if self.__streams[stream_id]['rx_port'] not in used_rx_port:
- rxbps_rates, rxpps_rates = self._retrieve_port_statistic(stream_id)
- used_rx_port.append(self.__streams[stream_id]['rx_port'])
- bps_rx.append(rxbps_rates)
- pps_rx.append(rxpps_rates)
- self._stop_transmission(stream_id)
- bps_rx_total = self._summary_statistic(bps_rx)
- pps_rx_total = self._summary_statistic(pps_rx)
-
- print "throughput: pps_rx %f, bps_rx %f" % (pps_rx_total, bps_rx_total)
-
- return bps_rx_total, pps_rx_total
-
- def _summary_statistic(self, array=[]):
- """
- Summary all values in statistic array
- """
- summary = 0.000
- for value in array:
- summary += value
-
- return summary
-
- def _get_stream(self, stream_id):
- return self.__streams[stream_id]
-
- def _get_generator_conf_instance(self):
- conf_inst = PktgenConf(self.pktgen_type)
- return conf_inst
-
- @abstractmethod
- def quit_generator(self):
- pass
-
-class TrexPacketGenerator(PacketGenerator):
- """
- Trex packet generator, detail usage can be seen at
- https://trex-tgn.cisco.com/trex/doc/trex_manual.html
- """
- def __init__(self, tester):
- self.pktgen_type = "trex"
- self._conn = None
- self.control_session = None
- self._ports = []
- self._traffic_ports = []
- self._transmit_streams = {}
- self.trex_app = "scripts/t-rex-64"
-
- self.conf_inst = self._get_generator_conf_instance()
- self.conf = self.conf_inst.load_pktgen_config()
- self.options_keys = [ 'rate', 'ip', 'vlan']
- self.ip_keys = ['start', 'end','action', 'mask', 'step']
- self.vlan_keys = ['start', 'end', 'action', 'step', 'count']
- super(TrexPacketGenerator, self).__init__(tester)
-
- def connect(self):
- self._conn = self.trex_client(server=self.conf["server"])
- self._conn.connect()
- for p in self._conn.get_all_ports():
- self._ports.append(p)
-
- logger.debug(self._ports)
-
- def _get_ports(self):
- """
- Return self ports information
- """
- ports = []
- for idx in range(len(self._ports)):
- ports.append('TREX:%d' % idx)
- return ports
-
- def disconnect(self):
- self._conn.disconnect()
-
- def _check_options(self, opts={}):
- for key in opts:
- if key in self.options_keys:
- if key == 'ip':
- ip = opts['ip']
- for ip_key in ip:
- if not ip_key in self.ip_keys:
- print " %s is invalid ip option" % ip_key
- return False
- if key == 'action':
- if not ip[key] == 'inc' or not ip[key] == 'dec':
- print " %s is invalid ip action" % ip[key]
- return False
- elif key == 'vlan':
- vlan = opts['vlan']
- for vlan_key in vlan:
- if not vlan_key in self.vlan_keys:
- print " %s is invalid vlan option" % vlan_key
- return False
- if key == 'action':
- if not vlan[key] == 'inc' or not ip[key] == 'dec':
- print " %s is invalid vlan action" % vlan[key]
- return False
- else:
- print " %s is invalid option" % key
- return False
- return True
-
- def create_vm (self, ip_src_range, ip_dst_range, action='inc', step=1):
- if not ip_src_range and not ip_dst_range:
- return None
-
- vm = []
-
- if ip_src_range:
- vm += [self.trex_vm_flow(name="src", min_value = ip_src_range['start'], max_value = ip_src_range['end'], size = 4, op = action),
- self.trex_vm_wr_flow(fv_name="src",pkt_offset= "IP.src")
- ]
-
- if ip_dst_range:
- vm += [self.trex_vm_flow(name="dst", min_value = ip_dst_range['start'], max_value = ip_dst_range['end'], size = 4, op = action),
- self.trex_vm_wr_flow(fv_name="dst",pkt_offset = "IP.dst")
- ]
-
- vm += [self.trex_vm_ipv4(offset = "IP")
- ]
-
- return vm
-
- def _prepare_generator(self):
- if self.conf.has_key('start_trex') and self.conf['start_trex']:
- app_param_temp = "-i"
-
- for key in self.conf:
- #key, value = pktgen_conf
- if key == 'config_file':
- app_param_temp = app_param_temp + " --cfg " + self.conf[key]
- elif key == 'core_num':
- app_param_temp = app_param_temp + " -c " + self.conf[key]
- app = self.conf['trex_root_path'] + os.sep + self.trex_app
- cmd = app + " " + app_param_temp
- self.control_session = self.tester.create_session('trex_control_session')
- self.control_session.send_expect('cd ' + self.conf['trex_root_path'] + os.sep + 'scripts', '# ')
- self.control_session.send_expect(app + " " + app_param_temp, '-Per port stats table', 30)
-
- # Insert Trex api library
- sys.path.insert(0, "{0}/scripts/automation/trex_control_plane/stl".format(self.conf['trex_root_path']))
- #from trex_stl_lib.api import *
- from trex_stl_lib.api import STLStream, STLPktBuilder, STLTXCont, STLVmFlowVar, STLVmWrFlowVar,\
- STLVmFixIpv4
-
- mod = __import__("trex_stl_lib.api")
- client_mod = getattr(mod, "trex_stl_client", None)
- self.trex_client = getattr(client_mod, "STLClient", None)
- self.trex_vm_flow = getattr(client_mod, "STLVmFlowVar", None)
- self.trex_vm_wr_flow = getattr(client_mod, "STLVmWrFlowVar", None)
- self.trex_vm_ipv4 = getattr(client_mod, "STLVmFixIpv4", None)
- self.trex_stream = getattr(client_mod, "STLStream", None)
- self.trex_pkt_builder = getattr(client_mod, "STLPktBuilder", None)
- self.trex_tx_count = getattr(client_mod, "STLTXCont", None)
-
- self.connect()
- #self.control_session.send_expect("cd " + cwd, "", 70)
-
- def _prepare_transmission(self, stream_ids=[]):
- # Create base packet and pad it to size
- streams = []
- ip_src_range = {}
- ip_dst_range = {}
- ip_src_range_temp = []
- ip_dst_range_temp = []
-
- # prepare stream configuration
- for stream_id in stream_ids:
- stream = self._get_stream(stream_id)
- tx_port = stream['tx_port']
- rx_port = stream['rx_port']
- rx_port_name = "port%d" % rx_port
- option = stream['options']
- pcap_file = stream["pcap_file"]
- #set rate
- rate = option['rate']
- if "ip" not in option:
- stl_stream = self.trex_stream(packet=self.trex_pkt_builder(pkt=pcap_file), mode=self.trex_tx_count(percentage=100))
- self._transmit_streams[stream_id] = stl_stream
+ self.packetLayers[pkt_object.name] = dict()
+ for curfield in pkt_object.fields_desc:
+ if isinstance(curfield, ConditionalField) and \
+ not curfield._evalcond(pkt_object):
continue
+ field_value = pkt_object.getfieldval(curfield.name)
+ if isinstance(field_value, scapyPacket) or (curfield.islist and \
+ curfield.holds_packets and type(field_value) is list):
+ continue
+ repr_value = curfield.i2repr(pkt_object, field_value)
+ if isinstance(repr_value, str):
+ repr_value = repr_value.replace(os.linesep,
+ os.linesep + " "*(len(curfield.name) +4))
+ self.packetLayers[pkt_object.name][curfield.name] = repr_value
- ip = option['ip']
- mask = ip['mask']
- step_temp = ip['step'].split('.')
-
- #get the subnet range of src and dst ip
- if self.conf.has_key("ip_src"):
- ip_src = self.conf['ip_src']
- ip_src_range_string = IPy.IP(IPy.IP(ip_src).make_net(mask).strNormal()).strNormal(3)
- ip_src_range_temp = ip_src_range_string.split('-')
- ip_src_range['start'] = ip_src_range_temp[0]
- ip_src_range['end'] = ip_src_range_temp[1]
-
- if self.conf.has_key("ip_dst"):
- ip_dst = self.conf['ip_dst']
- ip_dst_range_string = IPy.IP(IPy.IP(ip_dst).make_net(mask).strNormal()).strNormal(3)
- ip_dst_range_temp = ip_dst_range_string.split('-')
- ip_dst_range['start'] = ip_dst_range_temp[0]
- ip_dst_range['end'] = ip_dst_range_temp[1]
-
- # pcap_file = stream['pcap_file']
-
- vm = self.create_vm(ip_src_range, ip_dst_range, action=ip['action'], step=step_temp[3])
-
- stl_stream = self.trex_stream(
- packet=self.trex_pkt_builder(pkt=pcap_file, vm=vm),
- mode=self.trex_tx_count(percentage=100))
-
- self._transmit_streams[stream_id] = stl_stream
-
- def _start_transmission(self, stream_ids, delay=50):
- self._conn.reset(ports=self._ports)
- self._conn.clear_stats()
- self._conn.set_port_attr(self._ports, promiscuous=True)
- duration_int = int(self.conf["duration"])
- rate = "100%"
- warmup = 15
-
- if self.conf.has_key("warmup"):
- warmup = int(self.conf["warmup"])
-
- self._traffic_ports = []
- for stream_id in stream_ids:
- stream = self._get_stream(stream_id)
- # tester port to Trex port
- tx_port = stream["tx_port"]
- p = self._ports[tx_port]
- self._conn.add_streams(self._transmit_streams[stream_id], ports=[p])
- rate = stream["options"]["rate"]
- self._traffic_ports.append(p)
-
- print self._traffic_ports
-
- if self.conf.has_key("core_mask"):
- self._conn.start(ports=self._traffic_ports, mult=rate, duration=warmup, core_mask=self.conf["core_mask"])
- self._conn.wait_on_traffic(ports=self._traffic_ports, timeout=warmup+30)
+ if isinstance(pkt_object.payload, NoPayload):
+ return
else:
- self._conn.start(ports=self._traffic_ports, mult=rate, duration=warmup)
- self._conn.wait_on_traffic(ports=self._traffic_ports, timeout=warmup+30)
-
- self._conn.clear_stats()
-
- if self.conf.has_key("core_mask"):
- self._conn.start(ports=self._traffic_ports, mult=rate, duration=duration_int, core_mask=self.conf["core_mask"])
+ self._parse_packet_layer(pkt_object.payload)
+
+ def _parse_pcap(self, pcapFile, number=0):
+ ''' parse one packet content '''
+ pcap_pkts = []
+ if os.path.exists(pcapFile) == False:
+ warning = "{0} is not exist !".format(pcapFile)
+ raise Exception(warning)
+
+ pcap_pkts = rdpcap(pcapFile)
+ # parse packets' every layers and fields
+ if len(pcap_pkts) == 0:
+ warning = "{0} is empty".format(pcapFile)
+ raise Exception(warning)
+ elif number>= len(pcap_pkts):
+ warning = "{0} is missing No.{1} packet".format(pcapFile, number)
+ raise Exception(warning)
else:
- self._conn.start(ports=self._traffic_ports, mult=rate, duration=duration_int)
-
- if self._conn.get_warnings():
- for warning in self._conn.get_warnings():
- logger.warn(warning)
-
- def _stop_transmission(self, stream_id):
- self._conn.stop(ports=self._traffic_ports, rx_delay_ms=5000)
-
- def _retrieve_port_statistic(self, stream_id):
- stats = self._conn.get_stats()
- stream = self._get_stream(stream_id)
- port_id = stream["rx_port"]
- port_stats = stats[port_id]
- print "Port %d stats: %s " % (port_id,port_stats)
- rate_rx_pkts = port_stats["rx_pps"]
- rate_rx_bits = port_stats["rx_bps_L1"]
- print "rx_port: %d, rate_rx_pkts: %f, rate_rx_bits:%f " % (port_id,rate_rx_pkts,rate_rx_bits)
- return rate_rx_bits, rate_rx_pkts
-
- def quit_generator(self):
- if self._conn is not None:
- self.disconnect()
- if self.control_session is not None:
- self.tester.send_expect('pkill -f _t-rex-64', '# ')
- time.sleep(5)
- self.tester.destroy_session(self.control_session)
- self.control_session = None
-
-def getPacketGenerator(tester, pktgen_type="trex"):
+ self._parse_packet_layer(pcap_pkts[number])
+
+ def _set_pktgen_fields_config(self, pcap, suite_config):
+ '''
+ get default fields value from a pcap file and unify layer fields
+ variables for trex/ixia
+ '''
+ self._parse_pcap(pcap)
+ if not self.packetLayers:
+ msg = "pcap content is empty"
+ raise Exception(msg)
+ # suite fields config convert to pktgen fields config
+ fields_config = {}
+ # set ethernet protocol layer fields
+ layer_name = 'mac'
+ if layer_name in suite_config.keys() and \
+ 'Ethernet' in self.packetLayers:
+ fields_config[layer_name] = {}
+ suite_fields = suite_config.get(layer_name)
+ pcap_fields = self.packetLayers.get('Ethernet')
+ for name, config in suite_fields.iteritems():
+ action = config.get('action') or 'default'
+ range = config.get('range') or 64
+ step = config.get('step') or 1
+ start_mac = pcap_fields.get(name)
+ end_mac = convert_mac2str(convert_mac2long(start_mac) + range-1)
+ fields_config[layer_name][name] = {}
+ fields_config[layer_name][name]['start'] = start_mac
+ fields_config[layer_name][name]['end'] = end_mac
+ fields_config[layer_name][name]['step'] = step
+ fields_config[layer_name][name]['action'] = action
+ # set ip protocol layer fields
+ layer_name = 'ip'
+ if layer_name in suite_config.keys() and \
+ 'IP' in self.packetLayers:
+ fields_config[layer_name] = {}
+ suite_fields = suite_config.get(layer_name)
+ pcap_fields = self.packetLayers.get('IP')
+ for name, config in suite_fields.iteritems():
+ action = config.get('action') or 'default'
+ range = config.get('range') or 64
+ step = config.get('step') or 1
+ start_ip = pcap_fields.get(name)
+ end_ip = convert_int2ip(convert_ip2int(start_ip) + range - 1)
+ fields_config[layer_name][name] = {}
+ fields_config[layer_name][name]['start'] = start_ip
+ fields_config[layer_name][name]['end'] = end_ip
+ fields_config[layer_name][name]['step'] = step
+ fields_config[layer_name][name]['action'] = action
+ # set vlan protocol layer fields, only support one layer vlan here
+ layer_name = 'vlan'
+ if layer_name in suite_config.keys() and \
+ '802.1Q' in self.packetLayers:
+ fields_config[layer_name] = {}
+ suite_fields = suite_config.get(layer_name)
+ pcap_fields = self.packetLayers.get('802.1Q')
+ # only support one layer vlan here, so set name to `0`
+ name = 0
+ if name in suite_fields.keys():
+ config = suite_fields[name]
+ action = config.get('action') or 'default'
+ range = config.get('range') or 64
+ # ignore 'L' suffix
+ start_vlan = int(pcap_fields.get(layer_name)[:-1])
+ end_vlan = start_vlan + range - 1
+ fields_config[layer_name][name] = {}
+ fields_config[layer_name][name]['start'] = start_vlan
+ fields_config[layer_name][name]['end'] = end_vlan
+ fields_config[layer_name][name]['step'] = 1
+ fields_config[layer_name][name]['action'] = action
+
+ return fields_config
+
+ def prepare_stream_from_tginput(self, tgen_input, ratePercent,
+ vm_config, pktgen_inst):
+ ''' create streams for ports, one port one stream '''
+ # set stream in pktgen
+ stream_ids = []
+ for config in tgen_input:
+ stream_id = pktgen_inst.add_stream(*config)
+ pcap = config[2]
+ _options = deepcopy(self.default_opt)
+ _options['pcap'] = pcap
+ # if vm is set
+ if vm_config:
+ _options['fields_config'] = \
+ self._set_pktgen_fields_config(pcap, vm_config)
+ pktgen_inst.config_stream(stream_id, _options)
+ stream_ids.append(stream_id)
+ return stream_ids
+
+def getPacketGenerator(tester, pktgen_type=PKTGEN_IXIA):
"""
Get packet generator object
"""
pktgen_type = pktgen_type.lower()
- if pktgen_type == "dpdk":
- return DpdkPacketGenerator(tester)
- elif pktgen_type == "ixia":
- return IxiaPacketGenerator(tester)
- elif pktgen_type == "trex":
- return TrexPacketGenerator(tester)
-
-
-if __name__ == "__main__":
- # init pktgen stream options
-
- from tester import Tester
- options = {
- 'rate' : '100%',
- 'ip': {'action': 'inc', 'mask' : '255.255.255.0', 'step':'0.0.0.1'}
- }
- crbsconf = CrbsConf()
- crb = (crbsconf.load_crbs_config())[0]
- tester = Tester(crb, None)
- # framework initial
- trex = getPacketGenerator(tester, pktgen_type="trex")
-
- conf = conf_inst.load_pktgen_config()
- # prepare running environment
- trex.prepare_generator()
-
- #config stream and convert options into pktgen commands
- stream_id1 = trex.add_stream(0, 1, conf['pcap_file'])
- trex.config_stream(stream_id=stream_id1, opts=options)
- stream_id2 = trex.add_stream(1, 0, conf['pcap_file'])
- trex.config_stream(stream_id=stream_id2, opts=options)
- stream_id3 = trex.add_stream(0, 1, conf['pcap_file'])
- trex.config_stream(stream_id=stream_id3, opts=options)
- stream_id4 = trex.add_stream(1, 0, conf['pcap_file'])
- trex.config_stream(stream_id=stream_id4, opts=options)
- #pktgen.prepare_transmission(stream_ids=[stream_id])
- trex.measure_throughput(stream_ids=[stream_id1,stream_id2,stream_id3,stream_id4], delay=5)
- #trex.measure_throughput(stream_ids=[stream_id1,stream_id2], delay=5)
- # comeback to framework
- trex.quit_generator()
+ pktgen_cls = {
+ PKTGEN_DPDK: DpdkPacketGenerator,
+ PKTGEN_IXIA: IxiaPacketGenerator,
+ PKTGEN_TREX: TrexPacketGenerator,}
+
+ if pktgen_type in pktgen_cls.keys():
+ CLS = pktgen_cls.get(pktgen_type)
+ return CLS(tester)
+ else:
+ msg = "not support <{0}> packet generator".format(pktgen_type)
+ raise Exception(msg)
\ No newline at end of file
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 2/14] framework/pktgen: packet generator types definition
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (10 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 1/14] framework/pktgen: pktgen instance creation and helper yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 3/14] framework/pktgen: packet generator init and trex port yufengmx
` (2 subsequent siblings)
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
packet generator types definition
define packet generator types.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/settings.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/framework/settings.py b/framework/settings.py
index f8301c1..707a9ab 100644
--- a/framework/settings.py
+++ b/framework/settings.py
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -194,7 +194,11 @@ TIMEOUT = 15
Global macro for dts.
"""
IXIA = "ixia"
-
+PKTGEN = "pktgen"
+PKTGEN_DPDK = "dpdk"
+PKTGEN_TREX = "trex"
+PKTGEN_IXIA = "ixia"
+PKTGEN_GRP = frozenset([PKTGEN_DPDK, PKTGEN_TREX, PKTGEN_IXIA])
"""
The log name seperater.
"""
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 3/14] framework/pktgen: packet generator init and trex port
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (11 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 2/14] framework/pktgen: packet generator types definition yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 4/14] framework/pktgen: utils methods yufengmx
2019-05-29 2:45 ` [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api Tu, Lijuan
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
mapping
packet generator init and trex port mapping
Move pktgen init process in init_ext_gen method(). Add is_pktgen() to check if
pktgen module configuration is set by user. Add restore_trex_interfaces() to restore
ports used by trex tool after trex tool quit. Use crbs.cfg setting to avoid duplicate
parsing pktgen.cfg content. Set pktgen quit process before dts default sessions close.
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/tester.py | 136 +++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 113 insertions(+), 23 deletions(-)
mode change 100755 => 100644 framework/tester.py
diff --git a/framework/tester.py b/framework/tester.py
old mode 100755
new mode 100644
index ce02b7f..9a6f6aa
--- a/framework/tester.py
+++ b/framework/tester.py
@@ -1,6 +1,6 @@
# BSD LICENSE
#
-# Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+# Copyright(c) 2010-2019 Intel Corporation. All rights reserved.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
@@ -38,16 +38,18 @@ import subprocess
import os
from time import sleep
from settings import NICS, load_global_setting, PERF_SETTING
+from settings import IXIA, USERNAME, PKTGEN, PKTGEN_GRP
from crb import Crb
from net_device import GetNicObj
from etgen import IxiaPacketGenerator, SoftwarePacketGenerator
-from settings import IXIA, USERNAME
import random
from utils import GREEN, convert_int2ip, convert_ip2int
from exception import ParameterInvalidException
from multiprocessing import Process
-from pktgen import getPacketGenerator, PktgenConf
+from pktgen import getPacketGenerator
+from config import PktgenConf
+
class Tester(Crb):
"""
@@ -71,13 +73,19 @@ class Tester(Crb):
self.bgCmds = []
self.bgItf = ''
self.re_run_time = 0
+ self.pktgen = None
+ self.ixia_packet_gen = None
def init_ext_gen(self):
"""
Initialize tester packet generator object.
"""
if self.it_uses_external_generator():
- self.ixia_packet_gen = IxiaPacketGenerator(self)
+ if self.is_pktgen:
+ self.pktgen_init()
+ else:
+ self.ixia_packet_gen = IxiaPacketGenerator(self)
+ return
self.packet_gen = SoftwarePacketGenerator(self)
def set_re_run(self, re_run_time):
@@ -91,7 +99,7 @@ class Tester(Crb):
Get ip address of tester CRB.
"""
return self.crb['tester IP']
-
+
def get_username(self):
"""
Get login username of tester CRB.
@@ -104,12 +112,32 @@ class Tester(Crb):
"""
return self.crb['tester pass']
+ @property
+ def is_pktgen(self):
+ """
+ Check whether packet generator is configured.
+ """
+ if PKTGEN not in self.crb or not self.crb[PKTGEN]:
+ return False
+
+ if self.crb[PKTGEN].lower() in PKTGEN_GRP:
+ return True
+ else:
+ msg = os.linesep.join([
+ "Packet generator <{0}> is not supported".format(self.crb[PKTGEN]),
+ "Current supports: {0}".format(' | '.join(PKTGEN_GRP))])
+ self.logger.info(msg)
+ return False
+
def has_external_traffic_generator(self):
"""
Check whether performance test will base on IXIA equipment.
"""
try:
- if self.crb[IXIA] is not None:
+ # if pktgen_group is set, take pktgen config file as first selection
+ if self.is_pktgen:
+ return True
+ elif self.crb[IXIA] is not None:
return True
except Exception as e:
return False
@@ -140,7 +168,6 @@ class Tester(Crb):
self.pci_devices_information()
self.restore_interfaces()
self.scan_ports()
- self.pktgen_init()
def get_local_port(self, remotePort):
"""
@@ -251,6 +278,30 @@ class Tester(Crb):
sleep(2)
+ def restore_trex_interfaces(self):
+ """
+ Restore Linux interfaces used by trex
+ """
+ try:
+ for port_info in self.ports_info:
+ nic_type = port_info.get('type')
+ if nic_type is not 'trex':
+ continue
+ pci_bus = port_info.get('pci')
+ port_inst = port_info.get('port')
+ port_inst.bind_driver()
+ itf = port_inst.get_interface_name()
+ self.enable_ipv6(itf)
+ self.send_expect("ifconfig %s up" % itf, "# ")
+ if port_inst.get_interface2_name():
+ itf = port_inst.get_interface2_name()
+ self.enable_ipv6(itf)
+ self.send_expect("ifconfig %s up" % itf, "# ")
+ except Exception as e:
+ self.logger.error(" !!! Restore ITF: " + e.message)
+
+ sleep(2)
+
def set_promisc(self):
try:
for (pci_bus, pci_id) in self.pci_devices_info:
@@ -284,6 +335,28 @@ class Tester(Crb):
cached_ports_info.append(port_info)
self.serializer.save(self.PORT_INFO_CACHE_KEY, cached_ports_info)
+ def _scan_pktgen_ports(self):
+ ''' packet generator port setting
+ Currently, trex run on tester node
+ '''
+ pktgen_ports_info = self.pktgen.get_ports()
+ for pktgen_port_info in pktgen_ports_info:
+ pktgen_port_type = pktgen_port_info['type']
+ if pktgen_port_type.lower() == 'ixia':
+ self.ports_info.extend(pktgen_ports_info)
+ break
+ pktgen_port_name = pktgen_port_info['intf']
+ pktgen_pci = pktgen_port_info['pci']
+ pktgen_mac = pktgen_port_info['mac']
+ for port_info in self.ports_info:
+ dts_pci = port_info['pci']
+ if dts_pci != pktgen_pci:
+ continue
+ port_info['intf'] = pktgen_port_name
+ port_info['type'] = pktgen_port_type
+ port_info['mac'] = pktgen_mac
+ break
+
def scan_ports(self):
"""
Scan all ports on tester and save port's pci/mac/interface.
@@ -295,7 +368,10 @@ class Tester(Crb):
if not self.read_cache or self.ports_info is None:
self.scan_ports_uncached()
if self.it_uses_external_generator():
- self.ports_info.extend(self.ixia_packet_gen.get_ports())
+ if self.is_pktgen:
+ self._scan_pktgen_ports()
+ else:
+ self.ports_info.extend(self.ixia_packet_gen.get_ports())
self.save_serializer_ports()
for port_info in self.ports_info:
@@ -359,7 +435,7 @@ class Tester(Crb):
'type': pci_id,
'intf': intf,
'mac': macaddr,
- 'ipv4': ipv4,
+ 'ipv4': ipv4,
'ipv6': ipv6})
# return if port is not connect x3
@@ -382,14 +458,14 @@ class Tester(Crb):
'ipv6': ipv6})
def pktgen_init(self):
- pktgen = PktgenConf()
- pktgen_inst_type = pktgen.pktgen_conf.get_sections()
- if len(pktgen_inst_type) == 1 and pktgen_inst_type[0] == "TREX":
- pktgen_type = "TREX"
- # init packet generator instance
- self.pktgen = getPacketGenerator(self, pktgen_type)
- # prepare running environment
- self.pktgen.prepare_generator()
+ '''
+ initialize packet generator instance
+ '''
+ pktgen_type = self.crb[PKTGEN]
+ # init packet generator instance
+ self.pktgen = getPacketGenerator(self, pktgen_type)
+ # prepare running environment
+ self.pktgen.prepare_generator()
def send_ping(self, localPort, ipv4, mac):
"""
@@ -404,10 +480,14 @@ class Tester(Crb):
"""
Send ping6 packet from local port with destination ipv6 address.
"""
- if self.ports_info[localPort]['type'].lower() == 'ixia':
+ if self.is_pktgen:
+ if self.ports_info[localPort]['type'].lower() in 'ixia':
+ return self.packet_gen.send_ping6(
+ self.ports_info[localPort]['pci'], mac, ipv6)
+ elif self.ports_info[localPort]['type'].lower() == 'trex':
+ return "Not implemented yet"
+ elif self.ports_info[localPort]['type'].lower() in 'ixia':
return self.ixia_packet_gen.send_ping6(self.ports_info[localPort]['pci'], mac, ipv6)
- elif self.ports_info[localPort]['type'].lower() == 'trex':
- return "Not implemented yet"
else:
return self.send_expect("ping6 -w 5 -c 5 -A %s%%%s" % (ipv6, self.ports_info[localPort]['intf']), "# ", 10)
@@ -686,6 +766,10 @@ class Tester(Crb):
"""
Update packet generator function, will implement later.
"""
+ # packet generator has forbidden suite class to override parent class methods
+ if self.is_pktgen:
+ return
+ # discard this in future
if self.it_uses_external_generator():
self.ixia_packet_gen.__class__ = clazz
current_attrs = instance.__dict__
@@ -770,19 +854,25 @@ class Tester(Crb):
"""
Close ssh session and IXIA tcl session.
"""
+ if self.it_uses_external_generator():
+ if self.is_pktgen and self.pktgen:
+ self.pktgen.quit_generator()
+ self.restore_trex_interfaces()
+ self.pktgen = None
+ elif self.ixia_packet_gen:
+ self.ixia_packet_gen.close()
+ self.ixia_packet_gen = None
+
if self.session:
self.session.close()
self.session = None
if self.alt_session:
self.alt_session.close()
self.alt_session = None
- if self.it_uses_external_generator():
- self.ixia_packet_gen.close()
def crb_exit(self):
"""
Close all resource before crb exit
"""
- self.pktgen.quit_generator()
self.logger.logger_exit()
self.close()
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dts] [next][PATCH V1 4/14] framework/pktgen: utils methods
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (12 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 3/14] framework/pktgen: packet generator init and trex port yufengmx
@ 2019-04-28 2:49 ` yufengmx
2019-05-29 2:45 ` [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api Tu, Lijuan
14 siblings, 0 replies; 16+ messages in thread
From: yufengmx @ 2019-04-28 2:49 UTC (permalink / raw)
To: dts; +Cc: yufengmx
new utils methods used by pktgen
Set convert_int2ip/convert_ip2int input parameter with a default ipv4 value.
Add convert_mac2long/convert_mac2str methods to deal with mac convert(str <--> int).
Signed-off-by: yufengmx <yufengx.mo@intel.com>
---
framework/utils.py | 32 ++++++++++++++++++++++++++++++--
1 file changed, 30 insertions(+), 2 deletions(-)
diff --git a/framework/utils.py b/framework/utils.py
index ce97d5f..8f5a233 100644
--- a/framework/utils.py
+++ b/framework/utils.py
@@ -214,7 +214,11 @@ def create_mask(indexes):
return hex(val).rstrip("L")
-def convert_int2ip(value, ip_type):
+def convert_int2ip(value, ip_type=4):
+ '''
+ @change:
+ 2019.0403 set default value
+ '''
if ip_type == 4:
ip_str = socket.inet_ntop(socket.AF_INET, struct.pack('!I', value))
else:
@@ -224,7 +228,11 @@ def convert_int2ip(value, ip_type):
return ip_str
-def convert_ip2int(ip_str, ip_type):
+def convert_ip2int(ip_str, ip_type=4):
+ '''
+ @change:
+ 2019.0403 set default value
+ '''
if ip_type == 4:
ip_val = struct.unpack("!I", socket.inet_aton(ip_str))[0]
else:
@@ -234,6 +242,26 @@ def convert_ip2int(ip_str, ip_type):
return ip_val
+def convert_mac2long(mac_str):
+ """
+ convert the MAC type from the string into the int.
+ """
+ mac_hex = '0x'
+ for mac_part in mac_str.lower().split(':'):
+ mac_hex += mac_part
+ ret = long(mac_hex, 16)
+ return ret
+
+def convert_mac2str(mac_long):
+ """
+ convert the MAC type from the int into the string.
+ """
+ mac = hex(mac_long)[2:-1].zfill(12)
+ b = []
+ [b.append(mac[n:n+2]) for n in range(len(mac)) if n % 2 == 0 ]
+ new_mac = ":".join(b)
+ return new_mac
+
def get_backtrace_object(file_name, obj_name):
import inspect
frame = inspect.currentframe()
--
1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
` (13 preceding siblings ...)
2019-04-28 2:49 ` [dts] [next][PATCH V1 4/14] framework/pktgen: utils methods yufengmx
@ 2019-05-29 2:45 ` Tu, Lijuan
14 siblings, 0 replies; 16+ messages in thread
From: Tu, Lijuan @ 2019-05-29 2:45 UTC (permalink / raw)
To: Mo, YufengX, dts; +Cc: Mo, YufengX
Applied the series, thanks
> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of yufengmx
> Sent: Sunday, April 28, 2019 10:49 AM
> To: dts@dpdk.org
> Cc: Mo, YufengX <yufengx.mo@intel.com>
> Subject: [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api
>
> dts packet generator(pktgen) source code, including:
> *. packet generator configure file definition.
> *. packet generator configure file parse class.
> *. packet generator base class.
> *. ixia packet generator relevant classes.
> *. trex packet generator relevant classes.
> *. packet generator helper class.
> *. trex packet generator port mapping process.
> *. new utils methods used by pktgen.
> *. pktgen api program guide document.
> *. migrate suite source code from etgen api to pktgen api guide document.
> *. trex tool known issue document.
> *. change copyright year.
>
> yufengmx (14):
> conf/pktgen: add pktgen key word in crbs.cfg
> conf/pktgen: packet generator configure file definition
> doc/pktgen: trex tool known issues
> doc/pktgen: migrate from etgen api to pktgen api
> doc/pktgen: pktgen api program guide document
> framework/pktgen: packet generator configure file parse class
> framework/pktgen: initialize pktgen logger
> framework/pktgen: packet generator base class
> framework/pktgen: ixia packet generator relevant classes
> framework/pktgen: trex packet generator relevant classes
> framework/pktgen: pktgen instance creation and helper class
> framework/pktgen: packet generator types definition
> framework/pktgen: packet generator init and trex port mapping
> framework/pktgen: utils methods
>
> conf/crbs.cfg | 3 +
> conf/pktgen.cfg | 23 +-
> doc/dts_gsg/migrate_from_etgen_to_pktgen.rst | 214 ++++
> doc/dts_gsg/pktgen_prog_guide.rst | 518 ++++++++
> doc/dts_gsg/trex_known_issue.rst | 112 ++
> framework/config.py | 107 +-
> framework/dts.py | 21 +-
> framework/pktgen.py | 610 +++------
> framework/pktgen_base.py | 400 ++++++
> framework/pktgen_ixia.py | 1750
> ++++++++++++++++++++++++++
> framework/pktgen_trex.py | 857 +++++++++++++
> framework/settings.py | 8 +-
> framework/tester.py | 136 +-
> framework/utils.py | 32 +-
> 14 files changed, 4284 insertions(+), 507 deletions(-) create mode 100644
> doc/dts_gsg/migrate_from_etgen_to_pktgen.rst
> create mode 100644 doc/dts_gsg/pktgen_prog_guide.rst create mode
> 100644 doc/dts_gsg/trex_known_issue.rst create mode 100644
> framework/pktgen_base.py create mode 100644 framework/pktgen_ixia.py
> create mode 100644 framework/pktgen_trex.py mode change 100755 =>
> 100644 framework/tester.py
>
> --
> 1.9.3
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-05-29 2:45 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-28 2:48 [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 1/14] conf/pktgen: add pktgen key word in crbs.cfg yufengmx
2019-04-28 2:48 ` [dts] [next][PATCH V1 2/14] conf/pktgen: packet generator configure file definition yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 3/14] doc/pktgen: trex tool known issues yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 4/14] doc/pktgen: migrate from etgen api to pktgen api yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 5/14] doc/pktgen: pktgen api program guide document yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 6/14] framework/pktgen: packet generator configure file parse yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 7/14] framework/pktgen: initialize pktgen logger yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 8/14] framework/pktgen: packet generator base class yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 9/14] framework/pktgen: ixia packet generator relevant classes yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 0/14] framework/pktgen: trex " yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 1/14] framework/pktgen: pktgen instance creation and helper yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 2/14] framework/pktgen: packet generator types definition yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 3/14] framework/pktgen: packet generator init and trex port yufengmx
2019-04-28 2:49 ` [dts] [next][PATCH V1 4/14] framework/pktgen: utils methods yufengmx
2019-05-29 2:45 ` [dts] [next][PATCH V1 0/14] [dts/pktgen]: dts packet generator api Tu, Lijuan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).