test suite reviews and discussions
 help / color / Atom feed
* Re: [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test
  2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
@ 2020-11-04  5:37 ` Peng, Yuan
  2020-11-04  7:14 ` Sun, QinX
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Peng, Yuan @ 2020-11-04  5:37 UTC (permalink / raw)
  To: Sun, QinX, dts; +Cc: Sun, QinX

Acked-by Peng, Yuan <yuan.peng@intel.com>


-----Original Message-----
From: dts <dts-bounces@dpdk.org> On Behalf Of sunqin
Sent: Wednesday, November 4, 2020 6:25 PM
To: dts@dpdk.org
Cc: Sun, QinX <qinx.sun@intel.com>
Subject: [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test

These 8 cases take a long time to run, so split them from 3 already existed
 
suites (iavf_fdir, cvl_fdir, cvl_dcf_switch_filter) and make them as a new test suite

sunqin (3):
  test_plans/cvl_limit_value_test: add test plan
  tests/cvl_limit_value_test: add cvl_limit test suite
  conf/cvl_limit_value_test: add config file for new test suite

 conf/cvl_limit_value_test.cfg                 |   5 +
 test_plans/cvl_limit_value_test_test_plan.rst | 594 ++++++++++++
 tests/TestSuite_cvl_limit_value_test.py       | 917 ++++++++++++++++++
 3 files changed, 1516 insertions(+)
 create mode 100644 conf/cvl_limit_value_test.cfg  create mode 100644 test_plans/cvl_limit_value_test_test_plan.rst
 create mode 100644 tests/TestSuite_cvl_limit_value_test.py

--
2.17.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test
  2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
  2020-11-04  5:37 ` Peng, Yuan
@ 2020-11-04  7:14 ` Sun, QinX
  2020-11-04 10:25 ` [dts] [PATCH V1 1/3] test_plans/cvl_limit_value_test sunqin
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Sun, QinX @ 2020-11-04  7:14 UTC (permalink / raw)
  To: dts

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

Tested-by: Sun, QinX <qinx.sun@intel.com>
 
Regards,
Sun Qin

> -----Original Message-----
> From: sunqin <qinx.sun@intel.com>
> Sent: Wednesday, November 4, 2020 6:25 PM
> To: dts@dpdk.org
> Cc: Sun, QinX <qinx.sun@intel.com>
> Subject: [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test


[-- Attachment #2: TestCvlLimitValue.log --]
[-- Type: application/octet-stream, Size: 4419734 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dts]  [PATCH V1 0/3] add test suite cvl_limit_value_test
@ 2020-11-04 10:25 sunqin
  2020-11-04  5:37 ` Peng, Yuan
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: sunqin @ 2020-11-04 10:25 UTC (permalink / raw)
  To: dts; +Cc: sunqin

These 8 cases take a long time to run, so split them from 3 already existed
 
suites (iavf_fdir, cvl_fdir, cvl_dcf_switch_filter) and make them as a new test suite

sunqin (3):
  test_plans/cvl_limit_value_test: add test plan
  tests/cvl_limit_value_test: add cvl_limit test suite
  conf/cvl_limit_value_test: add config file for new test suite

 conf/cvl_limit_value_test.cfg                 |   5 +
 test_plans/cvl_limit_value_test_test_plan.rst | 594 ++++++++++++
 tests/TestSuite_cvl_limit_value_test.py       | 917 ++++++++++++++++++
 3 files changed, 1516 insertions(+)
 create mode 100644 conf/cvl_limit_value_test.cfg
 create mode 100644 test_plans/cvl_limit_value_test_test_plan.rst
 create mode 100644 tests/TestSuite_cvl_limit_value_test.py

-- 
2.17.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dts]  [PATCH V1 1/3] test_plans/cvl_limit_value_test
  2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
  2020-11-04  5:37 ` Peng, Yuan
  2020-11-04  7:14 ` Sun, QinX
@ 2020-11-04 10:25 ` sunqin
  2020-11-04 10:25 ` [dts] [PATCH V1 2/3] tests/cvl_limit_value_test add test suite sunqin
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: sunqin @ 2020-11-04 10:25 UTC (permalink / raw)
  To: dts; +Cc: sunqin

These 8 cases take a long time to run, so split them from 3 already existed

suites(iavf_fdir, cvl_fdir, cvl_dcf_switch_filter) and make them as a new test plan.

Signed-off-by: sunqin <qinx.sun@intel.com>
---
 test_plans/cvl_limit_value_test_test_plan.rst | 594 ++++++++++++++++++
 1 file changed, 594 insertions(+)
 create mode 100644 test_plans/cvl_limit_value_test_test_plan.rst

diff --git a/test_plans/cvl_limit_value_test_test_plan.rst b/test_plans/cvl_limit_value_test_test_plan.rst
new file mode 100644
index 0000000..4c141a1
--- /dev/null
+++ b/test_plans/cvl_limit_value_test_test_plan.rst
@@ -0,0 +1,594 @@
+.. Copyright (c) <2020>, Intel Corporation
+   All rights reserved.
+
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+
+Supported function type
+-----------------------
+
+    validate
+    create
+    destroy
+    flush
+    list
+
+Supported action type
+---------------------
+
+    queue index
+    drop
+    rss queues
+    passthru
+    mark
+    mark/rss
+
+
+Prerequisites
+=============
+
+1. Hardware:
+   columbiaville_25g/columbiaville_100g
+
+2. Software:
+   DPDK: http://dpdk.org/git/dpdk
+   scapy: http://www.secdev.org/projects/scapy/
+
+3. Copy specific ice package to /lib/firmware/intel/ice/ddp/ice.pkg
+   Then reboot server, and compile DPDK
+
+4. Generate 2 VFs on each PF and set mac address for each VF::
+
+    echo 2 > /sys/bus/pci/devices/0000:86:00.0/sriov_numvfs
+    echo 2 > /sys/bus/pci/devices/0000:86:00.1/sriov_numvfs
+    ip link set enp134s0f0 vf 0 mac 00:11:22:33:44:55
+    ip link set enp134s0f0 vf 1 mac 00:11:22:33:44:66
+    ip link set enp134s0f1 vf 0 mac 00:11:22:33:44:77
+    ip link set enp134s0f1 vf 1 mac 00:11:22:33:44:88
+
+   0000:86:00.0 generate 0000:86:01.0 and 0000:86:01.1
+   0000:86:00.1 generate 0000:86:11.0 and 0000:86:11.1
+   define 86:01.0 as vf00, 86:01.1 as vf01, 86:11.0 as vf10, 86:11.1 as vf11.
+   assign mac address of pf0 is 68:05:ca:a3:1a:60,
+   assign mac address of pf1 is 68:05:ca:a3:1a:61.
+
+5. Bind VFs to dpdk driver::
+
+    ./usertools/dpdk-devbind.py -b vfio-pci 86:01.0 86:01.1 86:11.0 86:11.1
+
+5. Launch the app ``testpmd`` with the following arguments::
+
+    ./testpmd -c 0xff -n 6 -w 86:01.0 -w 86:01.1 --file-prefix=vf -- -i --rxq=16 --txq=16
+    testpmd> set fwd rxonly
+    testpmd> set verbose 1
+
+6. on tester side, copy the layer python file to /root::
+
+    cp pfcp.py to /root
+
+   then import layers when start scapy::
+
+    >>> import sys
+    >>> sys.path.append('/root')
+    >>> from pfcp import PFCP
+    >>> from scapy.contrib.gtp import *
+    >>> from scapy.contrib.mpls import *
+
+Test case: Max number
+=====================
+All the max number cases are designed based on 2*100G NIC.
+If the hardware is 4*25G NIC, the guaranteed rule number of PF is 512.
+So in subcase 3 and subcase 4, there can be created at most 14848 rules on 1pf and 2vfs.
+
+Subcase 1: 14336 rules on 1 vf
+------------------------------
+
+1. create 14336 rules on vf00::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.1 / end actions queue index 1 / mark / end
+    ......
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.255 / end actions queue index 1 / mark / end
+
+   all the rules are created successfully.
+
+2. create one more rule::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 1 / mark / end
+
+   the rule failed to create. return the error message.
+
+3. check the rule list, there are 14336 rules listed.
+
+4. send matched packets for rule 0 and rule 14335::
+
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.0")/Raw('x' * 80)],iface="enp134s0f1")
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.55.255")/Raw('x' * 80)],iface="enp134s0f1")
+
+   check all packets are redirected to expected queue with FDIR matched ID=0x0
+
+5. create a rule on vf01, it failed,
+   check the error message, the rule number has expired the max rule number.
+
+6. create a rule on vf10, it failed,
+   check the error message, the rule number has expired the max rule number.
+
+7. flush all the rules, check the rule list,
+   there is no rule listed.
+
+8. verify matched packets for rule 0  and rule 14335 received without FDIR matched ID.
+
+Subcase 2: 14336 rules on 2 vfs of 2pfs
+---------------------------------------
+
+1. start testpmd on vf00::
+
+    ./testpmd -c 0xf -n 6 -w 86:01.0 --file-prefix=vf00 -- -i --rxq=4 --txq=4
+
+   create 1 rule on vf00::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+
+   created successfully, check the rule is listed.
+
+2. start testpmd on vf10::
+
+    ./testpmd -c 0xf0 -n 6 -w 86:0a.0 --file-prefix=vf10 -- -i --rxq=4 --txq=4
+
+   create 14336 rules on vf10::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.1 / end actions queue index 1 / mark / end
+    ......
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.255 / end actions queue index 1 / mark / end
+
+   all the rules except the last one are created successfully.
+   check the rule list, there listed 14335 rules.
+
+3. send matched packet to vf00 and matched packet for rule 14334 to vf10,
+   check all packets are redirected to expected queue with FDIR matched ID=0x0
+
+4. flush all the rules, check the rule list,
+   there is no rule listed.
+
+5. verify matched packet received without FDIR matched ID.
+
+Subcase 3: 1025 rules on 1pf and 14335 rules on 2vfs
+----------------------------------------------------
+each pf can create 1024 rules at least in 2 ports card.
+each pf can create 512 rules at least in 4 ports card.
+there are 14k rules shared by pfs and vfs.
+so 1 pf and 2 vfs can create 15360 rules at most on 2 ports card.
+1 pf and 2 vfs can create 14848 rules at most on 4 ports card.
+
+1. create 1025 rules on pf0::
+
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.0.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.0.1 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+    ......
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.3.255 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.4.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+
+   all the rules can be created successfully::
+
+    Added rule with ID <Rule ID>
+
+   List the rules on pf0::
+
+    ethtool -n enp134s0f0
+
+2. start testpmd on vf00::
+
+    ./testpmd -c 0xf -n 6 -w 86:01.0 --file-prefix=vf00 -- -i --rxq=4 --txq=4
+
+   create 1 rule on vf00::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+
+   created successfully, check the rule is listed.
+
+2. start testpmd on vf10::
+
+    ./testpmd -c 0xf0 -n 6 -w 86:0a.0 --file-prefix=vf10 -- -i --rxq=4 --txq=4
+
+   create 14335 rules on vf10::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.1 / end actions queue index 1 / mark / end
+    ......
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.254 / end actions queue index 1 / mark / end
+
+   all the rules except the last one are created successfully.
+   check the rule list, there listed 14334 rules.
+
+3. send matched packet to vf00 and matched packet for rule 14333 to vf10,
+   check all packets are redirected to expected queue with FDIR matched ID=0x0
+
+4. delete 1 rule on pf0::
+
+    ethtool -N enp134s0f0 delete <Rule ID>
+
+5. create one more rule on vf10::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.254 / end actions queue index 1 / mark / end
+
+   the rule can be created successfully.
+
+6. send matched packet to vf10, it can be redirected to queue 1 with FDIR matched ID=0x0.
+
+7. flush all the rules, check the rule list,
+   there is no rule listed.
+
+8. verify matched packet received without FDIR matched ID.
+
+Subcase 3: 15360 rules on 1pf and 0 rules on 2vfs
+-------------------------------------------------
+each pf can create 1024 rules at least in 2 ports card.
+each pf can create 512 rules at least in 4 ports card.
+there are 14k rules shared by pfs and vfs.
+so 1 pf and 2 vfs can create 15360 rules at most on 2 ports card.
+1 pf and 2 vfs can create 14848 rules at most on 4 ports card.
+so if create 15360/14848 rules on 1 pf, there can't create rule on vf successfully.
+
+1. create 15360 rules on pf0::
+
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.0.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.0.1 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+    ......
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.57.255 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+
+   all the rules can be created successfully::
+
+    Added rule with ID <Rule ID>
+
+2. failed to create one more rule on pf0::
+
+    ethtool -N enp134s0f0 flow-type tcp4 src-ip 192.168.58.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8
+
+3. start testpmd on vf00 and vf10::
+
+    ./testpmd -c 0xf -n 6 -w 86:01.0 -w 86:11.0 --file-prefix=vf00 -- -i --rxq=4 --txq=4
+
+   create 1 rule on vf00::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+
+   failed to create the rule, check there is no rule listed.
+
+   create 1 rule on vf10::
+
+    flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+
+   failed to create the rule, check there is no rule listed.
+
+4. delete 1 rule on pf0::
+
+    ethtool -N enp134s0f0 delete <Rule ID>
+
+5. create 1 rule on vf00::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.254 / end actions queue index 1 / mark / end
+
+   the rule can be created successfully.
+
+   create 1 rule on vf10::
+
+    flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+
+   failed to create the rule, check there is no rule listed.
+
+6. send matched packet to vf00, it can be redirected to queue 1 with FDIR matched ID=0x0.
+   send matched packet to vf10, it is received without FDIR matched ID.
+
+7. delete 1 more rule on pf0::
+
+    ethtool -N enp134s0f0 delete <Rule ID>
+
+8. create 1 rule on vf10::
+
+    flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 1 / mark / end
+
+   the rule can be created successfully.
+
+9. send matched packet to vf00, it can be redirected to queue 1 with FDIR matched ID=0x0.
+   send matched packet to vf10, it can be redirected to queue 1 with FDIR matched ID=0x0.
+
+
+Test case: Stress test
+======================
+
+Subcase 1: add/delete rules
+---------------------------
+
+1. create two rules::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / udp src is 22 dst is 23 / end actions queue index 1 / mark id 0 / end
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / tcp src is 22 dst is 23 / end actions rss queues 2 3 end / mark id 1 / end
+
+   return the message::
+
+    Flow rule #0 created
+    Flow rule #1 created
+
+   list the rules::
+
+    testpmd> flow list 0
+    ID      Group   Prio    Attr    Rule
+    0       0       0       i--     ETH IPV4 UDP => QUEUE MARK
+    1       0       0       i--     ETH IPV4 TCP => RSS MARK
+
+2. delete the rules::
+
+    testpmd> flow flush 0
+
+3. repeat the create and delete operations in step1-2 14336 times.
+
+4. create the two rules one more time, check the rules listed.
+
+5. send matched packet::
+
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/UDP(sport=22,dport=23)/Raw('x' * 80)],iface="enp134s0f1")
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/TCP(sport=22,dport=23)/Raw('x' * 80)],iface="enp134s0f1")
+
+   check packet 1 is redirected to queue 1 with FDIR matched ID=0x0
+   check packet 2 is redirected to queue 2 or queue 3 with FDIR matched ID=0x1
+
+Subcase 2: add/delete rules on two VFs
+--------------------------------------
+
+1. create a rule on each vf::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.0 dst is 192.1.0.0 tos is 4 / tcp src is 22 dst is 23 / end actions queue index 5 / end
+    flow create 1 ingress pattern eth / ipv4 src is 192.168.0.0 dst is 192.1.0.0 tos is 4 / tcp src is 22 dst is 23 / end actions queue index 5 / end
+
+   return the message::
+
+    Flow rule #0 created
+    Flow rule #0 created
+
+   list the rules::
+
+    testpmd> flow list 0
+    ID      Group   Prio    Attr    Rule
+    0       0       0       i--     ETH IPV4 TCP => QUEUE
+    testpmd> flow list 1
+    ID      Group   Prio    Attr    Rule
+    0       0       0       i--     ETH IPV4 TCP => QUEUE
+
+2. delete the rules::
+
+    flow destroy 0 rule 0
+    flow destroy 1 rule 0
+
+3. repeate the create and delete operations in step1-2 14336 times with different IP src address.
+
+4. create the rule on each vf one more time, check the rules listed::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.56.0 dst is 192.1.0.0 tos is 4 / tcp src is 22 dst is 23 / end actions queue index 5 / end
+    flow create 1 ingress pattern eth / ipv4 src is 192.168.56.0 dst is 192.1.0.0 tos is 4 / tcp src is 22 dst is 23 / end actions queue index 5 / end
+
+5. send matched packet::
+
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.56.0",dst="192.1.0.0",tos=4)/TCP(sport=22,dport=23)/Raw('x' * 80)],iface="enp134s0f1")
+    sendp([Ether(dst="00:11:22:33:44:66")/IP(src="192.168.56.0",dst="192.1.0.0",tos=4)/TCP(sport=22,dport=23)/Raw('x' * 80)],iface="enp134s0f1")
+
+   check the packet is redirected to queue 5 of two vfs.
+
+Prerequisites
+=============
+
+1. Hardware:
+   columbiaville_25g/columbiaville_100g
+   design the cases with 2 ports card.
+
+2. Software:
+   DPDK: http://dpdk.org/git/dpdk
+   scapy: http://www.secdev.org/projects/scapy/
+
+3. Copy specific ice package to /lib/firmware/intel/ice/ddp/ice.pkg
+   Then reboot server, and compile DPDK
+
+4. Bind the pf to dpdk driver::
+
+    ./usertools/dpdk-devbind.py -b igb_uio 86:00.0 86:00.1
+
+5. Launch the app ``testpmd`` with the following arguments::
+
+    ./testpmd -c 0xff -n 6 -w 86:00.0 --log-level="ice,7" -- -i --portmask=0xff --rxq=64 --txq=64 --port-topology=loop
+    testpmd> set fwd rxonly
+    testpmd> set verbose 1
+
+   If set UDP tunnel flow rule::
+
+    testpmd> port config 0 udp_tunnel_port add vxlan 4789
+    testpmd> start
+
+   Notes: if need two ports environment, launch ``testpmd`` with the following arguments::
+
+    ./testpmd -c 0xff -n 6 -w 86:00.0 -w 86:00.1 --log-level="ice,7" -- -i --portmask=0xff --rxq=64 --txq=64 --port-topology=loop
+
+Test case: add/delete rules
+---------------------------
+
+1. create two rules::
+
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / udp src is 22 dst is 23 / end actions queue index 1 / mark / end
+    flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / tcp src is 22 dst is 23 / end actions rss queues 2 3 end / mark id 1 / end
+
+   return the message::
+
+    Flow rule #0 created
+    Flow rule #1 created
+
+   list the rules::
+
+    testpmd> flow list 0
+    ID      Group   Prio    Attr    Rule
+    0       0       0       i--     ETH IPV4 UDP => QUEUE MARK
+    1       0       0       i--     ETH IPV4 TCP => RSS MARK
+
+2. delete the rules::
+
+    testpmd> flow flush 0
+
+3. repeate the create and delete operations in step1-2 15360 times.
+
+4. create the two rules one more time, check the rules listed.
+
+5. send matched packet::
+
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/UDP(sport=22,dport=23)/Raw('x' * 80)],iface="enp175s0f0")
+    sendp([Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/TCP(sport=22,dport=23)/Raw('x' * 80)],iface="enp175s0f0")
+
+   check packet 1 is redirected to queue 1 with FDIR matched ID=0x0
+   check packet 2 is redirected to queue 2 or queue 3 with FDIR matched ID=0x1
+
+Prerequisites
+=============
+
+1. Hardware:
+   columbiaville_25g/columbiaville_100g
+   design the cases with 2 ports card.
+
+2. Software:
+   dpdk: http://dpdk.org/git/dpdk
+   scapy: http://www.secdev.org/projects/scapy/
+
+3. Copy specific ice package to /lib/firmware/updates/intel/ice/ddp/ice.pkg,
+   then load driver::
+
+     rmmod ice
+     insmod ice.ko
+
+4. Get the pci device id of DUT, for example::
+
+     ./usertools/dpdk-devbind.py -s
+
+     0000:18:00.0 'Device 1593' if=enp24s0f0 drv=ice unused=vfio-pci
+     0000:18:00.1 'Device 1593' if=enp24s0f1 drv=ice unused=vfio-pci
+
+5. Generate 4 VFs on PF0::
+
+     echo 4 > /sys/bus/pci/devices/0000:18:00.0/sriov_numvfs
+
+     ./usertools/dpdk-devbind.py -s
+     0000:18:01.0 'Ethernet Adaptive Virtual Function 1889' if=enp24s1 drv=iavf unused=vfio-pci
+     0000:18:01.1 'Ethernet Adaptive Virtual Function 1889' if=enp24s1f1 drv=iavf unused=vfio-pci
+     0000:18:01.2 'Ethernet Adaptive Virtual Function 1889' if=enp24s1f2 drv=iavf unused=vfio-pci
+     0000:18:01.3 'Ethernet Adaptive Virtual Function 1889' if=enp24s1f3 drv=iavf unused=vfio-pci
+
+6. Set VF0 as trust::
+
+     ip link set enp24s0f0 vf 0 trust on
+
+7. Bind VFs to dpdk driver::
+
+     modprobe vfio-pci
+     ./usertools/dpdk-devbind.py -b vfio-pci 0000:18:01.0 0000:18:01.1 0000:18:01.2 0000:18:01.3
+
+8. Launch dpdk on VF0 and VF1, and VF0 request DCF mode::
+
+     ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 0000:18:01.0,cap=dcf -w 0000:18:01.1 -- -i
+     testpmd> set portlist 1
+     testpmd> set fwd rxonly
+     testpmd> set verbose 1
+     testpmd> start
+     testpmd> show port info all
+
+   check the VF0 driver is net_ice_dcf.
+
+9. on tester side, copy the layer python file to /root::
+
+      cp pfcp.py to /root
+
+    then import layers when start scapy::
+
+      >>> import sys
+      >>> sys.path.append('/root')
+      >>> from pfcp import PFCP
+      >>> from scapy.contrib.igmp import *
+
+Test case: max rule number
+==========================
+
+Description: 32k switch filter rules can be created on a CVL card,
+and all PFs and VFs share the 32k rules. But the system will first create
+some MAC_VLAN rules in switch table, and as the number of rules increased,
+the hash conflicts in the switch filter table are increased, so we can
+create a total of 32563 switch filter rules on a DCF.
+
+1. create 32563 rules with the same pattern, but different input set::
+
+     testpmd> flow create 0 ingress pattern eth / ipv4 src is 192.168.0.0 / end actions vf id 1 / end
+     testpmd> flow create 0 ingress pattern eth / ipv4 src is 192.168.0.1 / end actions vf id 1 / end
+     ......
+     testpmd> flow create 0 ingress pattern eth / ipv4 src is 192.168.127.177 / end actions vf id 1 / end
+     testpmd> flow list 0
+
+   check the rules exist in the list.
+
+2. create one more rule::
+
+     testpmd> flow create 0 ingress pattern eth / ipv4 src is 192.168.127.178 / end actions vf id 1 / end
+
+   check the rule can not be created successfully, and
+   testpmd provide a friendly output, showing::
+
+     ice_flow_create(): Failed to create flow
+     port_flow_complain(): Caught PMD error type 2 (flow rule (handle)): switch filter create flow fail: Invalid argument
+
+3. check the rule list
+
+     testpmd> flow list 0
+
+   check the rule in step 2 not exists in the list.
+
+4. send 32563 matched packets for rule 0-32562::
+
+     sendp([Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.168.0.0")/TCP(sport=25,dport=23)/("X"*480)], iface="ens786f0", count=1)
+     sendp([Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.168.0.1")/TCP(sport=25,dport=23)/("X"*480)], iface="ens786f0", count=1)
+     ......
+     sendp([Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.168.127.177")/TCP(sport=25,dport=23)/("X"*480)], iface="ens786f0", count=1)
+
+   check port 1 receive the 32563 packets.
+   send 1 mismatched packet::
+
+     sendp([Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.167.0.1")/TCP(sport=25,dport=23)/("X"*480)], iface="ens786f0", count=1)
+
+   check the packet are not to port 1.
+
+5. verify rules can be destroyed::
+
+     testpmd> flow flush 0
+     testpmd> flow list 0
+
+   check the rules not exist in the list.
+   send 32563 matched packets, check the packets are not to port 1.
\ No newline at end of file
-- 
2.17.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dts]  [PATCH V1 2/3] tests/cvl_limit_value_test add test suite
  2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
                   ` (2 preceding siblings ...)
  2020-11-04 10:25 ` [dts] [PATCH V1 1/3] test_plans/cvl_limit_value_test sunqin
@ 2020-11-04 10:25 ` sunqin
  2020-11-04 10:25 ` [dts] [PATCH V1 3/3] conf/cvl_limit_value_test add config file sunqin
  2020-11-11  3:18 ` [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test Tu, Lijuan
  5 siblings, 0 replies; 7+ messages in thread
From: sunqin @ 2020-11-04 10:25 UTC (permalink / raw)
  To: dts; +Cc: sunqin

These 8 cases take a long time to run, so split them from 3 already existed

suites(iavf_fdir, cvl_fdir, cvl_dcf_switch_filter) and make them as a new test suite.

Signed-off-by: sunqin <qinx.sun@intel.com>
---
 tests/TestSuite_cvl_limit_value_test.py | 917 ++++++++++++++++++++++++
 1 file changed, 917 insertions(+)
 create mode 100644 tests/TestSuite_cvl_limit_value_test.py

diff --git a/tests/TestSuite_cvl_limit_value_test.py b/tests/TestSuite_cvl_limit_value_test.py
new file mode 100644
index 0000000..77ce016
--- /dev/null
+++ b/tests/TestSuite_cvl_limit_value_test.py
@@ -0,0 +1,917 @@
+# BSD LICENSE
+#
+# Copyright(c) 2020 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+import re
+import time
+import os
+import copy
+from packet import Packet
+from pmd_output import PmdOutput
+from test_case import TestCase
+import rte_flow_common as rfc
+from rte_flow_common import CVL_TXQ_RXQ_NUMBER
+import utils
+
+# max rule number case: rte_flow_pattern and matched packets will be generated by code, and rte_flow_pattern will be writed to file.
+tv_max_rule_number = {
+    "name": "tv_max_rule_number",
+    "rte_flow_pattern": [],
+    "matched": {"scapy_str": [],
+                "check_func": {"func": rfc.check_vf_rx_packets_number,
+                               "param": {"expect_port": 1}},
+                "expect_results": {"expect_pkts": 32563}},
+    "mismatched": {
+        "scapy_str": ['Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.167.0.1")/TCP(sport=25,dport=23)/("X"*480)'],
+        "check_func": {"func": rfc.check_vf_rx_packets_number,
+                       "param": {"expect_port": 1}},
+        "expect_results": {"expect_pkts": 0}}
+}
+
+
+class TestCvlLimitValue(TestCase):
+
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        prerequisites.
+        """
+        # Based on h/w type, choose how many ports to use
+        self.dut_ports = self.dut.get_ports(self.nic)
+        self.verify(len(self.dut_ports) >= 2, "Insufficient ports for testing")
+        # Verify that enough threads are available
+        cores = self.dut.get_core_list("1S/4C/1T")
+        self.verify(cores is not None, "Insufficient cores for testing")
+        self.ports_socket = self.dut.get_numa_id(self.dut_ports[0])
+        self.portMask = utils.create_mask([self.dut_ports[0], self.dut_ports[1]])
+        localPort0 = self.tester.get_local_port(self.dut_ports[0])
+        localPort1 = self.tester.get_local_port(self.dut_ports[1])
+        self.tester_iface0 = self.tester.get_interface(localPort0)
+        self.tester_iface1 = self.tester.get_interface(localPort1)
+        self.pf0_intf = self.dut.ports_info[self.dut_ports[0]]['intf']
+        self.pf1_intf = self.dut.ports_info[self.dut_ports[1]]['intf']
+        self.pf0_mac = self.dut.get_mac_address(0)
+        self.pf1_mac = self.dut.get_mac_address(1)
+        self.pci0 = self.dut.ports_info[self.dut_ports[0]]['pci']
+        self.pci1 = self.dut.ports_info[self.dut_ports[1]]['pci']
+
+        self.pkt = Packet()
+        self.pmd_output = PmdOutput(self.dut)
+        self.path = self.dut.apps_name['test-pmd']
+
+        self.src_file_dir = 'dep/'
+        self.cvlq_num = CVL_TXQ_RXQ_NUMBER
+
+        #max_rule number
+        localPort = self.tester.get_local_port(self.dut_ports[0])
+        self.__tx_iface = self.tester.get_interface(localPort)
+        self.pkt = Packet()
+
+    def set_up_for_iavf_dir(self):
+        """
+        Run before each test case.
+        """
+        #bind pf to kernel
+        for port in self.dut_ports:
+            netdev = self.dut.ports_info[port]['port']
+            netdev.bind_driver(driver='ice')
+        #set vf driver
+        self.vf_driver = 'vfio-pci'
+        self.dut.send_expect('modprobe vfio-pci', '#')
+        self.suite_config = rfc.get_suite_config(self)
+        self.setup_2pf_4vf_env()
+
+    def setup_2pf_4vf_env(self, driver='default'):
+
+        # get PF interface name
+        self.used_dut_port_0 = self.dut_ports[0]
+        self.used_dut_port_1 = self.dut_ports[1]
+
+        # generate 2 VFs on PF
+        self.dut.generate_sriov_vfs_by_port(self.used_dut_port_0, 2, driver=driver)
+        self.dut.generate_sriov_vfs_by_port(self.used_dut_port_1, 2, driver=driver)
+        self.sriov_vfs_pf0 = self.dut.ports_info[self.used_dut_port_0]['vfs_port']
+        self.sriov_vfs_pf1 = self.dut.ports_info[self.used_dut_port_1]['vfs_port']
+
+        self.dut.send_expect('ip link set %s vf 0 mac 00:11:22:33:44:55' % self.pf0_intf, '#')
+        self.dut.send_expect('ip link set %s vf 1 mac 00:11:22:33:44:66' % self.pf0_intf, '#')
+        self.dut.send_expect('ip link set %s vf 0 mac 00:11:22:33:44:77' % self.pf1_intf, '#')
+        self.dut.send_expect('ip link set %s vf 1 mac 00:11:22:33:44:88' % self.pf1_intf, '#')
+
+        # bind VF0 and VF1 to dpdk driver
+        try:
+            for vf_port in self.sriov_vfs_pf0:
+                vf_port.bind_driver(self.vf_driver)
+            for vf_port in self.sriov_vfs_pf1:
+                vf_port.bind_driver(self.vf_driver)
+
+        except Exception as e:
+            self.destroy_env()
+            raise Exception(e)
+        out = self.dut.send_expect('./usertools/dpdk-devbind.py -s', '#')
+        print(out)
+
+    def setup_1pf_vfs_env(self, pf_port=0, driver='default'):
+
+        self.used_dut_port_0 = self.dut_ports[pf_port]
+        #get PF interface name
+        self.pf0_intf = self.dut.ports_info[self.used_dut_port_0]['intf']
+        out = self.dut.send_expect('ethtool -i %s' % self.pf0_intf, '#')
+        #generate 4 VFs on PF
+        self.dut.generate_sriov_vfs_by_port(self.used_dut_port_0, 4, driver=driver)
+        self.sriov_vfs_port_0 = self.dut.ports_info[self.used_dut_port_0]['vfs_port']
+        #set VF0 as trust
+        self.dut.send_expect('ip link set %s vf 0 trust on' % self.pf0_intf, '#')
+        #bind VFs to dpdk driver
+        for port in self.sriov_vfs_port_0:
+            port.bind_driver(self.vf_driver)
+        time.sleep(5)
+
+    def bind_nics_driver(self, ports, driver=""):
+        # modprobe vfio driver
+        if driver == "vfio-pci":
+            for port in ports:
+                netdev = self.dut.ports_info[port]['port']
+                driver = netdev.get_nic_driver()
+                if driver != 'vfio-pci':
+                    netdev.bind_driver(driver='vfio-pci')
+
+        elif driver == "igb_uio":
+            # igb_uio should insmod as default, no need to check
+            for port in ports:
+                netdev = self.dut.ports_info[port]['port']
+                driver = netdev.get_nic_driver()
+                if driver != 'igb_uio':
+                    netdev.bind_driver(driver='igb_uio')
+        else:
+            for port in ports:
+                netdev = self.dut.ports_info[port]['port']
+                driver_now = netdev.get_nic_driver()
+                if driver == "":
+                    driver = netdev.default_driver
+                if driver != driver_now:
+                    netdev.bind_driver(driver=driver)
+
+    def destroy_env(self):
+        """
+        This is to stop testpmd and destroy 1pf and 2vfs environment.
+        """
+        self.dut.send_expect("quit", "# ", 60)
+        time.sleep(2)
+        self.dut.destroy_sriov_vfs_by_port(self.dut_ports[0])
+        self.dut.destroy_sriov_vfs_by_port(self.dut_ports[1])
+
+    def re_load_ice_driver(self):
+        """
+        remove and reload the ice driver
+        """
+        ice_driver_file_location = self.suite_config["ice_driver_file_location"]
+        self.dut.send_expect("rmmod ice", "# ", 15)
+        self.dut.send_expect("insmod %s" % ice_driver_file_location, "# ", 60)
+        time.sleep(5)
+
+    def config_testpmd(self):
+        self.pmd_output.execute_cmd("set fwd rxonly")
+        self.pmd_output.execute_cmd("set verbose 1")
+        res = self.pmd_output.wait_link_status_up('all', timeout=15)
+        self.verify(res is True, 'there have port link is down')
+        self.pmd_output.execute_cmd("start")
+
+    def config_testpmd_cvl_fidr(self):
+        self.pmd_output.execute_cmd("set fwd rxonly")
+        self.pmd_output.execute_cmd("set verbose 1")
+        self.pmd_output.execute_cmd("port config 0 udp_tunnel_port add vxlan 4789")
+        self.pmd_output.execute_cmd("port config 1 udp_tunnel_port add vxlan 4789")
+        self.pmd_output.execute_cmd("port config all rss all")
+        # specify a fixed rss-hash-key for cvl ether
+        self.pmd_output.execute_cmd(
+            "port config 0 rss-hash-key ipv4 1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd")
+        self.pmd_output.execute_cmd(
+            "port config 1 rss-hash-key ipv4 1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd")
+        res = self.pmd_output.wait_link_status_up('all', timeout=15)
+        self.verify(res is True, 'there have port link is down')
+
+    def launch_testpmd(self):
+        self.pmd_output.start_testpmd(cores="1S/4C/1T",
+                                      param="--rxq={} --txq={}".format(self.cvlq_num, self.cvlq_num),
+                                      eal_param="-w %s -w %s" % (
+                                          self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf0[1].pci),
+                                      socket=self.ports_socket)
+        self.config_testpmd()
+
+    def send_packets(self, packets, pf_id=0):
+        self.pkt.update_pkt(packets)
+        tx_port = self.tester_iface0 if pf_id == 0 else self.tester_iface1
+        self.pkt.send_pkt(crb=self.tester, tx_port=tx_port)
+
+    def send_pkts_getouput(self, pkts, pf_id=0):
+        """
+        if pkt_info is True, we need to get packet infomation to check the RSS hash and FDIR.
+        if pkt_info is False, we just need to get the packet number and queue number.
+        """
+        self.send_packets(pkts, pf_id)
+        time.sleep(1)
+        out_info = self.dut.get_session_output(timeout=1)
+        out_pkt = self.pmd_output.execute_cmd("stop")
+        out = out_info + out_pkt
+        self.pmd_output.execute_cmd("start")
+        return out
+
+    def create_fdir_rule(self, rule, check_stats=None):
+        p = re.compile(r"Flow rule #(\d+) created")
+        rule_list = []
+        if isinstance(rule, list):
+            for i in rule:
+                out = self.pmd_output.execute_cmd(i)
+                m = p.search(out)
+                if m:
+                    rule_list.append(m.group(1))
+                else:
+                    rule_list.append(False)
+        elif isinstance(rule, str):
+            out = self.pmd_output.execute_cmd(rule)
+            m = p.search(out)
+            if m:
+                rule_list.append(m.group(1))
+            else:
+                rule_list.append(False)
+        else:
+            raise Exception("unsupported rule type, only accept list or str")
+        if check_stats:
+            self.verify(all(rule_list), "some rules create failed, result %s" % rule_list)
+        elif check_stats == False:
+            self.verify(not any(rule_list), "all rules should create failed, result %s" % rule_list)
+        return rule_list
+
+    def check_fdir_rule(self, port_id=0, stats=True, rule_list=None):
+        out = self.pmd_output.execute_cmd("flow list %s" % port_id)
+        p = re.compile(r"ID\s+Group\s+Prio\s+Attr\s+Rule")
+        if stats:
+            self.verify(p.search(out), "flow rule on port %s is not existed" % port_id)
+            if rule_list:
+                p = re.compile("^(\d+)\s")
+                li = out.splitlines()
+                res = list(filter(bool, list(map(p.match, li))))
+                result = [i.group(1) for i in res]
+                self.verify(sorted(result) == sorted(rule_list),
+                            "check rule list failed. expect %s, result %s" % (rule_list, result))
+        else:
+            self.verify(not p.search(out), "flow rule on port %s is existed" % port_id)
+
+    def check_switch_filter_rule_list(self, port_id, rule_list, session_name="", need_verify=True):
+        """
+        check the rules in list identical to ones in rule_list
+        """
+        if session_name == "":
+            session_name = self.dut
+        out = session_name.send_expect("flow list %d" % port_id, "testpmd> ", 15)
+        p = re.compile(r"ID\s+Group\s+Prio\s+Attr\s+Rule")
+        m = p.search(out)
+        if not m:
+            result = []
+        else:
+            p_spec = re.compile("^(\d+)\s")
+            out_lines = out.splitlines()
+            res = filter(bool, map(p_spec.match, out_lines))
+            result = [i.group(1) for i in res]
+        if need_verify:
+            self.verify(result == rule_list,
+                        "the rule list is not the same. expect %s, result %s" % (rule_list, result))
+        else:
+            return result
+
+    def check_rule_number(self, port_id=0, num=0):
+        out = self.dut.send_command("flow list %s" % port_id, timeout=30)
+        result_scanner = r'\d*.*?\d*.*?\d*.*?=>*'
+        scanner = re.compile(result_scanner, re.DOTALL)
+        li = scanner.findall(out)
+        if num == 0:
+            self.verify(not li, "there should be no rule listed")
+        else:
+            print(len(li))
+            self.verify(len(li) == num, "the amount of rules is wrong.")
+        return out
+
+    def send_and_check_packets(self, dic, session_name="", tx_iface=""):
+        """
+        general packets processing workflow.
+        """
+        if session_name == "":
+            session_name = self.dut
+        if tx_iface == "":
+            tx_iface = self.__tx_iface
+        session_name.send_expect("start", "testpmd> ", 15)
+        time.sleep(2)
+        #send packets
+        self.pkt.update_pkt(dic["scapy_str"])
+        self.pkt.send_pkt(self.tester, tx_port=tx_iface, count=1, timeout=370)
+        time.sleep(3)
+        out = session_name.send_expect("stop", "testpmd> ", 15)
+        dic["check_func"]["func"](out, dic["check_func"]["param"], dic["expect_results"])
+
+    def test_maxnum_14336rules_1vf(self):
+        """
+        vfs share 14336 rules table
+        """
+        dut_file_dir = '/tmp/'
+        self.set_up_for_iavf_dir()
+        self.dut.kill_all()
+        src_file = 'create_14336_rules'
+        flows = open(self.src_file_dir + src_file, mode='w')
+        count = 0
+        for i in range(56):
+            for j in range(256):
+                flows.write(
+                    'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.%d.%d / end actions queue index 5 / mark / end \n' % (
+                    i, j))
+                count = count + 1
+        flows.close()
+        self.verify(count == 14336, "failed to create 14336 fdir rules on vf.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file, dut_file_dir)
+
+        eal_param = "-c f -n 6 -w %s -w %s" % (self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf0[1].pci)
+        command = self.path + eal_param + " -- -i --rxq=%s --txq=%s" % (
+        self.cvlq_num, self.cvlq_num) + " --cmdline-file=%s" % (dut_file_dir + src_file)
+        self.dut.send_expect(command, "testpmd> ", 300)
+        self.config_testpmd()
+
+        # can't create more than 14336 rules on vf0
+        rule_14336_vf0 = "flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        self.create_fdir_rule(rule_14336_vf0, check_stats=False)
+        # check there are 14336 rules created.
+        out = self.check_rule_number(port_id=0, num=14336)
+        self.verify("14336" not in out, "more than 14336 rules can be created on 1vf")
+
+        # can't create rule on vf1
+        rule_0_vf1 = "flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        self.create_fdir_rule(rule_0_vf1, check_stats=False)
+        self.check_fdir_rule(port_id=1, stats=False)
+
+        pkt_0 = 'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.0")/Raw("x" * 80)'
+        pkt_14335 = 'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.55.255")/Raw("x" * 80)'
+        pkt_14336 = 'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.56.0")/Raw("x" * 80)'
+
+        # check packet match rule 0 and rule 14335 can be redirected to expected queue
+        out_0 = self.send_pkts_getouput(pkts=pkt_0, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_0, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5}, stats=True)
+        out_14335 = self.send_pkts_getouput(pkts=pkt_14335, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_14335, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5}, stats=True)
+        # check packet match rule 14336 can't be redirected to expected queue.
+        out_14336 = self.send_pkts_getouput(pkts=pkt_14336, pf_id=0)
+        # #check 'FDIR matched ID' not in out_14336,not check queque number sometime it may be same as 5.
+        rfc.check_iavf_fdir_mark(out_14336, pkt_num=1, check_param={"port_id": 0, "queue": 5}, stats=False)
+
+        # delete one rule of vf0
+        self.dut.send_expect("flow destroy 0 rule 0", "testpmd> ", timeout=200)
+        self.create_fdir_rule(rule_0_vf1, check_stats=True)
+        pkt_0_vf1 = 'Ether(dst="00:11:22:33:44:66")/IP(src="192.168.0.20",dst="192.168.56.0")/Raw("x" * 80)'
+        out_0_vf1 = self.send_pkts_getouput(pkts=pkt_0_vf1, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_0_vf1, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5}, stats=True)
+
+        # flush all the rules
+        self.dut.send_expect("flow flush 0", "testpmd> ", timeout=500)
+        self.check_fdir_rule(port_id=0, stats=False)
+        out_0 = self.send_pkts_getouput(pkts=pkt_0, pf_id=0)
+        out_14335 = self.send_pkts_getouput(pkts=pkt_14335, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_0, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5}, stats=False)
+        rfc.check_iavf_fdir_mark(out_14335, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5},
+                                 stats=False)
+
+        self.create_fdir_rule(rule_14336_vf0, check_stats=True)
+        out_14336 = self.send_pkts_getouput(pkts=pkt_14336, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_14336, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5}, stats=True)
+
+    def test_maxnum_14336rules_2vf(self):
+        """
+        vfs share 14336 rules table
+        """
+        dut_file_dir = '/tmp/'
+        self.dut.kill_all()
+        self.set_up_for_iavf_dir()
+        self.session_secondary = self.dut.new_session()
+        src_file = 'create_14336_rules_2vf'
+        flows = open(self.src_file_dir + src_file, mode='w')
+        flows.write(
+            'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 5 / mark / end \n')
+        count = 1
+        for i in range(55):
+            for j in range(256):
+                flows.write(
+                    'flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.%d.%d / end actions queue index 5 / mark / end \n' % (
+                    i, j))
+                count = count + 1
+        for j in range(255):
+            flows.write(
+                'flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.%d / end actions queue index 5 / mark / end \n' % j)
+            count = count + 1
+        flows.close()
+        self.verify(count == 14336, "failed to create 14336 fdir rules on 2 vfs.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file, dut_file_dir)
+
+        eal_param = "-c f -n 6 -w %s -w %s" % (self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf1[0].pci)
+        command = self.path + eal_param + " -- -i --rxq=%s --txq=%s" % (
+        self.cvlq_num, self.cvlq_num) + " --cmdline-file=%s" % (dut_file_dir + src_file)
+        self.dut.send_expect(command, "testpmd> ", 1200)
+
+        self.config_testpmd()
+        self.check_fdir_rule(port_id=0, rule_list=['0'])
+
+        # can't create more than 14336 rules on 2vf
+        rule_14335_vf1 = "flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        self.create_fdir_rule(rule_14335_vf1, check_stats=False)
+        # check there are 14336 rules created.
+        out = self.check_rule_number(port_id=1, num=14335)
+        self.verify("14335" not in out, "more than 14336 rules are created on 2vf")
+
+        # can't create new rule on vf0
+        rule_1_vf0 = "flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        self.create_fdir_rule(rule_1_vf0, check_stats=False)
+        self.check_rule_number(port_id=0, num=1)
+
+        pkt_0 = 'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.0")/Raw("x" * 80)'
+        pkt_1 = 'Ether(dst="00:11:22:33:44:77")/IP(src="192.168.0.20",dst="192.168.0.0")/Raw("x" * 80)'
+        pkt_14335 = 'Ether(dst="00:11:22:33:44:77")/IP(src="192.168.0.20",dst="192.168.55.254")/Raw("x" * 80)'
+        pkt_14336 = 'Ether(dst="00:11:22:33:44:77")/IP(src="192.168.0.20",dst="192.168.56.0")/Raw("x" * 80)'
+
+        self.session_secondary.send_expect("ifconfig %s up" % self.pf0_intf, "# ", 15)
+        self.session_secondary.send_expect("ifconfig %s up" % self.pf1_intf, "# ", 15)
+        time.sleep(1)
+
+        # check packet match rule 0 and rule 14335 can be redirected to expected queue
+        out_0 = self.send_pkts_getouput(pkts=pkt_0, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_0, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5}, stats=True)
+        out_1 = self.send_pkts_getouput(pkts=pkt_1, pf_id=1)
+        rfc.check_iavf_fdir_mark(out_1, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5}, stats=True)
+
+        out_14335 = self.send_pkts_getouput(pkts=pkt_14335, pf_id=1)
+        rfc.check_iavf_fdir_mark(out_14335, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5}, stats=True)
+        # check packet match rule 14336 can't be redirected to expected queue.
+        out_14336 = self.send_pkts_getouput(pkts=pkt_14336, pf_id=1)
+        rfc.check_iavf_fdir_mark(out_14336, pkt_num=1, check_param={"port_id": 1, "queue": 5}, stats=False)
+
+        # destroy rule 0 on vf0, then create a new rule on vf1 successfully.
+        self.dut.send_expect("flow flush 0", "testpmd> ")
+        self.create_fdir_rule(rule_14335_vf1, check_stats=True)
+        out_14336 = self.send_pkts_getouput(pkts=pkt_14336, pf_id=1)
+        rfc.check_iavf_fdir_mark(out_14336, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5}, stats=True)
+
+        self.dut.send_expect("flow flush 1", "testpmd> ", timeout=300)
+
+        self.check_fdir_rule(port_id=0, stats=False)
+        self.check_fdir_rule(port_id=1, stats=False)
+
+        out_0 = self.send_pkts_getouput(pkts=pkt_0, pf_id=0)
+        out_1 = self.send_pkts_getouput(pkts=pkt_1, pf_id=1)
+        out_14335 = self.send_pkts_getouput(pkts=pkt_14335, pf_id=1)
+        rfc.check_iavf_fdir_mark(out_0, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5}, stats=False)
+        rfc.check_iavf_fdir_mark(out_1, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5}, stats=False)
+        rfc.check_iavf_fdir_mark(out_14335, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5},
+                                 stats=False)
+        self.dut.close_session(self.session_secondary)
+
+    def test_maxnum_15360rules_1pf_2vf(self):
+        """
+        2*100G NIC, each pf can create 1024 rules at least, vfs share 14336 rules table
+        4*25G NIC, each pf can create 512 rules at least, vfs share 14336 rules table
+        so if 2*25G NIC, max number is 14848 on 1pf and 2vfs.
+        """
+        dut_file_dir = '/tmp/'
+        self.dut.kill_all()
+        self.set_up_for_iavf_dir()
+        self.session_secondary = self.dut.new_session()
+        # create 1025 rules on pf1
+        src_file = 'create_1025_kernel_rules'
+        flows = open(self.src_file_dir + src_file, mode='w')
+        count = 0
+        if self.nic in ["columbiaville_100g"]:
+            for i in range(4):
+                for j in range(256):
+                    flows.write(
+                        'ethtool -N %s flow-type tcp4 src-ip 192.168.%d.%d dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8 \n' % (
+                        self.pf1_intf, i, j))
+                    count = count + 1
+            flows.write(
+                'ethtool -N %s flow-type tcp4 src-ip 192.168.100.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8 \n' % self.pf1_intf)
+            count = count + 1
+            flows.close()
+            self.verify(count == 1025, "failed to create 1025 fdir rules on pf.")
+        elif self.nic in ["columbiaville_25g"]:
+            for i in range(2):
+                for j in range(256):
+                    flows.write(
+                        'ethtool -N %s flow-type tcp4 src-ip 192.168.%d.%d dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8 \n' % (
+                        self.pf1_intf, i, j))
+                    count = count + 1
+            flows.write(
+                'ethtool -N %s flow-type tcp4 src-ip 192.168.100.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8 \n' % self.pf1_intf)
+            count = count + 1
+            flows.close()
+            self.verify(count == 513, "failed to create 1025 fdir rules on pf.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file, dut_file_dir)
+        # create 1025 rules on pf1
+        fkr = open(self.src_file_dir + src_file, "r+")
+        kernel_rules = fkr.read()
+        fkr.close()
+        self.dut.send_expect(kernel_rules, "# ")
+        time.sleep(15)
+        # create 1 rule on vf00, and 14334 rules on vf01
+        src_file_vf = 'create_14335_rules_on_2vfs'
+        flows = open(self.src_file_dir + src_file_vf, mode='w')
+        flows.write(
+            'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.0 / end actions queue index 5 / mark / end \n')
+        count = 1
+        for i in range(55):
+            for j in range(256):
+                flows.write(
+                    'flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.%d.%d / end actions queue index 5 / mark / end \n' % (
+                    i, j))
+                count = count + 1
+        for j in range(254):
+            flows.write(
+                'flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.55.%d / end actions queue index 5 / mark / end \n' % j)
+            count = count + 1
+        flows.close()
+        self.verify(count == 14335, "failed to create 14335 fdir rules on vfs.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file_vf, dut_file_dir)
+        # start testpmd with creating rules in commandline
+        eal_param = "-c f -n 6 -w %s -w %s" % (self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf0[1].pci)
+        command = self.path + eal_param + " -- -i --rxq=%s --txq=%s" % (
+        self.cvlq_num, self.cvlq_num) + " --cmdline-file=%s" % (dut_file_dir + src_file_vf)
+        fdw = open("15360_rules_vf_result.txt", "w")
+        fdw.write(self.dut.send_expect(command, "testpmd> ", 360))
+        fdw.close()
+        self.config_testpmd()
+        # check there is 1 rule created on vf00
+        self.check_fdir_rule(port_id=0, rule_list=['0'])
+        # can't create more than 14335 rules on 2vf, the rule index is from 0
+        rule_14334_vf1 = "flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        pkt_14334 = 'Ether(dst="00:11:22:33:44:66")/IP(src="192.168.0.20",dst="192.168.56.0")/Raw("x" * 80)'
+        self.create_fdir_rule(rule_14334_vf1, check_stats=False)
+        # check there are 14334 rules created on vf01
+        out = self.check_rule_number(port_id=1, num=14334)
+        self.verify("14334" not in out, "more than 15360 rules are created on 2vf")
+        # delete a rule on pf0
+        self.session_secondary.send_expect("ethtool -N %s delete 14847" % self.pf1_intf, "# ")
+        # then can create one more rule on vf01
+        self.create_fdir_rule(rule_14334_vf1, check_stats=True)
+        out_14334 = self.send_pkts_getouput(pkts=pkt_14334, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_14334, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5}, stats=True)
+        self.dut.send_expect("flow flush 0", "testpmd> ", timeout=600)
+        self.dut.send_expect("flow flush 1", "testpmd> ", timeout=600)
+        self.check_fdir_rule(port_id=0, stats=False)
+        self.check_fdir_rule(port_id=1, stats=False)
+        out_14334 = self.send_pkts_getouput(pkts=pkt_14334, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_14334, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5},
+                                 stats=False)
+        self.dut.send_expect("quit", "# ")
+        self.dut.close_session(self.session_secondary)
+
+    def test_maxnum_15360rules_1pf_0_rules_vf(self):
+        """
+        2*100G NIC, each pf can create 1024 rules at least, vfs share 14336 rules table
+        4*25G NIC, each pf can create 512 rules at least, vfs share 14336 rules table
+        so if 2*25G NIC, max number is 14848 on 1pf and vfs.
+        create 15360/14848 rules on pf1, check failed to create rule on vf00 and vf10
+        """
+        dut_file_dir = '/tmp/'
+        self.dut.kill_all()
+        self.set_up_for_iavf_dir()
+        self.session_secondary = self.dut.new_session()
+        # create 15360 rules on pf1
+        src_file = 'create_15360_kernel_rules'
+        flows = open(self.src_file_dir + src_file, mode='w')
+        count = 0
+        if self.nic in ["columbiaville_100g"]:
+            for i in range(60):
+                for j in range(256):
+                    flows.write(
+                        'ethtool -N %s flow-type tcp4 src-ip 192.168.%d.%d dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8 \n' % (
+                        self.pf1_intf, i, j))
+                    count = count + 1
+            flows.close()
+            self.verify(count == 15360, "failed to create 15360 fdir rules on pf.")
+        elif self.nic in ["columbiaville_25g"]:
+            for i in range(58):
+                for j in range(256):
+                    flows.write(
+                        'ethtool -N %s flow-type tcp4 src-ip 192.168.%d.%d dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8 \n' % (
+                        self.pf1_intf, i, j))
+                    count = count + 1
+            flows.close()
+            self.verify(count == 14848, "failed to create 14848 fdir rules on pf.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file, dut_file_dir)
+        # create 15360 rules on pf1
+        fkr = open(self.src_file_dir + src_file, "r+")
+        kernel_rules = fkr.read()
+        fkr.close()
+        self.dut.send_expect(kernel_rules, "# ")
+        time.sleep(200)
+        # failed to create 1 more rule on pf1
+        self.dut.send_expect(
+            "ethtool -N %s flow-type tcp4 src-ip 192.168.100.0 dst-ip 192.168.100.2 src-port 32 dst-port 33 action 8" % self.pf1_intf,
+            "Cannot insert RX class rule: No space left on device")
+        # start testpmd with creating rules in commandline
+        eal_param = "-c f -n 6 -w %s -w %s" % (self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf1[0].pci)
+        command = self.path + eal_param + " -- -i --rxq=%s --txq=%s" % (self.cvlq_num, self.cvlq_num)
+        self.dut.send_expect(command, "testpmd> ", 20)
+
+        self.config_testpmd()
+        rule_0_vf00 = "flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        pkt_0_vf00 = 'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.56.0")/Raw("x" * 80)'
+        rule_0_vf10 = "flow create 1 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.56.0 / end actions queue index 5 / mark / end"
+        pkt_0_vf10 = 'Ether(dst="00:11:22:33:44:77")/IP(src="192.168.0.20",dst="192.168.56.0")/Raw("x" * 80)'
+
+        # failed to create rule on vf00 and vf10
+        self.create_fdir_rule(rule_0_vf00, check_stats=False)
+        self.create_fdir_rule(rule_0_vf10, check_stats=False)
+        self.check_rule_number(port_id=0, num=0)
+        self.check_rule_number(port_id=1, num=0)
+
+        # delete a rule on pf1
+        self.session_secondary.send_expect("ethtool -N %s delete 14847" % self.pf1_intf, "# ")
+
+        # then can create one rule on vf00
+        self.create_fdir_rule(rule_0_vf00, check_stats=True)
+        self.create_fdir_rule(rule_0_vf10, check_stats=False)
+        self.check_rule_number(port_id=0, num=1)
+        self.check_rule_number(port_id=1, num=0)
+
+        # delete a rule on pf1
+        self.session_secondary.send_expect("ethtool -N %s delete 14846" % self.pf1_intf, "# ")
+
+        # then can create one rule on vf10
+        self.create_fdir_rule(rule_0_vf10, check_stats=True)
+        self.check_rule_number(port_id=1, num=1)
+
+        out_0_vf00 = self.send_pkts_getouput(pkts=pkt_0_vf00, pf_id=0)
+        rfc.check_iavf_fdir_mark(out_0_vf00, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 5},
+                                 stats=True)
+
+        out_0_vf10 = self.send_pkts_getouput(pkts=pkt_0_vf10, pf_id=1)
+        rfc.check_iavf_fdir_mark(out_0_vf10, pkt_num=1, check_param={"port_id": 1, "mark_id": 0, "queue": 5},
+                                 stats=True)
+
+        self.dut.send_expect("quit", "# ")
+        self.dut.close_session(self.session_secondary)
+
+    def test_stress_add_delete_rules_1vf(self):
+        """
+        add/delete rules 14336 times on 1 vf
+        """
+        dut_file_dir = '/tmp/'
+        rules = [
+            "flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / udp src is 22 dst is 23 / end actions queue index 6 / mark / end",
+            "flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / tcp src is 22 dst is 23 / end actions rss queues 2 3 end / mark id 1 / end"]
+        pkts = [
+            'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/UDP(sport=22,dport=23)/Raw("x" * 80)',
+            'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/TCP(sport=22,dport=23)/Raw("x" * 80)']
+        self.dut.kill_all()
+        self.set_up_for_iavf_dir()
+        src_file = 'add_delete_rules_1vf'
+        flows = open(self.src_file_dir + src_file, mode='w')
+        count = 0
+        for i in range(14336):
+            flows.write('%s \n' % rules[0])
+            flows.write('%s \n' % rules[1])
+            flows.write('flow flush 0\n')
+            count = count + 1
+        flows.close()
+        self.verify(count == 14336, "failed to add/delete 14336 times of fdir rules on vf.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file, dut_file_dir)
+
+        eal_param = "-c f -n 6 -w %s -w %s" % (self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf0[1].pci)
+        command = self.path + eal_param + " -- -i --rxq=%s --txq=%s" % (
+        self.cvlq_num, self.cvlq_num) + " --cmdline-file=%s" % (dut_file_dir + src_file)
+        self.dut.send_expect(command, "testpmd> ", 900)
+        self.config_testpmd()
+        self.check_fdir_rule(port_id=0, stats=False)
+        self.create_fdir_rule(rules, check_stats=True)
+        out_0 = self.send_pkts_getouput(pkts=pkts[0])
+        rfc.check_iavf_fdir_mark(out_0, pkt_num=1, check_param={"port_id": 0, "mark_id": 0, "queue": 6}, stats=True)
+        out_1 = self.send_pkts_getouput(pkts=pkts[1])
+        rfc.check_iavf_fdir_mark(out_1, pkt_num=1, check_param={"port_id": 0, "mark_id": 1, "queue": [2, 3]},
+                                 stats=True)
+
+    def test_stress_add_delete_rules_2vf(self):
+        """
+        add/delete rules 14336 times on 2 vfs
+        """
+        dut_file_dir = '/tmp/'
+        rules = [
+            "flow create 0 ingress pattern eth / ipv4 src is 192.168.56.0 dst is 192.1.0.0 tos is 4 / tcp src is 22 dst is 23 / end actions queue index 5 / end",
+            "flow create 1 ingress pattern eth / ipv4 src is 192.168.56.0 dst is 192.1.0.0 tos is 4 / tcp src is 22 dst is 23 / end actions queue index 5 / end"]
+        pkts = [
+            'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.56.0",dst="192.1.0.0", tos=4)/TCP(sport=22,dport=23)/Raw("x" * 80)',
+            'Ether(dst="00:11:22:33:44:66")/IP(src="192.168.56.0",dst="192.1.0.0", tos=4)/TCP(sport=22,dport=23)/Raw("x" * 80)']
+        self.dut.kill_all()
+        self.set_up_for_iavf_dir()
+        src_file = 'add_delete_rules_2vfs'
+        flows = open(self.src_file_dir + src_file, mode='w')
+        count = 0
+        for i in range(14336):
+            flows.write('%s \n' % rules[0])
+            flows.write('%s \n' % rules[1])
+            flows.write('flow flush 0\n')
+            flows.write('flow flush 1\n')
+            count = count + 1
+        flows.close()
+        self.verify(count == 14336, "failed to add/delete 14336 times of fdir rules on 2 vfs.")
+        self.dut.session.copy_file_to(self.src_file_dir + src_file, dut_file_dir)
+
+        eal_param = "-c f -n 6 -w %s -w %s" % (self.sriov_vfs_pf0[0].pci, self.sriov_vfs_pf0[1].pci)
+        command = self.path + eal_param + " -- -i --rxq=%s --txq=%s" % (
+        self.cvlq_num, self.cvlq_num) + " --cmdline-file=%s" % (dut_file_dir + src_file)
+        self.dut.send_expect(command, "testpmd> ", 900)
+        self.config_testpmd()
+        self.check_fdir_rule(port_id=0, stats=False)
+        self.check_fdir_rule(port_id=1, stats=False)
+
+        self.create_fdir_rule(rules, check_stats=True)
+        out_0 = self.send_pkts_getouput(pkts=pkts[0], pf_id=0)
+        rfc.check_iavf_fdir_mark(out_0, pkt_num=1, check_param={"port_id": 0, "queue": 5}, stats=True)
+        out_1 = self.send_pkts_getouput(pkts=pkts[1], pf_id=0)
+        rfc.check_iavf_fdir_mark(out_1, pkt_num=1, check_param={"port_id": 1, "queue": 5}, stats=True)
+
+    def launch_testpmd_with_mark(self, rxq=64, txq=64):
+        self.pmd_output.start_testpmd(cores="1S/4C/1T",
+                                      param="--portmask=%s --rxq=%d --txq=%d --port-topology=loop" % (
+                                          self.portMask, rxq, txq),
+                                      eal_param="-w %s -w %s --log-level=ice,7" % (
+                                          self.pci0, self.pci1), socket=self.ports_socket)
+        self.config_testpmd_cvl_fidr()
+
+    #this case copy from cvl_fdir
+    def test_add_delete_rules(self):
+        self.launch_testpmd_with_mark()
+        self.pmd_output.execute_cmd("start")
+        self.pmd_output.execute_cmd("stop")
+        self.dut.send_command("quit", timeout=2)
+        cmd_path = '/tmp/add_delete_rules'
+        cmds = [
+                   'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / udp src is 22 dst is 23 / end actions queue index 1 / mark / end',
+                   'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / tcp src is 22 dst is 23 / end actions rss queues 2 3 end / mark id 1 / end',
+                   'flow list 0', 'flow flush 0'] * 15360
+        cmds_li = map(lambda x: x + os.linesep, cmds)
+        with open(cmd_path, 'w') as f:
+            f.writelines(cmds_li)
+        self.dut.session.copy_file_to(cmd_path, cmd_path)
+        try:
+            eal_param = self.dut.create_eal_parameters(cores="1S/4C/1T", ports=[self.pci0,self.pci1], socket=self.ports_socket)
+            param = " --log-level='ice,7' -- -i --portmask=%s --rxq=%d --txq=%d --port-topology=loop --cmdline-file=%s" % (
+                self.portMask, 64, 64, cmd_path)
+            command_line = self.dut.apps_name['test-pmd'] + eal_param + param
+            out = self.dut.send_expect(command_line, 'testpmd>', timeout=1200)
+            self.verify('Failed to create file' not in out, "create some rule failed: %s" % out)
+            self.config_testpmd_cvl_fidr()
+            self.pmd_output.execute_cmd('start')
+            rules = [
+                'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / udp src is 22 dst is 23 / end actions queue index 1 / mark / end',
+                'flow create 0 ingress pattern eth / ipv4 src is 192.168.0.20 dst is 192.168.0.21 / tcp src is 22 dst is 23 / end actions rss queues 2 3 end / mark id 1 / end']
+            rule_li = self.create_fdir_rule(rule=rules, check_stats=True)
+            self.check_fdir_rule(port_id=0, stats=True, rule_list=rule_li)
+            pkts = [
+                'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/UDP(sport=22,dport=23)/Raw("x" * 80)',
+                'Ether(dst="00:11:22:33:44:55")/IP(src="192.168.0.20",dst="192.168.0.21")/TCP(sport=22,dport=23)/Raw("x" * 80)']
+            out1 = self.send_pkts_getouput(pkts=pkts[0], pf_id=0, )
+            rfc.check_mark(out1, pkt_num=1, check_param={"port_id": 0, "queue": 1, "mark_id": 0}, stats=True)
+            out2 = self.send_pkts_getouput(pkts=pkts[1], pf_id=0, )
+            rfc.check_mark(out2, pkt_num=1, check_param={"port_id": 0, "queue": [2, 3], "mark_id": 1}, stats=True)
+        except Exception as e:
+            raise Exception(e)
+        finally:
+            self.dut.kill_all()
+
+    # this case copy from cvl_dcf_switch_filter
+    def test_max_rule_number(self):
+        #bind pf to kernel
+        self.bind_nics_driver(self.dut_ports, driver="ice")
+        #move comms package to package folder
+        self.suite_config = rfc.get_suite_config(self)
+        comms_package_location = self.suite_config["comms_package_file_location"]
+        package_location = self.suite_config["package_file_location"]
+        self.dut.send_expect("cp %s %s" % (comms_package_location, package_location), "# ")
+        self.re_load_ice_driver()
+        #set vf driver
+        self.vf_driver = 'vfio-pci'
+        self.dut.send_expect('modprobe vfio-pci', '#')
+        self.path = self.dut.apps_name['test-pmd']
+
+        # set up 4 vfs on 1 pf environment
+        self.setup_1pf_vfs_env()
+        # create 32563 rules with the same pattern, but different input set to file
+        src_file = 'dep/testpmd_cmds_32k_switch_rules'
+        flows = open(src_file, mode='w')
+        rule_count = 1
+        for i in range(0, 255):
+            for j in range(0, 255):
+                if not rule_count > 32563:
+                    flows.write(
+                        'flow create 0 ingress pattern eth / ipv4 src is 192.168.%d.%d / end actions vf id 1 / end \n' % (
+                        i, j))
+                    matched_scapy_str = 'Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.168.%d.%d")/TCP(sport=25,dport=23)/Raw("X"*480)' % (
+                    i, j)
+                    tv_max_rule_number["matched"]["scapy_str"].append(matched_scapy_str)
+                    rule_count += 1
+                else:
+                    break
+            if rule_count > 32563:
+                break
+        flows.close()
+        dut_file_dir = '/tmp/'
+        self.dut.session.copy_file_to(src_file, dut_file_dir)
+        # launch testpmd with 32563 rules
+        vf0_pci = self.sriov_vfs_port_0[0].pci
+        vf1_pci = self.sriov_vfs_port_0[1].pci
+        all_eal_param = self.dut.create_eal_parameters(cores="1S/4C/1T", ports=[vf0_pci, vf1_pci],
+                                                       port_options={vf0_pci: "cap=dcf"})
+        command = self.path + all_eal_param + " -- -i --cmdline-file=/tmp/testpmd_cmds_32k_switch_rules"
+        out = self.dut.send_expect(command, "testpmd> ", 360)
+        self.testpmd_status = "running"
+        self.dut.send_expect("set portlist 1", "testpmd> ", 15)
+        self.dut.send_expect("set fwd rxonly", "testpmd> ", 15)
+        # check the rule list with 32563 rules
+        rule_list_num = list(range(0, 32563))
+        rule_list = [str(x) for x in rule_list_num]
+        self.check_switch_filter_rule_list(0, rule_list)
+        # create other rules to make switch filter table full
+        m = i
+        t = j
+        p = re.compile(r"Flow rule #(\d+) created")
+        switch_table_full_flag = False
+        count = 0
+        for i in range(m, 255):
+            for j in range(t, 255):
+                rule = 'flow create 0 ingress pattern eth / ipv4 src is 192.168.%d.%d / end actions vf id 1 / end \n' % (
+                i, j)
+                matched_packet = 'Ether(dst="68:05:ca:8d:ed:a8")/IP(src="192.168.%d.%d")/TCP(sport=25,dport=23)/Raw("X"*480)' % (
+                i, j)
+                out = self.dut.send_expect(rule, "testpmd> ", timeout=2)  # create a rule
+                m1 = p.search(out)
+                if m1:
+                    rule_list.append(m1.group(1))
+                    tv_max_rule_number["matched"]["scapy_str"].append(matched_packet)
+                    count += 1
+                else:
+                    self.verify("Failed to create flow" in out,
+                                "Log not provide a friendly output to indicate that the rule failed to create.")
+                    switch_table_full_flag = True
+                    break
+            if switch_table_full_flag:
+                break
+        tv_max_rule_number["matched"]["expect_results"]["expect_pkts"] += count
+        # check the rule list
+        self.check_switch_filter_rule_list(0, rule_list)
+        # send matched packets and check
+        matched_dic = tv_max_rule_number["matched"]
+        self.send_and_check_packets(matched_dic)
+        # send mismatched packets and check
+        mismatched_dic = tv_max_rule_number["mismatched"]
+        self.send_and_check_packets(mismatched_dic)
+        # destroy rules and send matched packets
+        self.dut.send_expect("flow flush 0", "testpmd> ", 300)
+        self.check_switch_filter_rule_list(0, [])
+        # send matched packets and check
+        destroy_dict = copy.deepcopy(matched_dic)
+        destroy_dict["expect_results"]["expect_pkts"] = 0
+        self.send_and_check_packets(destroy_dict)
+
+    def tear_down(self):
+        # destroy all flow rule on port 0
+        self.dut.kill_all()
+        self.destroy_env()
+        if getattr(self, 'session_secondary', None):
+            self.dut.close_session(self.session_secondary)
+        if getattr(self, 'session_third', None):
+            self.dut.close_session(self.session_third)
+
+    def tear_down_all(self):
+        self.dut.kill_all()
+        self.destroy_env()
-- 
2.17.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dts]  [PATCH V1 3/3] conf/cvl_limit_value_test add config file
  2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
                   ` (3 preceding siblings ...)
  2020-11-04 10:25 ` [dts] [PATCH V1 2/3] tests/cvl_limit_value_test add test suite sunqin
@ 2020-11-04 10:25 ` sunqin
  2020-11-11  3:18 ` [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test Tu, Lijuan
  5 siblings, 0 replies; 7+ messages in thread
From: sunqin @ 2020-11-04 10:25 UTC (permalink / raw)
  To: dts; +Cc: sunqin

Add configure file for cvl_limit_value test suite

Signed-off-by: sunqin <qinx.sun@intel.com>
---
 conf/cvl_limit_value_test.cfg | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 conf/cvl_limit_value_test.cfg

diff --git a/conf/cvl_limit_value_test.cfg b/conf/cvl_limit_value_test.cfg
new file mode 100644
index 0000000..bb4b68f
--- /dev/null
+++ b/conf/cvl_limit_value_test.cfg
@@ -0,0 +1,5 @@
+[suite]
+ice_driver_file_location = "/lib/modules/4.18.0-193.14.2.el8_2.x86_64/updates/drivers/net/ethernet/intel/ice/ice.ko"
+os_default_package_file_location="/lib/firmware/updates/intel/ice/ddp/ice-1.3.18.0.pkg"
+comms_package_file_location="/lib/firmware/updates/intel/ice/ddp/ice_comms-1.3.22.0.pkg"
+package_file_location="/lib/firmware/updates/intel/ice/ddp/ice.pkg"
-- 
2.17.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test
  2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
                   ` (4 preceding siblings ...)
  2020-11-04 10:25 ` [dts] [PATCH V1 3/3] conf/cvl_limit_value_test add config file sunqin
@ 2020-11-11  3:18 ` Tu, Lijuan
  5 siblings, 0 replies; 7+ messages in thread
From: Tu, Lijuan @ 2020-11-11  3:18 UTC (permalink / raw)
  To: Sun, QinX, dts; +Cc: Sun, QinX

> 
> These 8 cases take a long time to run, so split them from 3 already existed
> 
> suites (iavf_fdir, cvl_fdir, cvl_dcf_switch_filter) and make them as a new test
> suite
> 
> sunqin (3):
>   test_plans/cvl_limit_value_test: add test plan
>   tests/cvl_limit_value_test: add cvl_limit test suite
>   conf/cvl_limit_value_test: add config file for new test suite
> 
>  conf/cvl_limit_value_test.cfg                 |   5 +
>  test_plans/cvl_limit_value_test_test_plan.rst | 594 ++++++++++++
>  tests/TestSuite_cvl_limit_value_test.py       | 917 ++++++++++++++++++
>  3 files changed, 1516 insertions(+)
>  create mode 100644 conf/cvl_limit_value_test.cfg  create mode 100644
> test_plans/cvl_limit_value_test_test_plan.rst
>  create mode 100644 tests/TestSuite_cvl_limit_value_test.py
> 

Applied, thanks

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-04 10:25 [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test sunqin
2020-11-04  5:37 ` Peng, Yuan
2020-11-04  7:14 ` Sun, QinX
2020-11-04 10:25 ` [dts] [PATCH V1 1/3] test_plans/cvl_limit_value_test sunqin
2020-11-04 10:25 ` [dts] [PATCH V1 2/3] tests/cvl_limit_value_test add test suite sunqin
2020-11-04 10:25 ` [dts] [PATCH V1 3/3] conf/cvl_limit_value_test add config file sunqin
2020-11-11  3:18 ` [dts] [PATCH V1 0/3] add test suite cvl_limit_value_test Tu, Lijuan

test suite reviews and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dts/0 dts/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dts dts/ http://inbox.dpdk.org/dts \
		dts@dpdk.org
	public-inbox-index dts


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dts


AGPL code for this site: git clone https://public-inbox.org/ public-inbox