Re: [dts] [PATCH V3] add test suite of packet_ordering

["Liu, Yong" <yong.liu@intel.com>]

Gang, 
In test plan you mentioned about performance measurement, but there's no such case in the suite.
And there's no function case which can check packet sequence which reorder library worked for. 
Only basic Rx/Tx function checked in your suite. I think it's not enough for overall sample validation.

Thanks,
Marvin

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of xu,gang
> Sent: Wednesday, November 23, 2016 3:46 PM
> To: dts@dpdk.org
> Cc: Xu, GangX
> Subject: [dts] [PATCH V3] add test suite of packet_ordering
> 
> Signed-off-by: xu,gang <gangx.xu@intel.com>
> ---
>  test_plans/packet_ordering_test_plan.rst |  75 ++++++++++++++++++++
>  tests/TestSuite_packet_ordering.py       | 113
> +++++++++++++++++++++++++++++++
>  2 files changed, 188 insertions(+)
>  create mode 100644 test_plans/packet_ordering_test_plan.rst
>  create mode 100644 tests/TestSuite_packet_ordering.py
> 
> diff --git a/test_plans/packet_ordering_test_plan.rst
> b/test_plans/packet_ordering_test_plan.rst
> new file mode 100644
> index 0000000..efe3304
> --- /dev/null
> +++ b/test_plans/packet_ordering_test_plan.rst
> @@ -0,0 +1,75 @@
> +..  BSD LICENSE
> +    Copyright(c) 2015 Intel Corporation. All rights reserved.
> +    All rights reserved.
> +
> +    Redistribution and use in source and binary forms, with or without
> +    modification, are permitted provided that the following conditions
> +    are met:
> +
> +    * Redistributions of source code must retain the above copyright
> +    notice, this list of conditions and the following disclaimer.
> +    * Redistributions in binary form must reproduce the above copyright
> +    notice, this list of conditions and the following disclaimer in
> +    the documentation and/or other materials provided with the
> +    distribution.
> +    * Neither the name of Intel Corporation nor the names of its
> +    contributors may be used to endorse or promote products derived
> +    from this software without specific prior written permission.
> +
> +    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +Packet Ordering Application
> +============================
> +
> +The Packet Ordering sample app simply shows the impact of reordering a
> stream.
> +It's meant to stress the library with different configurations for
> performance.
> +
> +Overview
> +--------
> +
> +The application uses at least three CPU cores:
> +
> +* RX core (maser core) receives traffic from the NIC ports and feeds
> Worker
> +  cores with traffic through SW queues.
> +
> +* Worker core (slave core) basically do some light work on the packet.
> +  Currently it modifies the output port of the packet for configurations
> with
> +  more than one port enabled.
> +
> +* TX Core (slave core) receives traffic from Worker cores through
> software queues,
> +  inserts out-of-order packets into reorder buffer, extracts ordered
> packets
> +  from the reorder buffer and sends them to the NIC ports for
> transmission.
> +case test_packet_ordering
> +---------------------------
> +Compiling the Application
> +
> +The application execution command line is:
> +
> +.. code-block:: console
> +
> +    ./packet_ordering [EAL options] -- -p PORTMASK [--disable-reorder]
> +
> +The -c EAL CPU_COREMASK option has to contain at least 3 CPU cores.
> +The first CPU core in the core mask is the master core and would be
> assigned to
> +RX core, the last to TX core and the rest to Worker cores.
> +
> +The PORTMASK parameter must contain either 1 or even enabled port numbers.
> +When setting more than 1 port, traffic would be forwarded in pairs.
> +For example, if we enable 4 ports, traffic from port 0 to 1 and from 1 to
> 0,
> +then the other pair from 2 to 3 and from 3 to 2, having [0,1] and [2,3]
> pairs.
> +
> +The disable-reorder long option does, as its name implies, disable the
> reordering
> +of traffic, which should help evaluate reordering performance impact.
> +
> +check::
> + change the number of cpu core,check the work normal
> diff --git a/tests/TestSuite_packet_ordering.py
> b/tests/TestSuite_packet_ordering.py
> new file mode 100644
> index 0000000..6c491aa
> --- /dev/null
> +++ b/tests/TestSuite_packet_ordering.py
> @@ -0,0 +1,113 @@
> +#BSD LICENSE
> +#
> +# Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +#   * Redistributions of source code must retain the above copyright
> +#     notice, this list of conditions and the following disclaimer.
> +#   * Redistributions in binary form must reproduce the above copyright
> +#     notice, this list of conditions and the following disclaimer in
> +#     the documentation and/or other materials provided with the
> +#     distribution.
> +#   * Neither the name of Intel Corporation nor the names of its
> +#     contributors may be used to endorse or promote products derived
> +#     from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +
> +"""
> +DPDK Test suite.
> +Test packet ordering.
> +"""
> +
> +import string
> +import time
> +import re
> +from test_case import TestCase
> +from plotting import Plotting
> +from settings import HEADER_SIZE
> +from etgen import IxiaPacketGenerator
> +from packet import Packet, sniff_packets, load_sniff_packets
> +import utils
> +class TestPacketOrdering(TestCase):
> +
> +    def set_up_all(self):
> +        """
> +        Run at the start of each test suite.
> +        """
> +        self.dut_ports = self.dut.get_ports(self.nic)
> +        self.verify(len(self.dut_ports) >= 2, "Insufficient ports")
> +        self.path = "./examples/packet_ordering/build/packet_ordering"
> +        self.portmask = utils.create_mask(self.dut_ports)
> +
> +        # build sample app
> +        out = self.dut.build_dpdk_apps("./examples/packet_ordering")
> +        self.verify("Error" not in out, "compilation error 1")
> +        self.verify("No such file" not in out, "compilation error 2")
> +
> +    def set_up(self):
> +        """
> +        Run before each test case.
> +        """
> +        pass
> +
> +    def test_packet_ordering(self):
> +        """
> +        test packet ordering on different core
> +        """
> +        for i in range(1,3):
> +            cores = self.dut.get_core_list("1S/%sC/1T" % i)
> +            coremask = utils.create_mask(cores)
> +            cmd = self.path + " -c %s -n %d -- -p %s" % (coremask,
> self.dut.get_memory_channels(), self.portmask)
> +            out = self.dut.send_expect(cmd,"# ",60)
> +            self.verify("Error" in out, "Wrong: can error package")
> +        for i in range(3,19):
> +            cores = self.dut.get_core_list("1S/%sC/1T" % i)
> +            coremask = utils.create_mask(cores)
> +            cmd = self.path + " -c %s -n %d -- -p %s" %
> (coremask,self.dut.get_memory_channels(), self.portmask)
> +            self.dut.send_expect(cmd,"send_thread()",60)
> +            self.send_packet()
> +            out = self.dut.send_expect("^C","# ",50)
> +            packet = re.search(" - Pkts rxd:\s*(\d*)",out)
> +            sum_packet = packet.group(1)
> +            self.verify("1" == sum_packet, "Wrong: can error package")
> +
> +    def send_packet(self):
> +        """
> +        Send a packet to port
> +        """
> +        self.dmac = self.dut.get_mac_address(self.dut_ports[0])
> +        txport = self.tester.get_local_port(self.dut_ports[0])
> +        self.txItf = self.tester.get_interface(txport)
> +        pkt = Packet(pkt_type='UDP')
> +        pkt.config_layer('ether', {'dst': self.dmac,})
> +        pkt.send_pkt(tx_port=self.txItf)
> +
> +    def tear_down(self):
> +        """
> +        Run after each test case.
> +        """
> +        pass
> +
> +    def tear_down_all(self):
> +        """
> +        Run after each test suite.
> +        """
> +        self.dut.kill_all()
> +        time.sleep(2)
> +
> --
> 1.9.3