[dts] [PATCH v1 1/2] test_plans: add test plan for elastic flow distributor

test suite reviews and discussions
 help / color / mirror / Atom feed

* [dts] [PATCH v1 1/2] test_plans: add test plan for elastic flow distributor
@ 2017-07-27 14:19 Marvin Liu
  2017-07-27 14:19 ` [dts] [PATCH v1 2/2] tests: add test suite " Marvin Liu
  0 siblings, 1 reply; 2+ messages in thread
From: Marvin Liu @ 2017-07-27 14:19 UTC (permalink / raw)
  To: dts; +Cc: Marvin Liu

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/test_plans/efd_test_plan.rst b/test_plans/efd_test_plan.rst
new file mode 100644
index 0000000..a6b5e3d
--- /dev/null
+++ b/test_plans/efd_test_plan.rst
@@ -0,0 +1,137 @@
+.. Copyright (c) <2010-2017>, Intel Corporation
+   All rights reserved.
+   
+   Redistribution and use in source and binary forms, with or without
+   modification, are permitted provided that the following conditions
+   are met:
+   
+   - Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+   
+   - Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in
+     the documentation and/or other materials provided with the
+     distribution.
+   
+   - Neither the name of Intel Corporation nor the names of its
+     contributors may be used to endorse or promote products derived
+     from this software without specific prior written permission.
+   
+   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+   FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+   COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+   INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+   (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+   SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+   STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+   ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+   OF THE POSSIBILITY OF SUCH DAMAGE.
+
+========================
+Elastic Flow Distributor
+========================
+
+Description
+-----------
+EFD is a distributor library that uses perfect hashing to determine a
+target/value for a given incoming flow key.
+It has the following advantages: 
+1. It uses perfect hashing it does not store the key itself and hence
+lookup performance is not dependent on the key size. 
+2. Target/value can be any arbitrary value hence the system designer
+and/or operator can better optimize service rates and inter-cluster
+network traffic locating. 
+3. Since the storage requirement is much smaller than a hash-based flow
+table (i.e. better fit for CPU cache), EFD can scale to millions of flow
+keys.
+4. With the current optimized library implementation, performance is fully
+scalable with any number of CPU cores.
+
+For more details, please reference to dpdk online programming guide.
+
+Prerequisites
+=============
+Two ports connect to packet generator.
+
+DUT board must be two sockets system and each cpu have more than 16 lcores.
+
+Test Case: EFD function unit test
+---------------------------------
+Start test application and run efd unit test::
+
+   test> efd_autotest
+
+Verify every function passed in unit test
+
+Test Case: EFD performance unit test
+------------------------------------
+Start test application and run EFD performance unit test::
+
+   test> efd_perf_autotest
+
+Verify lookup and lookup bulk cpu cycles are reasonable.
+Verify when key size increased, no significant increment in cpu cycles.
+Verify when value bits increased, no significant increment in cpu cycles.
+Compare with cuckoo hash performance result, lookup cycles should be less.
+
+Test Case: Load balancer performance based on EFD
+-------------------------------------------------
+In EFD sample, EFD work as a flow-level load balancer. Flows are received at
+a front end server before being forwarded to the target back end server for
+processing. This case will measure the performance of flow distribution with
+different parameters.
+
+Value bits: number of bits of value that be stored in EFD table
+Nodes: number of back end nodes
+Entries: number of flows to be added in EFD table
+
++--------------+-------+-----------+------------+
+| Value Bits   | Nodes | Entries   | Throughput |
++--------------+-------+-----------+------------+
+|  8           |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  16          |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  24          |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  32          |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+
++--------------+-------+-----------+------------+
+| Value Bits   | Nodes | Entries   | Throughput |
++--------------+-------+-----------+------------+
+|  8           |   1   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   3   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   4   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   5   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   6   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   7   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   8   |    2M     |            |
++--------------+-------+-----------+------------+
+
++--------------+-------+-----------+------------+
+| Value Bits   | Nodes | Entries   | Throughput |
++--------------+-------+-----------+------------+
+|  8           |   2   |    1M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    4M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    8M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    16M    |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    32M    |            |
++--------------+-------+-----------+------------+
-- 
1.9.3

^ permalink raw reply	[flat|nested] 2+ messages in thread

* [dts] [PATCH v1 2/2] tests: add test suite for elastic flow distributor
  2017-07-27 14:19 [dts] [PATCH v1 1/2] test_plans: add test plan for elastic flow distributor Marvin Liu
@ 2017-07-27 14:19 ` Marvin Liu
  0 siblings, 0 replies; 2+ messages in thread
From: Marvin Liu @ 2017-07-27 14:19 UTC (permalink / raw)
  To: dts; +Cc: Marvin Liu

Signed-off-by: Marvin Liu <yong.liu@intel.com>

diff --git a/tests/TestSuite_efd.py b/tests/TestSuite_efd.py
new file mode 100644
index 0000000..991f99a
--- /dev/null
+++ b/tests/TestSuite_efd.py
@@ -0,0 +1,244 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2017 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+#   * Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+#   * Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in
+#     the documentation and/or other materials provided with the
+#     distribution.
+#   * Neither the name of Intel Corporation nor the names of its
+#     contributors may be used to endorse or promote products derived
+#     from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+"""
+DPDK Test suite.
+"""
+import re
+import utils
+from test_case import TestCase
+from etgen import IxiaPacketGenerator
+
+
+class TestEFD(TestCase, IxiaPacketGenerator):
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        """
+        self.tester.extend_external_packet_generator(TestEFD, self)
+
+        out = self.dut.send_expect("make -C test -j", "#")
+        self.verify("Error" not in out, "Compilation error")
+        self.verify("No such" not in out, "Compilation error")
+
+        out = self.dut.build_dpdk_apps("./examples/server_node_efd")
+        self.verify("Error" not in out, "Compilation error")
+        self.verify("No such" not in out, "Compilation error")
+
+        self.dut_ports = self.dut.get_ports()
+        self.node_app = "./examples/server_node_efd/node/%s/node" % self.target
+        self.server_app = "./examples/server_node_efd/server/%s/server" % self.target
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def test_efd_unit(self):
+        """
+        Run EFD unit test
+        """
+        self.dut.send_expect("./test/test/test -n 1 -c f", "RTE>>", 60)
+        out = self.dut.send_expect("efd_autotest", "RTE>>", 60)
+        self.dut.send_expect("quit", "# ")
+        self.verify("Test OK" in out, "Test failed")
+
+    def test_efd_unit_perf(self):
+        """
+        Run EFD unit perf test
+        """
+        self.dut.send_expect("./test/test/test -n 1 -c f", "RTE>>", 60)
+        out = self.dut.send_expect("efd_perf_autotest", "RTE>>", 120)
+        self.logger.info(out)
+        self.dut.send_expect("quit", "# ")
+        self.verify("Test OK" in out, "Test failed")
+
+    def test_perf_efd_nodenum(self):
+        """
+        Run EFD perf evaluation for number of nodes
+        """
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        node_nums = [1, 2, 3, 4, 5, 6, 7, 8]
+
+        flow_num = 1024 * 1024 * 2
+
+        table_header = ["Value Bits",
+                        "Nodes",
+                        "Flow Entries",
+                        "Throughput(mpps)"]
+
+        self.result_table_create(table_header)
+        # perf of different nodes
+        for node_num in node_nums:
+            pps = self._efd_perf_evaluate(node_num, flow_num)
+
+            self.result_table_add([8, node_num, "2M",  pps])
+
+        self.result_table_print()
+
+    def test_perf_efd_flownums(self):
+        """
+        Run EFD perf evaluation for millions of flows
+        """
+        self.logger.warning("Millions of flow required huge memory, please allocate 16G hugepage")
+        self.dut.setup_memory_linux(hugepages=8192)
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        flow_nums = [1024 * 1024, 1024 * 1024 * 2, 1024 * 1024 * 4,
+                     1024 * 1024 * 8, 1024 * 1024 * 16, 1024 * 1024 * 32]
+
+        table_header = ["Value Bits",
+                        "Nodes",
+                        "Million Flows",
+                        "Throughput(mpps)"]
+
+        self.result_table_create(table_header)
+
+        # perf of different flow numbers
+        for flow_num in flow_nums:
+            pps = self._efd_perf_evaluate(2, flow_num)
+
+            self.result_table_add([8, 2, flow_num / (1024 * 1024),  pps])
+
+        self.result_table_print()
+
+    def test_perf_efd_valuesize(self):
+        """
+        Run EFD perf evaluation for different value size
+        """
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        val_bitnums = [8, 16, 24, 32]
+        flow_num = 1024 * 1024 * 2
+
+        table_header = ["Value Bits",
+                        "Nodes",
+                        "Flow Entries",
+                        "Throughput(mpps)"]
+
+        self.result_table_create(table_header)
+        # perf of different value bit lengths
+        for val_bitnum in val_bitnums:
+            # change value length and rebuild dpdk
+            self.dut.send_expect("sed -i -e 's/#define RTE_EFD_VALUE_NUM_BITS .*$/#define RTE_EFD_VALUE_NUM_BITS (%d)/' lib/librte_efd/rte_efd.h" % val_bitnum, "#")
+            self.dut.build_install_dpdk(self.target)
+            out = self.dut.build_dpdk_apps("./examples/server_node_efd")
+            self.verify("Error" not in out, "Compilation error")
+            self.verify("No such" not in out, "Compilation error")
+
+            pps = self._efd_perf_evaluate(2, flow_num)
+            self.result_table_add([val_bitnum, 2, "2M",  pps])
+
+        self.result_table_print()
+        self.dut.send_expect("sed -i -e 's/#define RTE_EFD_VALUE_NUM_BITS .*$/#define RTE_EFD_VALUE_NUM_BITS (8)' lib/librte_efd/rte_efd.h", "#")
+        self.dut.build_install_dpdk(self.target)
+        out = self.dut.build_dpdk_apps("./examples/server_node_efd")
+        self.verify("Error" not in out, "Compilation error")
+        self.verify("No such" not in out, "Compilation error")
+
+    def _efd_perf_evaluate(self, node_num, flow_num):
+        # extended flow number into etgen module
+        self.tester.ixia_packet_gen.flow_num = flow_num
+
+        # output port is calculated from overall ports number
+        server_cmd_fmt = "%s -c %s -n %d -w %s -w %s -- -p 0x3 -n %d -f %s"
+        node_cmd_fmt = "%s -c %s -n %d --proc-type=secondary -- -n %d"
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+
+        self.tester.scapy_append('wrpcap("efd.pcap", [Ether()/IP(src="0.0.0.0", dst="0.0.0.0")/("X"*26)])')
+        self.tester.scapy_execute()
+
+        tgen_input = []
+        rx_port = self.tester.get_local_port(self.dut_ports[0])
+        tx_port = self.tester.get_local_port(self.dut_ports[1])
+
+        tgen_input.append((tx_port, rx_port, "efd.pcap"))
+        tgen_input.append((rx_port, tx_port, "efd.pcap"))
+
+        cores = self.dut.get_core_list("1S/%dC/1T" % (node_num + 2), socket)
+
+        self.verify(len(cores), "Can't find enough cores")
+
+        server_cmd = server_cmd_fmt % (self.server_app, utils.create_mask(cores[0:2]),
+                                       self.dut.get_memory_channels(),
+                                       self.dut.get_port_pci(self.dut_ports[0]),
+                                       self.dut.get_port_pci(self.dut_ports[1]),
+                                       node_num, hex(flow_num))
+
+        # create table may need few minutes
+        self.dut.send_expect(server_cmd, "Finished Process Init", timeout=240)
+
+        node_sessions = []
+        for node in range(node_num):
+            node_cmd = node_cmd_fmt % (self.node_app, utils.create_mask([cores[2 + node]]),
+                                       self.dut.get_memory_channels(), node)
+            node_session = self.dut.new_session(suite="node%d" % node)
+            node_sessions.append(node_session)
+            node_session.send_expect(node_cmd, "Finished Process Init", timeout=30)
+
+        _, pps = self.tester.traffic_generator_throughput(tgen_input, delay=10)
+
+        for node_session in node_sessions:
+            node_session.send_expect("^C", "#")
+            self.dut.close_session(node_session)
+
+        self.dut.send_expect("^C", "#")
+
+        pps /= 1000000.0
+        return pps
+
+    def ip(self, port, frag, src, proto, tos, dst, chksum, len, options, version, flags, ihl, ttl, id):
+        self.add_tcl_cmd("protocol config -name ip")
+        self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src)
+        self.add_tcl_cmd("ip config -sourceIpAddrMode ipIdle")
+        self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst)
+        self.add_tcl_cmd("ip config -destIpAddrMode ipIncrHost")
+        # increase number equal to flow number
+        self.add_tcl_cmd("ip config -destIpAddrRepeatCount %d" % self.flow_num)
+        self.add_tcl_cmd("ip config -ttl %d" % ttl)
+        self.add_tcl_cmd("ip config -totalLength %d" % len)
+        self.add_tcl_cmd("ip config -fragment %d" % frag)
+        self.add_tcl_cmd("ip config -ipProtocol ipV4ProtocolReserved255")
+        self.add_tcl_cmd("ip config -identifier %d" % id)
+        self.add_tcl_cmd("stream config -framesize %d" % (len + 18))
+        self.add_tcl_cmd("ip set %d %d %d" % (self.chasId, port['card'], port['port']))
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        pass
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        pass
-- 
1.9.3

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2017-07-27 14:21 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-07-27 14:19 [dts] [PATCH v1 1/2] test_plans: add test plan for elastic flow distributor Marvin Liu
2017-07-27 14:19 ` [dts] [PATCH v1 2/2] tests: add test suite " Marvin Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).