test suite reviews and discussions
 help / color / mirror / Atom feed
* [dts][PATCH V1 1/7] tests/efd:Separated performance cases
@ 2022-09-22 14:29 Hongbo Li
  2022-09-22 14:29 ` [dts][PATCH V1 2/7] tests/kni:Separated " Hongbo Li
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Hongbo Li @ 2022-09-22 14:29 UTC (permalink / raw)
  To: dts; +Cc: Hongbo Li

Separated performance cases

Signed-off-by: Hongbo Li <hongbox.li@intel.com>
---
 test_plans/efd_test_plan.rst      |  75 -----------
 test_plans/perf_efd_test_plan.rst | 105 +++++++++++++++
 tests/TestSuite_efd.py            | 138 --------------------
 tests/TestSuite_perf_efd.py       | 210 ++++++++++++++++++++++++++++++
 4 files changed, 315 insertions(+), 213 deletions(-)
 create mode 100644 test_plans/perf_efd_test_plan.rst
 create mode 100644 tests/TestSuite_perf_efd.py

diff --git a/test_plans/efd_test_plan.rst b/test_plans/efd_test_plan.rst
index ee440c02..a37a78a8 100644
--- a/test_plans/efd_test_plan.rst
+++ b/test_plans/efd_test_plan.rst
@@ -50,78 +50,3 @@ Verify lookup and lookup bulk cpu cycles are reasonable.
 Verify when key size increased, no significant increment in cpu cycles.
 Verify when value bits increased, no significant increment in cpu cycles.
 Compare with cuckoo hash performance result, lookup cycles should be less.
-
-Performance test cases
-==============================
-In EFD sample, EFD work as a flow-level load balancer. Flows are received at
-a front end server before being forwarded to the target back end server for
-processing. This case will measure the performance of flow distribution with
-different parameters.
-
-Nodes: number of back end nodes
-Entries: number of flows to be added in EFD table
-Value bits: number of bits of value that be stored in EFD table
-
-Test Case: Load balancer performance based on node numbers
-----------------------------------------------------------------------
-This case will measure the performance based on node numbers.
-
-+--------------+-------+-----------+------------+
-| Value Bits   | Nodes | Entries   | Throughput |
-+--------------+-------+-----------+------------+
-|  8           |   1   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   3   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   4   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   5   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   6   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   7   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   8   |    2M     |            |
-+--------------+-------+-----------+------------+
-
-Test Case: Load balancer performance based on flow numbers
------------------------------------------------------------------------
-This case will measure the performance based on flow numbers.
-
-+--------------+-------+-----------+------------+
-| Value Bits   | Nodes | Entries   | Throughput |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    1M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    4M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    8M     |            |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    16M    |            |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    32M    |            |
-+--------------+-------+-----------+------------+
-
-Test Case: Load balancer performance based on value bits
------------------------------------------------------------------------
-Modify different value size which must be between 1 and 32, and it need
-to be configured with the ``-Dc_args=-DRTE_EFD_VALUE_NUM_BITS`` option
-at compile time.
-
-This case will measure the performance based on value bits.
-
-+--------------+-------+-----------+------------+
-| Value Bits   | Nodes | Entries   | Throughput |
-+--------------+-------+-----------+------------+
-|  8           |   2   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  16          |   2   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  24          |   2   |    2M     |            |
-+--------------+-------+-----------+------------+
-|  32          |   2   |    2M     |            |
-+--------------+-------+-----------+------------+
diff --git a/test_plans/perf_efd_test_plan.rst b/test_plans/perf_efd_test_plan.rst
new file mode 100644
index 00000000..fe3fedd8
--- /dev/null
+++ b/test_plans/perf_efd_test_plan.rst
@@ -0,0 +1,105 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright(c) 2010-2017 Intel Corporation
+
+==================================================
+Sample Application Tests: Elastic Flow Distributor
+==================================================
+
+Description
+-----------
+EFD is a distributor library that uses perfect hashing to determine a
+target/value for a given incoming flow key.
+It has the following advantages: 
+1. It uses perfect hashing it does not store the key itself and hence
+lookup performance is not dependent on the key size. 
+2. Target/value can be any arbitrary value hence the system designer
+and/or operator can better optimize service rates and inter-cluster
+network traffic locating. 
+3. Since the storage requirement is much smaller than a hash-based flow
+table (i.e. better fit for CPU cache), EFD can scale to millions of flow
+keys.
+4. With the current optimized library implementation, performance is fully
+scalable with any number of CPU cores.
+
+For more details, please reference to dpdk online programming guide.
+
+Prerequisites
+=============
+Two ports connect to packet generator.
+
+DUT board must be two sockets system and each cpu have more than 16 lcores.
+
+Performance test cases
+==============================
+In EFD sample, EFD work as a flow-level load balancer. Flows are received at
+a front end server before being forwarded to the target back end server for
+processing. This case will measure the performance of flow distribution with
+different parameters.
+
+Nodes: number of back end nodes
+Entries: number of flows to be added in EFD table
+Value bits: number of bits of value that be stored in EFD table
+
+Test Case: Load balancer performance based on node numbers
+----------------------------------------------------------------------
+This case will measure the performance based on node numbers.
+
++--------------+-------+-----------+------------+
+| Value Bits   | Nodes | Entries   | Throughput |
++--------------+-------+-----------+------------+
+|  8           |   1   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   3   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   4   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   5   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   6   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   7   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   8   |    2M     |            |
++--------------+-------+-----------+------------+
+
+Test Case: Load balancer performance based on flow numbers
+-----------------------------------------------------------------------
+This case will measure the performance based on flow numbers.
+
++--------------+-------+-----------+------------+
+| Value Bits   | Nodes | Entries   | Throughput |
++--------------+-------+-----------+------------+
+|  8           |   2   |    1M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    4M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    8M     |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    16M    |            |
++--------------+-------+-----------+------------+
+|  8           |   2   |    32M    |            |
++--------------+-------+-----------+------------+
+
+Test Case: Load balancer performance based on value bits
+-----------------------------------------------------------------------
+Modify different value size which must be between 1 and 32, and it need
+to be configured with the ``-Dc_args=-DRTE_EFD_VALUE_NUM_BITS`` option
+at compile time.
+
+This case will measure the performance based on value bits.
+
++--------------+-------+-----------+------------+
+| Value Bits   | Nodes | Entries   | Throughput |
++--------------+-------+-----------+------------+
+|  8           |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  16          |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  24          |   2   |    2M     |            |
++--------------+-------+-----------+------------+
+|  32          |   2   |    2M     |            |
++--------------+-------+-----------+------------+
diff --git a/tests/TestSuite_efd.py b/tests/TestSuite_efd.py
index 9d94a3d4..071a3564 100644
--- a/tests/TestSuite_efd.py
+++ b/tests/TestSuite_efd.py
@@ -68,144 +68,6 @@ class TestEFD(TestCase):
         self.dut.send_expect("quit", "# ")
         self.verify("Test OK" in out, "Test failed")
 
-    def test_perf_efd_nodenum(self):
-        """
-        Run EFD perf evaluation for number of nodes
-        """
-        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
-        node_nums = [1, 2, 3, 4, 5, 6, 7, 8]
-
-        flow_num = 1024 * 1024 * 2
-
-        table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"]
-
-        self.result_table_create(table_header)
-        # perf of different nodes
-        for node_num in node_nums:
-            pps = self._efd_perf_evaluate(node_num, flow_num)
-
-            self.result_table_add([8, node_num, "2M", pps])
-
-        self.result_table_print()
-
-    def test_perf_efd_flownums(self):
-        """
-        Run EFD perf evaluation for millions of flows
-        """
-        self.logger.warning(
-            "Millions of flow required huge memory, please allocate 16G hugepage"
-        )
-        self.dut.setup_memory_linux(hugepages=8192)
-        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
-        flow_nums = [
-            1024 * 1024,
-            1024 * 1024 * 2,
-            1024 * 1024 * 4,
-            1024 * 1024 * 8,
-            1024 * 1024 * 16,
-            1024 * 1024 * 32,
-        ]
-
-        table_header = ["Value Bits", "Nodes", "Million Flows", "Throughput(mpps)"]
-
-        self.result_table_create(table_header)
-
-        # perf of different flow numbers
-        for flow_num in flow_nums:
-            pps = self._efd_perf_evaluate(2, flow_num)
-
-            self.result_table_add([8, 2, flow_num / (1024 * 1024), pps])
-
-        self.result_table_print()
-
-    def test_perf_efd_valuesize(self):
-        """
-        Run EFD perf evaluation for different value size
-        """
-        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
-        val_bitnums = [8, 16, 24, 32]
-        flow_num = 1024 * 1024 * 2
-
-        table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"]
-
-        self.result_table_create(table_header)
-        # perf of different value bit lengths
-        for val_bitnum in val_bitnums:
-            # change value length and rebuild dpdk
-            extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=%d" % val_bitnum
-            self.dut.build_install_dpdk(self.target, extra_options=extra_options)
-            self.build_server_node_efd()
-
-            pps = self._efd_perf_evaluate(2, flow_num)
-            self.result_table_add([val_bitnum, 2, "2M", pps])
-
-        self.result_table_print()
-        extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=8"
-        self.dut.build_install_dpdk(self.target, extra_options=extra_options)
-        self.build_server_node_efd()
-
-    def _efd_perf_evaluate(self, node_num, flow_num):
-        # extended flow number into etgen module
-
-        # output port is calculated from overall ports number
-        server_cmd_fmt = "%s %s -- -p 0x3 -n %d -f %s"
-        node_cmd_fmt = "%s %s --proc-type=secondary -- -n %d"
-        socket = self.dut.get_numa_id(self.dut_ports[0])
-
-        pcap = os.sep.join([self.output_path, "efd.pcap"])
-        self.tester.scapy_append(
-            'wrpcap("%s", [Ether()/IP(src="0.0.0.0", dst="0.0.0.0")/("X"*26)])' % pcap
-        )
-        self.tester.scapy_execute()
-
-        tgen_input = []
-        rx_port = self.tester.get_local_port(self.dut_ports[0])
-        tx_port = self.tester.get_local_port(self.dut_ports[1])
-
-        pcap = os.sep.join([self.output_path, "efd.pcap"])
-        tgen_input.append((tx_port, rx_port, pcap))
-        tgen_input.append((rx_port, tx_port, pcap))
-
-        cores = self.dut.get_core_list("1S/%dC/1T" % (node_num + 2), socket)
-
-        self.verify(len(cores), "Can't find enough cores")
-
-        eal_para = self.dut.create_eal_parameters(cores=cores[0:2], ports=[0, 1])
-        server_cmd = server_cmd_fmt % (
-            self.server_app,
-            eal_para,
-            node_num,
-            hex(flow_num),
-        )
-        # create table may need few minutes
-        self.dut.send_expect(server_cmd, "Finished Process Init", timeout=240)
-
-        node_sessions = []
-        for node in range(node_num):
-
-            eal_para = self.dut.create_eal_parameters(cores=[cores[2 + node]])
-            node_cmd = node_cmd_fmt % (self.node_app, eal_para, node)
-            node_session = self.dut.new_session(suite="node%d" % node)
-            node_sessions.append(node_session)
-            node_session.send_expect(node_cmd, "Finished Process Init", timeout=30)
-
-        # clear streams before add new streams
-        self.tester.pktgen.clear_streams()
-        # run packet generator
-        streams = self.pktgen_helper.prepare_stream_from_tginput(
-            tgen_input, 100, None, self.tester.pktgen
-        )
-        _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
-
-        for node_session in node_sessions:
-            node_session.send_expect("^C", "#")
-            self.dut.close_session(node_session)
-
-        self.dut.send_expect("^C", "#")
-
-        pps /= 1000000.0
-        return pps
-
     def set_fields(self):
         """set ip protocol field behavior"""
         fields_config = {
diff --git a/tests/TestSuite_perf_efd.py b/tests/TestSuite_perf_efd.py
new file mode 100644
index 00000000..5d638d2d
--- /dev/null
+++ b/tests/TestSuite_perf_efd.py
@@ -0,0 +1,210 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2010-2017 Intel Corporation
+#
+
+"""
+DPDK Test suite.
+"""
+import os
+import re
+
+import framework.utils as utils
+from framework.pktgen import PacketGeneratorHelper
+from framework.test_case import TestCase
+
+
+class TestEFD(TestCase):
+    def set_up_all(self):
+        """
+        Run at the start of each test suite.
+        """
+
+        self.build_server_node_efd()
+
+        self.dut_ports = self.dut.get_ports()
+        self.node_app = self.dut.apps_name["node"]
+        self.server_app = self.dut.apps_name["server"]
+        self.app_test_path = self.dut.apps_name["test"]
+        # get dts output path
+        if self.logger.log_path.startswith(os.sep):
+            self.output_path = self.logger.log_path
+        else:
+            cur_path = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
+            self.output_path = os.sep.join([cur_path, self.logger.log_path])
+        # create an instance to set stream field setting
+        self.pktgen_helper = PacketGeneratorHelper()
+
+    def build_server_node_efd(self):
+        apps = ["node", "server"]
+        for app in apps:
+            out = self.dut.build_dpdk_apps("./examples/server_node_efd/%s" % app)
+            self.verify("Error" not in out, "Compilation %s error" % app)
+            self.verify("No such" not in out, "Compilation %s error" % app)
+
+    def set_up(self):
+        """
+        Run before each test case.
+        """
+        pass
+
+    def test_perf_efd_nodenum(self):
+        """
+        Run EFD perf evaluation for number of nodes
+        """
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        node_nums = [1, 2, 3, 4, 5, 6, 7, 8]
+
+        flow_num = 1024 * 1024 * 2
+
+        table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"]
+
+        self.result_table_create(table_header)
+        # perf of different nodes
+        for node_num in node_nums:
+            pps = self._efd_perf_evaluate(node_num, flow_num)
+
+            self.result_table_add([8, node_num, "2M", pps])
+
+        self.result_table_print()
+
+    def test_perf_efd_flownums(self):
+        """
+        Run EFD perf evaluation for millions of flows
+        """
+        self.logger.warning(
+            "Millions of flow required huge memory, please allocate 16G hugepage"
+        )
+        self.dut.setup_memory_linux(hugepages=8192)
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        flow_nums = [
+            1024 * 1024,
+            1024 * 1024 * 2,
+            1024 * 1024 * 4,
+            1024 * 1024 * 8,
+            1024 * 1024 * 16,
+            1024 * 1024 * 32,
+        ]
+
+        table_header = ["Value Bits", "Nodes", "Million Flows", "Throughput(mpps)"]
+
+        self.result_table_create(table_header)
+
+        # perf of different flow numbers
+        for flow_num in flow_nums:
+            pps = self._efd_perf_evaluate(2, flow_num)
+
+            self.result_table_add([8, 2, flow_num / (1024 * 1024), pps])
+
+        self.result_table_print()
+
+    def test_perf_efd_valuesize(self):
+        """
+        Run EFD perf evaluation for different value size
+        """
+        self.verify(len(self.dut_ports) >= 2, "Not enough ports")
+        val_bitnums = [8, 16, 24, 32]
+        flow_num = 1024 * 1024 * 2
+
+        table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"]
+
+        self.result_table_create(table_header)
+        # perf of different value bit lengths
+        for val_bitnum in val_bitnums:
+            # change value length and rebuild dpdk
+            extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=%d" % val_bitnum
+            self.dut.build_install_dpdk(self.target, extra_options=extra_options)
+            self.build_server_node_efd()
+
+            pps = self._efd_perf_evaluate(2, flow_num)
+            self.result_table_add([val_bitnum, 2, "2M", pps])
+
+        self.result_table_print()
+        extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=8"
+        self.dut.build_install_dpdk(self.target, extra_options=extra_options)
+        self.build_server_node_efd()
+
+    def _efd_perf_evaluate(self, node_num, flow_num):
+        # extended flow number into etgen module
+
+        # output port is calculated from overall ports number
+        server_cmd_fmt = "%s %s -- -p 0x3 -n %d -f %s"
+        node_cmd_fmt = "%s %s --proc-type=secondary -- -n %d"
+        socket = self.dut.get_numa_id(self.dut_ports[0])
+
+        pcap = os.sep.join([self.output_path, "efd.pcap"])
+        self.tester.scapy_append(
+            'wrpcap("%s", [Ether()/IP(src="0.0.0.0", dst="0.0.0.0")/("X"*26)])' % pcap
+        )
+        self.tester.scapy_execute()
+
+        tgen_input = []
+        rx_port = self.tester.get_local_port(self.dut_ports[0])
+        tx_port = self.tester.get_local_port(self.dut_ports[1])
+
+        pcap = os.sep.join([self.output_path, "efd.pcap"])
+        tgen_input.append((tx_port, rx_port, pcap))
+        tgen_input.append((rx_port, tx_port, pcap))
+
+        cores = self.dut.get_core_list("1S/%dC/1T" % (node_num + 2), socket)
+
+        self.verify(len(cores), "Can't find enough cores")
+
+        eal_para = self.dut.create_eal_parameters(cores=cores[0:2], ports=[0, 1])
+        server_cmd = server_cmd_fmt % (
+            self.server_app,
+            eal_para,
+            node_num,
+            hex(flow_num),
+        )
+        # create table may need few minutes
+        self.dut.send_expect(server_cmd, "Finished Process Init", timeout=240)
+
+        node_sessions = []
+        for node in range(node_num):
+
+            eal_para = self.dut.create_eal_parameters(cores=[cores[2 + node]])
+            node_cmd = node_cmd_fmt % (self.node_app, eal_para, node)
+            node_session = self.dut.new_session(suite="node%d" % node)
+            node_sessions.append(node_session)
+            node_session.send_expect(node_cmd, "Finished Process Init", timeout=30)
+
+        # clear streams before add new streams
+        self.tester.pktgen.clear_streams()
+        # run packet generator
+        streams = self.pktgen_helper.prepare_stream_from_tginput(
+            tgen_input, 100, None, self.tester.pktgen
+        )
+        _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams)
+
+        for node_session in node_sessions:
+            node_session.send_expect("^C", "#")
+            self.dut.close_session(node_session)
+
+        self.dut.send_expect("^C", "#")
+
+        pps /= 1000000.0
+        return pps
+
+    def set_fields(self):
+        """set ip protocol field behavior"""
+        fields_config = {
+            "ip": {
+                # self.flow_num not used by this suite
+                # 'dst': {'range': self.flow_num, 'action': 'inc'}
+                "dst": {"range": 64, "action": "inc"}
+            },
+        }
+        return fields_config
+
+    def tear_down(self):
+        """
+        Run after each test case.
+        """
+        self.dut.kill_all()
+        pass
+
+    def tear_down_all(self):
+        """
+        Run after each test suite.
+        """
+        pass
-- 
2.25.1


^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-09-22  6:26 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-22 14:29 [dts][PATCH V1 1/7] tests/efd:Separated performance cases Hongbo Li
2022-09-22 14:29 ` [dts][PATCH V1 2/7] tests/kni:Separated " Hongbo Li
2022-09-22 14:29 ` [dts][PATCH V1 3/7] tests/l2fwd:Separated " Hongbo Li
2022-09-22 14:29 ` [dts][PATCH V1 4/7] tests/tso:Separated " Hongbo Li
2022-09-22 14:29 ` [dts][PATCH V1 5/7] tests/vxlan:Separated " Hongbo Li
2022-09-22 14:29 ` [dts][PATCH V1 6/7] tests/ipfrag:Separated " Hongbo Li
2022-09-22 14:29 ` [PATCH V1 7/7] tests/multiprocess:Separated " Hongbo Li

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).