From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8951AA0540; Thu, 22 Sep 2022 08:17:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F0024067C; Thu, 22 Sep 2022 08:17:25 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 038E740156 for ; Thu, 22 Sep 2022 08:17:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663827443; x=1695363443; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=QMSDJ4OsGOqTtcD33EFFS41RQE3rQ6Iav4hmOg51j6Q=; b=ebjV2a6//EZvSsNMzAV5Nq1sK0eNfivqdgHzbjS0z5/fApPn2ncGYvXu jM/3VXtSX8+KpWExeRfCxC9wMhacMtJIJdT9L7tryF2TzsyBajlahLpUU QMloGiLcsyJSzVCxoSXlgJvFL8a968gXPqZCB6v/PJrFXHp3aWy6SqnTb cyOXOPjZ1FqRw4G5qRxuPfTNHom9e2IfFDMULbLHIIYxjxB5yc6owPPp1 CY9F2PDzIjdbLfswhiij+/OZPHOaDgNA2Lty3lA3kErmcNpGMgUzAeWYp xsCTcJNQJ6fKuqoSybjGXmXIqXnjyppyLalMYBlNLN8vC82Jop6jNOpDN A==; X-IronPort-AV: E=McAfee;i="6500,9779,10477"; a="364188954" X-IronPort-AV: E=Sophos;i="5.93,335,1654585200"; d="scan'208";a="364188954" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2022 23:17:21 -0700 X-IronPort-AV: E=Sophos;i="5.93,335,1654585200"; d="scan'208";a="948465956" Received: from unknown (HELO localhost.localdomain) ([10.239.252.92]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Sep 2022 23:17:16 -0700 From: Hongbo Li To: dts@dpdk.org Cc: Hongbo Li Subject: [dts][PATCH V1 1/7] tests/efd:Separated performance cases Date: Thu, 22 Sep 2022 14:29:44 +0000 Message-Id: <20220922142950.398902-1-hongbox.li@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Separated performance cases Signed-off-by: Hongbo Li --- test_plans/efd_test_plan.rst | 75 ----------- test_plans/perf_efd_test_plan.rst | 105 +++++++++++++++ tests/TestSuite_efd.py | 138 -------------------- tests/TestSuite_perf_efd.py | 210 ++++++++++++++++++++++++++++++ 4 files changed, 315 insertions(+), 213 deletions(-) create mode 100644 test_plans/perf_efd_test_plan.rst create mode 100644 tests/TestSuite_perf_efd.py diff --git a/test_plans/efd_test_plan.rst b/test_plans/efd_test_plan.rst index ee440c02..a37a78a8 100644 --- a/test_plans/efd_test_plan.rst +++ b/test_plans/efd_test_plan.rst @@ -50,78 +50,3 @@ Verify lookup and lookup bulk cpu cycles are reasonable. Verify when key size increased, no significant increment in cpu cycles. Verify when value bits increased, no significant increment in cpu cycles. Compare with cuckoo hash performance result, lookup cycles should be less. - -Performance test cases -============================== -In EFD sample, EFD work as a flow-level load balancer. Flows are received at -a front end server before being forwarded to the target back end server for -processing. This case will measure the performance of flow distribution with -different parameters. - -Nodes: number of back end nodes -Entries: number of flows to be added in EFD table -Value bits: number of bits of value that be stored in EFD table - -Test Case: Load balancer performance based on node numbers ----------------------------------------------------------------------- -This case will measure the performance based on node numbers. - -+--------------+-------+-----------+------------+ -| Value Bits | Nodes | Entries | Throughput | -+--------------+-------+-----------+------------+ -| 8 | 1 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 2 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 3 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 4 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 5 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 6 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 7 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 8 | 2M | | -+--------------+-------+-----------+------------+ - -Test Case: Load balancer performance based on flow numbers ------------------------------------------------------------------------ -This case will measure the performance based on flow numbers. - -+--------------+-------+-----------+------------+ -| Value Bits | Nodes | Entries | Throughput | -+--------------+-------+-----------+------------+ -| 8 | 2 | 1M | | -+--------------+-------+-----------+------------+ -| 8 | 2 | 2M | | -+--------------+-------+-----------+------------+ -| 8 | 2 | 4M | | -+--------------+-------+-----------+------------+ -| 8 | 2 | 8M | | -+--------------+-------+-----------+------------+ -| 8 | 2 | 16M | | -+--------------+-------+-----------+------------+ -| 8 | 2 | 32M | | -+--------------+-------+-----------+------------+ - -Test Case: Load balancer performance based on value bits ------------------------------------------------------------------------ -Modify different value size which must be between 1 and 32, and it need -to be configured with the ``-Dc_args=-DRTE_EFD_VALUE_NUM_BITS`` option -at compile time. - -This case will measure the performance based on value bits. - -+--------------+-------+-----------+------------+ -| Value Bits | Nodes | Entries | Throughput | -+--------------+-------+-----------+------------+ -| 8 | 2 | 2M | | -+--------------+-------+-----------+------------+ -| 16 | 2 | 2M | | -+--------------+-------+-----------+------------+ -| 24 | 2 | 2M | | -+--------------+-------+-----------+------------+ -| 32 | 2 | 2M | | -+--------------+-------+-----------+------------+ diff --git a/test_plans/perf_efd_test_plan.rst b/test_plans/perf_efd_test_plan.rst new file mode 100644 index 00000000..fe3fedd8 --- /dev/null +++ b/test_plans/perf_efd_test_plan.rst @@ -0,0 +1,105 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2010-2017 Intel Corporation + +================================================== +Sample Application Tests: Elastic Flow Distributor +================================================== + +Description +----------- +EFD is a distributor library that uses perfect hashing to determine a +target/value for a given incoming flow key. +It has the following advantages: +1. It uses perfect hashing it does not store the key itself and hence +lookup performance is not dependent on the key size. +2. Target/value can be any arbitrary value hence the system designer +and/or operator can better optimize service rates and inter-cluster +network traffic locating. +3. Since the storage requirement is much smaller than a hash-based flow +table (i.e. better fit for CPU cache), EFD can scale to millions of flow +keys. +4. With the current optimized library implementation, performance is fully +scalable with any number of CPU cores. + +For more details, please reference to dpdk online programming guide. + +Prerequisites +============= +Two ports connect to packet generator. + +DUT board must be two sockets system and each cpu have more than 16 lcores. + +Performance test cases +============================== +In EFD sample, EFD work as a flow-level load balancer. Flows are received at +a front end server before being forwarded to the target back end server for +processing. This case will measure the performance of flow distribution with +different parameters. + +Nodes: number of back end nodes +Entries: number of flows to be added in EFD table +Value bits: number of bits of value that be stored in EFD table + +Test Case: Load balancer performance based on node numbers +---------------------------------------------------------------------- +This case will measure the performance based on node numbers. + ++--------------+-------+-----------+------------+ +| Value Bits | Nodes | Entries | Throughput | ++--------------+-------+-----------+------------+ +| 8 | 1 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 2 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 3 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 4 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 5 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 6 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 7 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 8 | 2M | | ++--------------+-------+-----------+------------+ + +Test Case: Load balancer performance based on flow numbers +----------------------------------------------------------------------- +This case will measure the performance based on flow numbers. + ++--------------+-------+-----------+------------+ +| Value Bits | Nodes | Entries | Throughput | ++--------------+-------+-----------+------------+ +| 8 | 2 | 1M | | ++--------------+-------+-----------+------------+ +| 8 | 2 | 2M | | ++--------------+-------+-----------+------------+ +| 8 | 2 | 4M | | ++--------------+-------+-----------+------------+ +| 8 | 2 | 8M | | ++--------------+-------+-----------+------------+ +| 8 | 2 | 16M | | ++--------------+-------+-----------+------------+ +| 8 | 2 | 32M | | ++--------------+-------+-----------+------------+ + +Test Case: Load balancer performance based on value bits +----------------------------------------------------------------------- +Modify different value size which must be between 1 and 32, and it need +to be configured with the ``-Dc_args=-DRTE_EFD_VALUE_NUM_BITS`` option +at compile time. + +This case will measure the performance based on value bits. + ++--------------+-------+-----------+------------+ +| Value Bits | Nodes | Entries | Throughput | ++--------------+-------+-----------+------------+ +| 8 | 2 | 2M | | ++--------------+-------+-----------+------------+ +| 16 | 2 | 2M | | ++--------------+-------+-----------+------------+ +| 24 | 2 | 2M | | ++--------------+-------+-----------+------------+ +| 32 | 2 | 2M | | ++--------------+-------+-----------+------------+ diff --git a/tests/TestSuite_efd.py b/tests/TestSuite_efd.py index 9d94a3d4..071a3564 100644 --- a/tests/TestSuite_efd.py +++ b/tests/TestSuite_efd.py @@ -68,144 +68,6 @@ class TestEFD(TestCase): self.dut.send_expect("quit", "# ") self.verify("Test OK" in out, "Test failed") - def test_perf_efd_nodenum(self): - """ - Run EFD perf evaluation for number of nodes - """ - self.verify(len(self.dut_ports) >= 2, "Not enough ports") - node_nums = [1, 2, 3, 4, 5, 6, 7, 8] - - flow_num = 1024 * 1024 * 2 - - table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"] - - self.result_table_create(table_header) - # perf of different nodes - for node_num in node_nums: - pps = self._efd_perf_evaluate(node_num, flow_num) - - self.result_table_add([8, node_num, "2M", pps]) - - self.result_table_print() - - def test_perf_efd_flownums(self): - """ - Run EFD perf evaluation for millions of flows - """ - self.logger.warning( - "Millions of flow required huge memory, please allocate 16G hugepage" - ) - self.dut.setup_memory_linux(hugepages=8192) - self.verify(len(self.dut_ports) >= 2, "Not enough ports") - flow_nums = [ - 1024 * 1024, - 1024 * 1024 * 2, - 1024 * 1024 * 4, - 1024 * 1024 * 8, - 1024 * 1024 * 16, - 1024 * 1024 * 32, - ] - - table_header = ["Value Bits", "Nodes", "Million Flows", "Throughput(mpps)"] - - self.result_table_create(table_header) - - # perf of different flow numbers - for flow_num in flow_nums: - pps = self._efd_perf_evaluate(2, flow_num) - - self.result_table_add([8, 2, flow_num / (1024 * 1024), pps]) - - self.result_table_print() - - def test_perf_efd_valuesize(self): - """ - Run EFD perf evaluation for different value size - """ - self.verify(len(self.dut_ports) >= 2, "Not enough ports") - val_bitnums = [8, 16, 24, 32] - flow_num = 1024 * 1024 * 2 - - table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"] - - self.result_table_create(table_header) - # perf of different value bit lengths - for val_bitnum in val_bitnums: - # change value length and rebuild dpdk - extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=%d" % val_bitnum - self.dut.build_install_dpdk(self.target, extra_options=extra_options) - self.build_server_node_efd() - - pps = self._efd_perf_evaluate(2, flow_num) - self.result_table_add([val_bitnum, 2, "2M", pps]) - - self.result_table_print() - extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=8" - self.dut.build_install_dpdk(self.target, extra_options=extra_options) - self.build_server_node_efd() - - def _efd_perf_evaluate(self, node_num, flow_num): - # extended flow number into etgen module - - # output port is calculated from overall ports number - server_cmd_fmt = "%s %s -- -p 0x3 -n %d -f %s" - node_cmd_fmt = "%s %s --proc-type=secondary -- -n %d" - socket = self.dut.get_numa_id(self.dut_ports[0]) - - pcap = os.sep.join([self.output_path, "efd.pcap"]) - self.tester.scapy_append( - 'wrpcap("%s", [Ether()/IP(src="0.0.0.0", dst="0.0.0.0")/("X"*26)])' % pcap - ) - self.tester.scapy_execute() - - tgen_input = [] - rx_port = self.tester.get_local_port(self.dut_ports[0]) - tx_port = self.tester.get_local_port(self.dut_ports[1]) - - pcap = os.sep.join([self.output_path, "efd.pcap"]) - tgen_input.append((tx_port, rx_port, pcap)) - tgen_input.append((rx_port, tx_port, pcap)) - - cores = self.dut.get_core_list("1S/%dC/1T" % (node_num + 2), socket) - - self.verify(len(cores), "Can't find enough cores") - - eal_para = self.dut.create_eal_parameters(cores=cores[0:2], ports=[0, 1]) - server_cmd = server_cmd_fmt % ( - self.server_app, - eal_para, - node_num, - hex(flow_num), - ) - # create table may need few minutes - self.dut.send_expect(server_cmd, "Finished Process Init", timeout=240) - - node_sessions = [] - for node in range(node_num): - - eal_para = self.dut.create_eal_parameters(cores=[cores[2 + node]]) - node_cmd = node_cmd_fmt % (self.node_app, eal_para, node) - node_session = self.dut.new_session(suite="node%d" % node) - node_sessions.append(node_session) - node_session.send_expect(node_cmd, "Finished Process Init", timeout=30) - - # clear streams before add new streams - self.tester.pktgen.clear_streams() - # run packet generator - streams = self.pktgen_helper.prepare_stream_from_tginput( - tgen_input, 100, None, self.tester.pktgen - ) - _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams) - - for node_session in node_sessions: - node_session.send_expect("^C", "#") - self.dut.close_session(node_session) - - self.dut.send_expect("^C", "#") - - pps /= 1000000.0 - return pps - def set_fields(self): """set ip protocol field behavior""" fields_config = { diff --git a/tests/TestSuite_perf_efd.py b/tests/TestSuite_perf_efd.py new file mode 100644 index 00000000..5d638d2d --- /dev/null +++ b/tests/TestSuite_perf_efd.py @@ -0,0 +1,210 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2010-2017 Intel Corporation +# + +""" +DPDK Test suite. +""" +import os +import re + +import framework.utils as utils +from framework.pktgen import PacketGeneratorHelper +from framework.test_case import TestCase + + +class TestEFD(TestCase): + def set_up_all(self): + """ + Run at the start of each test suite. + """ + + self.build_server_node_efd() + + self.dut_ports = self.dut.get_ports() + self.node_app = self.dut.apps_name["node"] + self.server_app = self.dut.apps_name["server"] + self.app_test_path = self.dut.apps_name["test"] + # get dts output path + if self.logger.log_path.startswith(os.sep): + self.output_path = self.logger.log_path + else: + cur_path = os.path.dirname(os.path.dirname(os.path.realpath(__file__))) + self.output_path = os.sep.join([cur_path, self.logger.log_path]) + # create an instance to set stream field setting + self.pktgen_helper = PacketGeneratorHelper() + + def build_server_node_efd(self): + apps = ["node", "server"] + for app in apps: + out = self.dut.build_dpdk_apps("./examples/server_node_efd/%s" % app) + self.verify("Error" not in out, "Compilation %s error" % app) + self.verify("No such" not in out, "Compilation %s error" % app) + + def set_up(self): + """ + Run before each test case. + """ + pass + + def test_perf_efd_nodenum(self): + """ + Run EFD perf evaluation for number of nodes + """ + self.verify(len(self.dut_ports) >= 2, "Not enough ports") + node_nums = [1, 2, 3, 4, 5, 6, 7, 8] + + flow_num = 1024 * 1024 * 2 + + table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"] + + self.result_table_create(table_header) + # perf of different nodes + for node_num in node_nums: + pps = self._efd_perf_evaluate(node_num, flow_num) + + self.result_table_add([8, node_num, "2M", pps]) + + self.result_table_print() + + def test_perf_efd_flownums(self): + """ + Run EFD perf evaluation for millions of flows + """ + self.logger.warning( + "Millions of flow required huge memory, please allocate 16G hugepage" + ) + self.dut.setup_memory_linux(hugepages=8192) + self.verify(len(self.dut_ports) >= 2, "Not enough ports") + flow_nums = [ + 1024 * 1024, + 1024 * 1024 * 2, + 1024 * 1024 * 4, + 1024 * 1024 * 8, + 1024 * 1024 * 16, + 1024 * 1024 * 32, + ] + + table_header = ["Value Bits", "Nodes", "Million Flows", "Throughput(mpps)"] + + self.result_table_create(table_header) + + # perf of different flow numbers + for flow_num in flow_nums: + pps = self._efd_perf_evaluate(2, flow_num) + + self.result_table_add([8, 2, flow_num / (1024 * 1024), pps]) + + self.result_table_print() + + def test_perf_efd_valuesize(self): + """ + Run EFD perf evaluation for different value size + """ + self.verify(len(self.dut_ports) >= 2, "Not enough ports") + val_bitnums = [8, 16, 24, 32] + flow_num = 1024 * 1024 * 2 + + table_header = ["Value Bits", "Nodes", "Flow Entries", "Throughput(mpps)"] + + self.result_table_create(table_header) + # perf of different value bit lengths + for val_bitnum in val_bitnums: + # change value length and rebuild dpdk + extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=%d" % val_bitnum + self.dut.build_install_dpdk(self.target, extra_options=extra_options) + self.build_server_node_efd() + + pps = self._efd_perf_evaluate(2, flow_num) + self.result_table_add([val_bitnum, 2, "2M", pps]) + + self.result_table_print() + extra_options = "-Dc_args=-DRTE_EFD_VALUE_NUM_BITS=8" + self.dut.build_install_dpdk(self.target, extra_options=extra_options) + self.build_server_node_efd() + + def _efd_perf_evaluate(self, node_num, flow_num): + # extended flow number into etgen module + + # output port is calculated from overall ports number + server_cmd_fmt = "%s %s -- -p 0x3 -n %d -f %s" + node_cmd_fmt = "%s %s --proc-type=secondary -- -n %d" + socket = self.dut.get_numa_id(self.dut_ports[0]) + + pcap = os.sep.join([self.output_path, "efd.pcap"]) + self.tester.scapy_append( + 'wrpcap("%s", [Ether()/IP(src="0.0.0.0", dst="0.0.0.0")/("X"*26)])' % pcap + ) + self.tester.scapy_execute() + + tgen_input = [] + rx_port = self.tester.get_local_port(self.dut_ports[0]) + tx_port = self.tester.get_local_port(self.dut_ports[1]) + + pcap = os.sep.join([self.output_path, "efd.pcap"]) + tgen_input.append((tx_port, rx_port, pcap)) + tgen_input.append((rx_port, tx_port, pcap)) + + cores = self.dut.get_core_list("1S/%dC/1T" % (node_num + 2), socket) + + self.verify(len(cores), "Can't find enough cores") + + eal_para = self.dut.create_eal_parameters(cores=cores[0:2], ports=[0, 1]) + server_cmd = server_cmd_fmt % ( + self.server_app, + eal_para, + node_num, + hex(flow_num), + ) + # create table may need few minutes + self.dut.send_expect(server_cmd, "Finished Process Init", timeout=240) + + node_sessions = [] + for node in range(node_num): + + eal_para = self.dut.create_eal_parameters(cores=[cores[2 + node]]) + node_cmd = node_cmd_fmt % (self.node_app, eal_para, node) + node_session = self.dut.new_session(suite="node%d" % node) + node_sessions.append(node_session) + node_session.send_expect(node_cmd, "Finished Process Init", timeout=30) + + # clear streams before add new streams + self.tester.pktgen.clear_streams() + # run packet generator + streams = self.pktgen_helper.prepare_stream_from_tginput( + tgen_input, 100, None, self.tester.pktgen + ) + _, pps = self.tester.pktgen.measure_throughput(stream_ids=streams) + + for node_session in node_sessions: + node_session.send_expect("^C", "#") + self.dut.close_session(node_session) + + self.dut.send_expect("^C", "#") + + pps /= 1000000.0 + return pps + + def set_fields(self): + """set ip protocol field behavior""" + fields_config = { + "ip": { + # self.flow_num not used by this suite + # 'dst': {'range': self.flow_num, 'action': 'inc'} + "dst": {"range": 64, "action": "inc"} + }, + } + return fields_config + + def tear_down(self): + """ + Run after each test case. + """ + self.dut.kill_all() + pass + + def tear_down_all(self): + """ + Run after each test suite. + """ + pass -- 2.25.1