From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 4678E11A4 for ; Tue, 12 Mar 2019 07:46:59 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Mar 2019 23:46:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,470,1544515200"; d="scan'208";a="126180530" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga006.jf.intel.com with ESMTP; 11 Mar 2019 23:46:57 -0700 Received: from fmsmsx156.amr.corp.intel.com (10.18.116.74) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.408.0; Mon, 11 Mar 2019 23:46:57 -0700 Received: from shsmsx105.ccr.corp.intel.com (10.239.4.158) by fmsmsx156.amr.corp.intel.com (10.18.116.74) with Microsoft SMTP Server (TLS) id 14.3.408.0; Mon, 11 Mar 2019 23:46:56 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.158]) by SHSMSX105.ccr.corp.intel.com ([169.254.11.113]) with mapi id 14.03.0415.000; Tue, 12 Mar 2019 14:46:54 +0800 From: "Wan, Zhe" To: "Wu, ChangqingX" , "dts@dpdk.org" CC: "Wu, ChangqingX" , "Tu, Lijuan" Thread-Topic: [dts] [PATCH V3] test_plans:add test plan about loadbalancer Thread-Index: AQHU2IALtD62lRHQOECIJzcDq+XYjKYHUEiQ Date: Tue, 12 Mar 2019 06:46:53 +0000 Message-ID: <861C16A15685B44AA870C0D2A97B604589757B8D@SHSMSX101.ccr.corp.intel.com> References: <1552359361-28205-1-git-send-email-changqingx.wu@intel.com> In-Reply-To: <1552359361-28205-1-git-send-email-changqingx.wu@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZjNhNzI3MzYtYmZiNS00ZjBkLTkwMzUtMmUxNGUwYzMyZDIzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoibEdjcDZLRWNNeExFQ1FyNVhKWThZUHVPeGtFUHlCNGozdWNMZjVIb3RpWGUrWW42cEI0ZUdMWE9pWVJuV3k0TCJ9 dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V3] test_plans:add test plan about loadbalancer X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Mar 2019 06:47:00 -0000 acked-by: Wan, Zhe Thanks! BR, Wan,Zhe -----Original Message----- From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of changqingxwu Sent: Tuesday, March 12, 2019 10:56 AM To: dts@dpdk.org Cc: Wu, ChangqingX Subject: [dts] [PATCH V3] test_plans:add test plan about loadbalancer Signed-off-by: changqingxwu --- test_plans/loadbalancer_test_plan.rst | 194 ++++++++++++++++++++++++++ 1 file changed, 194 insertions(+) create mode 100644 test_plans/loadbalancer_test_plan.rst diff --git a/test_plans/loadbalancer_test_plan.rst b/test_plans/loadbalance= r_test_plan.rst new file mode 100644 index 0000000..b5cc769 --- /dev/null +++ b/test_plans/loadbalancer_test_plan.rst @@ -0,0 +1,194 @@ +.. Copyright (c) <2011>, Intel Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + - Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + - Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + + - Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Load Balancer +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This test case uses the Load Balancer sample application to benchmark=20 +the concept of isolating the packet I/O task from the application specific= workload. +A number of lcores are dedicated to handle the interaction with the NIC=20 +ports (I/O lcores), while the rest of the lcores are dedicated to=20 +performing the application processing (worker lcores). The worker=20 +lcores are totally oblivious to the intricacies of the packet I/O=20 +activity and use the NIC-agnostic interface provided by SW rings to exchan= ge packets with the I/O cores. + +Prerequisites +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +1. Hardware requirements + +- For each CPU socket, each memory channel should be populated with at le= ast + 1x DIMM. +- If the PCIe controller is on the CPU, then each CPU socket is populated= with + the same number of NICs (2x or 4x NICs per CPU socket); +- Special PCIe restrictions may be required for performance. For example,= the + following requirements should be met for 10GbE NICs: + + - NICs are plugged into PCIe Gen2 or Gen3 slots; + - For PCIe Gen2 slots, the number of lanes should be 8x or higher; + - A single port from each NIC card should be used, so for 4x ports, 4= x NICs + should be connected to the traffic generator. + +2. BIOS requirements + +- Hyper-Threading Technology is ENABLED +- Hardware Prefetcher is DISABLED +- Adjacent Cache Line Prefetch is DISABLED +- Direct Cache Access is DISABLED + +3. Linux kernel requirements + +- Linux kernel has the following features enabled: huge page support,=20 +UIO, HPET +- Appropriate number of huge pages are reserved at kernel boot time +- The IDs of the hardware threads (logical cores) per each CPU socket can= be + determined by parsing the file /proc/cpuinfo. The naming convention for= the + logical cores is: C{x.y.z} =3D hyper-thread z of physical core y of CPU= socket + x, with typical values of x =3D 0 .. 3, y =3D 0 .. 7, z =3D 0 .. 1. Log= ical cores + C{0.0.1} and C{0.0.1} should be avoided while executing the test, as th= ey + are used by the Linux kernel for running regular processes. + +4. Software application configuration + +The application configuration is done through the command line: + +- -c COREMASK: Reunion of all the cores specified by the --rx, --tx and -= -w + parameters. +- --rx and --tx: These parameters are used to provide the list of lcores = used + for packet I/O, plus the list of the RX ports & queues, as well as the = list + of TX ports, that are handled by each of the packet I/O lcores; +- The RX and TX of the NICs that are physically attached (through PCIe) t= o a + specific CPU socket should always be handled by lcores from the same=20 +socket; +- The RX and TX of the same port can be handled by different lcores, depe= nding + on the usecase, therefore the set of RX lcores can be different than th= e set + of TX lcores; +- Typical configurations enabled for the I/O cores for each CPU socket (a= s long + as the conditions below are met, the actual lcore IDs are irrelevant): + + - Single lcore handling the RX and TX for all the NICs connected to i= ts CPU + socket. Its sibling hyper-thread should not be used by the=20 + application; + +- One lcore handling the RX and TX for a single NIC port (with its siblin= g + hyper-thread not used by the application). For each CPU socket, there a= re N + physical cores used for packet I/O for N NIC ports; +- One lcore handling the RX for all the NIC ports connected to its CPU so= cket + and another lcore handling the TX for the same NIC ports (with the sibl= ing + hyper-threads not used by the application). For each CPU socket, there = are 2 + physical cores used for packet I/O for N NIC ports; +- --w: This parameter specifies the list of worker lcores; + + - A worker lcore cannot be a packet I/O lcore; + - Typical configurations enabled for each CPU socket: 1 / 2 / 4 / 8. = Each + worker should be allocated on a different physical core. For 8 work= ers + (per CPU socket), if not enough physical cores, both hyper-threads = of 4 + physical cores can be used. As long as these conditions are met, th= e + actual lcore IDs are irrelevant. + +- --lpm: IPv4 routing table; + - Typically, there is a single rule for each TX port and the address = spaces + of the rules do not overlap, e.g. for each TX_PORT used by the + application, the rule "10.0.TX_PORT.0/24 =3D> TX_PORT" is included = in the + list. +- --rsz: Ring sizes + - Typically, the default values are used (parameter not present in th= e + command line). +- --bsz: Burst sizes + - Typically, the default values are used (parameter not present in th= e + command line). +- --pos-lb: Position of the 1-byte header field within the input packet t= hat is + used to determine the worker ID for each packet + + - Typically, the default value is used (parameter not present in the + command line). + +5. Traffic generator configuration + +The application is used to benchmark the penalty of packets going=20 +across different CPU sockets. In the general case, the input packets=20 +are RX-ed by NIC connected to CPU socket X, dispatched to worker=20 +running on CPU socket Y, TX-ed by NIC connected to CPU socket Z. The=20 +typical cases under test are: AAA, AAB, ABB, ABC (for 2-socket systems, AB= C is actually ABA). + +The worker ID is determined by reading a 1-byte field from the input=20 +packet. Its position is specified using the --pos-lb command line=20 +argument. For convenience, the --pos-lb argument typically points to=20 +the last byte of the IPv4 source address, e.g. the IPv4 source address=20 +for a traffic flow that shoud be processed by WORKER_ID is: 0.0.0.WORKER_I= D. + +The TX port is determined by the LPM rule that is hit by the IPv4=20 +destination address field read from each packet. Therefore, the traffic=20 +generator configuration has to be in sync with the routing table of the=20 +application (provided using the --lpm parameter). Given the convention=20 +described above of LPM rules of: "10.0.TX_PORT.0/24 =3D> TX_PORT", then=20 +packets with IPv4 destination address of 10.0.TX_PORT.1 will be sent=20 +out by TX_PORT, regardless of the worker lcore processing them. + +For a specific test case, the recommended flow configuration for each=20 +traffic generator port (connected to a NIC attached to CPU socket X) is=20 +to create a traffic flow for each pair of (worker on CPU socket Y, NIC=20 +TX port on CPU socket Z) and equally divide the TX rate amongst all the=20 +traffic flows on the same traffic generator port. This guarantees that=20 +all the workers on CPU socket Y will be hit evenly and none of the NIC=20 +TX ports on CPU socket Z will be oversubscribed. + +In this case, the same set of application command lines (testing=20 +different packet I/O and worker set configurations) can be applied with=20 +no modifications to test scenarios AAA, AAB, ABB, ABC/ABA by simply=20 +modifying two fields within each of the traffic flows sent by the traffic = generator on each of its ports. + +Test Case: Load Balancer +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Assuming that Logical core 4, 5, 6, 7 are connected to a traffic=20 +generator, launch the ``load_balancer`` with the following arguments:: + + ./examples/load_balancer/build/load_balancer -l 3-7 -n 4 -- \ --rx=20 + "(0,0,3),(1,0,3),(2,0,3),(3,0,3)" \ --tx "(0,3),(1,3),(2,3),(3,3)"=20 + --w "4,5,6,7" \ --lpm=20 + "1.0.0.0/24=3D>0;1.0.1.0/24=3D>1;1.0.2.0/24=3D>2;1.0.3.0/24=3D>3; " \ --= bsz=20 + "(10, 10), (10, 10), (10, 10)" --pos-lb 29 + +If the app run successfully, it will be the same as the shown in the termi= nal. :: + + ... + LPM rules: + 0: 1.0.0.0/24 =3D> 0; + 1: 1.0.1.0/24 =3D> 1; + 2: 1.0.2.0/24 =3D> 2; + 3: 1.0.3.0/24 =3D> 3; + Ring sizes: NIC RX =3D 1024; Worker in =3D 1024; Worker out =3D 1024; NI= C=20 + TX =3D 1024; Burst sizes: I/O RX (rd =3D 10, wr =3D 10); Worker (rd =3D = 10,=20 + wr =3D 10); I/O TX (rd =3D 10, wr =3D 10) Logical core 4 (worker 0) main= loop. + Logical core 5 (worker 1) main loop. + Logical core 6 (worker 2) main loop. + Logical core 7 (worker 3) main loop. + Logical core 3 (I/O) main loop. -- 2.17.2