From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 20602A0096 for ; Thu, 14 Mar 2019 09:17:33 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F094F2C17; Thu, 14 Mar 2019 09:17:32 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 3B9DA2BD3 for ; Thu, 14 Mar 2019 09:17:31 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Mar 2019 01:17:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,477,1544515200"; d="scan'208";a="154913128" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga001.fm.intel.com with ESMTP; 14 Mar 2019 01:17:29 -0700 Received: from fmsmsx154.amr.corp.intel.com (10.18.116.70) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.408.0; Thu, 14 Mar 2019 01:17:29 -0700 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by FMSMSX154.amr.corp.intel.com (10.18.116.70) with Microsoft SMTP Server (TLS) id 14.3.408.0; Thu, 14 Mar 2019 01:17:29 -0700 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.158]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.74]) with mapi id 14.03.0415.000; Thu, 14 Mar 2019 16:17:27 +0800 From: "Wu, ChangqingX" To: "Tu, Lijuan" , "dts@dpdk.org" Thread-Topic: [dts] [PATCH V1] test_plans:add test plan about loadbalancer Thread-Index: AQHU2j486ziXATe580Oi09Uh5G/6bKYKyCWg Date: Thu, 14 Mar 2019 08:17:26 +0000 Message-ID: <7F81DD3887C58F49A6B2EFEC3C28E22E0B6B7815@SHSMSX101.ccr.corp.intel.com> References: <1551689958-17285-1-git-send-email-changqingx.wu@intel.com> <8CE3E05A3F976642AAB0F4675D0AD20E0BA42C2C@SHSMSX101.ccr.corp.intel.com> In-Reply-To: <8CE3E05A3F976642AAB0F4675D0AD20E0BA42C2C@SHSMSX101.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dts] [PATCH V1] test_plans:add test plan about loadbalancer X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" Thanks changqingx -----Original Message----- From: Tu, Lijuan=20 Sent: Thursday, March 14, 2019 4:17 PM To: Wu, ChangqingX ; dts@dpdk.org Cc: Wu, ChangqingX Subject: RE: [dts] [PATCH V1] test_plans:add test plan about loadbalancer Sorry, made a mistake, applied V3, not V1 > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of changqingxwu > Sent: Monday, March 4, 2019 4:59 PM > To: dts@dpdk.org > Cc: Wu, ChangqingX > Subject: [dts] [PATCH V1] test_plans:add test plan about loadbalancer >=20 > add new test plan about loadbalancer >=20 > Signed-off-by: changqingxwu > --- > test_plans/loadbalancer_test_plan.rst | 206=20 > ++++++++++++++++++++++++++ > 1 file changed, 206 insertions(+) > create mode 100644 test_plans/loadbalancer_test_plan.rst >=20 > diff --git a/test_plans/loadbalancer_test_plan.rst > b/test_plans/loadbalancer_test_plan.rst > new file mode 100644 > index 0000000..f4d2e4e > --- /dev/null > +++ b/test_plans/loadbalancer_test_plan.rst > @@ -0,0 +1,206 @@ > +.. Copyright (c) <2011>, Intel Corporation > + All rights reserved. > + > + Redistribution and use in source and binary forms, with or without > + modification, are permitted provided that the following conditions > + are met: > + > + - Redistributions of source code must retain the above copyright > + notice, this list of conditions and the following disclaimer. > + > + - Redistributions in binary form must reproduce the above copyright > + notice, this list of conditions and the following disclaimer in > + the documentation and/or other materials provided with the > + distribution. > + > + - Neither the name of Intel Corporation nor the names of its > + contributors may be used to endorse or promote products derived > + from this software without specific prior written permission. > + > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS > + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, > INDIRECT, > + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES > + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > OR > + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION > + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN > CONTRACT, > + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) > + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED > + OF THE POSSIBILITY OF SUCH DAMAGE. > + > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > +Load Balancer > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +This test case uses the Load Balancer sample application to benchmark=20 > +the concept of isolating the packet I/O task from the application=20 > +specific > workload. > +A number of lcores are dedicated to handle the interaction with the=20 > +NIC ports (I/O lcores), while the rest of the lcores are dedicated to=20 > +performing the application processing (worker lcores). The worker=20 > +lcores are totally oblivious to the intricacies of the packet I/O=20 > +activity and use the NIC-agnostic interface provided by SW rings to=20 > +exchange > packets with the I/O cores. > + > +Prerequisites > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +1. Hardware requirements > + > +- For each CPU socket, each memory channel should be populated with=20 > +at > least > + 1x DIMM. > +- If the PCIe controller is on the CPU, then each CPU socket is populat= ed with > + the same number of NICs (2x or 4x NICs per CPU socket); > +- Special PCIe restrictions may be required for performance. For exampl= e, the > + following requirements should be met for 10GbE NICs: > + > + - NICs are plugged into PCIe Gen2 or Gen3 slots; > + - For PCIe Gen2 slots, the number of lanes should be 8x or higher; > + - A single port from each NIC card should be used, so for 4x ports,= 4x NICs > + should be connected to the traffic generator. > + > +2. BIOS requirements > + > +- Hyper-Threading Technology is ENABLED > +- Hardware Prefetcher is DISABLED > +- Adjacent Cache Line Prefetch is DISABLED > +- Direct Cache Access is DISABLED > + > +3. Linux kernel requirements > + > +- Linux kernel has the following features enabled: huge page=20 > +support, UIO, HPET > +- Appropriate number of huge pages are reserved at kernel boot time > +- The IDs of the hardware threads (logical cores) per each CPU socket c= an be > + determined by parsing the file /proc/cpuinfo. The naming convention f= or the > + logical cores is: C{x.y.z} =3D hyper-thread z of physical core y of C= PU socket > + x, with typical values of x =3D 0 .. 3, y =3D 0 .. 7, z =3D 0 .. 1. L= ogical cores > + C{0.0.1} and C{0.0.1} should be avoided while executing the test, as = they > + are used by the Linux kernel for running regular processes. > + > +4. Software application configuration > + > +The application configuration is done through the command line: > + > +- -c COREMASK: Reunion of all the cores specified by the --rx, --tx and= --w > + parameters. > +- --rx and --tx: These parameters are used to provide the list of lcore= s used > + for packet I/O, plus the list of the RX ports & queues, as well as th= e list > + of TX ports, that are handled by each of the packet I/O lcores; > +- The RX and TX of the NICs that are physically attached (through PCIe)= to a > + specific CPU socket should always be handled by lcores from the=20 > +same socket; > +- The RX and TX of the same port can be handled by different lcores, de= pending > + on the usecase, therefore the set of RX lcores can be different than = the set > + of TX lcores; > +- Typical configurations enabled for the I/O cores for each CPU socket = (as long > + as the conditions below are met, the actual lcore IDs are irrelevant)= : > + > + - Single lcore handling the RX and TX for all the NICs connected to= its CPU > + socket. Its sibling hyper-thread should not be used by the=20 > + application; > + > +- One lcore handling the RX and TX for a single NIC port (with its sibl= ing > + hyper-thread not used by the application). For each CPU socket, there= are N > + physical cores used for packet I/O for N NIC ports; > +- One lcore handling the RX for all the NIC ports connected to its CPU = socket > + and another lcore handling the TX for the same NIC ports (with the si= bling > + hyper-threads not used by the application). For each CPU socket, ther= e are 2 > + physical cores used for packet I/O for N NIC ports; > +- --w: This parameter specifies the list of worker lcores; > + > + - A worker lcore cannot be a packet I/O lcore; > + - Typical configurations enabled for each CPU socket: 1 / 2 / 4 / 8= . Each > + worker should be allocated on a different physical core. For 8 wo= rkers > + (per CPU socket), if not enough physical cores, both hyper-thread= s of 4 > + physical cores can be used. As long as these conditions are met, = the > + actual lcore IDs are irrelevant. > + > +- --lpm: IPv4 routing table; > + - Typically, there is a single rule for each TX port and the addres= s spaces > + of the rules do not overlap, e.g. for each TX_PORT used by the > + application, the rule "10.0.TX_PORT.0/24 =3D> TX_PORT" is include= d in the > + list. > +- --rsz: Ring sizes > + - Typically, the default values are used (parameter not present in = the > + command line). > +- --bsz: Burst sizes > + - Typically, the default values are used (parameter not present in = the > + command line). > +- --pos-lb: Position of the 1-byte header field within the input packet= that is > + used to determine the worker ID for each packet > + > + - Typically, the default value is used (parameter not present in th= e > + command line). > + > +5. Traffic generator configuration > + > +The application is used to benchmark the penalty of packets going=20 > +across different CPU sockets. In the general case, the input packets=20 > +are RX-ed by NIC connected to CPU socket X, dispatched to worker=20 > +running on CPU socket Y, TX-ed by NIC connected to CPU socket Z. The=20 > +typical cases under test are: AAA, AAB, ABB, ABC (for 2-socket=20 > +systems, ABC is > actually ABA). > + > +The worker ID is determined by reading a 1-byte field from the input=20 > +packet. Its position is specified using the --pos-lb command line=20 > +argument. For convenience, the --pos-lb argument typically points to=20 > +the last byte of the IPv4 source address, e.g. the IPv4 source=20 > +address for a traffic flow that shoud be processed by WORKER_ID is: 0.0.= 0.WORKER_ID. > + > +The TX port is determined by the LPM rule that is hit by the IPv4=20 > +destination address field read from each packet. Therefore, the=20 > +traffic generator configuration has to be in sync with the routing=20 > +table of the application (provided using the --lpm parameter). Given=20 > +the convention described above of LPM rules of: "10.0.TX_PORT.0/24 =3D>= =20 > +TX_PORT", then packets with IPv4 destination address of=20 > +10.0.TX_PORT.1 will be sent out by TX_PORT, regardless of the worker lco= re processing them. > + > +For a specific test case, the recommended flow configuration for each=20 > +traffic generator port (connected to a NIC attached to CPU socket X)=20 > +is to create a traffic flow for each pair of (worker on CPU socket Y,=20 > +NIC TX port on CPU socket Z) and equally divide the TX rate amongst=20 > +all the traffic flows on the same traffic generator port. This=20 > +guarantees that all the workers on CPU socket Y will be hit evenly=20 > +and none of the NIC TX ports on CPU socket Z will be oversubscribed. > + > +In this case, the same set of application command lines (testing=20 > +different packet I/O and worker set configurations) can be applied=20 > +with no modifications to test scenarios AAA, AAB, ABB, ABC/ABA by=20 > +simply modifying two fields within each of the traffic flows sent by=20 > +the traffic > generator on each of its ports. > + > +Test Case: Load Balancer > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +The following packet I/O and worker lcore scenarios should be=20 > +benchmarked for the CPU socket configurations of AAA, AAB, ABB, ABC/ABA: > + > +- Packet I/O lcores per CPU socket: > + > +- One lcore does RX & TX for all NIC ports: detailed above; > + > +- One lcore does RX & TX per each NIC port: detailed above; > + > +- One lcore does RX for all NIC ports and one other lcore does TX for a= ll NIC > + ports: detailed above. > + > +Assuming that Logical core 4, 5, 6, 7 are connected to a traffic=20 > +generator, launch the ``load_balancer`` with the following arguments:: > + > + ./examples/load_balancer/build/load_balancer -l 3-7 -n 4 -- \ --rx=20 > + "(0,0,3),(1,0,3),(2,0,3),(3,0,3)" \ --tx "(0,3),(1,3),(2,3),(3,3)" > + --w "4,5,6,7" \ --lpm > + "1.0.0.0/24=3D>0;1.0.1.0/24=3D>1;1.0.2.0/24=3D>2;1.0.3.0/24=3D>3; " \ = --bsz=20 > + "(10, 10), (10, 10), (10, 10)" --pos-lb 29 > + > +If the app run successfully, it will be the same as the shown in the ter= minal. :: > + > + ... > + LPM rules: > + 0: 1.0.0.0/24 =3D> 0; > + 1: 1.0.1.0/24 =3D> 1; > + 2: 1.0.2.0/24 =3D> 2; > + 3: 1.0.3.0/24 =3D> 3; > + Ring sizes: NIC RX =3D 1024; Worker in =3D 1024; Worker out =3D 1024; = NIC=20 > + TX =3D 1024; Burst sizes: I/O RX (rd =3D 10, wr =3D 10); Worker (rd = =3D 10,=20 > + wr =3D 10); I/O TX (rd =3D 10, wr =3D 10) Logical core 4 (worker 0) ma= in loop. > + Logical core 5 (worker 1) main loop. > + Logical core 6 (worker 2) main loop. > + Logical core 7 (worker 3) main loop. > + Logical core 3 (I/O) main loop. > -- > 2.17.2