From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id AFB97DE0 for ; Fri, 10 Jan 2014 03:07:16 +0100 (CET) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP; 09 Jan 2014 18:08:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.95,635,1384329600"; d="scan'208";a="462938286" Received: from pgsmsx102.gar.corp.intel.com ([10.221.44.80]) by fmsmga002.fm.intel.com with ESMTP; 09 Jan 2014 18:08:27 -0800 Received: from pgsmsx101.gar.corp.intel.com ([169.254.1.232]) by PGSMSX102.gar.corp.intel.com ([10.221.44.80]) with mapi id 14.03.0123.003; Fri, 10 Jan 2014 10:07:30 +0800 From: "Choi, Sy Jong" To: "dev@dpdk.org" Thread-Topic: [dpdk-dev] Unable to get RSS to work in testpmd and load balancing question Thread-Index: Ac8NprgMZOIlTN1wQiCp6wY/J2ezAw== Date: Fri, 10 Jan 2014 02:07:30 +0000 Message-ID: <697F8B1B48670548A5BAB03E8283550F2395552F@PGSMSX101.gar.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.30.20.206] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [dpdk-dev] Unable to get RSS to work in testpmd and load balancing question X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 02:07:17 -0000 Hi Dan, I have tested with 6 flows with identical ip address, but varies UDP port n= umber. I can see both queues with traffic. Using the following command:- sudo ./app/testpmd -c 0x1f -n 4 -- -i -rss-udp --portmask=3D0x03 --nb-cores= =3D4 --rxq=3D2 --txq=3D2 I have started with RSS IPv4, which is enabled by default. The critical part is the traffic, since I only 2 queues, I am sending 6 flo= ws with different IP addresses in order to see the flow got distributed eve= nly. Or else you might see only 1 queues if you have 2 flows they might loa= d to a single queue only. =20 My Command:- sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=3D0x03 --nb-cores=3D4 --rx= q=3D2 --txq=3D2 - Using 4 cores - Rxq =3D 2 for each port, so 4 queues to 4 cores. testpmd> show port stats all =A0 ######################## NIC statistics for port 0=A0 #################= ####### =A0 RX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 6306519648=A0=A0=A0 = RX-errors:=A0 757945685=A0=A0=A0 RX-bytes: 309383840254 =A0 TX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 132592678=A0=A0= =A0 TX-errors:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 TX-bytes: 8485925376 =A0 Stats reg=A0 0 RX-packets: 2556150208=A0=A0=A0 RX-errors:=A0=A0=A0=A0= =A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes: 116477417471 =A0 Stats reg=A0 1 RX-packets: 3750369440=A0=A0=A0 RX-errors:=A0=A0=A0=A0= =A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes: 192906422783 =A0 Stats reg=A0 2 RX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-err= ors:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes:=A0=A0 =A0=A0=A0=A0=A0= =A0=A00 . . . =A0 Stats reg 15 RX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-error= s:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes:=A0=A0=A0=A0=A0=A0=A0=A0= =A0 0 =A0 #######################################################################= ##### =A0 ######################## NIC statistics for port 1=A0 #################= ####### =A0 RX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 132594048=A0=A0= =A0 RX-errors:=A0=A0 13825889=A0=A0=A0 RX-bytes: 8486020288 =A0 TX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 6306522739=A0=A0=A0 = TX-errors:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 TX-bytes: 231983528894 =A0 Stats reg=A0 0 RX-packets:=A0=A0 83615783=A0=A0=A0 RX-errors:=A0=A0=A0= =A0=A0=A0=A0=A0 =A00=A0=A0=A0 RX-bytes: 5351410624 =A0 Stats reg=A0 1 RX-packets:=A0=A0 48978265=A0=A0=A0 RX-errors:=A0=A0=A0= =A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes: 3134609664 =A0 Stats reg=A0 2 RX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-err= ors:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes:=A0=A0=A0=A0=A0=A0=A0= =A0=A0 0 . . . =A0 Stats reg 15 RX-packets:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-error= s:=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0 RX-bytes:=A0=A0=A0=A0=A0=A0=A0=A0= =A0 0 =A0 #######################################################################= ##### testpmd> My Command:- sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=3D0x03 --nb-cores=3D4 --rx= q=3D2 --txq=3D2 - Using 4 cores - Rxq =3D 2 for each port, so 4 queues to 4 cores. I use this command to map the queue statistic. testpmd> set stat_qmap rx 0 0 0 testpmd> set stat_qmap rx 0 1 1 testpmd> set stat_qmap rx 1 0 0 testpmd> set stat_qmap rx 1 1 1 testpmd> start =A0 io packet forwarding - CRC stripping disabled - packets/burst=3D16 =A0 nb forwarding cores=3D2 - nb forwarding ports=3D2 =A0 RX queues=3D2 - RX desc=3D128 - RX free threshold=3D0 =A0 RX threshold registers: pthresh=3D8 hthresh=3D8 wthresh=3D4 =A0 TX queues=3D2 - TX desc=3D512 - TX free threshold=3D0 =A0 TX threshold registers: pthresh=3D36 hthresh=3D0 wthresh=3D0 =A0 TX RS bit threshold=3D0 - TXQ flags=3D0x0 testpmd> show port stats all Regards, Choi, Sy Jong Platform Application Engineer From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dan Kan Sent: Wednesday, January 08, 2014 3:25 PM To: dev@dpdk.org Subject: [dpdk-dev] Unable to get RSS to work in testpmd and load balancing= question I'm evaluating DPDK using dpdk-1.5.1r1. I have been playing around with the= test-pmd sample app. I'm having a hard time to get RSS to work. I have a 2= -port 82599 Intel X540-DA2 NIC. I'm running the following command to start = the app. sudo ./testpmd -c 0x1f -n 2 -- -i --portmask=3D0x3 --nb-cores=3D4 --rxq=3D4 --txq=3D4 I have a packet generator that sends udp packets with various src IP. According testpmd, I'm only receiving packets in port 0's queue 0. Packets = are not going into any other queues. I have attached the output from testpm= d. ------- Forward Stats for RX Port=3D 0/Queue=3D 0 -> TX Port=3D 1/Queue= =3D 0 ------- RX-packets: 1000000 TX-packets: 1000000 TX-dropped: 0 ---------------------- Forward statistics for port 0 ---------------------- RX-packets: 1000000 RX-dropped: 0 RX-total: 1000000 TX-packets: 0 TX-dropped: 0 TX-total: 0 ---------------------------------------------------------------------------= - ---------------------- Forward statistics for port 1 ---------------------- RX-packets: 0 RX-dropped: 0 RX-total: 0 TX-packets: 1000000 TX-dropped: 0 TX-total: 1000000 ---------------------------------------------------------------------------= - +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++ RX-packets: 1000000 RX-dropped: 0 RX-total: 1000000 TX-packets: 1000000 TX-dropped: 0 TX-total: 1000000 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= + On a separate note, I also find that the CPU utilization using 1 forwarding= core for 2 ports seems to be better (in the aggregate sense) than using 2 = forwarding cores for 2 ports. Running at 10gbps line rate of pktlen=3D400, = with 1 core, the core's utilization is 40%. With 2 cores, each core's utili= zation would 30%, giving an aggregate of 60%. I have a use case of only doing rxonly packet processing. From my initial t= est, it seems that it's more efficient to have a single core read packets f= rom both ports, and distribute the packet using rte_ring instead of having = each core read from its port. The rte_eth_rx operations appear to be much C= PU intensive than rte_ring_dequeue operations. Thanks in advance. Dan