From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) by dpdk.org (Postfix) with ESMTP id 96923DE0 for ; Fri, 10 Jan 2014 17:03:13 +0100 (CET) Received: by mail-pd0-f179.google.com with SMTP id y13so2224883pdi.38 for ; Fri, 10 Jan 2014 08:04:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=I0TW/9GGHtrXnyWutFvQlDl89oIvrQXvCCgbX3xBMyY=; b=l9Oo1sh8hF2Eewczh3qNNddUc/Td+CjlfJcxFuEvYkl7g6Rve1C0z6SZkW6oyhqziH I3B0H4uHsMnGp+spGguHoKit3ES047phvwTearp6lnGsqNX3S7arqKa9n7crs3PT3cwp kGtsQbs/iixzPdJs/QODrRXaCa9P8uqvE5QNZt0myvuzcTtqiubUGDxEI7ErR2x95xwy XhmXk14ahL7u2UKjRu60odSosrGs6o2IfWnLt/IpunpIqkSVhRNRUTk15I5IxiuI55V+ g25ONdGYaiNFu0c+sEe/9NAJMSppbXsop3OauU9/S8rlcqsqPW1AA0MAe69rDf0K5+Nw IR1w== X-Received: by 10.68.226.70 with SMTP id rq6mr12375556pbc.107.1389369865643; Fri, 10 Jan 2014 08:04:25 -0800 (PST) MIME-Version: 1.0 Received: by 10.70.131.225 with HTTP; Fri, 10 Jan 2014 08:04:05 -0800 (PST) In-Reply-To: <57D3B97C-FA2A-445F-BF15-FC1380ED427B@nyansa.com> References: <697F8B1B48670548A5BAB03E8283550F2395552F@PGSMSX101.gar.corp.intel.com> <57D3B97C-FA2A-445F-BF15-FC1380ED427B@nyansa.com> From: Michael Quicquaro Date: Fri, 10 Jan 2014 11:04:05 -0500 Message-ID: To: Daniel Kan Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Unable to get RSS to work in testpmd and load balancing question X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 16:03:14 -0000 Why are there so many RX-errors? On Thu, Jan 9, 2014 at 9:35 PM, Daniel Kan wrote: > Thanks, Sy Jong. I couldn=92t reproduce your outcome on dpdk 1.5.1 with > ixgbe. As I sent in the earlier email, rxmode.mq_mode is defaulted to 0 > (i.e. ETH_MQ_RX_NONE); it should be set to ETH_MQ_RX_RSS. > > Dan > > On Jan 9, 2014, at 6:07 PM, Choi, Sy Jong wrote: > > > > > Hi Dan, > > > > I have tested with 6 flows with identical ip address, but varies UDP > port number. I can see both queues with traffic. > > Using the following command:- > > sudo ./app/testpmd -c 0x1f -n 4 -- -i -rss-udp --portmask=3D0x03 > --nb-cores=3D4 --rxq=3D2 --txq=3D2 > > > > > > I have started with RSS IPv4, which is enabled by default. > > The critical part is the traffic, since I only 2 queues, I am sending 6 > flows with different IP addresses in order to see the flow got distribute= d > evenly. Or else you might see only 1 queues if you have 2 flows they migh= t > load to a single queue only. > > > > My Command:- > > sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=3D0x03 --nb-cores=3D4 > --rxq=3D2 --txq=3D2 > > - Using 4 cores > > - Rxq =3D 2 for each port, so 4 queues to 4 cores. > > > > > > > > testpmd> show port stats all > > > > ######################## NIC statistics for port 0 > ######################## > > RX-packets: 6306519648 RX-errors: 757945685 > RX-bytes: 309383840254 > > TX-packets: 132592678 TX-errors: 0 > TX-bytes: 8485925376 > > > > Stats reg 0 RX-packets: 2556150208 RX-errors: 0 > RX-bytes: 116477417471 > > Stats reg 1 RX-packets: 3750369440 RX-errors: 0 > RX-bytes: 192906422783 > > Stats reg 2 RX-packets: 0 RX-errors: 0 > RX-bytes: 0 > > . > > . > > . > > Stats reg 15 RX-packets: 0 RX-errors: 0 > RX-bytes: 0 > > > #########################################################################= ### > > > > ######################## NIC statistics for port 1 > ######################## > > RX-packets: 132594048 RX-errors: 13825889 > RX-bytes: 8486020288 > > TX-packets: 6306522739 TX-errors: 0 > TX-bytes: 231983528894 > > > > Stats reg 0 RX-packets: 83615783 RX-errors: 0 > RX-bytes: 5351410624 > > Stats reg 1 RX-packets: 48978265 RX-errors: 0 > RX-bytes: 3134609664 > > Stats reg 2 RX-packets: 0 RX-errors: 0 > RX-bytes: 0 > > . > > . > > . > > Stats reg 15 RX-packets: 0 RX-errors: 0 > RX-bytes: 0 > > > #########################################################################= ### > > testpmd> > > > > > > > > > > My Command:- > > sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=3D0x03 --nb-cores=3D4 > --rxq=3D2 --txq=3D2 > > - Using 4 cores > > - Rxq =3D 2 for each port, so 4 queues to 4 cores. > > > > I use this command to map the queue statistic. > > testpmd> set stat_qmap rx 0 0 0 > > testpmd> set stat_qmap rx 0 1 1 > > testpmd> set stat_qmap rx 1 0 0 > > testpmd> set stat_qmap rx 1 1 1 > > testpmd> start > > io packet forwarding - CRC stripping disabled - packets/burst=3D16 > > nb forwarding cores=3D2 - nb forwarding ports=3D2 > > RX queues=3D2 - RX desc=3D128 - RX free threshold=3D0 > > RX threshold registers: pthresh=3D8 hthresh=3D8 wthresh=3D4 > > TX queues=3D2 - TX desc=3D512 - TX free threshold=3D0 > > TX threshold registers: pthresh=3D36 hthresh=3D0 wthresh=3D0 > > TX RS bit threshold=3D0 - TXQ flags=3D0x0 > > > > testpmd> show port stats all > > > > > > > > Regards, > > Choi, Sy Jong > > Platform Application Engineer > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dan Kan > > Sent: Wednesday, January 08, 2014 3:25 PM > > To: dev@dpdk.org > > Subject: [dpdk-dev] Unable to get RSS to work in testpmd and load > balancing question > > > > I'm evaluating DPDK using dpdk-1.5.1r1. I have been playing around with > the test-pmd sample app. I'm having a hard time to get RSS to work. I hav= e > a 2-port 82599 Intel X540-DA2 NIC. I'm running the following command to > start the app. > > > > sudo ./testpmd -c 0x1f -n 2 -- -i --portmask=3D0x3 --nb-cores=3D4 --rxq= =3D4 > > --txq=3D4 > > > > I have a packet generator that sends udp packets with various src IP. > > According testpmd, I'm only receiving packets in port 0's queue 0. > Packets are not going into any other queues. I have attached the output > from testpmd. > > > > > > ------- Forward Stats for RX Port=3D 0/Queue=3D 0 -> TX Port=3D 1/Queu= e=3D 0 > > ------- > > RX-packets: 1000000 TX-packets: 1000000 TX-dropped: > > 0 > > ---------------------- Forward statistics for port 0 > > ---------------------- > > RX-packets: 1000000 RX-dropped: 0 RX-total: 1000000 > > TX-packets: 0 TX-dropped: 0 TX-total: 0 > > > > > -------------------------------------------------------------------------= --- > > > > ---------------------- Forward statistics for port 1 > > ---------------------- > > RX-packets: 0 RX-dropped: 0 RX-total: 0 > > TX-packets: 1000000 TX-dropped: 0 TX-total: 1000000 > > > > > -------------------------------------------------------------------------= --- > > > > +++++++++++++++ Accumulated forward statistics for all > > ports+++++++++++++++ > > RX-packets: 1000000 RX-dropped: 0 RX-total: 1000000 > > TX-packets: 1000000 TX-dropped: 0 TX-total: 1000000 > > > > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= +++ > > > > On a separate note, I also find that the CPU utilization using 1 > forwarding core for 2 ports seems to be better (in the aggregate sense) > than using 2 forwarding cores for 2 ports. Running at 10gbps line rate of > pktlen=3D400, with 1 core, the core's utilization is 40%. With 2 cores, e= ach > core's utilization would 30%, giving an aggregate of 60%. > > > > I have a use case of only doing rxonly packet processing. From my > initial test, it seems that it's more efficient to have a single core rea= d > packets from both ports, and distribute the packet using rte_ring instead > of having each core read from its port. The rte_eth_rx operations appear = to > be much CPU intensive than rte_ring_dequeue operations. > > > > Thanks in advance. > > > > Dan > > > >