From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by dpdk.org (Postfix) with ESMTP id 105E4DE0 for ; Fri, 10 Jan 2014 03:34:14 +0100 (CET) Received: by mail-pa0-f54.google.com with SMTP id kl14so4104383pab.13 for ; Thu, 09 Jan 2014 18:35:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=zMlWV2anyDzkfv6PKXuIYvILG9t+4HgO3D7D5gqw8C8=; b=eznAMilJCLsHNit91LDul9cKaapIYaAWbrPIFGDZBrotrW4rz/2hUIp/jdVB4Gpy1t 1zSTMQdLI8/wEUdfuCfNruqqBCPGGvIXvEiVGF+wqeG9VUrITHknOubwPPcHrpgizF3l IGNlSm+O7v7E8WhwNKkYgD+inGjH2Q8mKdbOzkCfybZYluqz7sgyboieH2aVB9Ve2vJ9 u4k9SKP9BZ+iPnDud84zZpkJtSyNpMdtvlHtEENinmHKSFblP/KV3Ujwqrt4dPUXEPcz dD+/dCslDJHHwzo76U1LEiDi0m+Mywp25G2nyhqLfIacGO5FF3jDv9gMpIH90Hohy/8/ sCtg== X-Gm-Message-State: ALoCoQnoPTuKM+Erlr9kO6EF4iUXHRyjvya83eywi/4jMXMUthgaTjUterCKqoSUrPI5O8Iug6u5 X-Received: by 10.68.29.72 with SMTP id i8mr7947801pbh.116.1389321326966; Thu, 09 Jan 2014 18:35:26 -0800 (PST) Received: from [192.168.128.242] (50-76-35-254-ip-static.hfc.comcastbusiness.net. [50.76.35.254]) by mx.google.com with ESMTPSA id vn10sm13253511pbc.21.2014.01.09.18.35.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 09 Jan 2014 18:35:25 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) From: Daniel Kan In-Reply-To: <697F8B1B48670548A5BAB03E8283550F2395552F@PGSMSX101.gar.corp.intel.com> Date: Thu, 9 Jan 2014 18:35:19 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <57D3B97C-FA2A-445F-BF15-FC1380ED427B@nyansa.com> References: <697F8B1B48670548A5BAB03E8283550F2395552F@PGSMSX101.gar.corp.intel.com> To: "Choi, Sy Jong" X-Mailer: Apple Mail (2.1827) Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Unable to get RSS to work in testpmd and load balancing question X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jan 2014 02:34:15 -0000 Thanks, Sy Jong. I couldn=92t reproduce your outcome on dpdk 1.5.1 with = ixgbe. As I sent in the earlier email, rxmode.mq_mode is defaulted to 0 = (i.e. ETH_MQ_RX_NONE); it should be set to ETH_MQ_RX_RSS. Dan On Jan 9, 2014, at 6:07 PM, Choi, Sy Jong = wrote: >=20 > Hi Dan, >=20 > I have tested with 6 flows with identical ip address, but varies UDP = port number. I can see both queues with traffic. > Using the following command:- > sudo ./app/testpmd -c 0x1f -n 4 -- -i -rss-udp --portmask=3D0x03 = --nb-cores=3D4 --rxq=3D2 --txq=3D2 >=20 >=20 > I have started with RSS IPv4, which is enabled by default. > The critical part is the traffic, since I only 2 queues, I am sending = 6 flows with different IP addresses in order to see the flow got = distributed evenly. Or else you might see only 1 queues if you have 2 = flows they might load to a single queue only. =20 >=20 > My Command:- > sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=3D0x03 --nb-cores=3D4 = --rxq=3D2 --txq=3D2 > - Using 4 cores > - Rxq =3D 2 for each port, so 4 queues to 4 cores. >=20 >=20 >=20 > testpmd> show port stats all >=20 > ######################## NIC statistics for port 0 = ######################## > RX-packets: 6306519648 RX-errors: 757945685 = RX-bytes: 309383840254 > TX-packets: 132592678 TX-errors: 0 = TX-bytes: 8485925376 >=20 > Stats reg 0 RX-packets: 2556150208 RX-errors: 0 = RX-bytes: 116477417471 > Stats reg 1 RX-packets: 3750369440 RX-errors: 0 = RX-bytes: 192906422783 > Stats reg 2 RX-packets: 0 RX-errors: 0 = RX-bytes: 0 > . > . > . > Stats reg 15 RX-packets: 0 RX-errors: 0 = RX-bytes: 0 > = ##########################################################################= ## >=20 > ######################## NIC statistics for port 1 = ######################## > RX-packets: 132594048 RX-errors: 13825889 = RX-bytes: 8486020288 > TX-packets: 6306522739 TX-errors: 0 = TX-bytes: 231983528894 >=20 > Stats reg 0 RX-packets: 83615783 RX-errors: 0 = RX-bytes: 5351410624 > Stats reg 1 RX-packets: 48978265 RX-errors: 0 = RX-bytes: 3134609664 > Stats reg 2 RX-packets: 0 RX-errors: 0 = RX-bytes: 0 > . > . > . > Stats reg 15 RX-packets: 0 RX-errors: 0 = RX-bytes: 0 > = ##########################################################################= ## > testpmd> >=20 >=20 >=20 >=20 > My Command:- > sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=3D0x03 --nb-cores=3D4 = --rxq=3D2 --txq=3D2 > - Using 4 cores > - Rxq =3D 2 for each port, so 4 queues to 4 cores. >=20 > I use this command to map the queue statistic. > testpmd> set stat_qmap rx 0 0 0 > testpmd> set stat_qmap rx 0 1 1 > testpmd> set stat_qmap rx 1 0 0 > testpmd> set stat_qmap rx 1 1 1 > testpmd> start > io packet forwarding - CRC stripping disabled - packets/burst=3D16 > nb forwarding cores=3D2 - nb forwarding ports=3D2 > RX queues=3D2 - RX desc=3D128 - RX free threshold=3D0 > RX threshold registers: pthresh=3D8 hthresh=3D8 wthresh=3D4 > TX queues=3D2 - TX desc=3D512 - TX free threshold=3D0 > TX threshold registers: pthresh=3D36 hthresh=3D0 wthresh=3D0 > TX RS bit threshold=3D0 - TXQ flags=3D0x0 >=20 > testpmd> show port stats all >=20 >=20 >=20 > Regards, > Choi, Sy Jong > Platform Application Engineer >=20 > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dan Kan > Sent: Wednesday, January 08, 2014 3:25 PM > To: dev@dpdk.org > Subject: [dpdk-dev] Unable to get RSS to work in testpmd and load = balancing question >=20 > I'm evaluating DPDK using dpdk-1.5.1r1. I have been playing around = with the test-pmd sample app. I'm having a hard time to get RSS to work. = I have a 2-port 82599 Intel X540-DA2 NIC. I'm running the following = command to start the app. >=20 > sudo ./testpmd -c 0x1f -n 2 -- -i --portmask=3D0x3 --nb-cores=3D4 = --rxq=3D4 > --txq=3D4 >=20 > I have a packet generator that sends udp packets with various src IP. > According testpmd, I'm only receiving packets in port 0's queue 0. = Packets are not going into any other queues. I have attached the output = from testpmd. >=20 >=20 > ------- Forward Stats for RX Port=3D 0/Queue=3D 0 -> TX Port=3D = 1/Queue=3D 0 > ------- > RX-packets: 1000000 TX-packets: 1000000 TX-dropped: > 0 > ---------------------- Forward statistics for port 0 > ---------------------- > RX-packets: 1000000 RX-dropped: 0 RX-total: = 1000000 > TX-packets: 0 TX-dropped: 0 TX-total: 0 >=20 > = --------------------------------------------------------------------------= -- >=20 > ---------------------- Forward statistics for port 1 > ---------------------- > RX-packets: 0 RX-dropped: 0 RX-total: 0 > TX-packets: 1000000 TX-dropped: 0 TX-total: = 1000000 >=20 > = --------------------------------------------------------------------------= -- >=20 > +++++++++++++++ Accumulated forward statistics for all > ports+++++++++++++++ > RX-packets: 1000000 RX-dropped: 0 RX-total: = 1000000 > TX-packets: 1000000 TX-dropped: 0 TX-total: = 1000000 >=20 > = ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++= ++ >=20 > On a separate note, I also find that the CPU utilization using 1 = forwarding core for 2 ports seems to be better (in the aggregate sense) = than using 2 forwarding cores for 2 ports. Running at 10gbps line rate = of pktlen=3D400, with 1 core, the core's utilization is 40%. With 2 = cores, each core's utilization would 30%, giving an aggregate of 60%. >=20 > I have a use case of only doing rxonly packet processing. =46rom my = initial test, it seems that it's more efficient to have a single core = read packets from both ports, and distribute the packet using rte_ring = instead of having each core read from its port. The rte_eth_rx = operations appear to be much CPU intensive than rte_ring_dequeue = operations. >=20 > Thanks in advance. >=20 > Dan >=20