From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f173.google.com (mail-io0-f173.google.com [209.85.223.173]) by dpdk.org (Postfix) with ESMTP id 54FCB256 for ; Mon, 25 Jul 2016 12:04:40 +0200 (CEST) Received: by mail-io0-f173.google.com with SMTP id 38so157036595iol.0 for ; Mon, 25 Jul 2016 03:04:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=oOAzZi0oeAx9nYR/B9av4VRWzXGELp/UK3/uA2tS0Bo=; b=rXUoIdkv6evkr0ds7ff+2G8pR4aji9vMX1R4ne1LPsdj6enNvPBF4192pmTAIWj5/C U/umTCsnO66X3imR7POV8pY0J80QhL6dYT6OpR+ubRMCgHLAo0CJ4zYiB/q7BDkBJ1eK jVUxkdqe27QoDf4JUH6uLcwW49Y8ilzO848hU7tyGrBIfWa54au4nuJJUlyoEA2QRXDh vkRd4EeI+egSnhDCdsJDlwhcbcoswkY7OtPV8Hp+N3G8i2nRRBjz5gkaoGU+5XMzjtK0 bVwWrnJZjvWFSOJB0zdsaWejcioS1YZEdEKV1WXbpv/uUXI2jBQDUYORGI4fs7PDiwVU UH7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=oOAzZi0oeAx9nYR/B9av4VRWzXGELp/UK3/uA2tS0Bo=; b=UQY93iv6CvDDJZfenZW8hhpBw3ULzppH2lmk/H0eNOA9NLB+m7siwzo9EQ266yrfIL CjCQssM9o4eLytn10dH+ApvlqBVmc8sIXLOstuJKR3itPobJZ7clrNd40DnsS0Id8+Tn 1lX/Jsq5m1UBb1k0htTs8ijH4MNJ92j4SMLB/e2op4mFJlEdew94dSLqYvp5oREBGwL2 8UQFFlS98pAy1aqoznMuz/E4yIViTdfzbJABbhtEVi/6rTJ7xvgapIxv2qYGZ6MahlH1 4jhlmLyGqheH9H1A9JguqOu2bOEprJItMjTy2N+Qo7sruvnqtFEtYgsBxtWo6DeBz/b0 VyTQ== X-Gm-Message-State: AEkoouvO+HDBukcbe++XvTU1Mw+q9O2ZS3ycpsY+CZaER+rcUIbfOYA2VpUCnzfxRugWDtCWWTBrn0TmeGlAXw== X-Received: by 10.107.10.170 with SMTP id 42mr20746089iok.131.1469441079705; Mon, 25 Jul 2016 03:04:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.7.82 with HTTP; Mon, 25 Jul 2016 03:04:20 -0700 (PDT) In-Reply-To: <94479800C636CB44BD422CB454846E013B81E4@SHSMSX101.ccr.corp.intel.com> References: <94479800C636CB44BD422CB454846E013B5A15@SHSMSX101.ccr.corp.intel.com> <94479800C636CB44BD422CB454846E013B5ED3@SHSMSX101.ccr.corp.intel.com> <94479800C636CB44BD422CB454846E013B7479@SHSMSX101.ccr.corp.intel.com> <94479800C636CB44BD422CB454846E013B81E4@SHSMSX101.ccr.corp.intel.com> From: Take Ceara Date: Mon, 25 Jul 2016 12:04:20 +0200 Message-ID: To: "Xing, Beilei" Cc: "Zhang, Helin" , "Wu, Jingjing" , "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 Subject: Re: [dpdk-dev] [dpdk-users] RSS Hash not working for XL710/X710 NICs for some RX mbuf sizes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jul 2016 10:04:40 -0000 Hi Beilei, On Mon, Jul 25, 2016 at 5:24 AM, Xing, Beilei wrote: > Hi, > >> -----Original Message----- >> From: Take Ceara [mailto:dumitru.ceara@gmail.com] >> Sent: Friday, July 22, 2016 8:32 PM >> To: Xing, Beilei >> Cc: Zhang, Helin ; Wu, Jingjing ; >> dev@dpdk.org >> Subject: Re: [dpdk-dev] [dpdk-users] RSS Hash not working for XL710/X710 NICs >> for some RX mbuf sizes >> >> I was using the test-pmd "txonly" implementation which sends fixed UDP >> packets from 192.168.0.1:1024 -> 192.168.0.2:1024. >> >> I changed the test-pmd tx_only code so that it sends traffic with incremental >> destination IP: 192.168.0.1:1024 -> [192.168.0.2, >> 192.168.0.12]:1024 >> I also dumped the source and destination IPs in the "rxonly" >> pkt_burst_receive function. >> Then I see that packets are indeed sent to different queues but the >> mbuf->hash.rss value is still 0. >> >> ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i >> --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 1152 --txpkts 1024 >> --enable-rx-cksum --rss-udp >> >> [...] >> >> - Receive queue=0xf >> PKT_RX_RSS_HASH >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 - >> length=1024 - nb_segs=1 - RSS queue=0xa - (outer) L2 type: ETHER - >> (outer) L3 type: IPV4_EXT_UNKNOWN SIP=C0A80001 DIP=C0A80006 - (outer) >> L4 type: UDP - Tunnel type: Unknown - RSS hash=0x0 - Inner L2 type: >> Unknown - RSS queue=0xf - RSS queue=0x7 - (outer) L2 type: ETHER - >> (outer) L3 type: IPV4_EXT_UNKNOWN SIP=C0A80001 DIP=C0A80007 - (outer) >> L4 type: UDP - Tunnel type: Unknown - Inner L2 type: Unknown - Inner >> L3 type: Unknown - Inner L4 type: Unknown >> - Receive queue=0x7 >> PKT_RX_RSS_HASH >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - (outer) L2 type: >> ETHER - (outer) L3 type: IPV4_EXT_UNKNOWN SIP=C0A80001 DIP=C0A80009 - >> type=0x0800 - length=1024 - nb_segs=1 - Inner L3 type: Unknown - Inner >> L4 type: Unknown - RSS hash=0x0 - (outer) L4 type: UDP - Tunnel type: >> Unknown - Inner L2 type: Unknown - Inner L3 type: Unknown - RSS >> queue=0x7 - Inner L4 type: Unknown >> >> [...] >> >> testpmd> stop >> ------- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 1/Queue= 0 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 1 -> TX Port= 1/Queue= 1 ------- >> RX-packets: 59 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 2 -> TX Port= 1/Queue= 2 ------- >> RX-packets: 48 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 3 -> TX Port= 1/Queue= 3 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 4 -> TX Port= 1/Queue= 4 ------- >> RX-packets: 59 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 5 -> TX Port= 1/Queue= 5 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 6 -> TX Port= 1/Queue= 6 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 7 -> TX Port= 1/Queue= 7 ------- >> RX-packets: 48 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 8 -> TX Port= 1/Queue= 8 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue= 9 -> TX Port= 1/Queue= 9 ------- >> RX-packets: 59 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue=10 -> TX Port= 1/Queue=10 ------- >> RX-packets: 48 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue=11 -> TX Port= 1/Queue=11 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue=12 -> TX Port= 1/Queue=12 ------- >> RX-packets: 59 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue=13 -> TX Port= 1/Queue=13 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue=14 -> TX Port= 1/Queue=14 ------- >> RX-packets: 0 TX-packets: 32 TX-dropped: 0 >> ------- Forward Stats for RX Port= 0/Queue=15 -> TX Port= 1/Queue=15 ------- >> RX-packets: 48 TX-packets: 32 TX-dropped: 0 >> ---------------------- Forward statistics for port 0 ---------------------- >> RX-packets: 428 RX-dropped: 84 RX-total: 512 >> TX-packets: 0 TX-dropped: 0 TX-total: 0 >> ---------------------------------------------------------------------------- >> >> ---------------------- Forward statistics for port 1 ---------------------- >> RX-packets: 0 RX-dropped: 0 RX-total: 0 >> TX-packets: 512 TX-dropped: 0 TX-total: 512 >> ---------------------------------------------------------------------------- >> >> +++++++++++++++ Accumulated forward statistics for all >> ports+++++++++++++++ >> RX-packets: 428 RX-dropped: 84 RX-total: 512 >> TX-packets: 512 TX-dropped: 0 TX-total: 512 >> >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> +++++++++++++++ >> >> I checked all the RSS hash values for all the 10 different incoming streams and >> they're all 0. Also, the fact that the traffic is actually distributed seems to suggest >> that RSS itself is working but the mbuf hash field is (I guess) either not written or >> corrupted. >> > > I tried to reproduce the problem with the same steps you used on 16.04 and 16.07, but I really didn't replicate it. > I think you can try follow ways: > 1. apply the patch I told you last time and check if the problem still exists. I applied the changes in the patch manually to 16.04. The RSS=0 problem still exists while the FDIR issue is fixed. > 2. update the codebase and check if the problem still exists. I updated the codebase to the latest version on http://dpdk.org/git/dpdk. I still see the RSS=0 issue. > 3. disable vector when you run testpmd, and check if the problem still exists. I recompiled the latest dpdk code with CONFIG_RTE_LIBRTE_I40E_INC_VECTOR=n and the RSS=0 issue is still there. My current command line is: ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 1152 --txpkts 1024 --rss-udp Not sure if relevant but I'm running kernel 4.2.0-27: $ uname -a Linux jspg2 4.2.0-27-generic #32~14.04.1-Ubuntu SMP Fri Jan 22 15:32:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Is there anything else that might help you identify the cause of the problem? Thanks, Dumitru > >> > >> >> >> >> If I use a different mbuf-size, for example 2048, everything is fine: >> >> >> >> ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i >> >> --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 2048 --txpkts 1024 >> >> -- enable-rx-cksum --rss-udp [...] >> >> testpmd> set verbose 1 >> >> Change verbose level from 0 to 1 >> >> testpmd> set fwd rxonly >> >> Set rxonly packet forwarding mode >> >> testpmd> start tx_first >> >> rxonly packet forwarding - CRC stripping disabled - packets/burst=32 >> >> nb forwarding cores=16 - nb forwarding ports=2 >> >> RX queues=16 - RX desc=128 - RX free threshold=32 >> >> RX threshold registers: pthresh=8 hthresh=8 wthresh=0 >> >> TX queues=16 - TX desc=512 - TX free threshold=32 >> >> TX threshold registers: pthresh=32 hthresh=0 wthresh=0 >> >> TX RS bit threshold=32 - TXQ flags=0xf01 port 0/queue 1: received >> >> 32 packets >> >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 - >> >> length=1024 - nb_segs=1 - RSS hash=0x5263c3f2 - RSS queue=0x1 - >> >> (outer) L2 type: ETHER - (outer) L3 type: IPV4_EXT_UNKNOWN - (outer) >> >> L4 type: UDP - Tunnel type: Unknown - Inner L2 type: Unknown - Inner >> >> L3 type: Unknown - Inner L4 type: Unknown >> >> - Receive queue=0x1 >> >> PKT_RX_RSS_HASH >> >> >> > >> > Did you send the same packet as before to port 0? >> > >> >> Another weird thing I noticed is when I run test-pmd without >> >> --enable-rx- cksum (which is the default mode) then the RSS flag doesn get >> set at all. >> >> Instead the PKT_RX_FDIR flag gets set. This happens even though >> >> fdir_mode is set to RTE_FDIR_MODE_NONE in the device >> >> configuration: >> >> >> >> ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i >> >> --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 1152 --txpkts 1024 >> >> --rss- udp [...] >> >> testpmd> set verbose 1 >> >> Change verbose level from 0 to 1 >> >> testpmd> set fwd rxonly >> >> Set rxonly packet forwarding mode >> >> testpmd> start tx_first >> >> rxonly packet forwarding - CRC stripping disabled - packets/burst=32 >> >> nb forwarding cores=16 - nb forwarding ports=2 >> >> RX queues=16 - RX desc=128 - RX free threshold=32 >> >> RX threshold registers: pthresh=8 hthresh=8 wthresh=0 >> >> TX queues=16 - TX desc=512 - TX free threshold=32 >> >> TX threshold registers: pthresh=32 hthresh=0 wthresh=0 >> >> TX RS bit threshold=32 - TXQ flags=0xf01 port 0/queue 1: received >> >> 16 packets >> >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 - >> >> length=1024 - nb_segs=2 - FDIR matched hash=0xc3f2 ID=0x5263 Unknown >> >> packet type >> >> - Receive queue=0x1 >> >> PKT_RX_FDIR >> >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 - >> >> length=1024 - nb_segs=2 - FDIR matched hash=0xc3f2 ID=0x5263 Unknown >> >> packet type >> >> - Receive queue=0x1 >> >> PKT_RX_FDIR >> >> >> > >> > For this issue, I think following patch can solve your problem, please apply this >> patch. >> > http://dpdk.org/dev/patchwork/patch/13593/ >> > >> >> I tried to apply it directly on 16.04 but it can't be applied. I see it's been applied to >> dpdk-next-net/rel_16_07. Do you happen to have one that would work on the >> latest stable 16.04 release? > > Sorry, I haven't. If there's conflict with R16.04, I think you can resolve it by going through the patch. > > Beilei > >> >> Thanks, >> Dumitru >> >> > Beilei >> > >> >> Please let me know if there's more debug info that might be of >> >> interest in troubleshooting the RSS=0 issue. >> >> >> >> Thanks, >> >> Dumitru >> >> >> >> > /Beilei >> >> > >> >> >> Thanks, >> >> >> Dumitru >> >> >> -- Dumitru Ceara