From: "Xing, Beilei" <beilei.xing@intel.com>
To: Take Ceara <dumitru.ceara@gmail.com>
Cc: "Zhang, Helin" <helin.zhang@intel.com>,
"Wu, Jingjing" <jingjing.wu@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [dpdk-users] RSS Hash not working for XL710/X710 NICs for some RX mbuf sizes
Date: Mon, 25 Jul 2016 03:24:50 +0000 [thread overview]
Message-ID: <94479800C636CB44BD422CB454846E013B81E4@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <CAKKV4w9Ttnv2tR51KxVM=g9X_EWhfum3Eo+Rr6TKmFvWkOYp8Q@mail.gmail.com>
Hi,
> -----Original Message-----
> From: Take Ceara [mailto:dumitru.ceara@gmail.com]
> Sent: Friday, July 22, 2016 8:32 PM
> To: Xing, Beilei <beilei.xing@intel.com>
> Cc: Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>;
> dev@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-users] RSS Hash not working for XL710/X710 NICs
> for some RX mbuf sizes
>
> I was using the test-pmd "txonly" implementation which sends fixed UDP
> packets from 192.168.0.1:1024 -> 192.168.0.2:1024.
>
> I changed the test-pmd tx_only code so that it sends traffic with incremental
> destination IP: 192.168.0.1:1024 -> [192.168.0.2,
> 192.168.0.12]:1024
> I also dumped the source and destination IPs in the "rxonly"
> pkt_burst_receive function.
> Then I see that packets are indeed sent to different queues but the
> mbuf->hash.rss value is still 0.
>
> ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i
> --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 1152 --txpkts 1024
> --enable-rx-cksum --rss-udp
>
> [...]
>
> - Receive queue=0xf
> PKT_RX_RSS_HASH
> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 -
> length=1024 - nb_segs=1 - RSS queue=0xa - (outer) L2 type: ETHER -
> (outer) L3 type: IPV4_EXT_UNKNOWN SIP=C0A80001 DIP=C0A80006 - (outer)
> L4 type: UDP - Tunnel type: Unknown - RSS hash=0x0 - Inner L2 type:
> Unknown - RSS queue=0xf - RSS queue=0x7 - (outer) L2 type: ETHER -
> (outer) L3 type: IPV4_EXT_UNKNOWN SIP=C0A80001 DIP=C0A80007 - (outer)
> L4 type: UDP - Tunnel type: Unknown - Inner L2 type: Unknown - Inner
> L3 type: Unknown - Inner L4 type: Unknown
> - Receive queue=0x7
> PKT_RX_RSS_HASH
> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - (outer) L2 type:
> ETHER - (outer) L3 type: IPV4_EXT_UNKNOWN SIP=C0A80001 DIP=C0A80009 -
> type=0x0800 - length=1024 - nb_segs=1 - Inner L3 type: Unknown - Inner
> L4 type: Unknown - RSS hash=0x0 - (outer) L4 type: UDP - Tunnel type:
> Unknown - Inner L2 type: Unknown - Inner L3 type: Unknown - RSS
> queue=0x7 - Inner L4 type: Unknown
>
> [...]
>
> testpmd> stop
> ------- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 1/Queue= 0 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 1 -> TX Port= 1/Queue= 1 -------
> RX-packets: 59 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 2 -> TX Port= 1/Queue= 2 -------
> RX-packets: 48 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 3 -> TX Port= 1/Queue= 3 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 4 -> TX Port= 1/Queue= 4 -------
> RX-packets: 59 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 5 -> TX Port= 1/Queue= 5 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 6 -> TX Port= 1/Queue= 6 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 7 -> TX Port= 1/Queue= 7 -------
> RX-packets: 48 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 8 -> TX Port= 1/Queue= 8 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue= 9 -> TX Port= 1/Queue= 9 -------
> RX-packets: 59 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue=10 -> TX Port= 1/Queue=10 -------
> RX-packets: 48 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue=11 -> TX Port= 1/Queue=11 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue=12 -> TX Port= 1/Queue=12 -------
> RX-packets: 59 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue=13 -> TX Port= 1/Queue=13 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue=14 -> TX Port= 1/Queue=14 -------
> RX-packets: 0 TX-packets: 32 TX-dropped: 0
> ------- Forward Stats for RX Port= 0/Queue=15 -> TX Port= 1/Queue=15 -------
> RX-packets: 48 TX-packets: 32 TX-dropped: 0
> ---------------------- Forward statistics for port 0 ----------------------
> RX-packets: 428 RX-dropped: 84 RX-total: 512
> TX-packets: 0 TX-dropped: 0 TX-total: 0
> ----------------------------------------------------------------------------
>
> ---------------------- Forward statistics for port 1 ----------------------
> RX-packets: 0 RX-dropped: 0 RX-total: 0
> TX-packets: 512 TX-dropped: 0 TX-total: 512
> ----------------------------------------------------------------------------
>
> +++++++++++++++ Accumulated forward statistics for all
> ports+++++++++++++++
> RX-packets: 428 RX-dropped: 84 RX-total: 512
> TX-packets: 512 TX-dropped: 0 TX-total: 512
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> +++++++++++++++
>
> I checked all the RSS hash values for all the 10 different incoming streams and
> they're all 0. Also, the fact that the traffic is actually distributed seems to suggest
> that RSS itself is working but the mbuf hash field is (I guess) either not written or
> corrupted.
>
I tried to reproduce the problem with the same steps you used on 16.04 and 16.07, but I really didn't replicate it.
I think you can try follow ways:
1. apply the patch I told you last time and check if the problem still exists.
2. update the codebase and check if the problem still exists.
3. disable vector when you run testpmd, and check if the problem still exists.
> >
> >>
> >> If I use a different mbuf-size, for example 2048, everything is fine:
> >>
> >> ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i
> >> --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 2048 --txpkts 1024
> >> -- enable-rx-cksum --rss-udp [...]
> >> testpmd> set verbose 1
> >> Change verbose level from 0 to 1
> >> testpmd> set fwd rxonly
> >> Set rxonly packet forwarding mode
> >> testpmd> start tx_first
> >> rxonly packet forwarding - CRC stripping disabled - packets/burst=32
> >> nb forwarding cores=16 - nb forwarding ports=2
> >> RX queues=16 - RX desc=128 - RX free threshold=32
> >> RX threshold registers: pthresh=8 hthresh=8 wthresh=0
> >> TX queues=16 - TX desc=512 - TX free threshold=32
> >> TX threshold registers: pthresh=32 hthresh=0 wthresh=0
> >> TX RS bit threshold=32 - TXQ flags=0xf01 port 0/queue 1: received
> >> 32 packets
> >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 -
> >> length=1024 - nb_segs=1 - RSS hash=0x5263c3f2 - RSS queue=0x1 -
> >> (outer) L2 type: ETHER - (outer) L3 type: IPV4_EXT_UNKNOWN - (outer)
> >> L4 type: UDP - Tunnel type: Unknown - Inner L2 type: Unknown - Inner
> >> L3 type: Unknown - Inner L4 type: Unknown
> >> - Receive queue=0x1
> >> PKT_RX_RSS_HASH
> >>
> >
> > Did you send the same packet as before to port 0?
> >
> >> Another weird thing I noticed is when I run test-pmd without
> >> --enable-rx- cksum (which is the default mode) then the RSS flag doesn get
> set at all.
> >> Instead the PKT_RX_FDIR flag gets set. This happens even though
> >> fdir_mode is set to RTE_FDIR_MODE_NONE in the device
> >> configuration:
> >>
> >> ./testpmd -c ffff1 -n 4 -w 0000:01:00.3 -w 0000:81:00.3 -- -i
> >> --coremask=0xffff0 --rxq=16 --txq=16 --mbuf-size 1152 --txpkts 1024
> >> --rss- udp [...]
> >> testpmd> set verbose 1
> >> Change verbose level from 0 to 1
> >> testpmd> set fwd rxonly
> >> Set rxonly packet forwarding mode
> >> testpmd> start tx_first
> >> rxonly packet forwarding - CRC stripping disabled - packets/burst=32
> >> nb forwarding cores=16 - nb forwarding ports=2
> >> RX queues=16 - RX desc=128 - RX free threshold=32
> >> RX threshold registers: pthresh=8 hthresh=8 wthresh=0
> >> TX queues=16 - TX desc=512 - TX free threshold=32
> >> TX threshold registers: pthresh=32 hthresh=0 wthresh=0
> >> TX RS bit threshold=32 - TXQ flags=0xf01 port 0/queue 1: received
> >> 16 packets
> >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 -
> >> length=1024 - nb_segs=2 - FDIR matched hash=0xc3f2 ID=0x5263 Unknown
> >> packet type
> >> - Receive queue=0x1
> >> PKT_RX_FDIR
> >> src=68:05:CA:38:6D:63 - dst=02:00:00:00:00:01 - type=0x0800 -
> >> length=1024 - nb_segs=2 - FDIR matched hash=0xc3f2 ID=0x5263 Unknown
> >> packet type
> >> - Receive queue=0x1
> >> PKT_RX_FDIR
> >>
> >
> > For this issue, I think following patch can solve your problem, please apply this
> patch.
> > http://dpdk.org/dev/patchwork/patch/13593/
> >
>
> I tried to apply it directly on 16.04 but it can't be applied. I see it's been applied to
> dpdk-next-net/rel_16_07. Do you happen to have one that would work on the
> latest stable 16.04 release?
Sorry, I haven't. If there's conflict with R16.04, I think you can resolve it by going through the patch.
Beilei
>
> Thanks,
> Dumitru
>
> > Beilei
> >
> >> Please let me know if there's more debug info that might be of
> >> interest in troubleshooting the RSS=0 issue.
> >>
> >> Thanks,
> >> Dumitru
> >>
> >> > /Beilei
> >> >
> >> >> Thanks,
> >> >> Dumitru
> >> >>
next prev parent reply other threads:[~2016-07-25 3:24 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAKKV4w9uoN_X=0DKJHgcAHT7VCmeBHP=WrHfi+12o3ogA6htSQ@mail.gmail.com>
2016-07-18 15:15 ` Zhang, Helin
2016-07-18 16:14 ` Take Ceara
2016-07-19 9:31 ` Xing, Beilei
2016-07-19 14:58 ` Take Ceara
2016-07-20 1:59 ` Xing, Beilei
2016-07-21 10:58 ` Take Ceara
2016-07-22 9:04 ` Xing, Beilei
2016-07-22 12:31 ` Take Ceara
2016-07-22 12:35 ` Take Ceara
2016-07-25 3:24 ` Xing, Beilei [this message]
2016-07-25 10:04 ` Take Ceara
2016-07-26 8:38 ` Take Ceara
2016-07-26 8:47 ` Zhang, Helin
2016-07-26 8:57 ` Take Ceara
2016-07-26 9:23 Ananyev, Konstantin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=94479800C636CB44BD422CB454846E013B81E4@SHSMSX101.ccr.corp.intel.com \
--to=beilei.xing@intel.com \
--cc=dev@dpdk.org \
--cc=dumitru.ceara@gmail.com \
--cc=helin.zhang@intel.com \
--cc=jingjing.wu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).