From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5A4F1A00C4 for ; Mon, 14 Nov 2022 23:14:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D56741148; Mon, 14 Nov 2022 23:14:12 +0100 (CET) Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by mails.dpdk.org (Postfix) with ESMTP id DD310400D7 for ; Mon, 7 Nov 2022 21:03:58 +0100 (CET) Received: by mail-ed1-f52.google.com with SMTP id u24so19293964edd.13 for ; Mon, 07 Nov 2022 12:03:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ooBSaRrsNvOJ6+DOdXdJbW91Tv7vxbBISrdU+Tde+LE=; b=OG91hlU6iTioarNabkrC3pDE0q7GDR/Xd/RLtUNbMOXpaURZZaswbGCifhY6lEdj30 EVcg1Bp3svRunbykdjfvRF97/B7Jbcu7tNoVO7MAGtSFcl88SZM1G8HTUm4yJLgwvYzR uua1AHwJi/CygxA0Q8XhTdo9tcFlJOTGTXW7eZGx/tkesqN60+4q1Wa8xUDo5t1PpgMD lfm2rybF2loBGTBet2n6/YEJwYr0XPqizG9GwFzQxlNEGMJSGAjnhJD1a9Vh0J2JJx0H su1F7XL4Hrw8m6POEQW9+CmCQujoL9fJcoZm+SWKnBLyS0fYUrkkfOSxBsAMnBRFsjxG tQNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ooBSaRrsNvOJ6+DOdXdJbW91Tv7vxbBISrdU+Tde+LE=; b=0i5TvT/zk8i+wnf8bP2boIo9mITpr0/UsjBcfkhwepIEEJPxguHN0u+ryRvHFEIfQG ueTflMVZ2/aQMKp0c1qGtDHwsE7fniqxVhKAlZkViSg4379sl9/KN/TvXEtmyK2lb3wf NsRY5jS8xqA53AczEFqb+2yv6xNyYv6+J9HFsgyxgDhfv/9pYNmkMVfw3eEAU0g0ICit FefuJipXgMHzqqsYcy0ys9sYhQELcyu66g9mtsexMzjSPQYVAjjkEdh/JNrH9JNsIvrQ dOMAl5X6YBCsTaMkJvDO8ZPoLOSEoD20quaC/Yc6mm6WTDPlME+2leWJzaKxz4NiNy+b Zhfg== X-Gm-Message-State: ACrzQf1WCyMyIzG2n6Y4khEkqxIZEZsClDePTjZkLqe4L4L+BHj7kAF3 2GNU3A4BQtaVZAJaf1Tl9HRFr21VRWCS1zCyTTOZ048z+cQ= X-Google-Smtp-Source: AMsMyM5wVlXon/QyoJV9V13GG6TdN8VPmHPRjsh5KSQ4GCplZ+eZ3RgkCGYZo3xVhnUPcoXfgbnE3UVurHV3STV4WcU= X-Received: by 2002:aa7:c981:0:b0:461:522c:ce0d with SMTP id c1-20020aa7c981000000b00461522cce0dmr50588601edt.169.1667851438372; Mon, 07 Nov 2022 12:03:58 -0800 (PST) MIME-Version: 1.0 References: <20221104095202.32c55020@hermes.local> <20221104151803.5eb40caa@hermes.local> In-Reply-To: <20221104151803.5eb40caa@hermes.local> From: Yang Luan Date: Mon, 7 Nov 2022 12:03:47 -0800 Message-ID: Subject: Re: Wrong rx queue assignment running DPDK on Azure To: Stephen Hemminger Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="00000000000085ba2d05ece6ec52" X-Mailman-Approved-At: Mon, 14 Nov 2022 23:14:11 +0100 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000085ba2d05ece6ec52 Content-Type: text/plain; charset="UTF-8" Thanks Stephen. We reconfigure the RSS key via rte_eth_dev_rss_hash_update(). We just set it to the same default key value which is below just in case it changes in the future. 0x2c, 0xc6, 0x81, 0xd1, 0x5b, 0xdb, 0xf4, 0xf7, 0xfc, 0xa2, 0x83, 0x19, 0xdb, 0x1a, 0x3e, 0x94, 0x6b, 0x9e, 0x38, 0xd9, 0x2c, 0x9c, 0x03, 0xd1, 0xad, 0x99, 0x44, 0xa7, 0xd9, 0x56, 0x3d, 0x59, 0x06, 0x3c, 0x25, 0xf3, 0xfc, 0x1f, 0xdc, 0x2a, Yes, we use accelerated network and our DPDK build includes mlx. It happens on the first packet. However we don't know if subsequent packets will work as our application stops if the first is delivered incorrectly. We don't use layered encapsulation. Our packets are standard UDP packets. No, we use the standard ETH_RSS_NONFRAG_IPV4_UDP. Yes, we print the rss field in rte_mbuf. It's not byte swapped from what I can tell. 21.08 has a feature we wanted at the time. We haven't had time to switch to a LTS version. On Fri, Nov 4, 2022 at 3:18 PM Stephen Hemminger wrote: > On Fri, 4 Nov 2022 10:15:36 -0700 > Yang Luan wrote: > > > We use netvsc PMD (drivers/net/netvsc/). > > We don't explicitly configure the RETA table. We configure the device > with > > 40 rx queues (rte_eth_dev_configure) and use rte_eth_dev_rss_reta_query() > > to query the RETA table (result posted earlier). > > More questions to help someone (not me I don't have free Azure access now); > get a reproduction. > > Did you reconfigure the RSS key? > The default for the netvsc PMD should be the same default as Mellanox. > > > Have you enabled accelerated networking (ie VF). If so then does your DPDK > build support Mellanox. Probably yes to both. > > Is this the first packet, or later packets in the flow? > Are you using any layered encapsulation (like GRE or VXLAN), and/or IP > options. > > Are you changing RSS options so that is different than default L3/L4? > > Are you printing the RSS key in the mbuf? It might be byte swapped. > > Why 21.08? it is not a long term supported version, and therefore does not > receive bugfixes like: > > $ git log --oneline v21.08..v20.11.6 -- drivers/net/netvsc/ > 9d474a9565a5 net/netvsc: fix vmbus device reference in multi-process > a61bd9df25dc net/netvsc: fix calculation of checksums based on mbuf flag > 0b5a6c7b32c8 fix spelling in comments and strings > e97bb2a91151 net/netvsc: ignore unsupported packet on sync command > --00000000000085ba2d05ece6ec52 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks Stephen.

We reconfigure the RSS = key via=C2=A0rte_eth_dev_rss_hash_update(). We just set it to the same defa= ult key value which is below just in case it changes in the=C2=A0future.

=C2=A0 =C2=A0 0x2c, 0xc6, 0x81, 0xd1, 0x5b, 0xdb, 0x= f4, 0xf7,
=C2=A0 =C2=A0 0xfc, 0xa2, 0x83, 0x19, 0xdb, 0x1a, 0x3e, 0x94,<= br>=C2=A0 =C2=A0 0x6b, 0x9e, 0x38, 0xd9, 0x2c, 0x9c, 0x03, 0xd1,
=C2=A0 = =C2=A0 0xad, 0x99, 0x44, 0xa7, 0xd9, 0x56, 0x3d, 0x59,
=C2=A0 =C2=A0 0x0= 6, 0x3c, 0x25, 0xf3, 0xfc, 0x1f, 0xdc, 0x2a,

Y= es, we use accelerated=C2=A0network and our DPDK build includes mlx.
<= div>
It happens on the first packet. However we don't kno= w if subsequent packets will work as our application stops if the first is = delivered incorrectly.

We don't use layered en= capsulation. Our packets are standard UDP packets.

No, we use the standard=C2=A0ETH_RSS_NONFRAG_IPV4_UDP.

Yes, w= e print the rss field in rte_mbuf. It's not byte swapped from what I ca= n tell.

21.08 has a feature we wanted at the time.= We haven't had time to switch=C2=A0to a LTS version.

On Fri, Nov = 4, 2022 at 3:18 PM Stephen Hemminger <stephen@networkplumber.org> wrote:
=
On Fri, 4 Nov 2022 = 10:15:36 -0700
Yang Luan <lua= n.penny@gmail.com> wrote:

> We use netvsc PMD (drivers/net/netvsc/).
> We don't explicitly configure the RETA table. We configure the dev= ice with
> 40 rx queues (rte_eth_dev_configure) and use rte_eth_dev_rss_reta_quer= y()
> to query the RETA table (result posted earlier).

More questions to help someone (not me I don't have free Azure access n= ow);
get a reproduction.

Did you reconfigure the RSS key?
The default for the netvsc PMD should be the same default as Mellanox.


Have you enabled accelerated networking (ie VF). If so then does your DPDK<= br> build support Mellanox.=C2=A0 Probably yes to both.

Is this the first packet, or later packets in the flow?
Are you using any layered encapsulation (like GRE or VXLAN), and/or IP opti= ons.

Are you changing RSS options so that is different than default L3/L4?

Are you printing the RSS key in the mbuf?=C2=A0 It might be byte swapped.
Why 21.08? it is not a long term supported version, and therefore does not<= br> receive bugfixes like:

$ git log --oneline v21.08..v20.11.6 -- drivers/net/netvsc/
9d474a9565a5 net/netvsc: fix vmbus device reference in multi-process
a61bd9df25dc net/netvsc: fix calculation of checksums based on mbuf flag 0b5a6c7b32c8 fix spelling in comments and strings
e97bb2a91151 net/netvsc: ignore unsupported packet on sync command
--00000000000085ba2d05ece6ec52--