From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 16860465FC for ; Tue, 22 Apr 2025 14:50:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9F837402E7; Tue, 22 Apr 2025 14:50:00 +0200 (CEST) Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by mails.dpdk.org (Postfix) with ESMTP id 9EC34402E6 for ; Tue, 22 Apr 2025 14:49:58 +0200 (CEST) Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2240b4de12bso73916685ad.2 for ; Tue, 22 Apr 2025 05:49:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745326198; x=1745930998; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=BVjBwWb/G7/K1U99HTrWhH1/Lln/wdUDPBp6CsIoYgE=; b=GaUINU9xRNRZQjt4gTtSOtKiRE5Se0JNQkdYpAjGwsEsHEp9TNZyzE3C2yH4s1B/kr tWyH+UdV8QOr42GUSQg1Rts3oWUotGW+qf2K4fj1A9fUo0OsMGihaUz3Uo5rZ/y/Sd7f wW1zDEfbvIhZfm5Yy4ztM1TZRQwkT6xqPuVCQcw31aG/S6kgfQC6ZW7lG0IhYGP4ERu3 puSvyZRArx7xXX+wHWR4VLSW2EKoqutNR7GEWg9RTfHnD4X9uyds+U+wfzXkRnfkTQna JUu2oXoMcHQIkH7fFzs8bW+1igyEfHK7ZH5TlG/5yqDY6cyIXBJHj6t/Fjv2Y0SCY84A YomA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745326198; x=1745930998; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BVjBwWb/G7/K1U99HTrWhH1/Lln/wdUDPBp6CsIoYgE=; b=PAtWF9Sd02Htn1gT2g2EMqvoWMK451/Fnvg/EIVPmrq1vesXojY/WN1m61WmdOdgdd vLMaHzCwJpOow3MM9GzLyAQmiI80Ocr8NNaWQGICNe8oqKmwMQLMm1CHoHmp4c81+upu y2SjXF8ZZSmoJOwfjtUP2jUSgIh2xTz9yOAxDQ8v1JVywB7rK1FZrHtHxi4dTEsiJGPT 4BLFktg8PBwA3aOWDvEqdG2DiQEvxskA43d8uSRlrkRWIibuH2OsLS217QyGrztVM8SK UB+w1tNUJYlR44+Oii5CIxuPtgLur5T3pH+cVKB9s3R682BxgAgJ+cdOsspaeIHc+Y74 f13w== X-Forwarded-Encrypted: i=1; AJvYcCX+bNLXTmo+GIVx5rE6eAvO0MRWiAAEnCA8S9hyKxzRR6863kwBDo2OBD050OOAgjA2/7qTMQ==@dpdk.org X-Gm-Message-State: AOJu0YzA27rlG7BE4AIfQPCQwClMh20oljgEpFrIjdZ9skiKo0wsjc4t E2Gh5ktgmUGXsNoRGVY506r0tvNSaWjCRar/UJb8Fmcvty113CH9gMTwZ9HkpmwbmQHHcyUA6DW hjnd+t58LL7ktPqM6oUy6KKiUG6A= X-Gm-Gg: ASbGncvuiyq2TzF18K/WyTDx92yxX496xy4Gn1KrZhRRp9h1AoJt5NzelcAadyKjUDw YUvOcBBrmX2Q6mtSCzDWOhCc5p7tYoh6XI+7VfpwWFalngWKoxdMN0sLdiIY0qtnR7Ak7OT6VJa nousOFiloEyXjr/QHS7iy9MzY= X-Google-Smtp-Source: AGHT+IGoPyAigOJXLFfnwoMvwBZDd0c7/2L8zDpcnfsjhOaKziq7uWEE/xJ+3ANJr36erha/Ns302r77ka1nc0A9OHQ= X-Received: by 2002:a17:902:d50e:b0:224:1ce1:a3f4 with SMTP id d9443c01a7336-22c532858cdmr229323725ad.1.1745326197191; Tue, 22 Apr 2025 05:49:57 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: spyroot Date: Tue, 22 Apr 2025 16:49:45 +0400 X-Gm-Features: ATxdqUFbOj44IdmLOfS-PeFZeSGXAFx6V91P0GTlbTKpSfsgfTGwK1BJsEcoLQU Message-ID: Subject: Re: VMware run the DPDK with vmxnet3 NIC failed with the RSS config To: Fangyin Hu Cc: "Knight, Joshua" , "users@dpdk.org" Content-Type: multipart/alternative; boundary="000000000000003cc006335d6c39" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000003cc006335d6c39 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I hit the same issue, and it was brute force. The first version of VMXNET3 is something that has never been exposed directly. (It is implicit and based on the VM Hardware version.A) You can adjust it in the VM advanced settings. It makes sense to indicate exactly which flag must be in the VM. (Why? Sometimes, downgrading the VM Hardware version is not an option. ) While downgrading, the VMXNET3 version will run in compatibility mode. (I.e., DPDK version old, nic new, RSS must X, etc, etc) There are a few combinations where You can not assume a single queue, or take a four-queue, or assume a 32-queue. (This combination depends on Combination NIC - VM - ESXi - DPDK version So, asking a client to do four variations and find a combination is not a good option. (Note in kernel driver it's the same, you can use 32, but you have no idea how to enable the VMXNET3 version that will allow you to use 32 queues) Example. You have 810 - VMXNET 3 poll mode and enable 16 queues. What do you do? You have 810 - VMXNET 3 (the default queue in RSS 4). What do you do? --- This will fail on 23/24/25. You have 801 - VMXNET 3 - ESXi 8.0.2 and DPDK 22.xx app ( example people use trex it build 22 or 23) I just gave a few examples. Kind Regards, Mus On Sat, Apr 19, 2025 at 11:15=E2=80=AFAM Fangyin Hu wro= te: > Hi, Knight > > Thank you very much for your reply! > > I check our codes, my application config the 1 RX queue for vmxnet3 > device, so the call vmxnet3_v4_rss_configure() will failed due to the > single RX queue. > > I adjusted our codes and use 2 RX queue for vmxnet3 device, then retest m= y > application aga in. Now my application works fine and here are som= e > debug logs. > > > > [2025-04-18 07:37:02]:: Try to configure port 0, *RX Queue Number: 2*, TX > Queue Number: 1 Dev 0^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:310:init_options():DPDK interface > (net_vmxnet3): 0^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:312:init_options(): multicast_en: > 1^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:313:init_options(): num_rx_desc: > 4096^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:314:init_options(): num_tx_desc: > 512^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:315:init_options(): rx_drop_en: 0^M= ^M > > [2025-04-18 07:37:02]:: odp_packet_io.c:439:odp_pktio_open():interface: 0= , > driver: dpdk^M^M > > [2025-04-18 07:37:02]:: created pktio 1 idx( 0) (0)^M^M > > > > [2025-04-18 07:37:02]:: ### starting data port pktio's...^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:2277:dpdk_setup_eth_tx():Mapping per > TX queue statistics not supported: -95^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:2281:dpdk_setup_eth_tx():Mapped 0/16 > TX counters^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:2315:dpdk_setup_eth_rx():Mapping per > RX queue statistics not supported: -95^M^M > > [2025-04-18 07:37:02]:: pktio/dpdk.c:2319:dpdk_setup_eth_rx():Mapped 0/16 > RX counters^M^M > > [2025-04-18 07:37:02]:: --[vmxnet3_dev_start:963] hw->version: 6, rss_hf: > 3380 --^M^M > > [2025-04-18 07:37:02]:: --[vmxnet3_rss_configure:1430] rss_hf: 3380 --^M^= M > > [2025-04-18 07:37:02]:: --[vmxnet3_v4_rss_configure:1381] rss_hf: 3380 > --^M^M > > [2025-04-18 07:37:02]:: odp_packet_io.c:723:odp_pktio_start():interface: > 0, input queues: 2, output queues: 1^M^M > > [2025-04-18 07:37:03]:: pktio port 0 dpdk port 0 started!^M^M > > > > So the next step we need to modify our codes, it will not config the RSS > when the device RX queue number is 1 > > > > Thank you for your advise again and best wish to you! > > > > *From:* Knight, Joshua > *Sent:* Wednesday, April 16, 2025 9:59 PM > *To:* Fangyin Hu ; users@dpdk.org > *Subject:* Re: VMware run the DPDK with vmxnet3 NIC failed with the RSS > config > > > > *CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe.* > > > > We=E2=80=99ve hit the same problem setting up RSS on vmxnet3 on newer esx= i8 > releases. In our case we would see the failure if we only had a single r= x > queue. > > Snippet from my discussion with their support team: > > So to summarize, with VM Hardware version 14 in esxi 7 and 6.7, setting > the RSS hash function did not generate an error in the driver. With VM > Hardware version 14 in esxi 8U2, it generates an error in the driver. > > > > I had a ticket opened with Broadcom and their response was they made a > change in behavior where instead of silently rejecting the RSS > configuration (since, really you don=E2=80=99t need RSS with a single que= ue > although we still wanted it enabled) the nic now returns an error. The > biggest problem I had with the change is they applied it to **earlier** > vm hw versions, not just the latest/newest version. > > > > Broadcom=E2=80=99s response to my ticket was: > > >Regarding the difference in behavior you have observed with ESXi > versions, there were some enhancements introduced in vSphere version 7.0U= 1 > used along with VM hardware version >=3D14, it identifies the RSS > misconfiguration with Rx Queue and throws an error message similar to wha= t > has been observed. In older versions, it was not erroring out so you will > not observe any failures in the applications. > > > > > > > > *From: *Fangyin Hu > *Date: *Wednesday, April 16, 2025 at 4:23=E2=80=AFAM > *To: *users@dpdk.org > *Subject: *VMware run the DPDK with vmxnet3 NIC failed with the RSS confi= g > > Hi > > > > My application developed with the DPDK, we found out one error message > when my application run in the VMware platform. > > > > [2025-04-15 03:01:09]:: --[vmxnet3_dev_start:961] hw->version: 6, rss_hf: > 3380 --^M^M > > [2025-04-15 03:01:09]:: --[vmxnet3_rss_configure:1434] rss_hf: 3380 --^M^= M > > [2025-04-15 03:01:09]:: --[vmxnet3_v4_rss_configure:1381] rss_hf: 3380 > --^M^M > > [2025-04-15 03:01:09]:: vmxnet3_v4_rss_configure(): Set RSS fields (v4) > failed: 1^M^M > > [2025-04-15 03:01:09]:: vmxnet3_dev_start(): Failed to configure v4 RSS^M= ^M > > > > From the debug logs, it seems that the vmxnet3 device failed to configure > the v4 RSS. > > > > We can found these informations from the DPDK=E2=80=99s code snippet: > > > > rss_hf =3D 3380 =3D 0xd34 > > > > So the RSS configure is this: > > > > #define VMXNET3_RSS_OFFLOAD_ALL ( \ > > RTE_ETH_RSS_IPV4 | \ > > RTE_ETH_RSS_NONFRAG_IPV4_TCP | \ > > RTE_ETH_RSS_IPV6 | \ > > RTE_ETH_RSS_NONFRAG_IPV6_TCP) > > > > #define VMXNET3_V4_RSS_MASK ( \ > > RTE_ETH_RSS_NONFRAG_IPV4_UDP | \ > > RTE_ETH_RSS_NONFRAG_IPV6_UDP) > > > > The function call vmxnet3_rss_configure() is works fine but the function > call vmxnet3_v4_rss_configure() failed: > > > > if (VMXNET3_VERSION_GE_4(hw) && > > dev->data->dev_conf.rxmode.mq_mode =3D=3D RTE_ETH_MQ_RX_RSS) { > > /* Check for additional RSS */ > > ret =3D vmxnet3_v4_rss_configure(dev); > > if (ret !=3D VMXNET3_SUCCESS) { > > PMD_INIT_LOG(ERR, "Failed to configure v4 RSS"); > > return ret; > > } > > } > > > > /* > > * Additional RSS configurations based on vmxnet v4+ APIs > > */ > > int > > vmxnet3_v4_rss_configure(struct rte_eth_dev *dev) > > { > > struct vmxnet3_hw *hw =3D dev->data->dev_private; > > Vmxnet3_DriverShared *shared =3D hw->shared; > > Vmxnet3_CmdInfo *cmdInfo =3D &shared->cu.cmdInfo; > > struct rte_eth_rss_conf *port_rss_conf; > > uint64_t rss_hf; > > uint32_t ret; > > > > PMD_INIT_FUNC_TRACE(); > > > > cmdInfo->setRSSFields =3D 0; > > port_rss_conf =3D &dev->data->dev_conf.rx_adv_conf.rss_conf; > > > > printf("--[%s:%d] rss_hf: %llu --\n", __FUNCTION__, __LINE__, > port_rss_conf->rss_hf); > > > > if ((port_rss_conf->rss_hf & VMXNET3_MANDATORY_V4_RSS) !=3D > > VMXNET3_MANDATORY_V4_RSS) { > > PMD_INIT_LOG(WARNING, "RSS: IPv4/6 TCP is required for vmxnet3 v4 > RSS," > > "automatically setting it"); > > port_rss_conf->rss_hf |=3D VMXNET3_MANDATORY_V4_RSS; > > } > > > > rss_hf =3D port_rss_conf->rss_hf & > > (VMXNET3_V4_RSS_MASK | VMXNET3_RSS_OFFLOAD_ALL); > > > > if (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP) > > cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_TCPIP4; > > if (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP) > > cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_TCPIP6; > > if (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP) > > cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_UDPIP4; > > if (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP) > > cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_UDPIP6; > > > > VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, > > VMXNET3_CMD_SET_RSS_FIELDS); > > ret =3D VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); > > > > if (ret !=3D VMXNET3_SUCCESS) { > > PMD_DRV_LOG(ERR, "Set RSS fields (v4) failed: %d", ret); > > } > > > > return ret; > > } > > > > It seems like the VMXNET3_WRITE_BAR1_REG failed, but don=E2=80=99t know w= hy this > happened. > > > > My DPDK version is 22.11.1 and the vmxnet3 driver version is 1.6.0.0-k-NA= PI > > > > BTW we just upgraded VM compatibility to *ESXi 8.0 U2 and later (VM > version 21), *then our application run with this error logs. > > > > The previous VM compatibility is *ESXi 6.5 and later (VM version 13),* > and our application works fine. It is because the vmxnet3 hw version is 3= , > so it will not trigger to call the V4 RSS configure. > > > > Could someone help to check this issue? Or maybe I missed something? > --000000000000003cc006335d6c39 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

I hit=C2=A0the same issue, and it was=C2=A0brute force= .=C2=A0 The first version of VMXNET3 is something that
has never been= =C2=A0exposed directly.=C2=A0 (It is implicit=C2=A0and based on=C2=A0the VM Hardware version.A) You can adjust it=C2=A0in the VM advanced settings. It makes sense to indi= cate exactly=C2=A0which flag=C2=A0must be in the=C2=A0VM.
(Why? Sometime= s, downgrading the VM Hardware version is not an option. ) While downgradin= g, the VMXNET3 version will run in compatibility=C2=A0mode.
(I.e., DPDK = version old, nic new, RSS must X, etc, etc) There are a few combinations wh= ere=C2=A0
You can not assume=C2=A0a single queue, or take a four-queue, = or assume a 32-queue.=C2=A0 (This combination depends on=C2=A0

Combi= nation
NIC - VM - ESXi - DPDK version

So, asking a client to do f= our=C2=A0variations and find a combination=C2=A0is=C2=A0not a good option.<= br>(Note in kernel driver it's the same,=C2=A0 you can use 32, but you = have no idea how to enable the VMXNET3 version that will allow you to use 3= 2 queues)=C2=A0

Example.

You have 810 - VMXNET 3 poll mode=C2= =A0and=C2=A0enable 16 queues.=C2=A0 What do you do?
You have 810 - VMXNE= T 3 (the default queue in RSS 4). What do you=C2=A0do? --- This will fail o= n 23/24/25.
You have 801 - VMXNET 3 - ESXi 8.0.2 and DPDK 22.xx app ( ex= ample people use trex it build 22 or 23)

I just gave a few examples= .=C2=A0=C2=A0

Kind Regards,
Mus

On Sat, Apr 19, 2025 at 11:15=E2=80=AFAM Fangyin Hu <FHu@sonicwall.com> wrote:

Hi, Knight<= /u>

Thank you very much f= or your reply!

I check our codes, my= application config the 1 RX queue for vmxnet3 device, so the call vmxnet3_v4_rss_configure() will faile= d due to the single RX queue.

I adjusted our codes = and use 2 RX queue for vmxnet3 device, then retest my application aga=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 in. Now my application works fine and = here are some debug logs.

=C2=A0<= /span>

[2025-04-18 07:37:02]= :: Try to configure port 0, RX Queue Number: 2, TX Queue Number: 1 Dev 0^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:310:init_options():DPDK interface (net_vmxnet3): 0^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:312:init_options():=C2=A0 multicast_en: 1^M^M=

[2025-04-18 07:37:02]= :: pktio/dpdk.c:313:init_options():=C2=A0 num_rx_desc: 4096^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:314:init_options():=C2=A0 num_tx_desc: 512^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:315:init_options():=C2=A0 rx_drop_en: 0^M^M

[2025-04-18 07:37:02]= :: odp_packet_io.c:439:odp_pktio_open():interface: 0, driver: dpdk^M^M

[2025-04-18 07:37:02]= :: created pktio 1 idx( 0) (0)^M^M

=C2=A0<= /span>

[2025-04-18 07:37:02]= :: ### starting data port pktio's...^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:2277:dpdk_setup_eth_tx():Mapping per TX queue statistics no= t supported: -95^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:2281:dpdk_setup_eth_tx():Mapped 0/16 TX counters^M^M=

[2025-04-18 07:37:02]= :: pktio/dpdk.c:2315:dpdk_setup_eth_rx():Mapping per RX queue statistics no= t supported: -95^M^M

[2025-04-18 07:37:02]= :: pktio/dpdk.c:2319:dpdk_setup_eth_rx():Mapped 0/16 RX counters^M^M=

[2025-04-18 07:37:02]= :: --[vmxnet3_dev_start:963] hw->version: 6, rss_hf: 3380 --^M^M<= u>

[2025-04-18 07:37:02]= :: --[vmxnet3_rss_configure:1430] rss_hf: 3380 --^M^M<= /p>

[2025-04-18 07:37:02]= :: --[vmxnet3_v4_rss_configure:1381] rss_hf: 3380 --^M^M

[2025-04-18 07:37:02]= :: odp_packet_io.c:723:odp_pktio_start():interface: 0, input queues: 2, out= put queues: 1^M^M

[2025-04-18 07:37:03]= :: pktio port 0 dpdk port 0 started!^M^M

=C2=A0<= /span>

So the next step we n= eed to modify our codes, it will not config the RSS when the device RX queu= e number is 1

=C2=A0<= /span>

Thank you for your ad= vise again and best wish to you!

=C2=A0<= /span>

From: Knight, Joshua <Joshua.Knight@netscout.com>
Sent: Wednesday, April 16, 2025 9:59 PM
To: Fangyin Hu <FHu@SonicWall.com>; users@dpdk.org
Subject: Re: VMware run the DPDK with vmxnet3 NIC failed with the RS= S config

=C2=A0

CAUTION: This email origina= ted from outside of the organization. Do not click links or open attachment= s unless you recognize the sender and know the content is safe.

=C2=A0

We=E2=80=99ve hit the= same problem setting up RSS on vmxnet3 on newer esxi8 releases.=C2=A0 In o= ur case we would see the failure if we only had a single rx queue.

Snippet from my discussion with their support team:
> So to summarize, with VM Hardware = version 14 in esxi 7 and 6.7, setting the RSS hash function did not generat= e an error in the driver.=C2=A0 With VM Hardware version 14 in esxi 8U2, it= generates an error in the driver.

=C2=A0<= /span>

I had a ticket opened= with Broadcom and their response was they made a change in behavior where = instead of silently rejecting the RSS configuration (since, really you don= =E2=80=99t need RSS with a single queue although we still wanted it enabled) the nic now returns an error. The biggest prob= lem I had with the change is they applied it to *earlier* vm hw vers= ions, not just the latest/newest version.

=C2=A0<= /span>

Broadcom=E2=80=99s re= sponse to my ticket was:

>Regarding the difference in behavior you have observed wi= th ESXi versions, there were some enhancements introduced in vSphere versio= n 7.0U1 used along with VM hardware version >=3D14, it identifies the RSS misconfiguration with Rx Queue and throws= an error message similar to what has been observed. In older versions, it = was not erroring out so you will not observe any failures in the applicatio= ns.=C2=A0

=C2=A0

=C2=A0<= /span>

=C2=A0<= /span>

From: Fangyin Hu <FHu@SonicWall.com>
Date: Wednesday, April 16, 2025 at 4:23=E2=80=AFAM
To: users@dpdk.org <users@dpdk.org>
Subject: VMware run the DPDK with vmxnet3 NIC failed with the RSS co= nfig

Hi

=C2=A0

My application develo= ped with the DPDK, we found out one error message when my application run i= n the VMware platform.<= /span>

=C2=A0

[2025-04-15 03:01:09]= :: --[vmxnet3_dev_start:961] hw->version: 6, rss_hf: 3380 --^M^M<= span style=3D"font-size:12pt">

[2025-04-15 03:01:09]= :: --[vmxnet3_rss_configure:1434] rss_hf: 3380 --^M^M

[2025-04-15 03:01:09]= :: --[vmxnet3_v4_rss_configure:1381] rss_hf: 3380 --^M^M

[2025-04-15 03:01:09]= :: vmxnet3_v4_rss_configure(): Set RSS fields (v4) failed: 1^M^M

[2025-04-15 03:01:09]= :: vmxnet3_dev_start(): Failed to configure v4 RSS^M^M

=C2=A0

From the debug logs, = it seems that the vmxnet3 device failed to configure the v4 RSS.

=C2=A0

We can found these in= formations from the DPDK=E2=80=99s code snippet:

=C2=A0

rss_hf =3D 3380 =3D 0= xd34

=C2=A0

So the RSS configure = is this:

=C2=A0

#define VMXNET3_RSS_O= FFLOAD_ALL ( \

=C2=A0=C2=A0=C2=A0 RT= E_ETH_RSS_IPV4 | \

=C2=A0=C2=A0=C2=A0 RT= E_ETH_RSS_NONFRAG_IPV4_TCP | \=

=C2=A0=C2=A0=C2=A0 RT= E_ETH_RSS_IPV6 | \

=C2=A0=C2=A0=C2=A0 RT= E_ETH_RSS_NONFRAG_IPV6_TCP)=

=C2=A0

#define VMXNET3_V4_RS= S_MASK ( \

=C2=A0=C2=A0=C2=A0 RT= E_ETH_RSS_NONFRAG_IPV4_UDP | \=

RTE_ETH_RSS_NONFRAG_IPV6_UDP)=

=C2=A0

The function call vmx= net3_rss_configure() is works fine but the function call vmxnet3_v4_rss_con= figure() failed:=

=C2=A0

=C2=A0=C2=A0=C2=A0 if= (VMXNET3_VERSION_GE_4(hw) &&=

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 dev->data->dev_conf.rxmode.mq_mode =3D=3D RTE_E= TH_MQ_RX_RSS) {<= /p>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 /* Check for additional RSS=C2=A0 */

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 ret =3D vmxnet3_v4_rss_configure(dev);

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 if (ret !=3D VMXNET3_SUCCESS) {

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PMD_INIT_LOG(ERR, "Failed to configure v4 RSS");

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return ret;

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 }=

}

=C2=A0

/*

* Additional RSS conf= igurations based on vmxnet v4+ APIs

*/

int

vmxnet3_v4_rss_config= ure(struct rte_eth_dev *dev)

{

=C2=A0=C2=A0=C2=A0 st= ruct vmxnet3_hw *hw =3D dev->data->dev_private;

=C2=A0=C2=A0=C2=A0 Vm= xnet3_DriverShared *shared =3D hw->shared;

=C2=A0=C2=A0=C2=A0 Vm= xnet3_CmdInfo *cmdInfo =3D &shared->cu.cmdInfo;

=C2=A0=C2=A0=C2=A0 st= ruct rte_eth_rss_conf *port_rss_conf;=

=C2=A0=C2=A0=C2=A0 ui= nt64_t rss_hf;

=C2=A0=C2=A0=C2=A0 ui= nt32_t ret;

=C2=A0

=C2=A0=C2=A0=C2=A0 PM= D_INIT_FUNC_TRACE();

=C2=A0

=C2=A0=C2=A0=C2=A0 cm= dInfo->setRSSFields =3D 0;<= u>

=C2=A0=C2=A0=C2=A0 po= rt_rss_conf =3D &dev->data->dev_conf.rx_adv_conf.rss_conf;=

=C2=A0

=C2=A0=C2=A0=C2=A0 pr= intf("--[%s:%d] rss_hf: %llu --\n", __FUNCTION__, __LINE__, port_= rss_conf->rss_hf);

=C2=A0

=C2=A0=C2=A0=C2=A0 if= ((port_rss_conf->rss_hf & VMXNET3_MANDATORY_V4_RSS) !=3D

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 VMXNET3_MANDATORY_V4_RSS) {

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 PMD_INIT_LOG(WARNING, "RSS: IPv4/6 TCP is requir= ed for vmxnet3 v4 RSS,"

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= "automatically setting it");

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 port_rss_conf->rss_hf |=3D VMXNET3_MANDATORY_V4_RS= S;

=C2=A0=C2=A0=C2=A0 }<= /span>

=C2=A0

=C2=A0=C2=A0=C2=A0 rs= s_hf =3D port_rss_conf->rss_hf &

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 (VMXNET3_V4_RSS_MASK | VMXNET3_RSS_OFFLOAD_ALL);

=C2=A0

=C2=A0=C2=A0=C2=A0 if= (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_TCP)

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_TCPI= P4;

=C2=A0=C2=A0=C2=A0 if= (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_TCP)

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_TCPI= P6;

=C2=A0=C2=A0=C2=A0 if= (rss_hf & RTE_ETH_RSS_NONFRAG_IPV4_UDP)

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_UDPI= P4;

=C2=A0=C2=A0=C2=A0 if= (rss_hf & RTE_ETH_RSS_NONFRAG_IPV6_UDP)

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 cmdInfo->setRSSFields |=3D VMXNET3_RSS_FIELDS_UDPI= P6;

=C2=A0

=C2=A0=C2=A0=C2=A0 VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD,<= /span>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 VMXNET3_CMD_SET_RSS_FIELDS);

=C2=A0=C2=A0=C2=A0 re= t =3D VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD);

=C2=A0

=C2=A0=C2=A0=C2=A0 if= (ret !=3D VMXNET3_SUCCESS) {<= u>

=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 PMD_DRV_LOG(ERR, "Set = RSS fields (v4) failed: %d", ret);

=C2=A0=C2=A0=C2=A0 }<= /span>

=C2=A0

return ret;=

}

=C2=A0

It seems like the VMX= NET3_WRITE_BAR1_REG failed, but don=E2=80=99t know why this happened.

=C2=A0

My DPDK version is 22= .11.1 and the vmxnet3 driver version is 1.6.0.0-k-NAPI

=C2=A0

BTW we just upgraded = VM compatibility to ESXi 8.0 U2 and later (VM version 21), then our application run with= this error logs.

=C2=A0

The previous VM compa= tibility is ESXi 6.5 and later (VM version 13), and our application works fine. = It is because the vmxnet3 hw version is 3, so it will not trigger to call t= he V4 RSS configure.

=C2=A0

Could someone help to= check this issue? Or maybe I missed something?

--000000000000003cc006335d6c39--