From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AD31845972 for ; Fri, 13 Sep 2024 00:02:17 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7B9324027C; Fri, 13 Sep 2024 00:02:17 +0200 (CEST) Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by mails.dpdk.org (Postfix) with ESMTP id E1EAF40156 for ; Fri, 13 Sep 2024 00:02:16 +0200 (CEST) Received: by mail-lj1-f169.google.com with SMTP id 38308e7fff4ca-2f75428b9f8so5317071fa.3 for ; Thu, 12 Sep 2024 15:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1726178536; x=1726783336; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=7lIOnHypIKItXZVHTPysHb+RuPivTgJlx4R/JfsIiRE=; b=PwmSkn1hGyOe7e59eCK1/tkHIeNczJ1DcEUqbWuD0x+T2ZRvBiTonXb3IwKCwEHWbK RkAKEK3xDahg9qThJlK9KjFGSochSRlydvAcNkTo4MAzuEx12dwJNI40PI03cjXbBXE9 xZarHyXIk8kZRdjcg8FnTQ/KUum/OJdgLUMRo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726178536; x=1726783336; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7lIOnHypIKItXZVHTPysHb+RuPivTgJlx4R/JfsIiRE=; b=YpMw1GWmKwuI63PDxpEqX7kLp2R4T1gqNpY4IvzGVJ8IvOvAJYYEdGH2pCqU2mzFvY 8hETkbsnBUAQQRXPVtaAa4C9E8aDeSRf9ukke8RL/CYOj+4f8/czIWor6AXgmbcZGbTM ksfoatjq2zVGcI/ool1Uie54uLuNXrkJbgDXVMGqaZDQ2PjviFvtLzXyh4OW3chJgZGk uGTVfgCHn3T8TpoNQokK94Xj5VYdYDU+v8W+jxGA3r4OUX+TKtvS4Uvc4FTFLTrddx5E z7A6oTazjhc2OxTXsLOARqQSmaAM9twTn1MLntCghc3ZgMsBX9Fnyf4UEn6Ytmvjj3EE hFLw== X-Forwarded-Encrypted: i=1; AJvYcCUlZSDy+yDiVTxOFO8GOhKaT2KKXEpC4yTKOe49fQg4O5RBuyuJY0QMvv9LPP0Pt0x+/m6WNw==@dpdk.org X-Gm-Message-State: AOJu0YydVcUjprD8M3Jh351HNNt3BMSWprzQH8yiEFOgp3nWJuFOBe/u JtlueX9cxksd33g8oegxVHyJN2Miy/oOKw9ZpnyHPP81bModld2Cn5tZeDhVU4COF4nA4Ygt5AS gjhCoNofPlSAQ+pl1yuInkNquEn/MY5plV+Lnd8PnMOODQ1sOGEMFrSOy65n6j36JRDXWdUQW9R heZYDZekk= X-Google-Smtp-Source: AGHT+IHe0rW2I7QjY7QNWqYIWc1XoLkeeWor0rVkQD88+WhNhPoJLfDWaJkSbg4d1Ue4ix2p2OqCbyK9YcJhIjUWvnw= X-Received: by 2002:a2e:744:0:b0:2f3:f256:aeec with SMTP id 38308e7fff4ca-2f791a02b5cmr4415831fa.19.1726178535896; Thu, 12 Sep 2024 15:02:15 -0700 (PDT) MIME-Version: 1.0 References: <20240903170350.7e663864@hermes.local> <20240904154246.1c5bbb58@hermes.local> In-Reply-To: <20240904154246.1c5bbb58@hermes.local> From: Nandini Rangaswamy Date: Thu, 12 Sep 2024 15:02:04 -0700 Message-ID: Subject: Re: Netvsc vs Failsafe Performance To: Stephen Hemminger Cc: Long Li , users@dpdk.org Content-Type: multipart/alternative; boundary="00000000000073aeac0621f34220" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000073aeac0621f34220 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Stephen/Long, Thanks for the suggestions. I configured the RSS key to be 40 bytes as suggested. What NDIS version is being used by netvsc and could you please direct me to a microsoft document confirming that if TCP RSS is requested, UDP RSS is implied? Regards On Wed, Sep 4, 2024 at 3:42=E2=80=AFPM Stephen Hemminger wrote: > On Tue, 3 Sep 2024 17:21:48 -0700 > Nandini Rangaswamy wrote: > > > Hi Stephen/Long, > > dpdk_netvsc_port_configure:1873 Configure port eth2/2. I am testing usi= ng > > TCP traffic (iperf3 tool) generated between pair of client and servers > with > > DPDK app forward traffic between client and servers. > > These are the config being passed for configuring netvsc: > > lsc_intr=3D1 > > rxq/txq=3D2/2, > > rss is enabled with rss_hf=3D0x0000000000000c30 > > tx_ol=3D0x00000000000006 > > rx_ol=3D0x00000000080007 > > > > Rsskey len is 64. > > struct rte_eth_conf conf =3D { > > .intr_conf =3D { > > .lsc =3D !dpdk.lsc_intr_disable && !dpdk_if->lsc_intr_disable && > > !!(dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC), > > }, > > .rxmode =3D { > > .mq_mode =3D RTE_ETH_MQ_RX_RSS, > > .offloads =3D RTE_ETH_RX_OFFLOAD_VLAN_STRIP | > RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | > > RTE_ETH_RX_OFFLOAD_RSS_HASH | RTE_ETH_RX_OFFLOAD_UDP_CKSUM, > > }, > > .rx_adv_conf.rss_conf =3D { > > .rss_hf =3D RTE_ETH_RSS_NONFRAG_IPV4_UDP | RTE_ETH_RSS_NONFRAG_IPV4_TCP= | > > RTE_ETH_RSS_NONFRAG_IPV6_TCP, > > .rss_key =3D conf_rss_key, > > .rss_key_len =3D rss_key_len, > > }, > > .txmode =3D { > > .offloads =3D RTE_ETH_TX_OFFLOAD_UDP_CKSUM | RTE_ETH_TX_OFFLOAD_IPV4_CK= SUM, > > }, > > > > Regards, > > Nandini > > > > On Tue, Sep 3, 2024 at 5:03=E2=80=AFPM Stephen Hemminger < > stephen@networkplumber.org> > > wrote: > > > > > On Tue, 3 Sep 2024 14:43:28 -0700 > > > Nandini Rangaswamy wrote: > > > > > > > Hi Stephen and Long, > > > > I was going through one of the netvsc patches > > > > https://mails.dpdk.org/archives/dev/2018-August/110559.html which > > > mentioned > > > > that netvsc and failsafe give the same performance in VF path > whereas for > > > > some exception path tests, about 22% performance gain in seen. > > > > I ran some tests locally with my dpdk app integrated with netvsc PM= D > and > > > > observed that netvsc does give nearly the same performance as > failsafe in > > > > the VF path. > > > > Since the official document does not explicitly cite this, I would > like > > > to > > > > confirm if this holds good. > > > > Regards, > > > > Nandini > > > > > > > > > > Shouldn't be. What settings are you using. > > > Both failsafe and netvsc just pass packets to VF if present. > > > There is even more locks to go through with failsafe. > > > > > > Are you sure the test doesn't exercise something like checksumming > which > > > maybe different. > > > > > > > How many streams? RSS won't matter unless multiple streams. > The netvsc driver does not have RSS for UDP as a listed flag. > It turns out that for that version of NDIS, if you ask for TCP RSS, UDP > RSS is implied. > > RSS Key must be 40 bytes (Toeplitz) not 64 bytes. > Just use the default key (rss_key =3D=3D NULL rss_key_len =3D 0) to be sa= fe > > Check that packets are going to the VF. One way to do that is to look at > xstats on both netvsc and mlx5 device. > > > --=20 This electronic communication and the information and any files transmitted= =20 with it, or attached to it, are confidential and are intended solely for=20 the use of the individual or entity to whom it is addressed and may contain= =20 information that is confidential, legally privileged, protected by privacy= =20 laws, or otherwise restricted from disclosure to anyone else. If you are=20 not the intended recipient or the person responsible for delivering the=20 e-mail to the intended recipient, you are hereby notified that any use,=20 copying, distributing, dissemination, forwarding, printing, or copying of= =20 this e-mail is strictly prohibited. If you received this e-mail in error,= =20 please return the e-mail to the sender, delete it from your computer, and= =20 destroy any printed copy of it. --00000000000073aeac0621f34220 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Stephen/Long,
Thanks for the suggestions. I configu= red the RSS key to be 40 bytes as suggested.
What NDIS version is b= eing used by netvsc and could you please direct me to a microsoft document = confirming that if TCP RSS is requested, UDP RSS is implied?
Rega= rds

On Wed, Sep 4, 2024 at 3:42=E2=80=AFPM Stephen Hemminger &l= t;stephen@n= etworkplumber.org> wrote:
On Tue, 3 Sep 2024 17:21:48 -0700
Nandini Rangaswamy <nandini.rangaswamy@broadcom.com> wrote:

> Hi Stephen/Long,
> dpdk_netvsc_port_configure:1873 Configure port eth2/2. I am testing us= ing
> TCP traffic (iperf3 tool) generated between pair of client and servers= with
> DPDK app forward traffic between client and servers.
> These are the config being passed for configuring netvsc:
> lsc_intr=3D1
> rxq/txq=3D2/2,
> rss is enabled with rss_hf=3D0x0000000000000c30
> tx_ol=3D0x00000000000006
> rx_ol=3D0x00000000080007
>
> Rsskey len is 64.
> struct rte_eth_conf conf =3D {
> .intr_conf =3D {
> .lsc =3D !dpdk.lsc_intr_disable && !dpdk_if->lsc_intr_disab= le &&
> !!(dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC),
> },
> .rxmode =3D {
> .mq_mode =3D RTE_ETH_MQ_RX_RSS,
> .offloads =3D RTE_ETH_RX_OFFLOAD_VLAN_STRIP | RTE_ETH_RX_OFFLOAD_IPV4_= CKSUM |
> RTE_ETH_RX_OFFLOAD_RSS_HASH | RTE_ETH_RX_OFFLOAD_UDP_CKSUM,
> },
> .rx_adv_conf.rss_conf =3D {
> .rss_hf =3D RTE_ETH_RSS_NONFRAG_IPV4_UDP | RTE_ETH_RSS_NONFRAG_IPV4_TC= P |
> RTE_ETH_RSS_NONFRAG_IPV6_TCP,
> .rss_key =3D conf_rss_key,
> .rss_key_len =3D rss_key_len,
> },
> .txmode =3D {
> .offloads =3D RTE_ETH_TX_OFFLOAD_UDP_CKSUM | RTE_ETH_TX_OFFLOAD_IPV4_C= KSUM,
> },
>
> Regards,
> Nandini
>
> On Tue, Sep 3, 2024 at 5:03=E2=80=AFPM Stephen Hemminger <stephen@networkplu= mber.org>
> wrote:
>
> > On Tue, 3 Sep 2024 14:43:28 -0700
> > Nandini Rangaswamy <nandini.rangaswamy@broadcom.com> wrote: > >=C2=A0
> > > Hi Stephen and Long,
> > > I was going through one of the netvsc patches
> > > https://mails.dpdk.org/arc= hives/dev/2018-August/110559.html which=C2=A0
> > mentioned=C2=A0
> > > that netvsc and failsafe give the same performance in VF pat= h whereas for
> > > some exception path=C2=A0 tests, about 22% performance gain = in seen.
> > > I ran some tests locally with my dpdk app integrated with ne= tvsc PMD and
> > > observed that netvsc does give nearly the same performance a= s failsafe in
> > > the VF path.
> > > Since the official document does not explicitly cite this, I= would like=C2=A0
> > to=C2=A0
> > > confirm if this holds good.
> > > Regards,
> > > Nandini
> > >=C2=A0
> >
> > Shouldn't be. What settings are you using.
> > Both failsafe and netvsc just pass packets to VF if present.
> > There is even more locks to go through with failsafe.
> >
> > Are you sure the test doesn't exercise something like checksu= mming which
> > maybe different.
> >=C2=A0
>

How many streams? RSS won't matter unless multiple streams.
The netvsc driver does not have RSS for UDP as a listed flag.
It turns out that for that version of NDIS, if you ask for TCP RSS, UDP RSS= is implied.

RSS Key must be 40 bytes (Toeplitz) not 64 bytes.
Just use the default key (rss_key =3D=3D NULL rss_key_len =3D 0) to be safe=

Check that packets are going to the VF. One way to do that is to look at xs= tats on both netvsc and mlx5 device.



This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it. --00000000000073aeac0621f34220--