From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B397945972 for ; Thu, 12 Sep 2024 22:47:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3B48B42DBD; Thu, 12 Sep 2024 22:47:51 +0200 (CEST) Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by mails.dpdk.org (Postfix) with ESMTP id E0DB140265 for ; Thu, 12 Sep 2024 22:47:49 +0200 (CEST) Received: by mail-lj1-f179.google.com with SMTP id 38308e7fff4ca-2f7528f4658so2735771fa.3 for ; Thu, 12 Sep 2024 13:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1726174069; x=1726778869; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=1wm1KTl/VMANWM3et9uUG0zHs/EeQ6gKfZZR3jqcpdw=; b=LJ7i/N324ti1haTAu32k1Z5La3iZ3wYofypA6ED4iWdGaASeaiupbxYIcrZUhrvHMl Re3qdy5TtG727JGKLIV744NBb4kSKeXVBTodD1iRlfoEZSLfYTyvumnaSQt86koKKq36 zpWkJ/tXQ+A/36aioxaIBiW2ltMXcIZ+hNAlk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726174069; x=1726778869; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1wm1KTl/VMANWM3et9uUG0zHs/EeQ6gKfZZR3jqcpdw=; b=ewjYYXuO5kLZARWhQnYOGr6w7U5GjGu72ZQhRxNS7bSSUTt4zlzh2OlTK6g0HVqbEH JXMQQGNssR7BzKqsK7cJN5pbMrmoyQQDIkwrcVbajRt2Fy1Oa3wj3zQJOfd/zg4Td/LC cKI5TjE+kDem2pizHFFQ7MbVQVpiYQtvTF4aSUFi4/iJZPLLeIkEWPmLDZh9Uti5wugd 0UgNWkA3r21Ln14xyllUsGH7GSFs7bcKIooK47n/p2iC+xblQ74TcmRb2OEt5GmFGx8i +2jAfJkQAywKhfAsctLVemWQ3qUQN47wkekF3TaJ1tg+Mnjve18N/gGDiv9RGSxHs2AI S02w== X-Forwarded-Encrypted: i=1; AJvYcCXX309iO+BzeA+FA68Ey1qErkebKTru1VRZufhCBG+rWcCLk6h1PNvv7VYYu5lIs0iVG1/b3Q==@dpdk.org X-Gm-Message-State: AOJu0YwIYYYV9pPuezUW8IyvAg5+KEQod7bUAjvmj1Il6gvsSDH3TAbM hTEVt5bVDwoa6v9n4Brn4Qjst5QDB0riulMBNRMifkjCQHSNOHJErlRksC+XPBnt6melNlrygKG qQFrp2tkeiBCSusfORRliHmC8EDyTDeMkkI/McjAC2OOW1sfREie5Fz/+HajlKDH4EBRfzfCz/o zhBYFc8f4= X-Google-Smtp-Source: AGHT+IHJylFuFDP5z0RcpvFIiuPbnJgdp4QTDUXoy2LhcdHoDxkjFCl6WuDRfdvihURgqFSqYAAgbC8Gs5fNZ/TnfhY= X-Received: by 2002:a2e:f0a:0:b0:2f7:4e9b:93c4 with SMTP id 38308e7fff4ca-2f791a04410mr2680431fa.19.1726174068914; Thu, 12 Sep 2024 13:47:48 -0700 (PDT) MIME-Version: 1.0 References: <20240903170350.7e663864@hermes.local> <20240904154246.1c5bbb58@hermes.local> In-Reply-To: From: Nandini Rangaswamy Date: Thu, 12 Sep 2024 13:47:37 -0700 Message-ID: Subject: Re: Netvsc vs Failsafe Performance To: Long Li Cc: Stephen Hemminger , "users@dpdk.org" Content-Type: multipart/alternative; boundary="00000000000032f0670621f238c8" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000032f0670621f238c8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for your response Long Li. I see with netvsc the maximum number of Tx descriptors is restricted to 4096 whereas the number of Rx descriptors is restricted to 8192. But, for failsafe PMD , we see that both the number of Txd and Rxd is restricted to 8192. How is netvsc PMD giving the same performance as failsafe PMD ? Regards On Wed, Sep 4, 2024 at 7:30=E2=80=AFPM Long Li wrote= : > > Subject: Re: Netvsc vs Failsafe Performance > > > > On Tue, 3 Sep 2024 17:21:48 -0700 > > Nandini Rangaswamy wrote: > > > > > Hi Stephen/Long, > > > dpdk_netvsc_port_configure:1873 Configure port eth2/2. I am testing > > > using TCP traffic (iperf3 tool) generated between pair of client and > > > servers with DPDK app forward traffic between client and servers. > > > These are the config being passed for configuring netvsc: > > > lsc_intr=3D1 > > > rxq/txq=3D2/2, > > > rss is enabled with rss_hf=3D0x0000000000000c30 > > > tx_ol=3D0x00000000000006 > > > rx_ol=3D0x00000000080007 > > > > > > Rsskey len is 64. > > > struct rte_eth_conf conf =3D { > > > .intr_conf =3D { > > > .lsc =3D !dpdk.lsc_intr_disable && !dpdk_if->lsc_intr_disable && > > > !!(dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC), }, .rxmode =3D { > > > .mq_mode =3D RTE_ETH_MQ_RX_RSS, .offloads =3D > > > RTE_ETH_RX_OFFLOAD_VLAN_STRIP | > > RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | > > > RTE_ETH_RX_OFFLOAD_RSS_HASH | > > RTE_ETH_RX_OFFLOAD_UDP_CKSUM, }, > > > .rx_adv_conf.rss_conf =3D { .rss_hf =3D RTE_ETH_RSS_NONFRAG_IPV4_UDP = | > > > RTE_ETH_RSS_NONFRAG_IPV4_TCP | > > RTE_ETH_RSS_NONFRAG_IPV6_TCP, .rss_key > > > =3D conf_rss_key, .rss_key_len =3D rss_key_len, }, .txmode =3D { .off= loads =3D > > > RTE_ETH_TX_OFFLOAD_UDP_CKSUM | > > RTE_ETH_TX_OFFLOAD_IPV4_CKSUM, }, > > > > > > Regards, > > > Nandini > > > > > > On Tue, Sep 3, 2024 at 5:03=E2=80=AFPM Stephen Hemminger > > > > > > wrote: > > > > > > > On Tue, 3 Sep 2024 14:43:28 -0700 > > > > Nandini Rangaswamy wrote: > > > > > > > > > Hi Stephen and Long, > > > > > I was going through one of the netvsc patches > > > > > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F= %2F > > > > > mails.dpdk.org%2Farchives%2Fdev%2F2018- > > August%2F110559.html&data=3D0 > > > > > > > 5%7C02%7Clongli%40microsoft.com%7Ce91cca1ee99f4809138708dccd32e76 > > 7 > > > > > %7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638610865749 > > 361006%7 > > > > > > > CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB > > TiI > > > > > > > 6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=3Dms6yhMjBpT0IFu6e9 > > wmh4K1 > > > > > WDINzgjoRzFJmMJGJwuY%3D&reserved=3D0 which > > > > mentioned > > > > > that netvsc and failsafe give the same performance in VF path > > > > > whereas for some exception path tests, about 22% performance gai= n > in > > seen. > > > > > I ran some tests locally with my dpdk app integrated with netvsc > > > > > PMD and observed that netvsc does give nearly the same performanc= e > > > > > as failsafe in the VF path. > > > > > Since the official document does not explicitly cite this, I woul= d > > > > > like > > > > to > > > > > confirm if this holds good. > > > > > Regards, > > > > > Nandini > > > > > > > > > > > > > Shouldn't be. What settings are you using. > > > > Both failsafe and netvsc just pass packets to VF if present. > > > > There is even more locks to go through with failsafe. > > > > > > > > Are you sure the test doesn't exercise something like checksumming > > > > which maybe different. > > > > > > > > > > > How many streams? RSS won't matter unless multiple streams. > > The netvsc driver does not have RSS for UDP as a listed flag. > > It turns out that for that version of NDIS, if you ask for TCP RSS, UDP > RSS is > > implied. > > > > RSS Key must be 40 bytes (Toeplitz) not 64 bytes. > > Just use the default key (rss_key =3D=3D NULL rss_key_len =3D 0) to be = safe > > > > Check that packets are going to the VF. One way to do that is to look a= t > xstats > > on both netvsc and mlx5 device. > > > > If most traffic goes through the VF, you won't see much difference in > performance of netvsc vs failsafe because they are not used on the data > path. > > It seems the 20% performance gain is measured on synthetic path, meaning > those traffic does not go through the VF. In this scenario, netvsc has an > advantage over failsafe since the traffic data doesn't need to be copied > around in the kernel space. > --=20 This electronic communication and the information and any files transmitted= =20 with it, or attached to it, are confidential and are intended solely for=20 the use of the individual or entity to whom it is addressed and may contain= =20 information that is confidential, legally privileged, protected by privacy= =20 laws, or otherwise restricted from disclosure to anyone else. If you are=20 not the intended recipient or the person responsible for delivering the=20 e-mail to the intended recipient, you are hereby notified that any use,=20 copying, distributing, dissemination, forwarding, printing, or copying of= =20 this e-mail is strictly prohibited. If you received this e-mail in error,= =20 please return the e-mail to the sender, delete it from your computer, and= =20 destroy any printed copy of it. --00000000000032f0670621f238c8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for your response Long Li.
I see with netvsc th= e maximum number of Tx descriptors is restricted to 4096 whereas the number= of Rx descriptors is restricted to 8192.
But, for failsafe PMD ,= we see that both the=C2=A0number of Txd and Rxd is restricted to 8192.
How=C2=A0is netvsc PMD giving the same performance as failsafe PMD ?=

Regards

On Wed, Sep 4, 2024 at 7:30=E2=80=AF= PM Long Li <lo= ngli@microsoft.com> wrote:
> Subject: Re: Netvsc vs Failsafe Performance
>
> On Tue, 3 Sep 2024 17:21:48 -0700
> Nandini Rangaswamy <nandini.rangaswamy@broadcom.com> wrote:
>
> > Hi Stephen/Long,
> > dpdk_netvsc_port_configure:1873 Configure port eth2/2. I am testi= ng
> > using TCP traffic (iperf3 tool) generated between pair of client = and
> > servers with DPDK app forward traffic between client and servers.=
> > These are the config being passed for configuring netvsc:
> > lsc_intr=3D1
> > rxq/txq=3D2/2,
> > rss is enabled with rss_hf=3D0x0000000000000c30
> > tx_ol=3D0x00000000000006
> > rx_ol=3D0x00000000080007
> >
> > Rsskey len is 64.
> > struct rte_eth_conf conf =3D {
> > .intr_conf =3D {
> > .lsc =3D !dpdk.lsc_intr_disable && !dpdk_if->lsc_intr_= disable &&
> > !!(dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC), }, .rx= mode =3D {
> > .mq_mode =3D RTE_ETH_MQ_RX_RSS, .offloads =3D
> > RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
> RTE_ETH_RX_OFFLOAD_IPV4_CKSUM |
> > RTE_ETH_RX_OFFLOAD_RSS_HASH |
> RTE_ETH_RX_OFFLOAD_UDP_CKSUM, },
> > .rx_adv_conf.rss_conf =3D { .rss_hf =3D RTE_ETH_RSS_NONFRAG_IPV4_= UDP |
> > RTE_ETH_RSS_NONFRAG_IPV4_TCP |
> RTE_ETH_RSS_NONFRAG_IPV6_TCP, .rss_key
> > =3D conf_rss_key, .rss_key_len =3D rss_key_len, }, .txmode =3D { = .offloads =3D
> > RTE_ETH_TX_OFFLOAD_UDP_CKSUM |
> RTE_ETH_TX_OFFLOAD_IPV4_CKSUM, },
> >
> > Regards,
> > Nandini
> >
> > On Tue, Sep 3, 2024 at 5:03=E2=80=AFPM Stephen Hemminger
> > <stephen@networkplumber.org>
> > wrote:
> >
> > > On Tue, 3 Sep 2024 14:43:28 -0700
> > > Nandini Rangaswamy <nandini.rangaswamy@broadcom.com> wro= te:
> > >
> > > > Hi Stephen and Long,
> > > > I was going through one of the netvsc patches
> > > > https://nam0= 6.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2F
> > > > mails.dpdk.org%2Farchives%2Fdev%2F2018-
> August%2F110559.html&data=3D0
> > > >
> 5%7C02%7Clongli%40microsoft.com%7Ce91cca1ee99f4809138708dccd32e76
> 7
> > > > %7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638610865= 749
> 361006%7
> > > >
> CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB
> TiI
> > > >
> 6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=3Dms6yhMjBpT0IFu6e9
> wmh4K1
> > > > WDINzgjoRzFJmMJGJwuY%3D&reserved=3D0 which
> > > mentioned
> > > > that netvsc and failsafe give the same performance in V= F path
> > > > whereas for some exception path=C2=A0 tests, about 22% = performance gain in
> seen.
> > > > I ran some tests locally with my dpdk app integrated wi= th netvsc
> > > > PMD and observed that netvsc does give nearly the same = performance
> > > > as failsafe in the VF path.
> > > > Since the official document does not explicitly cite th= is, I would
> > > > like
> > > to
> > > > confirm if this holds good.
> > > > Regards,
> > > > Nandini
> > > >
> > >
> > > Shouldn't be. What settings are you using.
> > > Both failsafe and netvsc just pass packets to VF if present.=
> > > There is even more locks to go through with failsafe.
> > >
> > > Are you sure the test doesn't exercise something like ch= ecksumming
> > > which maybe different.
> > >
> >
>
> How many streams? RSS won't matter unless multiple streams.
> The netvsc driver does not have RSS for UDP as a listed flag.
> It turns out that for that version of NDIS, if you ask for TCP RSS, UD= P RSS is
> implied.
>
> RSS Key must be 40 bytes (Toeplitz) not 64 bytes.
> Just use the default key (rss_key =3D=3D NULL rss_key_len =3D 0) to be= safe
>
> Check that packets are going to the VF. One way to do that is to look = at xstats
> on both netvsc and mlx5 device.
>

If most traffic goes through the VF, you won't see much difference in p= erformance of netvsc vs failsafe because they are not used on the data path= .

It seems the 20% performance gain is measured on synthetic path, meaning th= ose traffic does not go through the VF. In this scenario, netvsc has an adv= antage over failsafe since the traffic data doesn't need to be copied a= round in the kernel space.

This ele= ctronic communication and the information and any files transmitted with it= , or attached to it, are confidential and are intended solely for the use o= f the individual or entity to whom it is addressed and may contain informat= ion that is confidential, legally privileged, protected by privacy laws, or= otherwise restricted from disclosure to anyone else. If you are not the in= tended recipient or the person responsible for delivering the e-mail to the= intended recipient, you are hereby notified that any use, copying, distrib= uting, dissemination, forwarding, printing, or copying of this e-mail is st= rictly prohibited. If you received this e-mail in error, please return the = e-mail to the sender, delete it from your computer, and destroy any printed= copy of it. --00000000000032f0670621f238c8--