[PATCH] net/i40e: increase descriptor queue length to 8160

[PATCH] net/i40e: increase descriptor queue length to 8160

[Igor Gutorov]

According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
as QLEN in table 8-12, page 1083.

I've tested this change with an XXV710 NIC and it has positive effect on
performance under high load scenarios. Where previously I'd get
~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.

Signed-off-by: Igor Gutorov <igootorov@gmail.com>
---
 drivers/net/i40e/i40e_rxtx.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
index 2f2f890855..33fc9770d9 100644
--- a/drivers/net/i40e/i40e_rxtx.h
+++ b/drivers/net/i40e/i40e_rxtx.h
@@ -25,7 +25,7 @@
 #define I40E_RX_MAX_DATA_BUF_SIZE	(16 * 1024 - 128)
 
 #define	I40E_MIN_RING_DESC	64
-#define	I40E_MAX_RING_DESC	4096
+#define	I40E_MAX_RING_DESC	8160
 
 #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
 #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
-- 
2.45.1

Re: [PATCH] net/i40e: increase descriptor queue length to 8160

[Igor Gutorov]

On Mon, May 27, 2024 at 7:20 PM Igor Gutorov <igootorov@gmail.com> wrote:
>
> According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> as QLEN in table 8-12, page 1083.
>
> I've tested this change with an XXV710 NIC and it has positive effect on
> performance under high load scenarios. Where previously I'd get
> ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
>
> Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> ---
>  drivers/net/i40e/i40e_rxtx.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> index 2f2f890855..33fc9770d9 100644
> --- a/drivers/net/i40e/i40e_rxtx.h
> +++ b/drivers/net/i40e/i40e_rxtx.h
> @@ -25,7 +25,7 @@
>  #define I40E_RX_MAX_DATA_BUF_SIZE      (16 * 1024 - 128)
>
>  #define        I40E_MIN_RING_DESC      64
> -#define        I40E_MAX_RING_DESC      4096
> +#define        I40E_MAX_RING_DESC      8160
>
>  #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
>  #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> --
> 2.45.1
>

CC'ing to a different email (got an address rejected error, the
MAINTAINERS file might need an update).

Also, I've noticed that an X722 NIC is also supported by the i40e
driver, but it seems the datasheet isn't on Intel's website for that
NIC. Does anybody know if X722 has 4096 or 8160 usable descriptors? If
someone could test this patch against X722, that would be great.

Re: [PATCH] net/i40e: increase descriptor queue length to 8160

[Igor Gutorov]

On Mon, Jun 3, 2024 at 1:41 PM Igor Gutorov <igootorov@gmail.com> wrote:
>
> On Mon, May 27, 2024 at 7:20 PM Igor Gutorov <igootorov@gmail.com> wrote:
> >
> > According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> > queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> > as QLEN in table 8-12, page 1083.
> >
> > I've tested this change with an XXV710 NIC and it has positive effect on
> > performance under high load scenarios. Where previously I'd get
> > ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> >
> > Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> > ---
> >  drivers/net/i40e/i40e_rxtx.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> > index 2f2f890855..33fc9770d9 100644
> > --- a/drivers/net/i40e/i40e_rxtx.h
> > +++ b/drivers/net/i40e/i40e_rxtx.h
> > @@ -25,7 +25,7 @@
> >  #define I40E_RX_MAX_DATA_BUF_SIZE      (16 * 1024 - 128)
> >
> >  #define        I40E_MIN_RING_DESC      64
> > -#define        I40E_MAX_RING_DESC      4096
> > +#define        I40E_MAX_RING_DESC      8160
> >
> >  #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> >  #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> > --
> > 2.45.1
> >
>
> CC'ing to a different email (got an address rejected error, the
> MAINTAINERS file might need an update).
>
> Also, I've noticed that an X722 NIC is also supported by the i40e
> driver, but it seems the datasheet isn't on Intel's website for that
> NIC. Does anybody know if X722 has 4096 or 8160 usable descriptors? If
> someone could test this patch against X722, that would be great.

Apologies for pinging again. Who should I send my patch to? The
MAINTAINERS file shows Yuying Zhang as the maintainer of the net/i40e
driver and contains 2 emails, but both Yuying.Zhang@intel.com and
yuying.zhang@intel.com emails are non-existent (550 Address rejected).

RE: [PATCH] net/i40e: increase descriptor queue length to 8160

[Morten Brørup]

> From: Igor Gutorov [mailto:igootorov@gmail.com]
> Sent: Wednesday, 5 June 2024 12.47
> 
> On Mon, Jun 3, 2024 at 1:41 PM Igor Gutorov <igootorov@gmail.com> wrote:
> >
> > On Mon, May 27, 2024 at 7:20 PM Igor Gutorov <igootorov@gmail.com> wrote:
> > >
> > > According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> > > queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> > > as QLEN in table 8-12, page 1083.
> > >
> > > I've tested this change with an XXV710 NIC and it has positive effect on
> > > performance under high load scenarios. Where previously I'd get
> > > ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> > >
> > > Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> > > ---
> > >  drivers/net/i40e/i40e_rxtx.h | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> > > index 2f2f890855..33fc9770d9 100644
> > > --- a/drivers/net/i40e/i40e_rxtx.h
> > > +++ b/drivers/net/i40e/i40e_rxtx.h
> > > @@ -25,7 +25,7 @@
> > >  #define I40E_RX_MAX_DATA_BUF_SIZE      (16 * 1024 - 128)
> > >
> > >  #define        I40E_MIN_RING_DESC      64
> > > -#define        I40E_MAX_RING_DESC      4096
> > > +#define        I40E_MAX_RING_DESC      8160
> > >
> > >  #define I40E_FDIR_NUM_TX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> > >  #define I40E_FDIR_NUM_RX_DESC   (I40E_FDIR_PRG_PKT_CNT << 1)
> > > --
> > > 2.45.1
> > >
> >
> > CC'ing to a different email (got an address rejected error, the
> > MAINTAINERS file might need an update).
> >
> > Also, I've noticed that an X722 NIC is also supported by the i40e
> > driver, but it seems the datasheet isn't on Intel's website for that
> > NIC. Does anybody know if X722 has 4096 or 8160 usable descriptors?

It does.
Refer to table 38-419, page 3086 in the Intel C620 Chipset datasheet:
https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/c620-series-chipset-datasheet.pdf

> > If
> > someone could test this patch against X722, that would be great.

I don't have one, so I cannot test it. But:
Acked-by: Morten Brørup <mb@smartsharesystems.com>

> 
> Apologies for pinging again. Who should I send my patch to? The
> MAINTAINERS file shows Yuying Zhang as the maintainer of the net/i40e
> driver and contains 2 emails, but both Yuying.Zhang@intel.com and
> yuying.zhang@intel.com emails are non-existent (550 Address rejected).

The i40e DPDK driver is currently without maintainers.
Sending the patch to the dev@dpdk.org mailing list suffices.

Re: [PATCH] net/i40e: increase descriptor queue length to 8160

[Bruce Richardson]

On Mon, May 27, 2024 at 07:19:21PM +0300, Igor Gutorov wrote:
> According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> as QLEN in table 8-12, page 1083.
> 
> I've tested this change with an XXV710 NIC and it has positive effect on
> performance under high load scenarios. Where previously I'd get
> ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> 
> Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> ---
>  drivers/net/i40e/i40e_rxtx.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> index 2f2f890855..33fc9770d9 100644
> --- a/drivers/net/i40e/i40e_rxtx.h
> +++ b/drivers/net/i40e/i40e_rxtx.h
> @@ -25,7 +25,7 @@
>  #define I40E_RX_MAX_DATA_BUF_SIZE	(16 * 1024 - 128)
>  
>  #define	I40E_MIN_RING_DESC	64
> -#define	I40E_MAX_RING_DESC	4096
> +#define	I40E_MAX_RING_DESC	8160
>  

since the ring memory allocation in the driver always seems to allocate
space for up to the max number of descriptors, this will lead to an
increase of memory footprint of ~128k per Rx Q, and ~64k per Tx Q. However,
I'd view this as relatively harmless given what we have now. If it is an
issue, we should look to fix it in a separate patch, rather than blocking
this simple change.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Re: [PATCH] net/i40e: increase descriptor queue length to 8160

[Bruce Richardson]

On Tue, Jun 18, 2024 at 01:12:13PM +0100, Bruce Richardson wrote:
> On Mon, May 27, 2024 at 07:19:21PM +0300, Igor Gutorov wrote:
> > According to the Intel X710/XXV710/XL710 Datasheet, the maximum receive
> > queue descriptor length is 0x1FE0 (8160 in base 10). This is specified
> > as QLEN in table 8-12, page 1083.
> > 
> > I've tested this change with an XXV710 NIC and it has positive effect on
> > performance under high load scenarios. Where previously I'd get
> > ~2000 packets/sec miss rate, now I get only ~40 packets/sec miss rate.
> > 
> > Signed-off-by: Igor Gutorov <igootorov@gmail.com>
> > ---
> >  drivers/net/i40e/i40e_rxtx.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
> > index 2f2f890855..33fc9770d9 100644
> > --- a/drivers/net/i40e/i40e_rxtx.h
> > +++ b/drivers/net/i40e/i40e_rxtx.h
> > @@ -25,7 +25,7 @@
> >  #define I40E_RX_MAX_DATA_BUF_SIZE	(16 * 1024 - 128)
> >  
> >  #define	I40E_MIN_RING_DESC	64
> > -#define	I40E_MAX_RING_DESC	4096
> > +#define	I40E_MAX_RING_DESC	8160
> >  
> 
> since the ring memory allocation in the driver always seems to allocate
> space for up to the max number of descriptors, this will lead to an
> increase of memory footprint of ~128k per Rx Q, and ~64k per Tx Q. However,
> I'd view this as relatively harmless given what we have now. If it is an
> issue, we should look to fix it in a separate patch, rather than blocking
> this simple change.
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Patch applied to dpdk-next-net-intel,

Thanks,
/Bruce