DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] Weird 2 KB MBUF data room requirement
@ 2020-07-10  8:21 Morten Brørup
  2020-07-10 10:26 ` Bruce Richardson
  0 siblings, 1 reply; 4+ messages in thread
From: Morten Brørup @ 2020-07-10  8:21 UTC (permalink / raw)
  To: dev; +Cc: Olivier Matz

Dear Ethernet PMD developers,

According to rte_mbuf_core.h, RTE_MBUF_DEFAULT_DATAROOM is 2048 bytes because some NICs need at least 2 KB buffer to receive standard Ethernet frames without splitting them into multiple segments.

This is a serious waste of memory, considering that standard Ethernet frames are max 1518 bytes.

How wide spread is this limitation... is it common or a rare exception?

Where is it documented which NICs suffer from this limitation?

Do any Intel NICs suffer from this limitation?


NB: We are targeting an MBUF total size (incl. memzone element overhead) of 2^N, and this limitation would increase our MBUF total size to 4 KB.


Med venlig hilsen / kind regards
- Morten Brørup


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] Weird 2 KB MBUF data room requirement
  2020-07-10  8:21 [dpdk-dev] Weird 2 KB MBUF data room requirement Morten Brørup
@ 2020-07-10 10:26 ` Bruce Richardson
  2020-07-10 12:58   ` Olivier Matz
  0 siblings, 1 reply; 4+ messages in thread
From: Bruce Richardson @ 2020-07-10 10:26 UTC (permalink / raw)
  To: Morten Brørup; +Cc: dev, Olivier Matz

On Fri, Jul 10, 2020 at 10:21:40AM +0200, Morten Brørup wrote:
> Dear Ethernet PMD developers,
> 
> According to rte_mbuf_core.h, RTE_MBUF_DEFAULT_DATAROOM is 2048 bytes because some NICs need at least 2 KB buffer to receive standard Ethernet frames without splitting them into multiple segments.
> 
> This is a serious waste of memory, considering that standard Ethernet frames are max 1518 bytes.
> 
> How wide spread is this limitation... is it common or a rare exception?
> 
> Where is it documented which NICs suffer from this limitation?
> 
> Do any Intel NICs suffer from this limitation?
> 
> 
> NB: We are targeting an MBUF total size (incl. memzone element overhead) of 2^N, and this limitation would increase our MBUF total size to 4 KB.
> 
> 
> Med venlig hilsen / kind regards
> - Morten Brørup
> 

AFAIK: the NICs supported by the ixgbe driver only allow the size to be
specified in KB granularity.

However, it may be safe to have a driver modification whereby anything over
1600 bytes is considered as 2KB if jumbo frame support is disabled. I don't
think anyone has actually looked into doing so though, or if there are
other hidden gotchas about attempting to do so.

/Bruce

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] Weird 2 KB MBUF data room requirement
  2020-07-10 10:26 ` Bruce Richardson
@ 2020-07-10 12:58   ` Olivier Matz
  2020-07-10 13:45     ` Morten Brørup
  0 siblings, 1 reply; 4+ messages in thread
From: Olivier Matz @ 2020-07-10 12:58 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: Morten Brørup, dev

Hi,

On Fri, Jul 10, 2020 at 11:26:09AM +0100, Bruce Richardson wrote:
> On Fri, Jul 10, 2020 at 10:21:40AM +0200, Morten Brørup wrote:
> > Dear Ethernet PMD developers,
> > 
> > According to rte_mbuf_core.h, RTE_MBUF_DEFAULT_DATAROOM is 2048 bytes because some NICs need at least 2 KB buffer to receive standard Ethernet frames without splitting them into multiple segments.
> > 
> > This is a serious waste of memory, considering that standard Ethernet frames are max 1518 bytes.
> > 
> > How wide spread is this limitation... is it common or a rare exception?
> > 
> > Where is it documented which NICs suffer from this limitation?
> > 
> > Do any Intel NICs suffer from this limitation?
> > 
> > 
> > NB: We are targeting an MBUF total size (incl. memzone element overhead) of 2^N, and this limitation would increase our MBUF total size to 4 KB.
> > 
> > 
> > Med venlig hilsen / kind regards
> > - Morten Brørup
> > 
> 
> AFAIK: the NICs supported by the ixgbe driver only allow the size to be
> specified in KB granularity.
> 
> However, it may be safe to have a driver modification whereby anything over
> 1600 bytes is considered as 2KB if jumbo frame support is disabled. I don't
> think anyone has actually looked into doing so though, or if there are
> other hidden gotchas about attempting to do so.

If I remember well, the niantic NICs (and probably some others) can
have their rx size configured with 512, 1024, 2048, ...
This is the size that should be available from the given data pointer,
i.e. it does not include the headroom.

I suppose that if we configure the NIC with 2K but give less than 2K, the NIC
may write after the buffer when receiving a large packet.


Olivier

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [dpdk-dev] Weird 2 KB MBUF data room requirement
  2020-07-10 12:58   ` Olivier Matz
@ 2020-07-10 13:45     ` Morten Brørup
  0 siblings, 0 replies; 4+ messages in thread
From: Morten Brørup @ 2020-07-10 13:45 UTC (permalink / raw)
  To: Olivier Matz, Bruce Richardson; +Cc: dev

> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Matz
> Sent: Friday, July 10, 2020 2:58 PM
> 
> Hi,
> 
> On Fri, Jul 10, 2020 at 11:26:09AM +0100, Bruce Richardson wrote:
> > On Fri, Jul 10, 2020 at 10:21:40AM +0200, Morten Brørup wrote:
> > > Dear Ethernet PMD developers,
> > >
> > > According to rte_mbuf_core.h, RTE_MBUF_DEFAULT_DATAROOM is 2048
> bytes because some NICs need at least 2 KB buffer to receive standard
> Ethernet frames without splitting them into multiple segments.
> > >
> > > This is a serious waste of memory, considering that standard
> Ethernet frames are max 1518 bytes.
> > >
> > > How wide spread is this limitation... is it common or a rare
> exception?
> > >
> > > Where is it documented which NICs suffer from this limitation?
> > >
> > > Do any Intel NICs suffer from this limitation?
> > >
> > >
> > > NB: We are targeting an MBUF total size (incl. memzone element
> overhead) of 2^N, and this limitation would increase our MBUF total
> size to 4 KB.
> > >
> > >
> > > Med venlig hilsen / kind regards
> > > - Morten Brørup
> > >
> >
> > AFAIK: the NICs supported by the ixgbe driver only allow the size to
> be
> > specified in KB granularity.
> >
> > However, it may be safe to have a driver modification whereby
> anything over
> > 1600 bytes is considered as 2KB if jumbo frame support is disabled. I
> don't
> > think anyone has actually looked into doing so though, or if there
> are
> > other hidden gotchas about attempting to do so.
> 
> If I remember well, the niantic NICs (and probably some others) can
> have their rx size configured with 512, 1024, 2048, ...
> This is the size that should be available from the given data pointer,
> i.e. it does not include the headroom.
> 
> I suppose that if we configure the NIC with 2K but give less than 2K,
> the NIC
> may write after the buffer when receiving a large packet.
> 
> 
> Olivier

Thanks for the quick and detailed replies, Bruce and Olivier.

In conclusion, this NIC limitation seems more likely than unlikely.

And a workaround is risky business.

We will abandon the idea of getting an 2^N total size MBUF. It's not a big deal.



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-07-10 13:46 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-10  8:21 [dpdk-dev] Weird 2 KB MBUF data room requirement Morten Brørup
2020-07-10 10:26 ` Bruce Richardson
2020-07-10 12:58   ` Olivier Matz
2020-07-10 13:45     ` Morten Brørup

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).