DPDK patches and discussions
 help / color / mirror / Atom feed
From: Thomas Monjalon <thomas@monjalon.net>
To: "Mattias Rönnblom" <hofors@lysator.liu.se>
Cc: "David Marchand" <david.marchand@redhat.com>,
	"Morten Brørup" <mb@smartsharesystems.com>,
	dev@dpdk.org, "Bruce Richardson" <bruce.richardson@intel.com>,
	"Stephen Hemminger" <stephen@networkplumber.org>,
	"Chengwen Feng" <fengchengwen@huawei.com>,
	"Konstantin Ananyev" <konstantin.ananyev@huawei.com>
Subject: Re: [PATCH] config: limit lcore variable maximum size to 4k
Date: Sat, 09 Nov 2024 00:11:14 +0100	[thread overview]
Message-ID: <6948023.R56niFO833@thomas> (raw)
In-Reply-To: <ffef5836-7ee1-44d0-8e06-cb058acd66cc@lysator.liu.se>

08/11/2024 23:34, Mattias Rönnblom:
> On 2024-11-08 23:13, Thomas Monjalon wrote:
> > 08/11/2024 20:53, Morten Brørup:
> >>> From: Morten Brørup [mailto:mb@smartsharesystems.com]
> >>> Sent: Friday, 8 November 2024 19.35
> >>>
> >>>> From: David Marchand [mailto:david.marchand@redhat.com]
> >>>> Sent: Friday, 8 November 2024 19.18
> >>>>
> >>>> OVS locks all pages to avoid page faults while processing packets.
> >>
> >> It sounds smart, so I just took a look at how it does this. I'm not sure, but it seems like it only locks pages that are actually mapped (current and future).
> >>
> >>>> 1M for each lcore translates to allocating 128M with default build
> >>>> options on x86.
> >>>> This resulted in OOM while running unit tests in parallel.
> >>
> >> Is the root cause the lcore variables library itself, or the unit test using a lot of memory for testing the lcore variables?
> >> We don't want to fix the library if the problem is elsewhere.
> > 
> > The fix works for our urgent issue and we want to make a release candidate soon.
> > 
> > 
> >>>> At the moment, the more demanding DPDK user of lcore variable is
> >>>> rte_service, with a 2112 bytes object.
> >>>>
> >>>> Limit the lcore variable maximum size to 4k which looks more
> >>>> reasonable.
> >>>
> >>> 4 KB is not future proof.
> >>>
> >>> Here's an example where 16 KB is cutting it close:
> >>> https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9F7D0@smart
> >>> server.smartshare.dk/
> >>>
> >>> Depends on how we are going to use it. 4 KB suffices if we only want to
> >>> use it for "small" structures.
> > 
> > This is what is stated in the doc:
> > "Lcore variables are suitable for small objects"
> > "The amount of data kept in lcore variables is projected to be small"
> > >>> Would 64 KB work as a compromise?
> > 
> > Let's consider based on the need.
> > The lcore variables are new and we don't want it to degrade the DPDK footprint,
> > at least not in this first version.
> > 4 KB is a memory page on common systems,
> > it looks reasonnable and big enough for a "variable".
> > 
> > Applied, thanks.
> 
> Why do you have maintainers if you just ignore them?

I didn't receive your replies when I started to write this.
Please be comprehensive.
We are in a hurry to stabilize this before the release candidate which is already late.

I'll change to 128 KB as you recommend before pushing to the repository.

PS: maybe I should not have merged this feature in 24.11.





  reply	other threads:[~2024-11-08 23:11 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-08 18:17 David Marchand
2024-11-08 18:35 ` Morten Brørup
2024-11-08 19:53   ` Morten Brørup
2024-11-08 22:13     ` Thomas Monjalon
2024-11-08 22:34       ` Mattias Rönnblom
2024-11-08 23:11         ` Thomas Monjalon [this message]
2024-11-11  6:31           ` Mattias Rönnblom
2024-11-08 22:49       ` Morten Brørup
2024-11-08 22:23     ` Mattias Rönnblom
2024-11-08 23:52       ` Morten Brørup
2024-11-11  7:22         ` Mattias Rönnblom
2024-11-11 16:54           ` Stephen Hemminger
2024-11-08 22:02 ` Mattias Rönnblom

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6948023.R56niFO833@thomas \
    --to=thomas@monjalon.net \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=fengchengwen@huawei.com \
    --cc=hofors@lysator.liu.se \
    --cc=konstantin.ananyev@huawei.com \
    --cc=mb@smartsharesystems.com \
    --cc=stephen@networkplumber.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).