DPDK usage discussions
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>, Jiany Wu <wujianyue000@gmail.com>
Cc: "users@dpdk.org" <users@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	"dmitry.kozliuk@gmail.com" <dmitry.kozliuk@gmail.com>,
	"stephen@networkplumber.org" <stephen@networkplumber.org>,
	"Mcnamara, John" <john.mcnamara@intel.com>
Subject: RE: [dpdk-users] can we reserve hugepage and not release
Date: Wed, 29 Sep 2021 12:40:33 +0000
Message-ID: <PH0PR11MB50935FEA5233F5DC875B99A3F7A99@PH0PR11MB5093.namprd11.prod.outlook.com> (raw)
In-Reply-To: <7391021.896Q4A1nnV@thomas>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Wednesday, September 29, 2021 12:11 PM
> To: Jiany Wu <wujianyue000@gmail.com>; Burakov, Anatoly
> <anatoly.burakov@intel.com>
> Cc: users@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> olivier.matz@6wind.com; dmitry.kozliuk@gmail.com;
> stephen@networkplumber.org; Mcnamara, John
> <john.mcnamara@intel.com>
> Subject: Re: [dpdk-users] can we reserve hugepage and not release
> 29/09/2021 12:16, Burakov, Anatoly:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 18/09/2021 04:37, Jiany Wu:
> > > > Hello,
> > > >
> > > > I met a scenario that, need to start and stop the container many
> > > > times for the hugepage. But after several times container start
> > > > and stop, the hugepage is not able to reserve.
> > > > Hugepage size is 2MB, and HW only support 2MB, can't support 1GB.
> > > > Is there anyway to make sure the hugepage is still kept continuous?
> > > > Thanks indeed.
> > >
> > > Interesting question.
> > > I think we need to address it in the DPDK documentation.
> > >
> > > Anatoly, Stephen, Bruce, any advice please?
> > >
> >
> > Hi,
> >
> > From description, I don't quite understand what's the issue here. Is the
> problem about "contiguousness of memory", or is it about inability to reserve
> more hugepages?
> I think the issue is that sometimes some pages are not properly released, so
> we cannot reserve them again.
> That's something I experienced myself.
> Any trick to reset hugepages state?

Under normal circumstances, the hugepage files would just be deleted and recreated the next time a primary process initializes even if there are leftover pages.

That said, assuming the pages are not "released" because DPDK has left a few files behind and a new container doesn't have permissions to delete them (or can't for some other reason), then there is very little DPDK can do other than track down all memory leaks. For example, using RX callback feature is intentionally causing a memory leak because by default the API will not free the callback structure as we cannot guarantee that there are no Dataplane threads still using that structure. Individual drivers/libraries might also leak memory due to e.g. secondary process usage. For example, in the general case, if something is meant to be shared by primary and secondary, but primary cannot free() it because it might still be used by secondaries, and secondaries have no way of verifying whether the resource is being used by other processes, or can be freed.

Put it simply, sometimes DPDK may leak memory either out of necessity, or even intentionally, to avoid potential use-after-free errors, and there's no general way to solve this problem that I'm aware of. This is the kind of stuff --in-memory mode was introduced to address, as when hugepages are in memory, there can never be leaks because we never touch the filesystem in the first place, so if there is a recommended way to address this at all, it would be --in-memory mode. 

> > How are hugepages assigned to your container?
> > Have you tried using --in-memory mode?

  reply	other threads:[~2021-09-29 12:40 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-18  2:37 Jiany Wu
2021-09-29 10:04 ` Thomas Monjalon
2021-09-29 10:16   ` Burakov, Anatoly
2021-09-29 11:11     ` Thomas Monjalon
2021-09-29 12:40       ` Burakov, Anatoly [this message]
2021-10-05 15:56         ` Jiany Wu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=PH0PR11MB50935FEA5233F5DC875B99A3F7A99@PH0PR11MB5093.namprd11.prod.outlook.com \
    --to=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dmitry.kozliuk@gmail.com \
    --cc=john.mcnamara@intel.com \
    --cc=olivier.matz@6wind.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=users@dpdk.org \
    --cc=wujianyue000@gmail.com \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK usage discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
	public-inbox-index users

Example config snippet for mirrors.
Newsgroup available over NNTP:

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git