From: Bruce Richardson <bruce.richardson@intel.com>
To: "Morten Brørup" <mb@smartsharesystems.com>
Cc: Patrick Robb <probb@iol.unh.edu>,
Nicholas Pratte <npratte@iol.unh.edu>, <paul.szczepanek@arm.com>,
<juraj.linkes@pantheon.tech>, <yoan.picchi@foss.arm.com>,
<thomas@monjalon.net>, <wathsala.vithanage@arm.com>,
<Honnappa.Nagarahalli@arm.com>, <dev@dpdk.org>,
Jeremy Spewock <jspewock@iol.unh.edu>
Subject: Re: [PATCH] dts: Change hugepage runtime config to 2MB Exclusively
Date: Mon, 8 Apr 2024 09:10:38 +0100 [thread overview]
Message-ID: <ZhOmfvfYaqWw1XEL@bricha3-mobl1.ger.corp.intel.com> (raw)
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F376@smartserver.smartshare.dk>
On Sun, Apr 07, 2024 at 12:04:24AM +0200, Morten Brørup wrote:
> > From: Patrick Robb [mailto:probb@iol.unh.edu]
> > Sent: Saturday, 6 April 2024 21.21
> >
> > On Sat, Apr 6, 2024 at 4:47 AM Morten Brørup <mb@smartsharesystems.com>
> > wrote:
> > >
> > >
> > > This change seems very CPU specific.
> > >
> > > E.g. in x86 32-bit mode, the hugepage size is 4 MB, not 2 MB.
> > >
> > > I don't know the recommended hugepage size on other architectures.
> > >
> >
> > Thanks Morten, that's an important insight which we weren't aware of
> > when we initially discussed this ticket.
> >
> > We read on some dpdk docs that 1gb hugepages should be set at boot (I
> > think the reason is because that's when you can guarantee there is gbs
> > of contiguous available memory), and that after boot, only 2gb
> > hugepages should be set. I assume that even for other arches which
> > don't support the 2mb pages specifically, we still want to allocate
> > hugepages using the smallest page size possible per arch (for the same
> > reason).
>
> Correct; for very large hugepages, they need to be set at boot.
> I don't remember why, but that's the way the Linux kernel works.
> 2 MB is way below that threshold, and 1 GB is above.
>
> You can also not set nr_overcommit_hugepages for those very large hugepages, only nr_hugepages.
>
> >
> > So I think we can add some dict which stores the smallest valid
> > hugepage size per arch. Then during config init, use the config's arch
> > value to determine that size, and set the total hugepages allocation
> > based on that size and the hugepages count set in the conf.yaml. Or
> > maybe we can store the list of all valid hugepgage sizes per arch
> > (which are also valid to be set post boot), allow for a new
> > hugepage_size value on the conf.yaml, validate the input at config
> > init, and then set according to those values. I prefer the former
> > option though as I don't think the added flexibility offered by the
> > latter seems important.
>
> I agree; the former option suffices.
>
> A tiny detail...
> ARM supports multiple (4, I think) different hugepage sizes, where the smallest size is 64 KB.
> So you might want to choose another hugepage size than the smallest; but I still agree with your proposed concept of using one specific hugepage size per arch.
>
> Like x86_64, ARM also supports 2 MB and 1 GB hugepage sizes, and 2 MB hugepages is also the typical default on Linux.
>
> I don't know which hugepage sizes are supported by other architectures.
> It might only be 32-bit x86 that needs a different hugepage size than 2 MB.
>
Even for 32-bit x86, I think most distros now use PAE mode to allow physical
addresses >4GB, so even then 32-bit hugepages are 2MB rather than 4MB.
/Bruce
next prev parent reply other threads:[~2024-04-08 8:11 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-04 15:31 Nicholas Pratte
2024-04-05 14:27 ` Patrick Robb
2024-04-06 8:47 ` Morten Brørup
2024-04-06 19:20 ` Patrick Robb
2024-04-06 22:04 ` Morten Brørup
2024-04-08 8:10 ` Bruce Richardson [this message]
2024-04-08 9:35 ` Morten Brørup
2024-04-09 16:57 ` Nicholas Pratte
2024-04-09 17:28 ` [PATCH v2] " Nicholas Pratte
2024-04-10 7:23 ` Morten Brørup
2024-04-10 13:50 ` Patrick Robb
2024-04-10 10:43 ` Juraj Linkeš
2024-04-16 18:18 ` [PATCH v3] " Nicholas Pratte
2024-04-16 18:25 ` Morten Brørup
2024-04-16 18:26 ` Nicholas Pratte
2024-04-17 13:46 ` Juraj Linkeš
2024-04-18 16:10 ` [PATCH v4] " Nicholas Pratte
2024-04-25 8:00 ` Juraj Linkeš
2024-04-29 17:26 ` Nicholas Pratte
2024-04-30 8:42 ` Juraj Linkeš
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZhOmfvfYaqWw1XEL@bricha3-mobl1.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=dev@dpdk.org \
--cc=jspewock@iol.unh.edu \
--cc=juraj.linkes@pantheon.tech \
--cc=mb@smartsharesystems.com \
--cc=npratte@iol.unh.edu \
--cc=paul.szczepanek@arm.com \
--cc=probb@iol.unh.edu \
--cc=thomas@monjalon.net \
--cc=wathsala.vithanage@arm.com \
--cc=yoan.picchi@foss.arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).