[dpdk-dev] logs about hugepages detection

[dpdk-dev] logs about hugepages detection

[Thomas Monjalon]

Hi,

I would like to discuss some issues in logging of hugepage lookup.
The issues to be discussed will be enumerated and numbered below.
I will take an example of an x86 machine with 2M and 1G pages.
I reserve only 2M pages:

	usertools/dpdk-hugepages.py -p 2M -r 80M

If I start a DPDK application with --log-level info
the only message I read makes me think something is wrong:

	EAL: No available 1048576 kB hugepages reported

1/ Log level is too high.

If I start with EAL in debug level, I can see which page size is used:

	--log-level debug --log-level lib.eal:debug

	EAL: No available 1048576 kB hugepages reported
	[...]
	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152

2/ The positive message should be at the same level as the negative one.

3/ The sizes are sometimes written in bytes, sometimes in kB.
It should be always the highest unit, including GB.

When using the --in-memory mode, things are worst:

	EAL: No available 1048576 kB hugepages reported
	EAL: In-memory mode enabled, hugepages of size 1073741824 bytes will be allocated anonymously
	EAL: No free 1048576 kB hugepages reported on node 0
	EAL: No available 1048576 kB hugepages reported
	[...]
	EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152

4/ The unavailability of 1G should be reported only once.

5/ If non-reserved pages can be used without reservation, it should be better documented.

Please correct me if I'm wrong, and give your opinion.
I could work on some patches if needed.

Re: [dpdk-dev] logs about hugepages detection

[Bruce Richardson]

On Wed, Sep 15, 2021 at 03:52:35PM +0200, Thomas Monjalon wrote:
> Hi,
> 
> I would like to discuss some issues in logging of hugepage lookup.
> The issues to be discussed will be enumerated and numbered below.
> I will take an example of an x86 machine with 2M and 1G pages.
> I reserve only 2M pages:
> 
> 	usertools/dpdk-hugepages.py -p 2M -r 80M
> 
> If I start a DPDK application with --log-level info
> the only message I read makes me think something is wrong:
> 
> 	EAL: No available 1048576 kB hugepages reported
> 
> 1/ Log level is too high.
> 

Agreed.

> If I start with EAL in debug level, I can see which page size is used:
> 
> 	--log-level debug --log-level lib.eal:debug
> 
> 	EAL: No available 1048576 kB hugepages reported
> 	[...]
> 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> 
> 2/ The positive message should be at the same level as the negative one.
> 

A bit uncertain about this, as I think it need not always be the case. I
think the log messages should be assessed independently.

> 3/ The sizes are sometimes written in bytes, sometimes in kB.
> It should be always the highest unit, including GB.
> 
> When using the --in-memory mode, things are worst:
> 
> 	EAL: No available 1048576 kB hugepages reported
> 	EAL: In-memory mode enabled, hugepages of size 1073741824 bytes will be allocated anonymously
> 	EAL: No free 1048576 kB hugepages reported on node 0
> 	EAL: No available 1048576 kB hugepages reported
> 	[...]
> 	EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> 

Yes, things should be consistent, having highest units is nice-to-have. If
everything is consistently reported in KB or MB it's probably fine.

> 4/ The unavailability of 1G should be reported only once.
> 
I'd actually suggest that the unavailability of 1G pages should not be
reported at all if 2MB pages are available. If we imagine a hypothetical
architecture with 15 hugepage sizes, if more than enough memory is
available for DPDK use via one page size, would we really want to know or
care about the fact that 14 page sizes are unavailable?

> 5/ If non-reserved pages can be used without reservation, it should be better documented.
> 
> Please correct me if I'm wrong, and give your opinion.
> I could work on some patches if needed.
> 

Regards,
/Bruce

Re: [dpdk-dev] logs about hugepages detection

[Thomas Monjalon]

15/09/2021 16:25, Bruce Richardson:
> On Wed, Sep 15, 2021 at 03:52:35PM +0200, Thomas Monjalon wrote:
> > Hi,
> > 
> > I would like to discuss some issues in logging of hugepage lookup.
> > The issues to be discussed will be enumerated and numbered below.
> > I will take an example of an x86 machine with 2M and 1G pages.
> > I reserve only 2M pages:
> > 
> > 	usertools/dpdk-hugepages.py -p 2M -r 80M
> > 
> > If I start a DPDK application with --log-level info
> > the only message I read makes me think something is wrong:
> > 
> > 	EAL: No available 1048576 kB hugepages reported
> > 
> > 1/ Log level is too high.
> > 
> 
> Agreed.
> 
> > If I start with EAL in debug level, I can see which page size is used:
> > 
> > 	--log-level debug --log-level lib.eal:debug
> > 
> > 	EAL: No available 1048576 kB hugepages reported
> > 	[...]
> > 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> > 
> > 2/ The positive message should be at the same level as the negative one.
> 
> A bit uncertain about this, as I think it need not always be the case. I
> think the log messages should be assessed independently.

Not sure what you mean. Which level for which message?

> > 3/ The sizes are sometimes written in bytes, sometimes in kB.
> > It should be always the highest unit, including GB.
> > 
> > When using the --in-memory mode, things are worst:
> > 
> > 	EAL: No available 1048576 kB hugepages reported
> > 	EAL: In-memory mode enabled, hugepages of size 1073741824 bytes will be allocated anonymously
> > 	EAL: No free 1048576 kB hugepages reported on node 0
> > 	EAL: No available 1048576 kB hugepages reported
> > 	[...]
> > 	EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> > 
> 
> Yes, things should be consistent, having highest units is nice-to-have. If
> everything is consistently reported in KB or MB it's probably fine.

Fine but not nice :)
I'm looking to improve the user experience, so "1GB" is definitely easier
to read than "1048576 kB", not talking about "1073741824".

> > 4/ The unavailability of 1G should be reported only once.
> > 
> I'd actually suggest that the unavailability of 1G pages should not be
> reported at all if 2MB pages are available. If we imagine a hypothetical
> architecture with 15 hugepage sizes, if more than enough memory is
> available for DPDK use via one page size, would we really want to know or
> care about the fact that 14 page sizes are unavailable?

I agree.

> > 5/ If non-reserved pages can be used without reservation, it should be better documented.
> > 
> > Please correct me if I'm wrong, and give your opinion.
> > I could work on some patches if needed.

Re: [dpdk-dev] logs about hugepages detection

[Bruce Richardson]

On Wed, Sep 15, 2021 at 04:39:21PM +0200, Thomas Monjalon wrote:
> 15/09/2021 16:25, Bruce Richardson:
> > On Wed, Sep 15, 2021 at 03:52:35PM +0200, Thomas Monjalon wrote:
> > > Hi,
> > > 
> > > I would like to discuss some issues in logging of hugepage lookup.
> > > The issues to be discussed will be enumerated and numbered below.
> > > I will take an example of an x86 machine with 2M and 1G pages.
> > > I reserve only 2M pages:
> > > 
> > > 	usertools/dpdk-hugepages.py -p 2M -r 80M
> > > 
> > > If I start a DPDK application with --log-level info
> > > the only message I read makes me think something is wrong:
> > > 
> > > 	EAL: No available 1048576 kB hugepages reported
> > > 
> > > 1/ Log level is too high.
> > > 
> > 
> > Agreed.
> > 
> > > If I start with EAL in debug level, I can see which page size is used:
> > > 
> > > 	--log-level debug --log-level lib.eal:debug
> > > 
> > > 	EAL: No available 1048576 kB hugepages reported
> > > 	[...]
> > > 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> > > 
> > > 2/ The positive message should be at the same level as the negative one.
> > 
> > A bit uncertain about this, as I think it need not always be the case. I
> > think the log messages should be assessed independently.
> 
> Not sure what you mean. Which level for which message?
> 

I mean the positive and negative log messages. I would assess the positive
log level independently of what the log level chosen for the negative one,
rather than saying they should be at the same level.

> > > 3/ The sizes are sometimes written in bytes, sometimes in kB.
> > > It should be always the highest unit, including GB.
> > > 
> > > When using the --in-memory mode, things are worst:
> > > 
> > > 	EAL: No available 1048576 kB hugepages reported
> > > 	EAL: In-memory mode enabled, hugepages of size 1073741824 bytes will be allocated anonymously
> > > 	EAL: No free 1048576 kB hugepages reported on node 0
> > > 	EAL: No available 1048576 kB hugepages reported
> > > 	[...]
> > > 	EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > > 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> > > 
> > 
> > Yes, things should be consistent, having highest units is nice-to-have. If
> > everything is consistently reported in KB or MB it's probably fine.
> 
> Fine but not nice :)
> I'm looking to improve the user experience, so "1GB" is definitely easier
> to read than "1048576 kB", not talking about "1073741824".
> 

Yes, agreed. The one small advantage of always just reporting in kB is
that it is the units used by the kernel in reporting the page sizes:

  $ ls /sys/kernel/mm/hugepages/
  hugepages-1048576kB  hugepages-2048kB

/Bruce

Re: [dpdk-dev] logs about hugepages detection

[Stephen Hemminger]

On Wed, 15 Sep 2021 15:59:38 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Wed, Sep 15, 2021 at 04:39:21PM +0200, Thomas Monjalon wrote:
> > 15/09/2021 16:25, Bruce Richardson:  
> > > On Wed, Sep 15, 2021 at 03:52:35PM +0200, Thomas Monjalon wrote:  
> > > > Hi,
> > > > 
> > > > I would like to discuss some issues in logging of hugepage lookup.
> > > > The issues to be discussed will be enumerated and numbered below.
> > > > I will take an example of an x86 machine with 2M and 1G pages.
> > > > I reserve only 2M pages:
> > > > 
> > > > 	usertools/dpdk-hugepages.py -p 2M -r 80M
> > > > 
> > > > If I start a DPDK application with --log-level info
> > > > the only message I read makes me think something is wrong:
> > > > 
> > > > 	EAL: No available 1048576 kB hugepages reported
> > > > 
> > > > 1/ Log level is too high.
> > > >   
> > > 
> > > Agreed.
> > >   
> > > > If I start with EAL in debug level, I can see which page size is used:
> > > > 
> > > > 	--log-level debug --log-level lib.eal:debug
> > > > 
> > > > 	EAL: No available 1048576 kB hugepages reported
> > > > 	[...]
> > > > 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> > > > 
> > > > 2/ The positive message should be at the same level as the negative one.  
> > > 
> > > A bit uncertain about this, as I think it need not always be the case. I
> > > think the log messages should be assessed independently.  
> > 
> > Not sure what you mean. Which level for which message?
> >   
> 
> I mean the positive and negative log messages. I would assess the positive
> log level independently of what the log level chosen for the negative one,
> rather than saying they should be at the same level.
> 
> > > > 3/ The sizes are sometimes written in bytes, sometimes in kB.
> > > > It should be always the highest unit, including GB.
> > > > 
> > > > When using the --in-memory mode, things are worst:
> > > > 
> > > > 	EAL: No available 1048576 kB hugepages reported
> > > > 	EAL: In-memory mode enabled, hugepages of size 1073741824 bytes will be allocated anonymously
> > > > 	EAL: No free 1048576 kB hugepages reported on node 0
> > > > 	EAL: No available 1048576 kB hugepages reported
> > > > 	[...]
> > > > 	EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
> > > > 	EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
> > > >   
> > > 
> > > Yes, things should be consistent, having highest units is nice-to-have. If
> > > everything is consistently reported in KB or MB it's probably fine.  
> > 
> > Fine but not nice :)
> > I'm looking to improve the user experience, so "1GB" is definitely easier
> > to read than "1048576 kB", not talking about "1073741824".
> >   
> 
> Yes, agreed. The one small advantage of always just reporting in kB is
> that it is the units used by the kernel in reporting the page sizes:
> 
>   $ ls /sys/kernel/mm/hugepages/
>   hugepages-1048576kB  hugepages-2048kB

Agree the current messages are awkward.
They are too noisy in normal (healthy case); I prefer if every thing is
normal that EAL should print as little as possible, like one line.
And if there is a config problem the current messages don't give the right
diagnostic information for users.