Re: [dpdk-dev] A question about hugepage initialization time

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

Hi

> Hey Folks,
> 
> Our DPDK application deals with very large in memory data structures, and
> can potentially use tens or even hundreds of gigabytes of hugepage
> memory.
> During the course of development, we've noticed that as the number of
> huge pages increases, the memory initialization time during EAL init gets to
> be quite long, lasting several minutes at present.  The growth in init time
> doesn't appear to be linear, which is concerning.
> 
> This is a minor inconvenience for us and our customers, as memory
> initialization makes our boot times a lot longer than it would otherwise be.
> Also, my experience has been that really long operations often are hiding
> errors - what you think is merely a slow operation is actually a timeout of
> some sort, often due to misconfiguration. This leads to two
> questions:
> 
> 1. Does the long initialization time suggest that there's an error happening
> under the covers?
> 2. If not, is there any simple way that we can shorten memory initialization
> time?
> 
> Thanks in advance for your insights.

I've seen similar behavior on some systems with large amounts of memory. Basically, the more hugepages there are, the longer it takes for EAL to process them - map them, sort them, remap them, find contiguous segments, etc. The slowdown is not linear because on each stage there are multiple loops over hugepage list taking place, with trying to find contiguous memory segments taking the longest. I am not aware of any mechanism to speed up EAL startup in such cases (not without code changes, anyway), the only thing I could suggest is to avoid using 2MB pages in this scenario.

Thanks,
Anatoly