DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Matt Laswell <laswell@infiniteio.com>, "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] A question about hugepage initialization time
Date: Tue, 9 Dec 2014 16:50:21 +0000
Message-ID: <C6ECDF3AB251BE4894318F4E4512369780C245D2@IRSMSX109.ger.corp.intel.com> (raw)
In-Reply-To: <CA+GnqArTJoVd9Hh2xZ-fFhHRnUdbgvxB5Tp+rvi2crUi0-9g9A@mail.gmail.com>

Hi

> Hey Folks,
> 
> Our DPDK application deals with very large in memory data structures, and
> can potentially use tens or even hundreds of gigabytes of hugepage
> memory.
> During the course of development, we've noticed that as the number of
> huge pages increases, the memory initialization time during EAL init gets to
> be quite long, lasting several minutes at present.  The growth in init time
> doesn't appear to be linear, which is concerning.
> 
> This is a minor inconvenience for us and our customers, as memory
> initialization makes our boot times a lot longer than it would otherwise be.
> Also, my experience has been that really long operations often are hiding
> errors - what you think is merely a slow operation is actually a timeout of
> some sort, often due to misconfiguration. This leads to two
> questions:
> 
> 1. Does the long initialization time suggest that there's an error happening
> under the covers?
> 2. If not, is there any simple way that we can shorten memory initialization
> time?
> 
> Thanks in advance for your insights.

I've seen similar behavior on some systems with large amounts of memory. Basically, the more hugepages there are, the longer it takes for EAL to process them - map them, sort them, remap them, find contiguous segments, etc. The slowdown is not linear because on each stage there are multiple loops over hugepage list taking place, with trying to find contiguous memory segments taking the longest. I am not aware of any mechanism to speed up EAL startup in such cases (not without code changes, anyway), the only thing I could suggest is to avoid using 2MB pages in this scenario.

Thanks,
Anatoly

  reply	other threads:[~2014-12-09 16:52 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-12-09 16:33 Matt Laswell
2014-12-09 16:50 ` Burakov, Anatoly [this message]
2014-12-09 19:06 ` Matthew Hall
2014-12-09 22:05   ` Matt Laswell
2014-12-09 19:45 ` &rew
2014-12-09 22:10   ` Stephen Hemminger
2014-12-10 10:32     ` Bruce Richardson
2014-12-10 14:29       ` Neil Horman
2014-12-10 14:35         ` Bruce Richardson
2014-12-10 19:16           ` László Vadkerti
2014-12-11 10:14             ` Bruce Richardson
2014-12-12  4:07               ` László Vadkerti
2014-12-12  9:59                 ` Bruce Richardson
2014-12-12 15:50                   ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=C6ECDF3AB251BE4894318F4E4512369780C245D2@IRSMSX109.ger.corp.intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    --cc=laswell@infiniteio.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git