DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Alejandro Lucero <alejandro.lucero@netronome.com>
Cc: dev <dev@dpdk.org>, dpdk stable <stable@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 17.11] mem: fix memory initialization time
Date: Fri, 16 Nov 2018 15:56:16 +0000
Message-ID: <f8fbaac5-c385-f8f4-df3e-6def6a162ed0@intel.com> (raw)
In-Reply-To: <CAD+H992sDbY77ajcNaeF5oYsU5=R-mjTiC3VG748GLZX3kxkTQ@mail.gmail.com>

On 16-Nov-18 2:42 PM, Alejandro Lucero wrote:
> 
> 
> On Fri, Nov 16, 2018 at 1:35 PM Burakov, Anatoly 
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> 
>     On 16-Nov-18 12:49 PM, Alejandro Lucero wrote:
>      >
>      >
>      > On Thu, Nov 15, 2018 at 1:16 PM Burakov, Anatoly
>      > <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>
>     <mailto:anatoly.burakov@intel.com
>     <mailto:anatoly.burakov@intel.com>>> wrote:
>      >
>      >     On 12-Nov-18 11:18 AM, Alejandro Lucero wrote:
>      >      > When using large amount of hugepage based memory, doing
>     all the
>      >      > hugepages mapping can take quite significant time.
>      >      >
>      >      > The problem is hugepages being initially mmaped to virtual
>     addresses
>      >      > which will be tried later for the final hugepage mmaping.
>     This causes
>      >      > the final mapping requiring calling mmap with another hint
>      >     address which
>      >      > can happen several times, depending on the amount of memory to
>      >     mmap, and
>      >      > which each mmmap taking more than a second.
>      >      >
>      >      > This patch changes the hint for the initial hugepage
>     mmaping using
>      >      > a starting address which will not collide with the final
>     mmaping.
>      >      >
>      >      > Fixes: 293c0c4b957f ("mem: use address hint for mapping
>     hugepages")
>      >      >
>      >      > Signed-off-by: Alejandro Lucero
>     <alejandro.lucero@netronome.com <mailto:alejandro.lucero@netronome.com>
>      >     <mailto:alejandro.lucero@netronome.com
>     <mailto:alejandro.lucero@netronome.com>>>
>      >      > ---
>      >
>      >     Hi Alejandro,
>      >
>      >     I'm not sure i understand the purpose. When final mapping is
>     performed,
>      >     we reserve new memory area, and map pages into it. (i don't quite
>      >     understand why we unmap the area before mapping pages, but
>     it's how
>      >     it's
>      >     always been and i didn't change it in the legacy code)
>      >
>      >     Which addresses are causing the collision?
>      >
>      >
>      > Because the hint for the final mapping is at 4GB address, and the
>      > hugepages are initially individually mapped starting at low virtual
>      > addresses, when the memory to map is 4GB or higher, the hugepages
>     will
>      > end using that hint address and higher. The more the hugepages to
>     mmap,
>      > the more addresses above the hint address are used, and the more
>     mmaps
>      > failed for getting the virtual addresses for the final mmap.
> 
>     Yes, but i still don't understand what the problem is.
> 
>     Before the final mapping, all of the pages get unmapped. They no longer
>     occupy any VA space at all. Then, we create a VA-area the size of
>     IOVA-contiguous chunk we have, but then we also unmap *that* (again, no
>     idea why we actually do that, but that's how it works). So, the final
>     mapping is performed with the knowledge that there are no pages at
>     specified addresses, and mapping for specified addresses is performed
>     when the first mapping has already been unmapped.
> 
>     As far as i understand, at no point do we hold addresses for initial
>     and
>     final mappings concurrently. So, where does the conflict come in?
> 
> 
> Are you sure about this? Because I can see calling 
> unmap_all_hugepage_init happens after the second call to map_all_hugepages.
> 
> Maybe you are looking at the legacy code in a newer version which is not 
> exactly doing the same steps.

Ah yes, you're right - we do remap the pages before we unmap the 
original mappings. This patch perfect makes sense then. It'd still 
collide with mappings with --base-virtaddr set to the same address, but 
it's not going to fail (just be slow again), so it's OK.

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

> 
>      >
>      >     --
>      >     Thanks,
>      >     Anatoly
>      >
> 
> 
>     -- 
>     Thanks,
>     Anatoly
> 


-- 
Thanks,
Anatoly

      reply	other threads:[~2018-11-16 15:56 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-12 11:18 Alejandro Lucero
2018-11-12 11:26 ` [dpdk-dev] [dpdk-stable] " Eelco Chaudron
2018-11-14 12:45   ` Eelco Chaudron
2018-11-15 13:16 ` [dpdk-dev] " Burakov, Anatoly
2018-11-16 12:49   ` Alejandro Lucero
2018-11-16 13:35     ` Burakov, Anatoly
2018-11-16 14:42       ` Alejandro Lucero
2018-11-16 15:56         ` Burakov, Anatoly [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f8fbaac5-c385-f8f4-df3e-6def6a162ed0@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=dev@dpdk.org \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git