DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Kamaraj P <pkamaraj@gmail.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>, dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK hugepage memory fragmentation
Date: Wed, 16 Sep 2020 12:19:19 +0100
Message-ID: <0b3a544d-4765-b284-1755-8b95eb86725d@intel.com> (raw)
In-Reply-To: <CAG8PAargDvknkbvTxfW4NsSoufJ5q4rGyjYK++bOH9KOW9=3zA@mail.gmail.com>

On 16-Sep-20 5:32 AM, Kamaraj P wrote:
> Hi Anatoly,
> 
> We just dump the memory contents when it fails to allocate the memory.
> 

Hi Kamaraj,

Yes, i can see that the memory is fragmented. That's not what i was 
asking though, because memory fragmentation is *expected* if you're 
using igb_uio. You're not using VFIO and IOVA as VA addressing, so 
you're at the mercy of your kernel when it comes to getting 
IOVA-contiguous address. We in DPDK cannot do anything about it, because 
we don't control which pages we get and what addresses they get 
assigned. There's nothing to fix on our side in this situation.

Here are the things you can do to avoid allocation failure in your case:

1) Drop the IOVA-contiguous allocation flag [1]
2) Use legacy mode [2] [3]
3) Switch to using VFIO [4]
4) Use bigger page size

The first point is crucial! Your allocation *wouldn't have failed* if 
your code didn't specify for the allocation to require 
IOVA-contiguousness. The fact that it has failed means that your 
allocation has requested such memory, so it's on you to ensure that 
whatever you're allocating, IOVA-contiguousness is *required*. I cannot 
decide that for you as it is your code, so it is up to you to figure out 
if whatever you're allocating actually requires such memory, or if you 
can safely remove this allocation flag from your code.

[1] 
http://doc.dpdk.org/api/rte__memzone_8h.html#a3ccbea77ccab608c6e683817a3eb170f
[2] 
http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
[3] http://doc.dpdk.org/guides/linux_gsg/linux_eal_parameters.html#id3
[4] I understand the requirement for PF driver, but i think support for 
PF in VFIO is coming in 20.11 release

-- 
Thanks,
Anatoly

  reply	other threads:[~2020-09-16 11:19 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-10  9:22 Kamaraj P
2020-07-10 10:28 ` Bruce Richardson
2020-07-10 15:44   ` Burakov, Anatoly
2020-07-11  7:51     ` Kamaraj P
2020-07-13  9:27       ` Burakov, Anatoly
2020-07-27 15:30         ` Kamaraj P
2020-07-28 10:10           ` Burakov, Anatoly
2020-09-16  4:32             ` Kamaraj P
2020-09-16 11:19               ` Burakov, Anatoly [this message]
2020-09-16 11:47                 ` Burakov, Anatoly

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0b3a544d-4765-b284-1755-8b95eb86725d@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=pkamaraj@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git