From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Kamaraj P <pkamaraj@gmail.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>, dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK hugepage memory fragmentation
Date: Wed, 16 Sep 2020 12:47:12 +0100 [thread overview]
Message-ID: <4defb35b-ba04-6629-e740-2a47fa0fb121@intel.com> (raw)
In-Reply-To: <0b3a544d-4765-b284-1755-8b95eb86725d@intel.com>
On 16-Sep-20 12:19 PM, Burakov, Anatoly wrote:
> On 16-Sep-20 5:32 AM, Kamaraj P wrote:
>> Hi Anatoly,
>>
>> We just dump the memory contents when it fails to allocate the memory.
>>
>
> Hi Kamaraj,
>
> Yes, i can see that the memory is fragmented. That's not what i was
> asking though, because memory fragmentation is *expected* if you're
> using igb_uio. You're not using VFIO and IOVA as VA addressing, so
> you're at the mercy of your kernel when it comes to getting
> IOVA-contiguous address. We in DPDK cannot do anything about it, because
> we don't control which pages we get and what addresses they get
> assigned. There's nothing to fix on our side in this situation.
>
> Here are the things you can do to avoid allocation failure in your case:
>
> 1) Drop the IOVA-contiguous allocation flag [1]
> 2) Use legacy mode [2] [3]
> 3) Switch to using VFIO [4]
> 4) Use bigger page size
>
> The first point is crucial! Your allocation *wouldn't have failed* if
> your code didn't specify for the allocation to require
> IOVA-contiguousness. The fact that it has failed means that your
> allocation has requested such memory, so it's on you to ensure that
> whatever you're allocating, IOVA-contiguousness is *required*. I cannot
> decide that for you as it is your code, so it is up to you to figure out
> if whatever you're allocating actually requires such memory, or if you
> can safely remove this allocation flag from your code.
>
> [1]
> http://doc.dpdk.org/api/rte__memzone_8h.html#a3ccbea77ccab608c6e683817a3eb170f
>
> [2]
> http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#memory-mapping-discovery-and-memory-reservation
>
> [3] http://doc.dpdk.org/guides/linux_gsg/linux_eal_parameters.html#id3
> [4] I understand the requirement for PF driver, but i think support for
> PF in VFIO is coming in 20.11 release
>
Also, to add to this, we still haven't established that the allocation
error is in fact due to memory fragmentation. You *claim* that is the
case, but memory allocations can fail for other reasons, and you haven't
provided me with anything that would confirm this, as *merely* the fact
that the memory is fragmented will not, in and of itself, necessarily
cause any allocation failures *unless* you are trying to allocate memory
area that is bigger than your page size *and* is required to be contiguous.
If you can't tell me *what* you're allocating, at least tell me the size
of the allocation, the memzone flags it is allocated with, and any debug
log messages around the allocation, so that we can at least be sure that
we know the reason why the allocation fails. You're forcing me to fly
blind here, i can't help you if i don't know what i'm dealing with.
--
Thanks,
Anatoly
prev parent reply other threads:[~2020-09-16 11:47 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-10 9:22 Kamaraj P
2020-07-10 10:28 ` Bruce Richardson
2020-07-10 15:44 ` Burakov, Anatoly
2020-07-11 7:51 ` Kamaraj P
2020-07-13 9:27 ` Burakov, Anatoly
2020-07-27 15:30 ` Kamaraj P
2020-07-28 10:10 ` Burakov, Anatoly
2020-09-16 4:32 ` Kamaraj P
2020-09-16 11:19 ` Burakov, Anatoly
2020-09-16 11:47 ` Burakov, Anatoly [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4defb35b-ba04-6629-e740-2a47fa0fb121@intel.com \
--to=anatoly.burakov@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=pkamaraj@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).