From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Jim Harris <james.r.harris@intel.com>, dev@dpdk.org
Cc: dariusz.stojaczyk@intel.com, benjamin.walker@intel.com,
seth.howell@intel.com
Subject: Re: [dpdk-dev] [PATCH v2] mem: add --match-allocations
Date: Mon, 17 Dec 2018 09:40:00 +0000 [thread overview]
Message-ID: <677d302e-6583-babe-edc7-3c462aa09a62@intel.com> (raw)
In-Reply-To: <20181214171303.21817-1-james.r.harris@intel.com>
On 14-Dec-18 5:13 PM, Jim Harris wrote:
> SPDK uses the rte_mem_event_callback_register API to
> create RDMA memory regions (MRs) for newly allocated regions
> of memory. This is used in both the SPDK NVMe-oF target
> and the NVMe-oF host driver.
>
> DPDK creates internal malloc_elem structures for these
> allocated regions. As users malloc and free memory, DPDK
> will sometimes merge malloc_elems that originated from
> different allocations that were notified through the
> registered mem_event callback routine. This results
> in subsequent allocations that can span across multiple
> RDMA MRs. This requires SPDK to check each DPDK buffer to
> see if it crosses an MR boundary, and if so, would have to
> add considerable logic and complexity to describe that
> buffer before it can be accessed by the RNIC. It is somewhat
> analagous to rte_malloc returning a buffer that is not
> IOVA-contiguous.
>
> As a malloc_elem gets split and some of these elements
> get freed, it can also result in DPDK sending an
> RTE_MEM_EVENT_FREE notification for a subset of the
> original RTE_MEM_EVENT_ALLOC notification. This is also
> problematic for RDMA memory regions, since unregistering
> the memory region is all-or-nothing. It is not possible
> to unregister part of a memory region.
>
> To support these types of applications, this patch adds
> a new --match-allocations EAL init flag. When this
> flag is specified, malloc elements from different
> hugepage allocations will never be merged. Memory will
> also only be freed back to the system (with the requisite
> memory event callback) exactly as it was originally
> allocated.
>
> Since part of this patch is extending the size of struct
> malloc_elem, we also fix up the malloc autotests so they
> do not assume its size exactly fits in one cacheline.
>
> Signed-off-by: Jim Harris <james.r.harris@intel.com>
> ---
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
--
Thanks,
Anatoly
next prev parent reply other threads:[~2018-12-17 9:40 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-14 17:13 Jim Harris
2018-12-17 9:40 ` Burakov, Anatoly [this message]
2018-12-17 9:45 ` Burakov, Anatoly
2018-12-20 12:00 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=677d302e-6583-babe-edc7-3c462aa09a62@intel.com \
--to=anatoly.burakov@intel.com \
--cc=benjamin.walker@intel.com \
--cc=dariusz.stojaczyk@intel.com \
--cc=dev@dpdk.org \
--cc=james.r.harris@intel.com \
--cc=seth.howell@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).