From: Stephen Hemminger <stephen@networkplumber.org>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Cc: dev@dpdk.org, andras.kovacs@ericsson.com,
laszlo.vadkeri@ericsson.com, keith.wiles@intel.com,
benjamin.walker@intel.com, bruce.richardson@intel.com,
thomas@monjalon.net
Subject: Re: [dpdk-dev] [RFC v2 00/23] Dynamic memory allocation for DPDK
Date: Tue, 19 Dec 2017 08:06:22 -0800 [thread overview]
Message-ID: <20171219080622.1c1bd5ca@xeon-e3> (raw)
In-Reply-To: <6d025303-f974-077f-511a-51cd62203925@intel.com>
On Tue, 19 Dec 2017 16:02:51 +0000
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:
> On 19-Dec-17 3:46 PM, Stephen Hemminger wrote:
> > On Tue, 19 Dec 2017 11:14:27 +0000
> > Anatoly Burakov <anatoly.burakov@intel.com> wrote:
> >
> >> This patchset introduces a prototype implementation of dynamic memory allocation
> >> for DPDK. It is intended to start a conversation and build consensus on the best
> >> way to implement this functionality. The patchset works well enough to pass all
> >> unit tests, and to work with traffic forwarding, provided the device drivers are
> >> adjusted to ensure contiguous memory allocation where it matters.
> >
> >
> > What exact functionality is this patchset trying to enable.
> > It isn't clear what is broken now. Is it a cleanup or something that
> > is being motivated by memory layout issues?
> >
>
> Hi Stephen,
>
> Apologies for not making that clear enough in the cover letter.
>
> The big issue this patchset is trying to solve is the static-ness of
> DPDK's memory allocation. I.e. you reserve memory on startup, and that's
> it - you can't allocate any more memory from the system, and you can't
> free it back without stopping the application.
>
> With this patchset, you can do exactly that. You can basically start
> with zero memory preallocated, and allocate (and free) as you go. For
> example, if you apply this patchset and run malloc autotest, after
> startup you will have used perhaps a single 2MB page. While the test is
> running, you are going to allocate something to the tune of 14MB per
> socket, and at the end you're back at eating 2MB of hugepage memory,
> while all of the memory you used for autotest will be freed back to the
> system. That's the main use case this patchset is trying to address.
>
> Down the line, there are other issues to be solved, which are outlined
> in the cover letter (the aforementioned "discussion points"), but for
> this iteration, dynamic allocation/free of DPDK memory is the one issue
> that is being addressed.
>
Ok, maybe name it "memory hot add/remove" since dynamic memory allocation
to me implies redoing malloc.
next prev parent reply other threads:[~2017-12-19 16:06 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-19 11:14 Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 01/23] eal: move get_virtual_area out of linuxapp eal_memory.c Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 02/23] eal: add function to report number of detected sockets Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 03/23] eal: add rte_fbarray Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 04/23] eal: move all locking to heap Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 05/23] eal: protect malloc heap stats with a lock Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 06/23] eal: make malloc a doubly-linked list Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 07/23] eal: make malloc_elem_join_adjacent_free public Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 08/23] eal: add "single file segments" command-line option Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 09/23] eal: add "legacy memory" option Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 10/23] eal: read hugepage counts from node-specific sysfs path Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 11/23] eal: replace memseg with memseg lists Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 12/23] eal: add support for dynamic memory allocation Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 13/23] eal: make use of dynamic memory allocation for init Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 14/23] eal: add support for dynamic unmapping of pages Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 15/23] eal: add API to check if memory is physically contiguous Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 16/23] eal: enable dynamic memory allocation/free on malloc/free Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 17/23] eal: add backend support for contiguous memory allocation Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 18/23] eal: add rte_malloc support for allocating contiguous memory Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 19/23] eal: enable reserving physically contiguous memzones Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 20/23] eal: make memzones use rte_fbarray Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 21/23] mempool: add support for the new memory allocation methods Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 22/23] vfio: allow to map other memory regions Anatoly Burakov
2017-12-19 11:14 ` [dpdk-dev] [RFC v2 23/23] eal: map/unmap memory with VFIO when alloc/free pages Anatoly Burakov
2017-12-19 15:46 ` [dpdk-dev] [RFC v2 00/23] Dynamic memory allocation for DPDK Stephen Hemminger
2017-12-19 16:02 ` Burakov, Anatoly
2017-12-19 16:06 ` Stephen Hemminger [this message]
2017-12-19 16:09 ` Burakov, Anatoly
2017-12-21 21:38 ` Walker, Benjamin
2017-12-22 9:13 ` Burakov, Anatoly
2017-12-26 17:19 ` Walker, Benjamin
2018-02-02 19:28 ` Yongseok Koh
2018-02-05 10:03 ` Burakov, Anatoly
2018-02-05 10:18 ` Nélio Laranjeiro
2018-02-05 10:36 ` Burakov, Anatoly
2018-02-06 9:10 ` Nélio Laranjeiro
2018-02-14 2:01 ` Yongseok Koh
2018-02-14 9:32 ` Burakov, Anatoly
2018-02-14 18:13 ` Yongseok Koh
2018-01-13 14:13 ` Burakov, Anatoly
2018-01-23 22:33 ` Yongseok Koh
2018-01-25 16:18 ` Burakov, Anatoly
2018-02-14 8:04 ` Thomas Monjalon
2018-02-14 10:07 ` Burakov, Anatoly
2018-04-25 16:02 ` Burakov, Anatoly
2018-04-25 16:12 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171219080622.1c1bd5ca@xeon-e3 \
--to=stephen@networkplumber.org \
--cc=anatoly.burakov@intel.com \
--cc=andras.kovacs@ericsson.com \
--cc=benjamin.walker@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=keith.wiles@intel.com \
--cc=laszlo.vadkeri@ericsson.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).