From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
To: Yongseok Koh <yskoh@mellanox.com>, Shahaf Shuler <shahafs@mellanox.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
Thomas Monjalon <thomas@monjalon.net>,
"shreyansh.jain@nxp.com" <shreyansh.jain@nxp.com>
Subject: Re: [dpdk-dev] [PATCH 0/4] Allow using external memory without malloc
Date: Mon, 3 Dec 2018 10:23:03 +0000 [thread overview]
Message-ID: <57d7ad23-bebd-f788-5ac9-540804ecf5da@intel.com> (raw)
In-Reply-To: <7F52E2C1-4552-48E7-9AFA-BEAF4D8E45D7@mellanox.com>
On 02-Dec-18 11:28 PM, Yongseok Koh wrote:
>
>> On Dec 1, 2018, at 9:48 PM, Shahaf Shuler <shahafs@mellanox.com> wrote:
>>
>> Hi Anatoly,
>>
>> Thursday, November 29, 2018 3:49 PM, Anatoly Burakov:
>>> Subject: [PATCH 0/4] Allow using external memory without malloc
>>>
>>> Currently, the only way to use externally allocated memory is through
>>> rte_malloc API's. While this is fine for a lot of use cases, it may not be suitable
>>> for certain other use cases like manual memory management, etc.
>>>
>>> This patchset adds another API to register memory segments with DPDK (so
>>> that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and
>>> such), but not create a malloc heap out of them.
>>>
>>> Aside from the obvious (not adding memory to a heap), the other major
>>> difference between this API and the ``rte_malloc_heap_*`` external memory
>>> functions is the fact that no DMA mapping is performed automatically.
>>>
>>> This really draws a line in the sand, and there are now two ways of doing
>>> things - do everything automatically (using the ``rte_malloc_heap_*`` API's),
>>> or do everything manually (``rte_extmem_*`` and future DMA mapping API
>>> [1] that would replace ``rte_vfio_dma_map``). This way, the consistency of
>>> API is kept, and flexibility is also allowed.
>>>
>>
>> As you know I like the idea.
>> One question though, do you see a use case for application to have externally allocated memory which needs to be registered to the DPDK subsystem however not being used for DMA?
>> My only guess would be so helper libraries which requires the memory allocation from user (however it doesn't seems like a good API).
>>
>> If no use case, maybe it is better to merge between the two (rte_extmem_* and rte_dma_map) to have a single call for app to register and DMA map the memory. The rte_mem_virt2memseg is not something application needs to understand, it is used internally by PMDs or other libs.
>
> Just FYI.
> My implementation for mlx4/5 doesn't need to have a separate registration for
> DMA by rte_dma_map() as long as it is included in the memseg list. Registration
> is done only if Lkey lookup misses and only mem free event is taken to release
> it. From my end, the reason why we wanted to have a generic DMA registration was
> because some people doesn't want to use the new API to make the ext mem included
> in the memseg list but want to simply call the API for DMA registration.
>
> In a nutshell, mlx4/5 needs users use either rte_extmem_register() or
> rte_dma_map(). However, it is no problem to call both.
It would be good to create a segment when using rte_dma_map().
Unfortunately, that's not realistic :)
Registering memory within DPDK does not necessarily have to be performed
by the primary process - whichever process that wants to create the
table, can do so, and later processes have to attach to the memory area.
There's also no way to know if memory segment can be attached to - this
is a question only application can answer.
In other words, there's no way to combine rte_dma_map() and
rte_extmem_register() into one call and keep multiprocess support.
>
>
> Thanks,
> Yongseok
>
>>> [1]
>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma
>>> ils.dpdk.org%2Farchives%2Fdev%2F2018-
>>> November%2F118175.html&data=02%7C01%7Cshahafs%40mellanox.co
>>> m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492
>>> 56f461b%7C0%7C0%7C636790961244424277&sdata=YqwcPEEhJM3I7Toe
>>> Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=0
>>>
>>> Note: at the time of this writing, there's no release notes
>>> template, so no release notes updates in this patchset.
>>> They will be added in later revisions.
>>>
>>> Anatoly Burakov (4):
>>> malloc: separate creating memseg list and malloc heap
>>> malloc: separate destroying memseg list and heap data
>>> mem: allow registering external memory areas
>>> mem: allow usage of non-heap external memory in multiprocess
>>>
>>> .../prog_guide/env_abstraction_layer.rst | 63 +++++++--
>>> lib/librte_eal/common/eal_common_memory.c | 116
>>> +++++++++++++++++
>>> lib/librte_eal/common/include/rte_memory.h | 122
>>> ++++++++++++++++++
>>> lib/librte_eal/common/malloc_heap.c | 104 +++++++++++----
>>> lib/librte_eal/common/malloc_heap.h | 15 ++-
>>> lib/librte_eal/common/rte_malloc.c | 115 +++++++----------
>>> lib/librte_eal/rte_eal_version.map | 4 +
>>> 7 files changed, 434 insertions(+), 105 deletions(-)
>>>
>>> --
>>> 2.17.1
>
>
--
Thanks,
Anatoly
next prev parent reply other threads:[~2018-12-03 10:23 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-29 13:48 Anatoly Burakov
2018-11-29 13:48 ` [dpdk-dev] [PATCH 1/4] malloc: separate creating memseg list and malloc heap Anatoly Burakov
2018-12-14 9:33 ` Yongseok Koh
2018-11-29 13:48 ` [dpdk-dev] [PATCH 2/4] malloc: separate destroying memseg list and heap data Anatoly Burakov
2018-12-14 9:34 ` Yongseok Koh
2018-11-29 13:48 ` [dpdk-dev] [PATCH 3/4] mem: allow registering external memory areas Anatoly Burakov
2018-12-14 9:55 ` Yongseok Koh
2018-12-14 11:03 ` Burakov, Anatoly
2018-11-29 13:48 ` [dpdk-dev] [PATCH 4/4] mem: allow usage of non-heap external memory in multiprocess Anatoly Burakov
2018-12-14 9:56 ` Yongseok Koh
2018-12-02 5:48 ` [dpdk-dev] [PATCH 0/4] Allow using external memory without malloc Shahaf Shuler
2018-12-02 23:28 ` Yongseok Koh
2018-12-03 10:23 ` Burakov, Anatoly [this message]
2018-12-12 12:55 ` Yongseok Koh
2018-12-12 13:17 ` Burakov, Anatoly
2018-12-14 11:50 ` [dpdk-dev] [PATCH v2 " Anatoly Burakov
2018-12-20 15:32 ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2018-12-20 16:16 ` Stephen Hemminger
2018-12-20 17:18 ` Thomas Monjalon
2018-12-21 9:17 ` Burakov, Anatoly
2018-12-20 17:17 ` Thomas Monjalon
2018-12-20 15:32 ` [dpdk-dev] [PATCH v3 1/4] malloc: separate creating memseg list and malloc heap Anatoly Burakov
2018-12-20 15:32 ` [dpdk-dev] [PATCH v3 2/4] malloc: separate destroying memseg list and heap data Anatoly Burakov
2018-12-20 15:32 ` [dpdk-dev] [PATCH v3 3/4] mem: allow registering external memory areas Anatoly Burakov
2018-12-20 15:32 ` [dpdk-dev] [PATCH v3 4/4] mem: allow usage of non-heap external memory in multiprocess Anatoly Burakov
2018-12-14 11:50 ` [dpdk-dev] [PATCH v2 1/4] malloc: separate creating memseg list and malloc heap Anatoly Burakov
2018-12-14 11:50 ` [dpdk-dev] [PATCH v2 2/4] malloc: separate destroying memseg list and heap data Anatoly Burakov
2018-12-14 11:50 ` [dpdk-dev] [PATCH v2 3/4] mem: allow registering external memory areas Anatoly Burakov
2018-12-14 11:50 ` [dpdk-dev] [PATCH v2 4/4] mem: allow usage of non-heap external memory in multiprocess Anatoly Burakov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=57d7ad23-bebd-f788-5ac9-540804ecf5da@intel.com \
--to=anatoly.burakov@intel.com \
--cc=dev@dpdk.org \
--cc=shahafs@mellanox.com \
--cc=shreyansh.jain@nxp.com \
--cc=thomas@monjalon.net \
--cc=yskoh@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).