[dpdk-dev] [RFC 00/11] Support externally allocated memory in DPDK

[Anatoly Burakov <anatoly.burakov@intel.com>]

This is a proposal to enable using externally allocated memory
in DPDK.

In a nutshell, here is what is being done here:

- Index malloc heaps by NUMA node index, rather than NUMA node itself
- Add identifier string to malloc heap, to uniquely identify it
- Allow creating named heaps and add/remove memory to/from those heaps
- Allocate memseg lists at runtime, to keep track of IOVA addresses
  of externally allocated memory
  - If IOVA addresses aren't provided, use RTE_BAD_IOVA
- Allow malloc and memzones to allocate from named heaps

The responsibility to ensure memory is accessible before using it is
on the shoulders of the user - there is no checking done with regards
to validity of the memory (nor could there be...).

The following limitations are present:

- No multiprocess support
- No thread safety

There is currently no way to allocate memory during initialization
stage, so even if multiprocess support is added, it is not guaranteed
to work because of underlying issues with mapping fbarrays in
secondary processes. This is not an issue in single process scenario,
but it may be an issue in a multiprocess scenario in case where
primary doesn't intend to share the externally allocated memory, yet
adding such memory could fail because some other process failed to
attach to this shared memory when it wasn't needed.

Anatoly Burakov (11):
  mem: allow memseg lists to be marked as external
  eal: add function to rerieve socket index by socket ID
  malloc: index heaps using heap ID rather than NUMA node
  malloc: add name to malloc heaps
  malloc: enable retrieving statistics from named heaps
  malloc: enable allocating from named heaps
  malloc: enable creating new malloc heaps
  malloc: allow adding memory to named heaps
  malloc: allow removing memory from named heaps
  malloc: allow destroying heaps
  memzone: enable reserving memory from named heaps

 config/common_base                            |   1 +
 lib/librte_eal/common/eal_common_lcore.c      |  15 +
 lib/librte_eal/common/eal_common_memory.c     |  51 +++-
 lib/librte_eal/common/eal_common_memzone.c    | 283 ++++++++++++++----
 .../common/include/rte_eal_memconfig.h        |   5 +-
 lib/librte_eal/common/include/rte_lcore.h     |  19 +-
 lib/librte_eal/common/include/rte_malloc.h    | 158 +++++++++-
 .../common/include/rte_malloc_heap.h          |   2 +
 lib/librte_eal/common/include/rte_memzone.h   | 183 +++++++++++
 lib/librte_eal/common/malloc_heap.c           | 277 +++++++++++++++--
 lib/librte_eal/common/malloc_heap.h           |  26 ++
 lib/librte_eal/common/rte_malloc.c            | 197 +++++++++++-
 lib/librte_eal/rte_eal_version.map            |  10 +
 13 files changed, 1118 insertions(+), 109 deletions(-)

-- 
2.17.1