Re: [dpdk-dev] memory allocation requirements

["Wiles, Keith" <keith.wiles@intel.com>]

>After looking at the patches for container support, it appears that
>some changes are needed in the memory management:
>http://thread.gmane.org/gmane.comp.networking.dpdk.devel/32786/focus=32788
>
>I think it is time to collect what are the needs and expectations of
>the DPDK memory allocator. The goal is to satisfy every needs while
>cleaning the API.
>Here is a first try to start the discussion.
>
>The memory allocator has 2 classes of API in DPDK.
>First the user/application allows or requires DPDK to take over some
>memory resources of the system. The characteristics can be:
>	- numa node
>	- page size
>	- swappable or not
>	- contiguous (cannot be guaranteed) or not
>	- physical address (as root only)
>Then the drivers or other libraries use the memory through
>	- rte_malloc
>	- rte_memzone
>	- rte_mempool

Do not forget about rte_pktmbuf_create() which replies on rte_mempool and rte_memzone for high performance. We need to make sure we do not break this area.

We need to draw a good diagram showing the relationship to memory allocation and API’s at least I need something like this to help understand the bigger picture. Then we can start looking at how to modify memory allocation.

What I want is to reduce the primary APIs related to the complexity of the APIs and start using less arguments and handle the most command cases. The more complex memory configurations should not be hidden in the API IMO, but more explicit using APIs to configure the memory allocation in the case of rte_mempool_create(). 

>I think we can integrate the characteristics of the requested memory
>in rte_malloc. Then rte_memzone would be only a named rte_malloc.
>The rte_mempool still focus on collection of objects with cache.
>
>If a rework happens, maybe that the build options CONFIG_RTE_LIBRTE_IVSHMEM
>and CONFIG_RTE_EAL_SINGLE_FILE_SEGMENTS can be removed.
These need to be remove or at least moved to a runtime configuration.

>The Xen support should also be better integrated.

The Xen support and just external memory manager support is coming in 16.07, which I hope helps with the external memory management support by adding a better structure around how external memory is managed.

>
>Currently, the first class of API is directly implemented as command line
>parameters. Please let's think of C functions first.

I would also like to start thinking in terms of config-file read on startup instead of command line options. This would help create the C functions you are talking about IMO.

>The EAL parameters should simply wrap some API functions and let the
>applications tune the memory initialization with a well documented API.
>
>Probably that I forget some needs, e.g. for the secondary processes.
>Please comment.

This will be a big change and will effect many applications, but I hope it can be kept to a minimum.
>


Regards,
Keith