From: "Hunt, David" <david.hunt@intel.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 0/5] add external mempool manager
Date: Fri, 29 Jan 2016 13:40:40 +0000 [thread overview]
Message-ID: <56AB6BD8.9000403@intel.com> (raw)
In-Reply-To: <20160128172631.GA11992@localhost.localdomain>
On 28/01/2016 17:26, Jerin Jacob wrote:
> On Tue, Jan 26, 2016 at 05:25:50PM +0000, David Hunt wrote:
>> Hi all on the list.
>>
>> Here's a proposed patch for an external mempool manager
>>
>> The External Mempool Manager is an extension to the mempool API that allows
>> users to add and use an external mempool manager, which allows external memory
>> subsystems such as external hardware memory management systems and software
>> based memory allocators to be used with DPDK.
>
> I like this approach.It will be useful for external hardware memory
> pool managers.
>
> BTW, Do you encounter any performance impact on changing to function
> pointer based approach?
Jerin,
Thanks for your comments.
The performance impacts I've seen depends on whether I'm using an object
cache for the mempool or not. Without object cache, I see between 0-10%
degradation. With object cache, I see a slight performance gain of
between 0-5%. But that will most likely vary from system to system.
>> The existing API to the internal DPDK mempool manager will remain unchanged
>> and will be backward compatible.
>>
>> There are two aspects to external mempool manager.
>> 1. Adding the code for your new mempool handler. This is achieved by adding a
>> new mempool handler source file into the librte_mempool library, and
>> using the REGISTER_MEMPOOL_HANDLER macro.
>> 2. Using the new API to call rte_mempool_create_ext to create a new mempool
>> using the name parameter to identify which handler to use.
>>
>> New API calls added
>> 1. A new mempool 'create' function which accepts mempool handler name.
>> 2. A new mempool 'rte_get_mempool_handler' function which accepts mempool
>> handler name, and returns the index to the relevant set of callbacks for
>> that mempool handler
>>
>> Several external mempool managers may be used in the same application. A new
>> mempool can then be created by using the new 'create' function, providing the
>> mempool handler name to point the mempool to the relevant mempool manager
>> callback structure.
>>
>> The old 'create' function can still be called by legacy programs, and will
>> internally work out the mempool handle based on the flags provided (single
>> producer, single consumer, etc). By default handles are created internally to
>> implement the built-in DPDK mempool manager and mempool types.
>>
>> The external mempool manager needs to provide the following functions.
>> 1. alloc - allocates the mempool memory, and adds each object onto a ring
>> 2. put - puts an object back into the mempool once an application has
>> finished with it
>> 3. get - gets an object from the mempool for use by the application
>> 4. get_count - gets the number of available objects in the mempool
>> 5. free - frees the mempool memory
>>
>> Every time a get/put/get_count is called from the application/PMD, the
>> callback for that mempool is called. These functions are in the fastpath,
>> and any unoptimised handlers may limit performance.
>>
>> The new APIs are as follows:
>>
>> 1. rte_mempool_create_ext
>>
>> struct rte_mempool *
>> rte_mempool_create_ext(const char * name, unsigned n,
>> unsigned cache_size, unsigned private_data_size,
>> int socket_id, unsigned flags,
>> const char * handler_name);
>>
>> 2. rte_get_mempool_handler
>>
>> int16_t
>> rte_get_mempool_handler(const char *name);
>
> Do we need above public API as, in any case we need rte_mempool* pointer to
> operate on mempools(which has the index anyway)?
>
> May a similar functional API with different name/return will be
> better to figure out, given "name" registered or not in ethernet driver
> which has dependency on a particular HW pool manager.
Good point. An earlier revision required getting the index first, then
passing that to the create_ext call. Now that the call is by name, the
'get' is mostly redundant. As you suggest, we may need an API for
checking the existence of a particular manager/handler. Then again, we
could always return an error from the create_ext api if it fails to find
that handler. I'll remove the 'get' for the moment.
Thanks,
David.
next prev parent reply other threads:[~2016-01-29 13:40 UTC|newest]
Thread overview: 237+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-26 17:25 David Hunt
2016-01-26 17:25 ` [dpdk-dev] [PATCH 1/5] mempool: add external mempool manager support David Hunt
2016-01-28 17:52 ` Jerin Jacob
2016-02-03 14:16 ` Hunt, David
2016-02-04 13:23 ` Jerin Jacob
2016-02-04 14:52 ` Olivier MATZ
2016-02-04 16:47 ` Hunt, David
2016-02-08 11:02 ` Olivier MATZ
2016-02-04 17:34 ` Hunt, David
2016-02-05 9:26 ` Olivier MATZ
2016-03-01 13:32 ` Hunt, David
2016-03-04 9:05 ` Olivier MATZ
2016-03-08 10:04 ` Hunt, David
2016-01-26 17:25 ` [dpdk-dev] [PATCH 2/5] memool: add stack (lifo) based external mempool handler David Hunt
2016-01-26 17:25 ` [dpdk-dev] [PATCH 3/5] mempool: add custom external mempool handler example David Hunt
2016-01-28 17:54 ` Jerin Jacob
2016-01-26 17:25 ` [dpdk-dev] [PATCH 4/5] mempool: add autotest for external mempool custom example David Hunt
2016-01-26 17:25 ` [dpdk-dev] [PATCH 5/5] mempool: allow rte_pktmbuf_pool_create switch between memool handlers David Hunt
2016-02-05 10:11 ` Olivier MATZ
2016-01-28 17:26 ` [dpdk-dev] [PATCH 0/5] add external mempool manager Jerin Jacob
2016-01-29 13:40 ` Hunt, David [this message]
2016-01-29 17:16 ` Jerin Jacob
2016-02-16 14:48 ` [dpdk-dev] [PATCH 0/6] " David Hunt
2016-02-16 14:48 ` [dpdk-dev] [PATCH 1/6] mempool: add external mempool manager support David Hunt
2016-02-16 19:27 ` [dpdk-dev] [dpdk-dev, " Jan Viktorin
2016-02-19 13:30 ` [dpdk-dev] [PATCH " Olivier MATZ
2016-02-29 11:11 ` Hunt, David
2016-03-04 9:04 ` Olivier MATZ
2016-02-16 14:48 ` [dpdk-dev] [PATCH 2/6] mempool: add stack (lifo) based external mempool handler David Hunt
2016-02-19 13:31 ` Olivier MATZ
2016-02-29 11:04 ` Hunt, David
2016-03-04 9:04 ` Olivier MATZ
2016-03-08 20:45 ` Venkatesan, Venky
2016-03-09 14:53 ` Olivier MATZ
2016-02-16 14:48 ` [dpdk-dev] [PATCH 3/6] mempool: adds a simple ring-based mempool handler using mallocs for objects David Hunt
2016-02-16 14:48 ` [dpdk-dev] [PATCH 4/6] mempool: add autotest for external mempool custom example David Hunt
2016-02-16 14:48 ` [dpdk-dev] [PATCH 5/6] mempool: allow rte_pktmbuf_pool_create switch between memool handlers David Hunt
2016-02-16 14:48 ` [dpdk-dev] [PATCH 6/6] mempool: add in the RTE_NEXT_ABI protection for ABI breakages David Hunt
2016-02-19 13:33 ` Olivier MATZ
2016-02-19 13:25 ` [dpdk-dev] [PATCH 0/6] external mempool manager Olivier MATZ
2016-02-29 10:55 ` Hunt, David
2016-03-09 9:50 ` [dpdk-dev] [PATCH v3 0/4] " David Hunt
2016-03-09 9:50 ` [dpdk-dev] [PATCH v3 1/4] mempool: add external mempool manager support David Hunt
2016-04-11 22:52 ` Yuanhan Liu
2016-03-09 9:50 ` [dpdk-dev] [PATCH v3 2/4] mempool: add custom mempool handler example David Hunt
2016-03-09 9:50 ` [dpdk-dev] [PATCH v3 3/4] mempool: allow rte_pktmbuf_pool_create switch between memool handlers David Hunt
2016-03-09 10:54 ` Panu Matilainen
2016-03-09 11:38 ` Hunt, David
2016-03-09 11:44 ` Panu Matilainen
2016-03-09 9:50 ` [dpdk-dev] [PATCH v3 4/4] mempool: add in the RTE_NEXT_ABI for ABI breakages David Hunt
2016-03-09 10:46 ` Panu Matilainen
2016-03-09 11:30 ` Hunt, David
2016-03-09 14:59 ` Olivier MATZ
2016-03-09 16:28 ` Hunt, David
2016-03-09 16:31 ` Olivier MATZ
2016-03-09 16:39 ` Hunt, David
2016-03-09 11:10 ` [dpdk-dev] [PATCH v3 0/4] external mempool manager Hunt, David
2016-04-11 22:46 ` Yuanhan Liu
2016-04-14 13:57 ` [dpdk-dev] [PATCH v4 0/3] " Olivier Matz
2016-04-14 13:57 ` [dpdk-dev] [PATCH v4 1/3] mempool: support external handler Olivier Matz
2016-04-14 13:57 ` [dpdk-dev] [PATCH v4 2/3] app/test: test external mempool handler Olivier Matz
2016-04-14 13:57 ` [dpdk-dev] [PATCH v4 3/3] mbuf: get default mempool handler from configuration Olivier Matz
2016-05-19 13:44 ` [dpdk-dev] mempool: external mempool manager David Hunt
2016-05-19 13:44 ` [dpdk-dev] [PATCH v5 1/3] mempool: support external handler David Hunt
2016-05-23 12:35 ` [dpdk-dev] [dpdk-dev,v5,1/3] " Jan Viktorin
2016-05-24 14:04 ` Hunt, David
2016-05-31 9:09 ` Hunt, David
2016-05-31 12:06 ` Jan Viktorin
2016-05-31 13:47 ` Hunt, David
2016-05-31 20:40 ` Olivier MATZ
2016-06-01 9:39 ` Hunt, David
2016-06-01 12:30 ` Jan Viktorin
2016-05-24 15:35 ` [dpdk-dev] [PATCH v5 1/3] " Jerin Jacob
2016-05-27 9:52 ` Hunt, David
2016-05-27 10:33 ` Jerin Jacob
2016-05-27 14:44 ` Hunt, David
2016-05-30 9:41 ` Jerin Jacob
2016-05-30 11:27 ` Hunt, David
2016-05-31 8:53 ` Jerin Jacob
2016-05-31 15:37 ` Hunt, David
2016-05-31 16:03 ` Jerin Jacob
2016-05-31 20:41 ` Olivier MATZ
2016-05-31 21:11 ` Jerin Jacob
2016-06-01 10:46 ` Hunt, David
2016-06-01 11:18 ` Jerin Jacob
2016-05-19 13:45 ` [dpdk-dev] [PATCH v5 2/3] app/test: test external mempool handler David Hunt
2016-05-23 12:45 ` [dpdk-dev] [dpdk-dev, v5, " Jan Viktorin
2016-05-31 9:17 ` Hunt, David
2016-05-31 12:14 ` Jan Viktorin
2016-05-31 20:40 ` Olivier MATZ
2016-05-19 13:45 ` [dpdk-dev] [PATCH v5 3/3] mbuf: get default mempool handler from configuration David Hunt
2016-05-23 12:40 ` [dpdk-dev] [dpdk-dev, v5, " Jan Viktorin
2016-05-31 9:26 ` Hunt, David
2016-06-01 16:19 ` [dpdk-dev] [PATCH v6 0/5] mempool: add external mempool manager David Hunt
2016-06-01 16:19 ` [dpdk-dev] [PATCH v6 1/5] mempool: support external handler David Hunt
2016-06-01 16:29 ` Hunt, David
2016-06-01 17:54 ` Jan Viktorin
2016-06-02 9:11 ` Hunt, David
2016-06-02 11:23 ` Hunt, David
2016-06-02 13:43 ` Jan Viktorin
2016-06-01 16:19 ` [dpdk-dev] [PATCH v6 2/5] mempool: remove rte_ring from rte_mempool struct David Hunt
2016-06-01 16:19 ` [dpdk-dev] [PATCH v6 3/5] mempool: add default external mempool handler David Hunt
2016-06-01 16:19 ` [dpdk-dev] [PATCH v6 4/5] app/test: test " David Hunt
2016-06-01 16:19 ` [dpdk-dev] [PATCH v6 5/5] mbuf: get default mempool handler from configuration David Hunt
2016-06-02 13:27 ` [dpdk-dev] [PATCH v7 0/5] mempool: add external mempool manager David Hunt
2016-06-02 13:27 ` [dpdk-dev] [PATCH v7 1/5] mempool: support external mempool operations David Hunt
2016-06-02 13:38 ` [dpdk-dev] [PATCH v7 0/5] mempool: add external mempool manager Hunt, David
2016-06-03 6:38 ` [dpdk-dev] [PATCH v7 1/5] mempool: support external mempool operations Jerin Jacob
2016-06-03 10:28 ` Hunt, David
2016-06-03 10:49 ` Jerin Jacob
2016-06-03 11:07 ` Olivier MATZ
2016-06-03 11:42 ` Jan Viktorin
2016-06-03 12:10 ` Hunt, David
2016-06-03 12:28 ` Olivier MATZ
2016-06-02 13:27 ` [dpdk-dev] [PATCH v7 2/5] mempool: remove rte_ring from rte_mempool struct David Hunt
2016-06-03 12:28 ` Olivier MATZ
2016-06-03 14:17 ` Hunt, David
2016-06-02 13:27 ` [dpdk-dev] [PATCH v7 3/5] mempool: add default external mempool ops David Hunt
2016-06-02 13:27 ` [dpdk-dev] [PATCH v7 4/5] app/test: test external mempool manager David Hunt
2016-06-02 13:27 ` [dpdk-dev] [PATCH v7 5/5] mbuf: allow apps to change default mempool ops David Hunt
2016-06-03 12:28 ` Olivier MATZ
2016-06-03 14:06 ` Hunt, David
2016-06-03 14:10 ` Olivier Matz
2016-06-03 14:14 ` Hunt, David
2016-06-03 14:58 ` [dpdk-dev] [PATCH v8 0/5] mempool: add external mempool manager David Hunt
2016-06-03 14:58 ` [dpdk-dev] [PATCH v8 1/3] mempool: support external mempool operations David Hunt
2016-06-06 14:32 ` Shreyansh Jain
2016-06-06 14:38 ` Shreyansh Jain
2016-06-07 9:25 ` Hunt, David
2016-06-08 13:48 ` Shreyansh Jain
2016-06-09 9:39 ` Hunt, David
2016-06-09 10:31 ` Jerin Jacob
2016-06-09 11:06 ` Hunt, David
2016-06-09 11:49 ` Shreyansh Jain
2016-06-09 12:30 ` Jerin Jacob
2016-06-09 13:03 ` Shreyansh Jain
2016-06-09 13:18 ` Hunt, David
2016-06-09 13:37 ` Jerin Jacob
2016-06-09 11:41 ` Shreyansh Jain
2016-06-09 12:55 ` Hunt, David
2016-06-09 13:09 ` Jan Viktorin
2016-06-10 7:29 ` Olivier Matz
2016-06-10 8:49 ` Jan Viktorin
2016-06-10 9:02 ` Hunt, David
2016-06-10 9:34 ` Hunt, David
2016-06-10 11:29 ` Shreyansh Jain
2016-06-10 11:13 ` Jerin Jacob
2016-06-10 11:37 ` Shreyansh Jain
2016-06-07 9:05 ` Shreyansh Jain
2016-06-08 12:13 ` Olivier Matz
2016-06-09 10:33 ` Hunt, David
2016-06-08 14:28 ` Shreyansh Jain
2016-06-03 14:58 ` [dpdk-dev] [PATCH v8 2/3] app/test: test external mempool manager David Hunt
2016-06-03 14:58 ` [dpdk-dev] [PATCH v8 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-10 15:16 ` [dpdk-dev] [PATCH v9 0/3] mempool: add external mempool manager David Hunt
2016-06-10 15:16 ` [dpdk-dev] [PATCH v9 1/3] mempool: support external mempool operations David Hunt
2016-06-13 12:16 ` Olivier Matz
2016-06-13 13:46 ` Hunt, David
2016-06-10 15:16 ` [dpdk-dev] [PATCH v9 2/3] app/test: test external mempool manager David Hunt
2016-06-10 15:16 ` [dpdk-dev] [PATCH v9 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-14 9:46 ` [dpdk-dev] [PATCH v10 0/3] mempool: add external mempool manager David Hunt
2016-06-14 9:46 ` [dpdk-dev] [PATCH v10 1/3] mempool: support external mempool operations David Hunt
2016-06-14 11:38 ` Shreyansh Jain
2016-06-14 12:55 ` Thomas Monjalon
2016-06-14 13:20 ` Hunt, David
2016-06-14 13:29 ` Thomas Monjalon
2016-06-14 9:46 ` [dpdk-dev] [PATCH v10 2/3] app/test: test external mempool manager David Hunt
2016-06-14 11:39 ` Shreyansh Jain
2016-06-14 9:46 ` [dpdk-dev] [PATCH v10 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-14 11:45 ` Shreyansh Jain
2016-06-14 12:32 ` [dpdk-dev] [PATCH v10 0/3] mempool: add external mempool manager Olivier MATZ
2016-06-14 15:48 ` [dpdk-dev] [PATCH v11 " David Hunt
2016-06-14 15:48 ` [dpdk-dev] [PATCH v11 1/3] mempool: support external mempool operations David Hunt
2016-06-14 16:08 ` Thomas Monjalon
2016-06-14 15:49 ` [dpdk-dev] [PATCH v11 2/3] app/test: test external mempool manager David Hunt
2016-06-14 15:49 ` [dpdk-dev] [PATCH v11 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-15 7:47 ` [dpdk-dev] [PATCH v12 0/3] mempool: add external mempool manager David Hunt
2016-06-15 7:47 ` [dpdk-dev] [PATCH v12 1/3] mempool: support external mempool operations David Hunt
2016-06-15 10:14 ` Jan Viktorin
2016-06-15 10:29 ` Hunt, David
2016-06-15 11:26 ` Jan Viktorin
2016-06-15 11:38 ` Thomas Monjalon
2016-06-15 7:47 ` [dpdk-dev] [PATCH v12 2/3] app/test: test external mempool manager David Hunt
2016-06-15 7:47 ` [dpdk-dev] [PATCH v12 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-15 10:13 ` [dpdk-dev] [PATCH v12 0/3] mempool: add external mempool manager Jan Viktorin
2016-06-15 11:47 ` Hunt, David
2016-06-15 12:03 ` Olivier MATZ
2016-06-15 12:38 ` Hunt, David
2016-06-15 13:50 ` Olivier MATZ
2016-06-15 14:02 ` Hunt, David
2016-06-15 14:10 ` Olivier MATZ
2016-06-15 14:47 ` Jan Viktorin
2016-06-15 16:03 ` Hunt, David
2016-06-15 16:34 ` Hunt, David
2016-06-15 16:40 ` Olivier MATZ
2016-06-16 4:35 ` Shreyansh Jain
2016-06-16 7:04 ` Hunt, David
2016-06-16 7:47 ` Hunt, David
2016-06-16 8:47 ` Olivier MATZ
2016-06-16 8:55 ` Hunt, David
2016-06-16 8:58 ` Olivier MATZ
2016-06-16 11:34 ` Hunt, David
2016-06-16 12:30 ` [dpdk-dev] [PATCH v13 " David Hunt
2016-06-16 12:30 ` [dpdk-dev] [PATCH v13 1/3] mempool: support external mempool operations David Hunt
2016-06-17 6:58 ` Hunt, David
2016-06-17 8:08 ` Olivier Matz
2016-06-17 8:42 ` Hunt, David
2016-06-17 9:09 ` Thomas Monjalon
2016-06-17 9:24 ` Hunt, David
2016-06-17 10:19 ` Olivier Matz
2016-06-17 10:18 ` Olivier Matz
2016-06-17 10:47 ` Hunt, David
2016-06-16 12:30 ` [dpdk-dev] [PATCH v13 2/3] app/test: test external mempool manager David Hunt
2016-06-16 12:30 ` [dpdk-dev] [PATCH v13 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-17 13:53 ` [dpdk-dev] [PATCH v14 0/3] mempool: add mempool handler feature David Hunt
2016-06-17 13:53 ` [dpdk-dev] [PATCH v14 1/3] mempool: support mempool handler operations David Hunt
2016-06-17 14:35 ` Jan Viktorin
2016-06-19 11:44 ` Hunt, David
2016-06-17 13:53 ` [dpdk-dev] [PATCH v14 2/3] app/test: test mempool handler David Hunt
2016-06-17 14:37 ` Jan Viktorin
2016-06-17 13:53 ` [dpdk-dev] [PATCH v14 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-17 14:41 ` Jan Viktorin
2016-06-19 12:05 ` [dpdk-dev] [PATCH v15 0/3] mempool: add mempool handler feature David Hunt
2016-06-19 12:05 ` [dpdk-dev] [PATCH v15 1/3] mempool: support mempool handler operations David Hunt
2016-06-19 12:05 ` [dpdk-dev] [PATCH v15 2/3] app/test: test mempool handler David Hunt
2016-06-19 12:05 ` [dpdk-dev] [PATCH v15 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-22 7:56 ` [dpdk-dev] [PATCH v15 0/3] mempool: add mempool handler feature Thomas Monjalon
2016-06-22 8:02 ` Thomas Monjalon
2016-06-22 9:27 ` [dpdk-dev] [PATCH v16 " David Hunt
2016-06-22 9:27 ` [dpdk-dev] [PATCH v16 1/3] mempool: support mempool handler operations David Hunt
2016-06-22 9:27 ` [dpdk-dev] [PATCH v16 2/3] app/test: test mempool handler David Hunt
2016-06-22 9:27 ` [dpdk-dev] [PATCH v16 3/3] mbuf: make default mempool ops configurable at build David Hunt
2016-06-23 21:22 ` [dpdk-dev] [PATCH v16 0/3] mempool: add mempool handler feature Thomas Monjalon
2016-06-24 4:55 ` Wiles, Keith
2016-06-24 11:20 ` Jan Viktorin
2016-06-24 11:24 ` Thomas Monjalon
2016-06-24 13:10 ` Jan Viktorin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56AB6BD8.9000403@intel.com \
--to=david.hunt@intel.com \
--cc=dev@dpdk.org \
--cc=jerin.jacob@caviumnetworks.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).