[dpdk-dev] [PATCH] doc: add multi-proc shared lib mempool note

[dpdk-dev] [PATCH] doc: add multi-proc shared lib mempool note

[Gage Eads]

The mempool library assigns handler ops indexes based on the dynamic load
order of mempool handlers. Indexes are used so a mempool can be used by
multiple processes, but this only works if all processes agree on the
mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load
mempool handlers in different orders, and thus have different
index->handler mappings. Using a mempool in multiple of such processes will
result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning
users against this.

Fixes: 449c49b93a6b ("mempool: support handler operations")
Cc: stable@dpdk.org

Signed-off-by: Gage Eads <gage.eads@intel.com>
---
 doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst
index 52a569f57..4470f6b38 100644
--- a/doc/guides/prog_guide/mempool_lib.rst
+++ b/doc/guides/prog_guide/mempool_lib.rst
@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting
 (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of
 an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool handler
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with shared
+    libraries, the -d arguments for mempool handlers *must be specified in the
+    same order for all processes* to ensure correct operation.
 
 Use Cases
 ---------
-- 
2.13.6

Re: [dpdk-dev] [PATCH] doc: add multi-proc shared lib mempool note

[Andrew Rybchenko]

On 6/18/19 9:48 PM, Gage Eads wrote:
> The mempool library assigns handler ops indexes based on the dynamic load
> order of mempool handlers. Indexes are used so a mempool can be used by
> multiple processes, but this only works if all processes agree on the
> mapping from index to mempool handler.
>
> When using the '-d' argument, it's possible for different processes to load
> mempool handlers in different orders, and thus have different
> index->handler mappings. Using a mempool in multiple of such processes will
> result in undefined behavior.
>
> This commit adds a note to the mempool library programmer's guide warning
> users against this.
>
> Fixes: 449c49b93a6b ("mempool: support handler operations")
> Cc: stable@dpdk.org
>
> Signed-off-by: Gage Eads <gage.eads@intel.com>
> ---
>   doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
>   1 file changed, 7 insertions(+)
>
> diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst
> index 52a569f57..4470f6b38 100644
> --- a/doc/guides/prog_guide/mempool_lib.rst
> +++ b/doc/guides/prog_guide/mempool_lib.rst
> @@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting
>   (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of
>   an alternative mempool handler.
>   
> +  .. note::
> +
> +    When running a DPDK application with shared libraries, mempool handler
> +    shared objects specified with the '-d' EAL command-line parameter are
> +    dynamically loaded. When running a multi-process application with shared
> +    libraries, the -d arguments for mempool handlers *must be specified in the
> +    same order for all processes* to ensure correct operation.
>   

One more empty line is required here, other than that:
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

I think it is OK as a fix for stable branches.

In theory I think it is still technically possible to guarantee ops 
indices correctness
using dedicated memzone (as Olivier suggested), but with sync up on
rte_mempool_{get,set}_ops() (which should be called when EAL is 
initialized) and
reordering. However, it requires API breakage to avoid returning of 
ops_index
on register (since it may change in secondary process on resync when EAL is
already initialized).

Re: [dpdk-dev] [PATCH] doc: add multi-proc shared lib mempool note

[Eads, Gage]

> -----Original Message-----
> From: Andrew Rybchenko [mailto:arybchenko@solarflare.com]
> Sent: Thursday, June 20, 2019 1:01 PM
> To: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org
> Cc: olivier.matz@6wind.com; stable@dpdk.org
> Subject: Re: [PATCH] doc: add multi-proc shared lib mempool note
> 
> On 6/18/19 9:48 PM, Gage Eads wrote:
> > The mempool library assigns handler ops indexes based on the dynamic
> > load order of mempool handlers. Indexes are used so a mempool can be
> > used by multiple processes, but this only works if all processes agree
> > on the mapping from index to mempool handler.
> >
> > When using the '-d' argument, it's possible for different processes to
> > load mempool handlers in different orders, and thus have different
> > index->handler mappings. Using a mempool in multiple of such processes
> > index->will
> > result in undefined behavior.
> >
> > This commit adds a note to the mempool library programmer's guide
> > warning users against this.
> >
> > Fixes: 449c49b93a6b ("mempool: support handler operations")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Gage Eads <gage.eads@intel.com>
> > ---
> >   doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
> >   1 file changed, 7 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/mempool_lib.rst
> > b/doc/guides/prog_guide/mempool_lib.rst
> > index 52a569f57..4470f6b38 100644
> > --- a/doc/guides/prog_guide/mempool_lib.rst
> > +++ b/doc/guides/prog_guide/mempool_lib.rst
> > @@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``,
> there is a config setting
> >   (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to
> make use of
> >   an alternative mempool handler.
> >
> > +  .. note::
> > +
> > +    When running a DPDK application with shared libraries, mempool
> handler
> > +    shared objects specified with the '-d' EAL command-line parameter are
> > +    dynamically loaded. When running a multi-process application with
> shared
> > +    libraries, the -d arguments for mempool handlers *must be specified in
> the
> > +    same order for all processes* to ensure correct operation.
> >
> 
> One more empty line is required here, other than that:
> Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

Can do. Just for my understanding, why is the extra empty line required?

> 
> I think it is OK as a fix for stable branches.
> 
> In theory I think it is still technically possible to guarantee ops indices
> correctness using dedicated memzone (as Olivier suggested), but with sync
> up on
> rte_mempool_{get,set}_ops() (which should be called when EAL is
> initialized) and
> reordering. However, it requires API breakage to avoid returning of
> ops_index on register (since it may change in secondary process on resync
> when EAL is already initialized).

Re: [dpdk-dev] [PATCH] doc: add multi-proc shared lib mempool note

[Andrew Rybchenko]

On 6/20/19 10:07 PM, Eads, Gage wrote:
>> -----Original Message-----
>> From: Andrew Rybchenko [mailto:arybchenko@solarflare.com]
>> Sent: Thursday, June 20, 2019 1:01 PM
>> To: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org
>> Cc: olivier.matz@6wind.com; stable@dpdk.org
>> Subject: Re: [PATCH] doc: add multi-proc shared lib mempool note
>>
>> On 6/18/19 9:48 PM, Gage Eads wrote:
>>> The mempool library assigns handler ops indexes based on the dynamic
>>> load order of mempool handlers. Indexes are used so a mempool can be
>>> used by multiple processes, but this only works if all processes agree
>>> on the mapping from index to mempool handler.
>>>
>>> When using the '-d' argument, it's possible for different processes to
>>> load mempool handlers in different orders, and thus have different
>>> index->handler mappings. Using a mempool in multiple of such processes
>>> index->will
>>> result in undefined behavior.
>>>
>>> This commit adds a note to the mempool library programmer's guide
>>> warning users against this.
>>>
>>> Fixes: 449c49b93a6b ("mempool: support handler operations")
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Gage Eads <gage.eads@intel.com>
>>> ---
>>>    doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
>>>    1 file changed, 7 insertions(+)
>>>
>>> diff --git a/doc/guides/prog_guide/mempool_lib.rst
>>> b/doc/guides/prog_guide/mempool_lib.rst
>>> index 52a569f57..4470f6b38 100644
>>> --- a/doc/guides/prog_guide/mempool_lib.rst
>>> +++ b/doc/guides/prog_guide/mempool_lib.rst
>>> @@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``,
>> there is a config setting
>>>    (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to
>> make use of
>>>    an alternative mempool handler.
>>>
>>> +  .. note::
>>> +
>>> +    When running a DPDK application with shared libraries, mempool
>> handler
>>> +    shared objects specified with the '-d' EAL command-line parameter are
>>> +    dynamically loaded. When running a multi-process application with
>> shared
>>> +    libraries, the -d arguments for mempool handlers *must be specified in
>> the
>>> +    same order for all processes* to ensure correct operation.
>>>
>> One more empty line is required here, other than that:
>> Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
> Can do. Just for my understanding, why is the extra empty line required?

https://doc.dpdk.org/guides/contributing/documentation.html#whitespace
    * Add 2 blank lines before each section header.

[dpdk-dev] [PATCH v2] doc: add multi-proc shared lib mempool note

[Gage Eads]

The mempool library assigns handler ops indexes based on the dynamic load
order of mempool handlers. Indexes are used so a mempool can be used by
multiple processes, but this only works if all processes agree on the
mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load
mempool handlers in different orders, and thus have different
index->handler mappings. Using a mempool in multiple of such processes will
result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning
users against this.

Fixes: 449c49b93a6b ("mempool: support handler operations")
Cc: stable@dpdk.org

Signed-off-by: Gage Eads <gage.eads@intel.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/prog_guide/mempool_lib.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

v2: add another empty line

diff --git a/doc/guides/prog_guide/mempool_lib.rst b/doc/guides/prog_guide/mempool_lib.rst
index 52a569f57..3bb84b0a6 100644
--- a/doc/guides/prog_guide/mempool_lib.rst
+++ b/doc/guides/prog_guide/mempool_lib.rst
@@ -133,6 +133,14 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting
 (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of
 an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool handler
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with shared
+    libraries, the -d arguments for mempool handlers *must be specified in the
+    same order for all processes* to ensure correct operation.
+
 
 Use Cases
 ---------
-- 
2.13.6

Re: [dpdk-dev] [dpdk-stable] [PATCH v2] doc: add multi-proc shared lib mempool note

[Thomas Monjalon]

21/06/2019 00:07, Gage Eads:
> The mempool library assigns handler ops indexes based on the dynamic load
> order of mempool handlers. Indexes are used so a mempool can be used by
> multiple processes, but this only works if all processes agree on the
> mapping from index to mempool handler.
> 
> When using the '-d' argument, it's possible for different processes to load
> mempool handlers in different orders, and thus have different
> index->handler mappings. Using a mempool in multiple of such processes will
> result in undefined behavior.
> 
> This commit adds a note to the mempool library programmer's guide warning
> users against this.
> 
> Fixes: 449c49b93a6b ("mempool: support handler operations")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Gage Eads <gage.eads@intel.com>
> Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>

Applied, thanks