Re: [dpdk-dev] [PATCH dpdk-dev v3 2/2] mempool: use shared memzone for rte_mempool_ops

[Olivier Matz <olivier.matz@6wind.com>]

Hi,

On Tue, Apr 28, 2020 at 09:22:37PM +0800, Tonghao Zhang wrote:
> On Mon, Apr 27, 2020 at 8:51 PM Tonghao Zhang <xiangxia.m.yue@gmail.com> wrote:
> >
> > On Mon, Apr 27, 2020 at 7:40 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >
> > > 27/04/2020 10:03, Tonghao Zhang:
> > > > On Fri, Apr 17, 2020 at 6:27 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> > > > >
> > > > > 13/04/2020 16:21, xiangxia.m.yue@gmail.com:
> > > > > > The order of mempool initiation affects mempool index in the
> > > > > > rte_mempool_ops_table. For example, when building APPs with:
> > > > > >
> > > > > > $ gcc -lrte_mempool_bucket -lrte_mempool_ring ...
> > > > > >
> > > > > > The "bucket" mempool will be registered firstly, and its index
> > > > > > in table is 0 while the index of "ring" mempool is 1. DPDK
> > > > > > uses the mk/rte.app.mk to build APPs, and others, for example,
> > > > > > Open vSwitch, use the libdpdk.a or libdpdk.so to build it.
> > > > > > The mempool lib linked in dpdk and Open vSwitch is different.
> > > > >
> > > > > We are supposed to use pkg-config to link DPDK.
> > > > > Does the problem appear between a DPDK compiled with meson
> > > > > and an application linked with pkg-config information?
> > Hi Thomas,
> > The library mempool linked order can trigger that problem. but when
> > the library is loaded
> > dynamically, trigger that problem too.
> > as Olivier Matz said:
> > The fact that the ops index changes during mempool driver lifetime is
> > indeed frightening, especially knowning that this is a dynamic
> > registration that could happen at any moment in the life of the
> > application.
> >
> > the message in https://mails.dpdk.org/archives/dev/2020-March/159354.html
> Hi Thomas,
> For first patch, I guess we support a callback for other library, it
> make the codes much cleaner
> at eal layer. Otherwise, if we init for library, we may include their
> header file.
> There is a better solution ?

To summarize my understanding of the issu encountered by Tonghao:

Currently, it is not possible to call memzone_register() from an init
function (registered with RTE_INIT()). This is needed if we want to
store the list of registered mempool ops in a shared memory, available
from multiprocess.

Tonghao's patch 1/2 solves this issue. I tried to find alternatives
to this approach, but none of them seems satisfying:

- use RTE_PMD_REGISTER_VDEV() and rte_vdev_add_custom_scan() instead of
  RTE_INIT() in the MEMPOOL_REGISTER_OPS() macro to delay the
  initialization after eal_init(). This looks too complex (I made a POC
  of it, it someone is interested).

- synchronize mempool ops in shared memory when mempool_create() is
  called in the primary: this would probably works most of the time, but
  it is not a perfect solution as we cannot ensure that the primary
  application will create a mempool before the secondary comes up.

- introduce a mandatory call to rte_mempool_lib_init(): despite it's the
  usual way to initialize libs, this will break compatibility.

> > > > > If the problem really needs to be solved,
> > > > > the EAL patch (first of this series) needs to be discussed
> > > > > and reviewed carefully. I don't imagine it being done in 20.05.
> > > >

OK, let's discuss it once 20.05 is out.

Regards,
Olivier