From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id C19DDA052E;
Mon, 9 Mar 2020 09:56:07 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 8F6641BECF;
Mon, 9 Mar 2020 09:56:07 +0100 (CET)
Received: from mail-qv1-f67.google.com (mail-qv1-f67.google.com
[209.85.219.67]) by dpdk.org (Postfix) with ESMTP id 71014FEB
for <dev@dpdk.org>; Mon, 9 Mar 2020 09:56:06 +0100 (CET)
Received: by mail-qv1-f67.google.com with SMTP id r15so4005169qve.3
for <dev@dpdk.org>; Mon, 09 Mar 2020 01:56:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=LJ8xxFvW0FhzhwcZHhfKM1MhG3J+0DICnJAHjBx93O0=;
b=QfnPo7J4SJDZ8rvOHLedy+u+3+igdm41CfSmoKN0xVuInRkq30H76q7DoMX+oTZhSb
SlYPhk3HpxVygdqwLuuLGG8FEQ63/4yH6v+9rS7/UZW61RxmmZCsScQKsln7ftxKj9l3
v7gTF4eelPXDpz73bBHA2QmWfACywXC79CTJBvx197Jhv1Ng8San12WjQO4HL1gEEHHr
JLYgTMLjCDvmXu/RyHLUVoIiry36wWbgcmuRUGq2NN4pbXC2F0THTU+TMguADpe8jRxp
WhaORM+iEtkp0ZtxYgAf6SFOxBdtQV0v8WLuqDaV5TzhYIkaqUEPwWPOkU1E22y6x/I/
mQIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=LJ8xxFvW0FhzhwcZHhfKM1MhG3J+0DICnJAHjBx93O0=;
b=RFAV2sct24bO7mV0vDp3xi4LcaRYEn969FD9FgSFTeuUWLX3LyPgZlqi7cj/drdLfH
5uo41vR5AJHwXNo1+yMMP+MHFkkB9i4MFQKmvDELULiZnfuEIefrRX2P58UYxpkIXeO6
z1AXKheQpp0AKHCvY3QrHa8cNkWnussE8mL5Yu4gwy4qUY0FyPof+VvWpVTlRr69XU4n
hXCM+VGvrjwcOSQnhPYM/fkc/yBz+Lar4oymat9CfsTIhQ22fbjRvxkwgO5Pc360GYYR
AenkeX3wUlyGOyw3MF+/DP8rJ05GB4t2lHsuQpVOlihU7aOM7jnYO6AqLlmTgkFTy0kt
TdEw==
X-Gm-Message-State: ANhLgQ1gaDjMgaHxUZKaRqpFtEsTbplopLoE1OVWF3f2kVpIAjee9pox
LcGYUuqJjixY3tVf9Ym0IgqdGeZwD72eh2qm7N8=
X-Google-Smtp-Source: ADFU+vthH9ckpCmPX3WsWQNs8hF7a+cmq0hgYxf23v+yQ2hsD96w7+0X4i7e8vAroSVDljCUnoU33JIZcSHzYkmTvV8=
X-Received: by 2002:a05:6214:17c3:: with SMTP id
cu3mr8313328qvb.135.1583744165684;
Mon, 09 Mar 2020 01:56:05 -0700 (PDT)
MIME-Version: 1.0
References: <1583114253-15345-1-git-send-email-xiangxia.m.yue@gmail.com>
<1583501776-9958-1-git-send-email-xiangxia.m.yue@gmail.com>
<CALBAE1MUEgG_fZXyvHA70PZSp0uoez8tT7dXkasqe=G0qQSZ0g@mail.gmail.com>
<7420c590-4906-34e2-b0b8-d412df9005c8@solarflare.com>
<c15eead5-8ed9-7d6f-8f87-6963d640371b@solarflare.com>
<CAMDZJNVu08PFvygj6Uo+Xt2rkGQ9i6SQ47-THfZsTKhP98LB0g@mail.gmail.com>
<20200309082705.GM13822@platinum>
In-Reply-To: <20200309082705.GM13822@platinum>
From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Date: Mon, 9 Mar 2020 16:55:28 +0800
Message-ID: <CAMDZJNU7B9qhJp0Hq49fSzivqf2budAYj+YENVkbsqPjNJxSRw@mail.gmail.com>
To: Olivier Matz <olivier.matz@6wind.com>
Cc: Andrew Rybchenko <arybchenko@solarflare.com>,
Jerin Jacob <jerinjacobk@gmail.com>,
dpdk-dev <dev@dpdk.org>, Gage Eads <gage.eads@intel.com>,
"Artem V. Andreev" <artem.andreev@oktetlabs.ru>,
Jerin Jacob <jerinj@marvell.com>,
Nithin Dabilpuram <ndabilpuram@marvell.com>,
Vamsi Attunuru <vattunuru@marvell.com>,
Hemant Agrawal <hemant.agrawal@nxp.com>
Content-Type: text/plain; charset="UTF-8"
Subject: Re: [dpdk-dev] [PATCH dpdk-dev v3] mempool: sort the
rte_mempool_ops by name
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On Mon, Mar 9, 2020 at 4:27 PM Olivier Matz <olivier.matz@6wind.com> wrote:
>
> Hi,
>
> On Mon, Mar 09, 2020 at 11:01:25AM +0800, Tonghao Zhang wrote:
> > On Sat, Mar 7, 2020 at 8:54 PM Andrew Rybchenko
> > <arybchenko@solarflare.com> wrote:
> > >
> > > On 3/7/20 3:51 PM, Andrew Rybchenko wrote:
> > > > On 3/6/20 4:37 PM, Jerin Jacob wrote:
> > > >> On Fri, Mar 6, 2020 at 7:06 PM <xiangxia.m.yue@gmail.com> wrote:
> > > >>> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> > > >>>
> > > >>> The order of mempool initiation affects mempool index in the
> > > >>> rte_mempool_ops_table. For example, when building APPs with:
> > > >>>
> > > >>> $ gcc -lrte_mempool_bucket -lrte_mempool_ring ...
> > > >>>
> > > >>> The "bucket" mempool will be registered firstly, and its index
> > > >>> in table is 0 while the index of "ring" mempool is 1. DPDK
> > > >>> uses the mk/rte.app.mk to build APPs, and others, for example,
> > > >>> Open vSwitch, use the libdpdk.a or libdpdk.so to build it.
> > > >>> The mempool lib linked in dpdk and Open vSwitch is different.
> > > >>>
> > > >>> The mempool can be used between primary and secondary process,
> > > >>> such as dpdk-pdump and pdump-pmd/Open vSwitch(pdump enabled).
> > > >>> There will be a crash because dpdk-pdump creates the "ring_mp_mc"
> > > >>> ring which index in table is 0, but the index of "bucket" ring
> > > >>> is 0 in Open vSwitch. If Open vSwitch use the index 0 to get
> > > >>> mempool ops and malloc memory from mempool. The crash will occur:
> > > >>>
> > > >>> bucket_dequeue (access null and crash)
> > > >>> rte_mempool_get_ops (should get "ring_mp_mc",
> > > >>> but get "bucket" mempool)
> > > >>> rte_mempool_ops_dequeue_bulk
> > > >>> ...
> > > >>> rte_pktmbuf_alloc
> > > >>> rte_pktmbuf_copy
> > > >>> pdump_copy
> > > >>> pdump_rx
> > > >>> rte_eth_rx_burst
> > > >>>
> > > >>> To avoid the crash, there are some solution:
> > > >>> * constructor priority: Different mempool uses different
> > > >>> priority in RTE_INIT, but it's not easy to maintain.
> > > >>>
> > > >>> * change mk/rte.app.mk: Change the order in mk/rte.app.mk to
> > > >>> be same as libdpdk.a/libdpdk.so, but when adding a new mempool
> > > >>> driver in future, we must make sure the order.
> > > >>>
> > > >>> * register mempool orderly: Sort the mempool when registering,
> > > >>> so the lib linked will not affect the index in mempool table.
> > > >>>
> > > >>> Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> > > >>> Acked-by: Olivier Matz <olivier.matz@6wind.com>
> > > >> Acked-by: Jerin Jacob <jerinj@marvell.com>
> > > >
> > > > The patch is OK, but the fact that ops index changes during
> > > > mempool driver lifetime is frightening. In fact it breaks
> > > > rte_mempool_register_ops() return value semantics (read
> > > > as API break). The return value is not used in DPDK, but it
> > > > is a public function. If I'm not mistaken it should be taken
> > > > into account.
>
> Good points.
>
> The fact that the ops index changes during mempool driver lifetime is
> indeed frightening, especially knowning that this is a dynamic
> registration that could happen at any moment in the life of the
> application. Also, breaking the ABI is not desirable.
That solution is better.
> Let me try to propose something else to solve your issue:
>
> 1/ At init, the primary process allocates a struct in shared memory
> (named memzone):
>
> struct rte_mempool_shared_ops {
> size_t num_mempool_ops;
> struct {
> char name[RTE_MEMPOOL_OPS_NAMESIZE];
> } mempool_ops[RTE_MEMPOOL_MAX_OPS_IDX];
> char *mempool_ops_name[RTE_MEMPOOL_MAX_OPS_IDX];
> rte_spinlock_t mempool;
> }
>
> 2/ When we register a mempool ops, we first get a name and id from the
> shared struct: with the lock held, lookup for the registered name and
> return its index, else get the last id and copy the name in the struct.
>
> 3/ Then do as before (in the per-process global table), except that we
> reuse the registered id.
>
> We can remove the num_ops field from rte_mempool_ops_table.
>
> Thoughts?
>
>
> > Yes, should update the doc: how about this:
> >
> > diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h
> > index c90cf31..5a9c8a7 100644
> > --- a/lib/librte_mempool/rte_mempool.h
> > +++ b/lib/librte_mempool/rte_mempool.h
> > @@ -904,7 +904,9 @@ int rte_mempool_ops_get_info(const struct rte_mempool *mp,
> > * @param ops
> > * Pointer to an ops structure to register.
> > * @return
> > - * - >=0: Success; return the index of the ops struct in the table.
> > + * - >=0: Success; return the index of the last ops struct in the table.
> > + * The number of the ops struct registered is equal to index
> > + * returned + 1.
> > * - -EINVAL - some missing callbacks while registering ops struct.
> > * - -ENOSPC - the maximum number of ops structs has been reached.
> > */
> > diff --git a/lib/librte_mempool/rte_mempool_ops.c
> > b/lib/librte_mempool/rte_mempool_ops.c
> > index b0da096..053f340 100644
> > --- a/lib/librte_mempool/rte_mempool_ops.c
> > +++ b/lib/librte_mempool/rte_mempool_ops.c
> > @@ -26,7 +26,11 @@ struct rte_mempool_ops_table rte_mempool_ops_table = {
> > return strcmp(m_a->name, m_b->name);
> > }
> >
> > -/* add a new ops struct in rte_mempool_ops_table, return its index. */
> > +/*
> > + * add a new ops struct in rte_mempool_ops_table.
> > + * on success, return the index of the last ops
> > + * struct in the table.
> > + */
> > int
> > rte_mempool_register_ops(const struct rte_mempool_ops *h)
> > {
> > > > Also I remember patches which warn about above behaviour
> > > > in documentation. If behaviour changes, corresponding
> > > > documentation must be updated.
> > >
> > > One more point. If the patch is finally accepted it definitely
> > > deserves few lines in release notes.
> > OK, a separate patch should be sent before DPDK 20.05 release ?
> > >
> >
> >
> > --
> > Thanks,
> > Tonghao
--
Thanks,
Tonghao