From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 318A5A052E; Mon, 9 Mar 2020 10:06:01 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9690B1C011; Mon, 9 Mar 2020 10:06:00 +0100 (CET) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by dpdk.org (Postfix) with ESMTP id 77C1A1C00D for ; Mon, 9 Mar 2020 10:05:59 +0100 (CET) Received: by mail-wr1-f66.google.com with SMTP id n7so9945716wrt.11 for ; Mon, 09 Mar 2020 02:05:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind.com; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=fBI2ccjn1+ApY4BmzBT4Lp92kD4QBqfIqSGoC+sNF3w=; b=eo3+JvnAXSIKYCdQaDRzajpeM3kLPq2iK9UF51gHDHNKFL0z84NS8kufn59SfUtXPz dDqGnh3SgDIXSTcYifzGb6tNscXdMtdO+cJogkXpXacnGkskDUXPpqjM8goe465objXC rYQvfENlC062qSRtVRVLW6E0G6UJVJMH+nG/IkCX0+W9ZAcC6EQCROf6jucbiPNIzBz4 j2bdoQW6VY7bjhaYjsXX3DSnVWS9Gth/GS4DMmwhzT0uBSCYs9wVprddxzDfNrfEjmkx 8HfwCXNdt7CVzaat2/soW0IPkK7i1qquQ2AL7BOvudt5yS5MBUUpdvC1GWy9DaSgH+JJ 3Wvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=fBI2ccjn1+ApY4BmzBT4Lp92kD4QBqfIqSGoC+sNF3w=; b=e24uEvqCDdUddj9c8nFJZ8UcT4tOtq8fHl4Q/ZSVz2tdpC/SP88h+rw+yY+lhxf0UP DgG3/SN3tu94PU+pDKidZ0aVwjMCmYv8Nq+5S819c6EqDcKePO1/jDviWdAt9owc5g4k Lto7hxrQwx4RKhKmJIeqU6ZrgzO/3gNZjL6inXsawSRQnnXvbc/YKFzKSYvG+jGH2mgq 17+LiTEeYswxb9uBxFrzl9DpRiaA+Uw7N6yXrh/UTr0Rcczq45TozPUoQa8L6AJq6CRJ c+wENkJCuU3tJDYYC2JqHnd9Cp4gdORAiG3OhAtdiimv0Jwqy/t52OEo2FKrbx4ORI2N ogEg== X-Gm-Message-State: ANhLgQ04C/kcmeStq03FMRsaM2Uz2/sQOjCogmUMfTHHKjHNyXi6yi59 mcd4yhTJPn8KWsZzcNg6EGb6tg== X-Google-Smtp-Source: ADFU+vvU6W7DJb4/Fums5nJRDYx6vZEf1HXAiaXojjIG110MncyMI23X0I7vKwUoGKQqdrjpdt//HA== X-Received: by 2002:adf:ee4f:: with SMTP id w15mr20959544wro.254.1583744759158; Mon, 09 Mar 2020 02:05:59 -0700 (PDT) Received: from 6wind.com (2a01cb0c0005a600345636f7e65ed1a0.ipv6.abo.wanadoo.fr. [2a01:cb0c:5:a600:3456:36f7:e65e:d1a0]) by smtp.gmail.com with ESMTPSA id w22sm25816885wmk.34.2020.03.09.02.05.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Mar 2020 02:05:58 -0700 (PDT) Date: Mon, 9 Mar 2020 10:05:57 +0100 From: Olivier Matz To: Tonghao Zhang Cc: Andrew Rybchenko , Jerin Jacob , dpdk-dev , Gage Eads , "Artem V. Andreev" , Jerin Jacob , Nithin Dabilpuram , Vamsi Attunuru , Hemant Agrawal Message-ID: <20200309090557.GP13822@platinum> References: <1583114253-15345-1-git-send-email-xiangxia.m.yue@gmail.com> <1583501776-9958-1-git-send-email-xiangxia.m.yue@gmail.com> <7420c590-4906-34e2-b0b8-d412df9005c8@solarflare.com> <20200309082705.GM13822@platinum> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Subject: Re: [dpdk-dev] [PATCH dpdk-dev v3] mempool: sort the rte_mempool_ops by name X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Mar 09, 2020 at 04:55:28PM +0800, Tonghao Zhang wrote: > On Mon, Mar 9, 2020 at 4:27 PM Olivier Matz wrote: > > > > Hi, > > > > On Mon, Mar 09, 2020 at 11:01:25AM +0800, Tonghao Zhang wrote: > > > On Sat, Mar 7, 2020 at 8:54 PM Andrew Rybchenko > > > wrote: > > > > > > > > On 3/7/20 3:51 PM, Andrew Rybchenko wrote: > > > > > On 3/6/20 4:37 PM, Jerin Jacob wrote: > > > > >> On Fri, Mar 6, 2020 at 7:06 PM wrote: > > > > >>> From: Tonghao Zhang > > > > >>> > > > > >>> The order of mempool initiation affects mempool index in the > > > > >>> rte_mempool_ops_table. For example, when building APPs with: > > > > >>> > > > > >>> $ gcc -lrte_mempool_bucket -lrte_mempool_ring ... > > > > >>> > > > > >>> The "bucket" mempool will be registered firstly, and its index > > > > >>> in table is 0 while the index of "ring" mempool is 1. DPDK > > > > >>> uses the mk/rte.app.mk to build APPs, and others, for example, > > > > >>> Open vSwitch, use the libdpdk.a or libdpdk.so to build it. > > > > >>> The mempool lib linked in dpdk and Open vSwitch is different. > > > > >>> > > > > >>> The mempool can be used between primary and secondary process, > > > > >>> such as dpdk-pdump and pdump-pmd/Open vSwitch(pdump enabled). > > > > >>> There will be a crash because dpdk-pdump creates the "ring_mp_mc" > > > > >>> ring which index in table is 0, but the index of "bucket" ring > > > > >>> is 0 in Open vSwitch. If Open vSwitch use the index 0 to get > > > > >>> mempool ops and malloc memory from mempool. The crash will occur: > > > > >>> > > > > >>> bucket_dequeue (access null and crash) > > > > >>> rte_mempool_get_ops (should get "ring_mp_mc", > > > > >>> but get "bucket" mempool) > > > > >>> rte_mempool_ops_dequeue_bulk > > > > >>> ... > > > > >>> rte_pktmbuf_alloc > > > > >>> rte_pktmbuf_copy > > > > >>> pdump_copy > > > > >>> pdump_rx > > > > >>> rte_eth_rx_burst > > > > >>> > > > > >>> To avoid the crash, there are some solution: > > > > >>> * constructor priority: Different mempool uses different > > > > >>> priority in RTE_INIT, but it's not easy to maintain. > > > > >>> > > > > >>> * change mk/rte.app.mk: Change the order in mk/rte.app.mk to > > > > >>> be same as libdpdk.a/libdpdk.so, but when adding a new mempool > > > > >>> driver in future, we must make sure the order. > > > > >>> > > > > >>> * register mempool orderly: Sort the mempool when registering, > > > > >>> so the lib linked will not affect the index in mempool table. > > > > >>> > > > > >>> Signed-off-by: Tonghao Zhang > > > > >>> Acked-by: Olivier Matz > > > > >> Acked-by: Jerin Jacob > > > > > > > > > > The patch is OK, but the fact that ops index changes during > > > > > mempool driver lifetime is frightening. In fact it breaks > > > > > rte_mempool_register_ops() return value semantics (read > > > > > as API break). The return value is not used in DPDK, but it > > > > > is a public function. If I'm not mistaken it should be taken > > > > > into account. > > > > Good points. > > > > The fact that the ops index changes during mempool driver lifetime is > > indeed frightening, especially knowning that this is a dynamic > > registration that could happen at any moment in the life of the > > application. Also, breaking the ABI is not desirable. > That solution is better. > > > Let me try to propose something else to solve your issue: > > > > 1/ At init, the primary process allocates a struct in shared memory > > (named memzone): > > > > struct rte_mempool_shared_ops { > > size_t num_mempool_ops; > > struct { > > char name[RTE_MEMPOOL_OPS_NAMESIZE]; > > } mempool_ops[RTE_MEMPOOL_MAX_OPS_IDX]; > > char *mempool_ops_name[RTE_MEMPOOL_MAX_OPS_IDX]; oops I forgot to remove this line (replaced by mini-struct just above). > > rte_spinlock_t mempool; > > } > > > > 2/ When we register a mempool ops, we first get a name and id from the > > shared struct: with the lock held, lookup for the registered name and > > return its index, else get the last id and copy the name in the struct. > > > > 3/ Then do as before (in the per-process global table), except that we > > reuse the registered id. > > > > We can remove the num_ops field from rte_mempool_ops_table. > > > > Thoughts? > > > > > > > Yes, should update the doc: how about this: > > > > > > diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_mempool.h > > > index c90cf31..5a9c8a7 100644 > > > --- a/lib/librte_mempool/rte_mempool.h > > > +++ b/lib/librte_mempool/rte_mempool.h > > > @@ -904,7 +904,9 @@ int rte_mempool_ops_get_info(const struct rte_mempool *mp, > > > * @param ops > > > * Pointer to an ops structure to register. > > > * @return > > > - * - >=0: Success; return the index of the ops struct in the table. > > > + * - >=0: Success; return the index of the last ops struct in the table. > > > + * The number of the ops struct registered is equal to index > > > + * returned + 1. > > > * - -EINVAL - some missing callbacks while registering ops struct. > > > * - -ENOSPC - the maximum number of ops structs has been reached. > > > */ > > > diff --git a/lib/librte_mempool/rte_mempool_ops.c > > > b/lib/librte_mempool/rte_mempool_ops.c > > > index b0da096..053f340 100644 > > > --- a/lib/librte_mempool/rte_mempool_ops.c > > > +++ b/lib/librte_mempool/rte_mempool_ops.c > > > @@ -26,7 +26,11 @@ struct rte_mempool_ops_table rte_mempool_ops_table = { > > > return strcmp(m_a->name, m_b->name); > > > } > > > > > > -/* add a new ops struct in rte_mempool_ops_table, return its index. */ > > > +/* > > > + * add a new ops struct in rte_mempool_ops_table. > > > + * on success, return the index of the last ops > > > + * struct in the table. > > > + */ > > > int > > > rte_mempool_register_ops(const struct rte_mempool_ops *h) > > > { > > > > > Also I remember patches which warn about above behaviour > > > > > in documentation. If behaviour changes, corresponding > > > > > documentation must be updated. > > > > > > > > One more point. If the patch is finally accepted it definitely > > > > deserves few lines in release notes. > > > OK, a separate patch should be sent before DPDK 20.05 release ? > > > > > > > > > > > > > -- > > > Thanks, > > > Tonghao > > > > -- > Thanks, > Tonghao