From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4AA29A034F; Wed, 10 Nov 2021 18:15:51 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0E27440DDA; Wed, 10 Nov 2021 18:15:51 +0100 (CET) Received: from mail-il1-f171.google.com (mail-il1-f171.google.com [209.85.166.171]) by mails.dpdk.org (Postfix) with ESMTP id 34B0240683 for ; Wed, 10 Nov 2021 18:15:50 +0100 (CET) Received: by mail-il1-f171.google.com with SMTP id j28so3291570ila.1 for ; Wed, 10 Nov 2021 09:15:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TF2mbEOHm2z5gP5RpJ4io1UJOxesctktXQ+VzjZOQGE=; b=j76aYzF3eo/ow9VW1shoCr5uLEwhtFjTpqCXa5tc/hAHC09y3y3+W/sWHMGtOtSUhB zV3726vQgxBeZRzpkv9ieAE9UtgiAiGmSUGEl17zubdf68LkbdyLHBMAuYoJCJA5/NBY RAyUwRZoN4yAdJHkuZp0ov+uMLstAOQfM0+pHA6hIlhvCG0e3NvXsis/dXE96uWk/c9l 1UXUnXrRYyqXlsZtXCRXBubt04rX+vKPE7zCMWIA2wuFsdt5uWvfnL4Rs47Z7Jf/m1J8 5ZSKUMstTjbuCuXY/TCZk89epmlFdkTg7E27ppINggDF2WAYOatBZ3dGQcApefJGHWQs bA6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=TF2mbEOHm2z5gP5RpJ4io1UJOxesctktXQ+VzjZOQGE=; b=okMlEPa6dCydH0dD3ob5o9TYX2zwWwWRqIYZ296dsap4O7IYrSWdrwg6E57tpxwM6L N1QmuHnbfm1BwSWg/ezcLwocYyzTkM+EgbKqpa9wPJrHfOkgCFkYKxMXZ+UAK4DyMD81 K5NyUGpFnmkoeHFgF62maFwR1rO7PWXt0LYUuKvcDlFTfPWjPGgf9pDF9LV3EqHpUCll PkZD9eWC1pyOixQw7GKCEKYKcyT6CqRYkiEX0obE3scTMriENrK9ZiipjoPKsSIv4fRr hDIJ399uwvboHhZIeU/TniZSdiHsxr4QergJDEQE/kwBUwq+GoMlgWWATwwfZyhSHz9n 8Rjw== X-Gm-Message-State: AOAM530k9XcI+FzprUF0/pmUHhpASzcyOhDZ1w2Uy27I9JOQ5n0L2Ftg gE+o2Hk7N79eQHPbPUZC01Fr9wiaSvGDb+azTn4= X-Google-Smtp-Source: ABdhPJyPsxqmua0jDHKa5QqqMg6/wzWMzujXDT0siC7026zYrfXWfXUWo66n3WcFWkTwAZJ8qrdEOq98CpCX53Lb4ck= X-Received: by 2002:a92:d6ca:: with SMTP id z10mr431289ilp.294.1636564549448; Wed, 10 Nov 2021 09:15:49 -0800 (PST) MIME-Version: 1.0 References: <1636559839-6553-1-git-send-email-laitianli@tom.com> In-Reply-To: <1636559839-6553-1-git-send-email-laitianli@tom.com> From: Jerin Jacob Date: Wed, 10 Nov 2021 22:45:22 +0530 Message-ID: Subject: Re: [PATCH] mempool: fix rte primary program coredump To: Tianli Lai Cc: dpdk-dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Nov 10, 2021 at 9:38 PM Tianli Lai wrote: > > the primary program(such as ofp app) run first, then run the secondary > program(such as dpdk-pdump), the primary program would receive signal > SIGSEGV. the function stack as follow: > > aived signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7fffee60e700 (LWP 112613)] > 0x00007ffff5f2cc0b in bucket_stack_pop (stack=3D0xffff00010000) at > /ofp/dpdk/drivers/mempool/bucket/rte_mempool_bucket.c:95 > 95 if (stack->top =3D=3D 0) > Missing separate debuginfos, use: debuginfo-install > glibc-2.17-196.el7.x86_64 libatomic-4.8.5-16.el7.x86_64 > libconfig-1.4.9-5.el7.x86_64 libgcc-4.8.5-16.el7.x86_64 > libpcap-1.5.3-12.el7.x86_64 numactl-libs-2.0.9-6.el7_2.x86_64 > openssl-libs-1.0.2k-8.el7.x86_64 zlib-1.2.7-17.el7.x86_64 > (gdb) bt > #0 0x00007ffff5f2cc0b in bucket_stack_pop (stack=3D0xffff00010000) at /= ofp/dpdk/drivers/mempool/bucket/rte_mempool_bucket.c:95 > #1 0x00007ffff5f2e5dc in bucket_dequeue_orphans (bd=3D0x2209e5fac0,obj_= table=3D0x220b083710, n_orphans=3D251) at /ofp/dpdk/drivers/mempool/bucket/= rte_mempool_bucket.c:190 > #2 0x00007ffff5f30192 in bucket_dequeue (mp=3D0x220b07d5c0,obj_table=3D= 0x220b083710, n=3D251) at /ofp/dpdk/drivers/mempool/bucket/rte_mempool_buck= et.c:288 > #3 0x00007ffff5f47e18 in rte_mempool_ops_dequeue_bulk (mp=3D0x220b07d5c= 0,obj_table=3D0x220b083710, n=3D251) at /ofp/dpdk/x86_64-native-linuxapp-gc= c/include/rte_mempool.h:739 > #4 0x00007ffff5f4819d in __mempool_generic_get (cache=3D0x220b083700, n= =3D1, obj_table=3D0x7fffee5deb18, mp=3D0x220b07d5c0) at /ofp/dpdk/x86_64-na= tive-linuxapp-gcc/include/rte_mempool.h:1443 > #5 rte_mempool_generic_get (cache=3D0x220b083700, n=3D1, obj_table=3D0x= 7fffee5deb18, mp=3D0x220b07d5c0) at /ofp/dpdk/x86_64-native-linuxapp-gcc/in= clude/rte_mempool.h:1506 > #6 rte_mempool_get_bulk (n=3D1, obj_table=3D0x7fffee5deb18, mp=3D0x220b= 07d5c0) at /ofp/dpdk/x86_64-native-linuxapp-gcc/include/rte_mempool.h:1539 > #7 rte_mempool_get (obj_p=3D0x7fffee5deb18, mp=3D0x220b07d5c0) at /ofp/= dpdk/x86_64-native-linuxapp-gcc/include/rte_mempool.h:1565 > #8 rte_mbuf_raw_alloc (mp=3D0x220b07d5c0) at /ofp/dpdk/x86_64-native-li= nuxapp-gcc/include/rte_mbuf.h:551 > #9 0x00007ffff5f483a4 in rte_pktmbuf_alloc (mp=3D0x220b07d5c0) at /ofp/= dpdk/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:804 > #10 0x00007ffff5f4c9d9 in pdump_pktmbuf_copy (m=3D0x220746ad80, mp=3D0x2= 20b07d5c0) at /ofp/dpdk/lib/librte_pdump/rte_pdump.c:99 > #11 0x00007ffff5f4e42e in pdump_copy (pkts=3D0x7fffee5dfdf0, nb_pkts=3D1= , user_params=3D0x7ffff76d7cc0 ) at /ofp/dpdk/lib/librte_pdump/rte_= pdump.c:151 > #12 0x00007ffff5f4eadd in pdump_rx (port=3D0, qidx=3D0, pkts=3D0x7fffee5= dfdf0, nb_pkts=3D1, max_pkts=3D16, user_params=3D0x7ffff76d7cc0 ) a= t /ofp/dpdk/lib/librte_pdump/rte_pdump.c:172 > #13 0x00007ffff5d0e9e8 in rte_eth_rx_burst (port_id=3D0, queue_id=3D0, r= x_pkts=3D0x7fffee5dfdf0, nb_pkts=3D16) at /ofp/dpdk/x86_64-native-linuxapp-= gcc/usr/local/include/dpdk/rte_ethdev.h:4396 > #14 0x00007ffff5d114c3 in recv_pkt_dpdk (pktio_entry=3D0x22005436c0, ind= ex=3D0, pkt_table=3D0x7fffee5dfdf0, num=3D16) at odp_packet_dpdk.c:1081 > #15 0x00007ffff5d2f931 in odp_pktin_recv (queue=3D...,packets=3D0x7fffee= 5dfdf0, num=3D16) at ../linux-generic/odp_packet_io.c:1896 > #16 0x000000000040a344 in rx_burst (pktin=3D...) at app_main.c:223 > #17 0x000000000040aca4 in run_server_single (arg=3D0x7fffffffe2b0) at ap= p_main.c:417 > #18 0x00007ffff7bd6883 in run_thread (arg=3D0x7fffffffe3b8) at threads.c= :67 > #19 0x00007ffff53c8e25 in start_thread () from /lib64/libpthread.so.0 > #20 0x00007ffff433e34d in clone () from /lib64/libc.so.6.c:67 > > The program crash down reason is: > > In primary program and secondary program , the global array rte_mempool_o= ps.ops[]: > primary name secondary name > [0]: "bucket" "ring_mp_mc" > [1]: "dpaa" "ring_sp_sc" > [2]: "dpaa2" "ring_mp_sc" > [3]: "octeontx_fpavf" "ring_sp_mc" > [4]: "octeontx2_npa" "octeontx2_npa" > [5]: "ring_mp_mc" "bucket" > [6]: "ring_sp_sc" "stack" > [7]: "ring_mp_sc" "if_stack" > [8]: "ring_sp_mc" "dpaa" > [9]: "stack" "dpaa2" > [10]: "if_stack" "octeontx_fpavf" > [11]: NULL NULL > > this array in primary program is different with secondary program. > so when secondary program call rte_pktmbuf_pool_create_by_ops() with > mempool name =E2=80=9Cring_mp_mc=E2=80=9D, but the primary program use "= bucket" type > to alloc rte_mbuf. > > so sort this array both primary program and secondary program when init > memzone. > > Signed-off-by: Tianli Lai > --- > lib/librte_eal/common/eal_common_memzone.c | 2 +- > lib/librte_mempool/rte_mempool.h | 6 ++++++ > lib/librte_mempool/rte_mempool_ops.c | 31 ++++++++++++++++++++++++= ++++++ > 3 files changed, 38 insertions(+), 1 deletion(-) > mode change 100644 =3D> 100755 lib/librte_mempool/rte_mempool_ops.c > > diff --git a/lib/librte_eal/common/eal_common_memzone.c b/lib/librte_eal/= common/eal_common_memzone.c > index 99b8d65..b59f3f5 100644 > --- a/lib/librte_eal/common/eal_common_memzone.c > +++ b/lib/librte_eal/common/eal_common_memzone.c > @@ -384,7 +384,7 @@ > } > > rte_rwlock_write_unlock(&mcfg->mlock); > - > + rte_sort_mempool_ops(); > return ret; > } > > diff --git a/lib/librte_mempool/rte_mempool.h b/lib/librte_mempool/rte_me= mpool.h > index f81152a..a22850b 100644 > --- a/lib/librte_mempool/rte_mempool.h > +++ b/lib/librte_mempool/rte_mempool.h > @@ -910,6 +910,12 @@ int rte_mempool_ops_get_info(const struct rte_mempoo= l *mp, > int rte_mempool_register_ops(const struct rte_mempool_ops *ops); > > /** > + * Sort global array rte_mempool_ops_table.ops[] . > + * Used by rte_eal_memzone_init() > + */ > +int rte_sort_mempool_ops(void); Since it is an internal API, No need for rte_ prefix. > + > +/** > * Macro to statically register the ops of a mempool handler. > * Note that the rte_mempool_register_ops fails silently here when > * more than RTE_MEMPOOL_MAX_OPS_IDX is registered. > diff --git a/lib/librte_mempool/rte_mempool_ops.c b/lib/librte_mempool/rt= e_mempool_ops.c > old mode 100644 > new mode 100755 > index 22c5251..8e10488 > --- a/lib/librte_mempool/rte_mempool_ops.c > +++ b/lib/librte_mempool/rte_mempool_ops.c > @@ -68,6 +68,37 @@ struct rte_mempool_ops_table rte_mempool_ops_table =3D= { > return ops_index; > } > > + > +int rte_sort_mempool_ops(void) > +{ > + /* same with rte_mempool_ops.name */ > + static const char *memops_name[RTE_MEMPOOL_MAX_OPS_IDX] =3D { > + "ring_mp_mc", "ring_sp_sc", "ring_mp_sc", "ring_sp_mc", > + "stack", "lf_stack", "octeontx2_npa", "octeontx_fpavf", > + "dpaa2", "dpaa", "bucket", I think, it is not foolproof. I think, either 1) you can use primary - secondary communication mechanism to get the library order from the primary. OR 2) At end of primary rte_eal_init or so, copy the array to memzone and then lookup the memzone by name(string) in secondary to fill this array. > + }; > + struct rte_mempool_ops_table tmp_mempool_ops_table =3D { > + .sl =3D rte_mempool_ops_table.sl, > + .num_ops =3D rte_mempool_ops_table.num_ops > + }; > + uint32_t i =3D 0, j=3D 0; > + struct rte_mempool_ops *ops =3D NULL; > + for (i =3D 0; i < 16; i++) { > + const char* name =3D memops_name[i]; > + if(name && strlen(name)) { > + for(j =3D 0; j < rte_mempool_ops_table.num_ops; j= ++) { > + if(strcmp(name, rte_mempool_ops_table.ops= [j].name)) > + continue; > + ops =3D &rte_mempool_ops_table.ops[j]; > + memcpy(&tmp_mempool_ops_table.ops[i], ops= , sizeof(*ops)); > + break; > + } > + } > + } > + memcpy(&rte_mempool_ops_table, &tmp_mempool_ops_table, sizeof(tmp= _mempool_ops_table)); > + return 0; > +} > + > /* wrapper to allocate an external mempool's private (pool) data. */ > int > rte_mempool_ops_alloc(struct rte_mempool *mp) > -- > 1.8.3.1 >