From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id E22281288 for ; Thu, 22 Jan 2015 10:52:51 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP; 22 Jan 2015 01:52:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="443625956" Received: from irsmsx154.ger.corp.intel.com ([163.33.192.96]) by FMSMGA003.fm.intel.com with ESMTP; 22 Jan 2015 01:39:27 -0800 Received: from irsmsx104.ger.corp.intel.com ([169.254.5.229]) by IRSMSX154.ger.corp.intel.com ([169.254.12.111]) with mapi id 14.03.0195.001; Thu, 22 Jan 2015 09:52:48 +0000 From: "Walukiewicz, Miroslaw" To: "Liang, Cunming" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL thread Thread-Index: AQHQNhwPyqRd0DBtV0SaZJL7/aG+LZzL4v+g Date: Thu, 22 Jan 2015 09:52:47 +0000 Message-ID: <7C4248CAE043B144B1CD242D275626532FE3BEE9@IRSMSX104.ger.corp.intel.com> References: <1417589628-43666-1-git-send-email-cunming.liang@intel.com> <1421914598-2747-1-git-send-email-cunming.liang@intel.com> <1421914598-2747-14-git-send-email-cunming.liang@intel.com> In-Reply-To: <1421914598-2747-14-git-send-email-cunming.liang@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2015 09:52:52 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Cunming Liang > Sent: Thursday, January 22, 2015 9:17 AM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL > thread >=20 > For non-EAL thread, bypass per lcore cache, directly use ring pool. > It allows using rte_mempool in either EAL thread or any user pthread. > As in non-EAL thread, it directly rely on rte_ring and it's none preempti= ve. > It doesn't suggest to run multi-pthread/cpu which compete the > rte_mempool. > It will get bad performance and has critical risk if scheduling policy is= RT. >=20 > Signed-off-by: Cunming Liang > --- > lib/librte_mempool/rte_mempool.h | 18 +++++++++++------- > 1 file changed, 11 insertions(+), 7 deletions(-) >=20 > diff --git a/lib/librte_mempool/rte_mempool.h > b/lib/librte_mempool/rte_mempool.h > index 3314651..4845f27 100644 > --- a/lib/librte_mempool/rte_mempool.h > +++ b/lib/librte_mempool/rte_mempool.h > @@ -198,10 +198,12 @@ struct rte_mempool { > * Number to add to the object-oriented statistics. > */ > #ifdef RTE_LIBRTE_MEMPOOL_DEBUG > -#define __MEMPOOL_STAT_ADD(mp, name, n) do { \ > - unsigned __lcore_id =3D rte_lcore_id(); \ > - mp->stats[__lcore_id].name##_objs +=3D n; \ > - mp->stats[__lcore_id].name##_bulk +=3D 1; \ > +#define __MEMPOOL_STAT_ADD(mp, name, n) do { \ > + unsigned __lcore_id =3D rte_lcore_id(); \ > + if (__lcore_id < RTE_MAX_LCORE) { \ > + mp->stats[__lcore_id].name##_objs +=3D n; \ > + mp->stats[__lcore_id].name##_bulk +=3D 1; \ > + } \ > } while(0) > #else > #define __MEMPOOL_STAT_ADD(mp, name, n) do {} while(0) > @@ -767,8 +769,9 @@ __mempool_put_bulk(struct rte_mempool *mp, > void * const *obj_table, > __MEMPOOL_STAT_ADD(mp, put, n); >=20 > #if RTE_MEMPOOL_CACHE_MAX_SIZE > 0 > - /* cache is not enabled or single producer */ > - if (unlikely(cache_size =3D=3D 0 || is_mp =3D=3D 0)) > + /* cache is not enabled or single producer or none EAL thread */ I don't understand this limitation.=20 I see that the rte_membuf.h defines table per RTE_MAX_LCORE like below=20 #if RTE_MEMPOOL_CACHE_MAX_SIZE > 0 /** Per-lcore local cache. */ struct rte_mempool_cache local_cache[RTE_MAX_LCORE]; #endif But why we cannot extent the size of the local cache table to something lik= e RTE_MAX_THREADS that does not exceed max value of rte_lcore_id() Keeping this condition here is a real performance killer!!!!!!.=20 I saw in my test application spending more 95% of CPU time reading the atom= ic in M C/MP ring utilizing access to mempool.=20 Same comment for get operation below > + if (unlikely(cache_size =3D=3D 0 || is_mp =3D=3D 0 || > + lcore_id >=3D RTE_MAX_LCORE)) > goto ring_enqueue; >=20 > /* Go straight to ring if put would overflow mem allocated for cache > */ > @@ -952,7 +955,8 @@ __mempool_get_bulk(struct rte_mempool *mp, void > **obj_table, > uint32_t cache_size =3D mp->cache_size; >=20 > /* cache is not enabled or single consumer */ > - if (unlikely(cache_size =3D=3D 0 || is_mc =3D=3D 0 || n >=3D cache_size= )) > + if (unlikely(cache_size =3D=3D 0 || is_mc =3D=3D 0 || > + n >=3D cache_size || lcore_id >=3D RTE_MAX_LCORE)) > goto ring_dequeue; >=20 > cache =3D &mp->local_cache[lcore_id]; > -- > 1.8.1.4