From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 74F945A15 for ; Thu, 22 Jan 2015 13:21:53 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP; 22 Jan 2015 04:16:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="443676915" Received: from kmsmsx153.gar.corp.intel.com ([172.21.73.88]) by FMSMGA003.fm.intel.com with ESMTP; 22 Jan 2015 04:08:28 -0800 Received: from shsmsx104.ccr.corp.intel.com (10.239.110.15) by KMSMSX153.gar.corp.intel.com (172.21.73.88) with Microsoft SMTP Server (TLS) id 14.3.195.1; Thu, 22 Jan 2015 20:20:20 +0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.238]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.231]) with mapi id 14.03.0195.001; Thu, 22 Jan 2015 20:20:12 +0800 From: "Liang, Cunming" To: "Walukiewicz, Miroslaw" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL thread Thread-Index: AQHQNhvw6L2lzBswKkGTZyutKRNjT5zLYAWAgACsdwA= Date: Thu, 22 Jan 2015 12:20:12 +0000 Message-ID: References: <1417589628-43666-1-git-send-email-cunming.liang@intel.com> <1421914598-2747-1-git-send-email-cunming.liang@intel.com> <1421914598-2747-14-git-send-email-cunming.liang@intel.com> <7C4248CAE043B144B1CD242D275626532FE3BEE9@IRSMSX104.ger.corp.intel.com> In-Reply-To: <7C4248CAE043B144B1CD242D275626532FE3BEE9@IRSMSX104.ger.corp.intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jan 2015 12:21:54 -0000 > -----Original Message----- > From: Walukiewicz, Miroslaw > Sent: Thursday, January 22, 2015 5:53 PM > To: Liang, Cunming; dev@dpdk.org > Subject: RE: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL > thread >=20 >=20 >=20 > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Cunming Liang > > Sent: Thursday, January 22, 2015 9:17 AM > > To: dev@dpdk.org > > Subject: [dpdk-dev] [PATCH v1 13/15] mempool: add support to non-EAL > > thread > > > > For non-EAL thread, bypass per lcore cache, directly use ring pool. > > It allows using rte_mempool in either EAL thread or any user pthread. > > As in non-EAL thread, it directly rely on rte_ring and it's none preemp= tive. > > It doesn't suggest to run multi-pthread/cpu which compete the > > rte_mempool. > > It will get bad performance and has critical risk if scheduling policy = is RT. > > > > Signed-off-by: Cunming Liang > > --- > > lib/librte_mempool/rte_mempool.h | 18 +++++++++++------- > > 1 file changed, 11 insertions(+), 7 deletions(-) > > > > diff --git a/lib/librte_mempool/rte_mempool.h > > b/lib/librte_mempool/rte_mempool.h > > index 3314651..4845f27 100644 > > --- a/lib/librte_mempool/rte_mempool.h > > +++ b/lib/librte_mempool/rte_mempool.h > > @@ -198,10 +198,12 @@ struct rte_mempool { > > * Number to add to the object-oriented statistics. > > */ > > #ifdef RTE_LIBRTE_MEMPOOL_DEBUG > > -#define __MEMPOOL_STAT_ADD(mp, name, n) do { \ > > - unsigned __lcore_id =3D rte_lcore_id(); \ > > - mp->stats[__lcore_id].name##_objs +=3D n; \ > > - mp->stats[__lcore_id].name##_bulk +=3D 1; \ > > +#define __MEMPOOL_STAT_ADD(mp, name, n) do { \ > > + unsigned __lcore_id =3D rte_lcore_id(); \ > > + if (__lcore_id < RTE_MAX_LCORE) { \ > > + mp->stats[__lcore_id].name##_objs +=3D n; \ > > + mp->stats[__lcore_id].name##_bulk +=3D 1; \ > > + } \ > > } while(0) > > #else > > #define __MEMPOOL_STAT_ADD(mp, name, n) do {} while(0) > > @@ -767,8 +769,9 @@ __mempool_put_bulk(struct rte_mempool *mp, > > void * const *obj_table, > > __MEMPOOL_STAT_ADD(mp, put, n); > > > > #if RTE_MEMPOOL_CACHE_MAX_SIZE > 0 > > - /* cache is not enabled or single producer */ > > - if (unlikely(cache_size =3D=3D 0 || is_mp =3D=3D 0)) > > + /* cache is not enabled or single producer or none EAL thread */ >=20 > I don't understand this limitation. >=20 > I see that the rte_membuf.h defines table per RTE_MAX_LCORE like below > #if RTE_MEMPOOL_CACHE_MAX_SIZE > 0 > /** Per-lcore local cache. */ > struct rte_mempool_cache local_cache[RTE_MAX_LCORE]; > #endif >=20 > But why we cannot extent the size of the local cache table to something l= ike > RTE_MAX_THREADS that does not exceed max value of rte_lcore_id() >=20 > Keeping this condition here is a real performance killer!!!!!!. > I saw in my test application spending more 95% of CPU time reading the at= omic > in M C/MP ring utilizing access to mempool. [Liang, Cunming] This is the first step to make it work. By Konstantin's comments, shall prevent to allocate unique id by ourselves. And the return value from gettid() is too large as an index. For non-EAL thread performance gap, will think about additional fix patch h= ere. If care about performance, still prefer to choose EAL thread now. >=20 > Same comment for get operation below >=20 > > + if (unlikely(cache_size =3D=3D 0 || is_mp =3D=3D 0 || > > + lcore_id >=3D RTE_MAX_LCORE)) > > goto ring_enqueue; > > > > /* Go straight to ring if put would overflow mem allocated for cache > > */ > > @@ -952,7 +955,8 @@ __mempool_get_bulk(struct rte_mempool *mp, void > > **obj_table, > > uint32_t cache_size =3D mp->cache_size; > > > > /* cache is not enabled or single consumer */ > > - if (unlikely(cache_size =3D=3D 0 || is_mc =3D=3D 0 || n >=3D cache_si= ze)) > > + if (unlikely(cache_size =3D=3D 0 || is_mc =3D=3D 0 || > > + n >=3D cache_size || lcore_id >=3D RTE_MAX_LCORE)) > > goto ring_dequeue; > > > > cache =3D &mp->local_cache[lcore_id]; > > -- > > 1.8.1.4