From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 36DDC376E for ; Fri, 27 Feb 2015 12:17:32 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga102.fm.intel.com with ESMTP; 27 Feb 2015 03:17:30 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,659,1418112000"; d="scan'208";a="658044412" Received: from irsmsx110.ger.corp.intel.com ([163.33.3.25]) by orsmga001.jf.intel.com with ESMTP; 27 Feb 2015 03:17:30 -0800 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.117]) by irsmsx110.ger.corp.intel.com ([169.254.15.236]) with mapi id 14.03.0195.001; Fri, 27 Feb 2015 11:17:28 +0000 From: "Ananyev, Konstantin" To: "vadim.suraev@gmail.com" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] rte_mbuf: scattered pktmbufs freeing optimization Thread-Index: AQHQUhpF34jL5PPo00ycUKUBx6MD550EVXVg Date: Fri, 27 Feb 2015 11:17:28 +0000 Message-ID: <2601191342CEEE43887BDE71AB977258213F2C93@irsmsx105.ger.corp.intel.com> References: <1424992506-20484-1-git-send-email-vadim.suraev@gmail.com> In-Reply-To: <1424992506-20484-1-git-send-email-vadim.suraev@gmail.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] rte_mbuf: scattered pktmbufs freeing optimization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 11:17:32 -0000 Hi Vadim, > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of vadim.suraev@gmail.c= om > Sent: Thursday, February 26, 2015 11:15 PM > To: dev@dpdk.org > Subject: [dpdk-dev] [PATCH] rte_mbuf: scattered pktmbufs freeing optimiza= tion >=20 > From: "vadim.suraev@gmail.com" >=20 > new function - rte_pktmbuf_free_bulk makes freeing long > scattered (chained) pktmbufs belonging to the same pool > more optimal using rte_mempool_put_bulk rather than calling > rte_mempool_put for each segment. > Inlike rte_pktmbuf_free, which calls rte_pktmbuf_free_seg, > this function calls __rte_pktmbuf_prefree_seg. If non-NULL > returned, the pointer is placed in an array. When array is > filled or when the last segment is processed, rte_mempool_put_bulk > is called. In case of multiple producers, performs 3 times better. >=20 >=20 > Signed-off-by: vadim.suraev@gmail.com > --- > lib/librte_mbuf/rte_mbuf.h | 55 ++++++++++++++++++++++++++++++++++++++= ++++++ > 1 file changed, 55 insertions(+) >=20 > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h > index 17ba791..1d6f848 100644 > --- a/lib/librte_mbuf/rte_mbuf.h > +++ b/lib/librte_mbuf/rte_mbuf.h > @@ -824,6 +824,61 @@ static inline void rte_pktmbuf_free(struct rte_mbuf = *m) > } > } >=20 > +/* This macro defines the size of max bulk of mbufs to free for rte_pktm= buf_free_bulk */ > +#define MAX_MBUF_FREE_SIZE 32 > + > +/* If RTE_LIBRTE_MBUF_DEBUG is enabled, checks if all mbufs must belong = to the same mempool */ > +#ifdef RTE_LIBRTE_MBUF_DEBUG > + > +#define RTE_MBUF_MEMPOOL_CHECK1(m) struct rte_mempool *first_buffers_mem= pool =3D (m) ? (m)->pool : NULL > + > +#define RTE_MBUF_MEMPOOL_CHECK2(m) RTE_MBUF_ASSERT(first_buffers_mempool= =3D=3D (m)->pool) > + > +#else > + > +#define RTE_MBUF_MEMPOOL_CHECK1(m) > + > +#define RTE_MBUF_MEMPOOL_CHECK2(m) > + > +#endif > + > +/** > + * Free chained (scattered) mbuf into its original mempool. > + * > + * All the mbufs in the chain must belong to the same mempool. Seems really useful. One thought - why to introduce the limitation that all mbufs have to be fro= m the same mempool? I think you can reorder it a bit, so it can handle situation when chained m= bufs belong to different mempools. Something like: ... mbufs[mbufs_count] =3D head; if (unlikely (head->mp !=3D mbufs[0]->mp || mbufs_count =3D=3D RTE_DIM(mbuf= s) - 1)) { rte_mempool_put_bulk(mbufs[0]->pool, mbufs, mbufs_count); mbufs[0] =3D mbufs[mbufs_count]; mbufs_count =3D 0; }=20 mbufs_count++; ... =20 Another nit: probably better name it rte_pktmbuf_free_chain() or something? For me _bulk implies that we have an array of mbufs that we need to free. Actually probably would be another useful function to have: rte_pktmbuf_free_seg_bulk(struct rte_mbuf *m[], uint32_t num); Konstantin > + * > + * @param head > + * The head of mbufs to be freed chain > + */ > + > +static inline void __attribute__((always_inline)) > +rte_pktmbuf_free_bulk(struct rte_mbuf *head) > +{ > + void *mbufs[MAX_MBUF_FREE_SIZE]; > + unsigned mbufs_count =3D 0; > + struct rte_mbuf *next; > + > + RTE_MBUF_MEMPOOL_CHECK1(head); > + > + while(head) { > + next =3D head->next; > + head->next =3D NULL; > + if(__rte_pktmbuf_prefree_seg(head)) { > + RTE_MBUF_ASSERT(rte_mbuf_refcnt_read(head) =3D=3D 0); > + RTE_MBUF_MEMPOOL_CHECK2(head); > + mbufs[mbufs_count++] =3D head; > + } > + head =3D next; > + if(mbufs_count =3D=3D MAX_MBUF_FREE_SIZE) { > + rte_mempool_put_bulk(((struct rte_mbuf *)mbufs[0])->pool,mbu= fs,mbufs_count); > + mbufs_count =3D 0; > + } > + } > + if(mbufs_count > 0) { > + rte_mempool_put_bulk(((struct rte_mbuf *)mbufs[0])->pool,mbufs,m= bufs_count); > + } > +} > + > /** > * Creates a "clone" of the given packet mbuf. > * > -- > 1.7.9.5