From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 56CC2212 for ; Tue, 7 Oct 2014 17:35:47 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP; 07 Oct 2014 08:42:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,671,1406617200"; d="scan'208";a="610865784" Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159]) by fmsmga002.fm.intel.com with ESMTP; 07 Oct 2014 08:42:50 -0700 Received: from irsmsx151.ger.corp.intel.com (163.33.192.59) by IRSMSX104.ger.corp.intel.com (163.33.3.159) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 7 Oct 2014 16:42:49 +0100 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.174]) by IRSMSX151.ger.corp.intel.com ([169.254.4.146]) with mapi id 14.03.0195.001; Tue, 7 Oct 2014 16:42:48 +0100 From: "Ananyev, Konstantin" To: "Wiles, Roger Keith (Wind River)" Thread-Topic: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() Thread-Index: AQHP4Ch2aL1BNwlnX0CzjWDe7U+wDpwitckAgABi6wCAAADNAIAAF86A///+ewCAADtJgIAABkgAgADpxACAAEf5gIAAF4gw Date: Tue, 7 Oct 2014 15:42:47 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725821391240@IRSMSX105.ger.corp.intel.com> References: <1412464229-125521-1-git-send-email-keith.wiles@windriver.com> <1412464229-125521-2-git-send-email-keith.wiles@windriver.com> <59AF69C657FD0841A61C55336867B5B03441BE9E@IRSMSX103.ger.corp.intel.com> <5DD5FF6E-C045-4764-A5B1-877C88B023F5@windriver.com> <20141006145330.GA2548@BRICHA3-MOBL> <2601191342CEEE43887BDE71AB97725821390E75@IRSMSX105.ger.corp.intel.com> <545592DF-3306-49F7-8685-10BD021B9854@windriver.com> <1AAECD5E-9A22-481D-9712-C75B8C1FAFC1@windriver.com> <844D44A2-27B2-47F9-BB6D-5A3A2F1757F6@windriver.com> <2601191342CEEE43887BDE71AB97725821391035@IRSMSX105.ger.corp.intel.com> <1CB2412C-DC76-438D-A35F-BB03BA6A2A3A@windriver.com> In-Reply-To: <1CB2412C-DC76-438D-A35F-BB03BA6A2A3A@windriver.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 15:35:48 -0000 Hi Keith, > -----Original Message----- > From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] > Sent: Tuesday, October 07, 2014 3:22 PM > To: Ananyev, Konstantin > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc= _bulk() and rte_pktmbuf_free_bulk() >=20 >=20 > On Oct 7, 2014, at 4:09 AM, Ananyev, Konstantin wrote: >=20 > > > > > >> -----Original Message----- > >> From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] > >> Sent: Monday, October 06, 2014 9:08 PM > >> To: Ananyev, Konstantin > >> Cc: dev@dpdk.org > >> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_al= loc_bulk() and rte_pktmbuf_free_bulk() > >> > >> Attaching to the list does not work. If you want the code let me know = it is only about 5K in size. > >> > >> On Oct 6, 2014, at 2:45 PM, Wiles, Roger Keith wrote: > >> > >>> > >>> On Oct 6, 2014, at 11:13 AM, Wiles, Roger Keith wrote: > >>> > >>>> > >>>> On Oct 6, 2014, at 10:54 AM, Ananyev, Konstantin wrote: > >>>> > >>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richard= son > >>>>>> Sent: Monday, October 06, 2014 3:54 PM > >>>>>> To: Wiles, Roger Keith (Wind River) > >>>>>> Cc: dev@dpdk.org > >>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbu= f_alloc_bulk() and rte_pktmbuf_free_bulk() > >>>>>> > >>>>>> On Mon, Oct 06, 2014 at 03:50:38PM +0100, Wiles, Roger Keith wrote= : > >>>>>>> Hi Bruce, > >>>>>>> > >>>>>>> Do I need to reject the for the new routines or just make sure th= e vector driver does not get updated to use those routines? > >>>>>>> > >>>>>> > >>>>>> The new routines are probably useful in the general case. I see no= issue > >>>>>> with having them in the code, so long as the vector driver is not = modified > >>>>>> to use them. > >>>>> > >>>>> I 'd say the same thing for non-vector RX/TX PMD code-paths too. > >>>>> > >>>>> BTW, are the new functions comments valid? > >>>>> > >>>>> + * @return > >>>>> + * - 0 if the number of mbufs allocated was ok > >>>>> + * - <0 is an ERROR. > >>>>> + */ > >>>>> +static inline int __rte_mbuf_raw_alloc_bulk( > >>>>> > >>>>> Though, as I can see __rte_mbuf_raw_alloc_bulk() returns either: > >>>>> - number of allocated mbuf (cnt) > >>>>> - negative error code > >>>> > >>>> Let me fix up the comments. > >>>>> > >>>>> And: > >>>>> + * @return > >>>>> + * - The number of valid mbufs pointers in the m_list array. > >>>>> + * - Zero if the request cnt could not be allocated. > >>>>> + */ > >>>>> +static inline int __attribute__((always_inline)) > >>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_= list[], int16_t cnt) > >>>>> +{ > >>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); > >>>>> +} > >>>>> > >>>>> Shouldn't be "less than zero if the request cnt could not be alloca= ted."? > >>>>> > >>>>> BTW, is there any point to have __rte_mbuf_raw_alloc_bulk() at all? > >>>>> After all, as you are calling rte_pktmbuf_reset() inside it, it doe= sn't look __raw__ any more. > >>>>> Might be just put its content into rte_pktmbuf_alloc_bulk() and get= rid of it. > >>>>> > >>>> I was just following the non-bulk routine style __rte_mbuf_raw_alloc= (), but I can pull that into a single routine. > >>>> > >>>>> Also wonder, what is the advantage of having multiple counters insi= de the same loop? > >>>>> i.e: > >>>>> + for(i =3D 0; i < cnt; i++) { > >>>>> + m =3D *m_list++; > >>>>> > >>>>> Why not just: > >>>>> > >>>>> for(i =3D 0; i < cnt; i++) { > >>>>> m =3D &m_list[i]; > >>>>> > >>>>> Same for free: > >>>>> + while(npkts--) > >>>>> + rte_pktmbuf_free(*m_list++); > >>>>> > >>>>> While not just: > >>>>> for (i =3D 0; i < npkts; i++) > >>>>> rte_pktmbuf_free(&m_list[i]); > >>>> > >>>> Maybe I have it wrong or the compilers are doing the right thing now= , but at one point the &m_list[i] would cause the compiler > to > >> generate a shift or multiple of 'i' and then add it to the base of m_l= ist. If that is not the case anymore then I can update the code as > >> you suggested. Using the *m_list++ just adds the size of a pointer to = a register and continues. > >>> > >>> I compared the clang assembler (.s file) output from an example test = code I wrote to see if we have any differences in the code > >> using the two styles and I found no difference and the code looked the= same. I am not a Intel assembler expert and I would > suggest > >> someone else determine if it generates different code. I tried to comp= are the GCC outputs and it did look the same to me. > > > > That's was my question: > > Modern compilers are able to generate a good code for a simple loop as = above. > > So what's the point to use 2 iterators inside the loop, when just one i= s enough? > > Nothing wrong technically, but makes code a bit harder to follow. > > Plus, in general, it is a good practise to minimise number of iterators= inside the loop, when possible. > > > > Konstantin >=20 > Hi Konstantin, >=20 > I really do not understand the concern if the code is the same, as it app= ears to me the current patch is very clean and simple. Maybe > you have not seen the v2 patch and now v3 patch I sent this morning to fi= x Bruce's comment suggestion. >=20 > For the case of the free routine your suggestion would require an extra c= ounter/variable a bit more code a 'for' loop instead of a > 'while' loop. My point was that just one iterator for both loops is enough. In general, it is a good practise to minimise number of iterators per loop = if possible: in some cases compiler might get confused and wouldn't be able to eliminat= e redundant iterators itself. =20 Though yes - technically there is nothing wrong with your approach. So if you prefer to keep it as it is - I wouldn't insist.=20 Konstantin > +static inline void __attribute__((always_inline)) > +rte_pktmbuf_free_bulk(struct rte_mbuf *m_list[], int16_t npkts) > +{ > + while(npkts--) > + rte_pktmbuf_free(*m_list++); > +} >=20 > For the case of the alloc routine I did remove the rte_mbuf * m variable = and now I believe it is very clean and changing it to use index > variables is just a personal preference. I personal preference of this ty= pe is not useful IMO and does not cause any harm. Unless you > can suggest a good technical reason to change I am going to leave the pat= ch as is. >=20 > +static inline int __attribute__((always_inline)) > +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_list[]= , int16_t cnt) > +{ > + int ret; > + > + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); > + if ( ret =3D=3D 0 ) { > + ret =3D cnt; > + while(cnt--) { > +#ifdef RTE_MBUF_REFCNT > + rte_mbuf_refcnt_set(*m_list, 1); > +#endif /* RTE_MBUF_REFCNT */ > + rte_pktmbuf_reset(*m_list++); > + } > + } > + return ret; > +} >=20 > >>> > >>> I have attached the code and output, please let me know if I did some= thing wrong, but as it stands using the original style is what I > >> want to go with. > >>> > >>>>> > >>>>> Konstantin > >>>>> > >>>>>> > >>>>>> /Bruce > >>>>>> > >>>>>>> Thanks > >>>>>>> ++Keith > >>>>>>> > >>>>>>> On Oct 6, 2014, at 3:56 AM, Richardson, Bruce wrote: > >>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> -----Original Message----- > >>>>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Keith Wile= s > >>>>>>>>> Sent: Sunday, October 05, 2014 12:10 AM > >>>>>>>>> To: dev@dpdk.org > >>>>>>>>> Subject: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf= _alloc_bulk() > >>>>>>>>> and rte_pktmbuf_free_bulk() > >>>>>>>>> > >>>>>>>>> Minor helper routines to mirror the mempool routines and remove= the code > >>>>>>>>> from applications. The ixgbe_rxtx_vec.c routine could be change= d to use > >>>>>>>>> the ret_pktmbuf_alloc_bulk() routine inplace of rte_mempool_get= _bulk(). > >>>>>>>>> > >>>>>>>> > >>>>>>>> I believe such a change would cause a performance regression, as= the extra init code in the alloc_bulk() function would > take > >>>>>> additional cycles and is not needed. The vector routines use the m= empool function directly, so that there is no overhead of > >> mbuf > >>>>>> initialization, as the vector routines use their additional "knowl= edge" of what the mbufs will be used for to init them in a faster > >> manner > >>>>>> than can be done inside the mbuf library. > >>>>>>>> > >>>>>>>> /Bruce > >>>>>>>> > >>>>>>>>> Signed-off-by: Keith Wiles > >>>>>>>>> --- > >>>>>>>>> lib/librte_mbuf/rte_mbuf.h | 77 > >>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++ > >>>>>>>>> 1 file changed, 77 insertions(+) > >>>>>>>>> > >>>>>>>>> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_m= buf.h > >>>>>>>>> index 1c6e115..f298621 100644 > >>>>>>>>> --- a/lib/librte_mbuf/rte_mbuf.h > >>>>>>>>> +++ b/lib/librte_mbuf/rte_mbuf.h > >>>>>>>>> @@ -546,6 +546,41 @@ static inline void rte_pktmbuf_reset(struc= t rte_mbuf > >>>>>>>>> *m) > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> /** > >>>>>>>>> + * @internal Allocate a list of mbufs from mempool *mp*. > >>>>>>>>> + * The use of that function is reserved for RTE internal needs= . > >>>>>>>>> + * Please use rte_pktmbuf_alloc_bulk(). > >>>>>>>>> + * > >>>>>>>>> + * @param mp > >>>>>>>>> + * The mempool from which mbuf is allocated. > >>>>>>>>> + * @param m_list > >>>>>>>>> + * The array to place the allocated rte_mbufs pointers. > >>>>>>>>> + * @param cnt > >>>>>>>>> + * The number of mbufs to allocate > >>>>>>>>> + * @return > >>>>>>>>> + * - 0 if the number of mbufs allocated was ok > >>>>>>>>> + * - <0 is an ERROR. > >>>>>>>>> + */ > >>>>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk(struct rte_mempool= *mp, struct > >>>>>>>>> rte_mbuf *m_list[], int cnt) > >>>>>>>>> +{ > >>>>>>>>> + struct rte_mbuf *m; > >>>>>>>>> + int ret; > >>>>>>>>> + > >>>>>>>>> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); > >>>>>>>>> + if ( ret =3D=3D 0 ) { > >>>>>>>>> + int i; > >>>>>>>>> + for(i =3D 0; i < cnt; i++) { > >>>>>>>>> + m =3D *m_list++; > >>>>>>>>> +#ifdef RTE_MBUF_REFCNT > >>>>>>>>> + rte_mbuf_refcnt_set(m, 1); > >>>>>>>>> +#endif /* RTE_MBUF_REFCNT */ > >>>>>>>>> + rte_pktmbuf_reset(m); > >>>>>>>>> + } > >>>>>>>>> + ret =3D cnt; > >>>>>>>>> + } > >>>>>>>>> + return ret; > >>>>>>>>> +} > >>>>>>>>> + > >>>>>>>>> +/** > >>>>>>>>> * Allocate a new mbuf from a mempool. > >>>>>>>>> * > >>>>>>>>> * This new mbuf contains one segment, which has a length of 0. = The pointer > >>>>>>>>> @@ -671,6 +706,32 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf = *m) > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> /** > >>>>>>>>> + * Allocate a list of mbufs from a mempool into a mbufs array. > >>>>>>>>> + * > >>>>>>>>> + * This mbuf list contains one segment per mbuf, which has a l= ength of 0. The > >>>>>>>>> pointer > >>>>>>>>> + * to data is initialized to have some bytes of headroom in th= e buffer > >>>>>>>>> + * (if buffer size allows). > >>>>>>>>> + * > >>>>>>>>> + * The routine is just a simple wrapper routine to reduce code= in the application > >>>>>>>>> and > >>>>>>>>> + * provide a cleaner API for multiple mbuf requests. > >>>>>>>>> + * > >>>>>>>>> + * @param mp > >>>>>>>>> + * The mempool from which the mbuf is allocated. > >>>>>>>>> + * @param m_list > >>>>>>>>> + * An array of mbuf pointers, cnt must be less then or equal= to the size of the > >>>>>>>>> list. > >>>>>>>>> + * @param cnt > >>>>>>>>> + * Number of slots in the m_list array to fill. > >>>>>>>>> + * @return > >>>>>>>>> + * - The number of valid mbufs pointers in the m_list array. > >>>>>>>>> + * - Zero if the request cnt could not be allocated. > >>>>>>>>> + */ > >>>>>>>>> +static inline int __attribute__((always_inline)) > >>>>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf= *m_list[], > >>>>>>>>> int16_t cnt) > >>>>>>>>> +{ > >>>>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); > >>>>>>>>> +} > >>>>>>>>> + > >>>>>>>>> +/** > >>>>>>>>> * Free a segment of a packet mbuf into its original mempool. > >>>>>>>>> * > >>>>>>>>> * Free an mbuf, without parsing other segments in case of chain= ed > >>>>>>>>> @@ -708,6 +769,22 @@ static inline void rte_pktmbuf_free(struct= rte_mbuf > >>>>>>>>> *m) > >>>>>>>>> } > >>>>>>>>> } > >>>>>>>>> > >>>>>>>>> +/** > >>>>>>>>> + * Free a list of packet mbufs back into its original mempool. > >>>>>>>>> + * > >>>>>>>>> + * Free a list of mbufs by calling rte_pktmbuf_free() in a loo= p as a wrapper > >>>>>>>>> function. > >>>>>>>>> + * > >>>>>>>>> + * @param m_list > >>>>>>>>> + * An array of rte_mbuf pointers to be freed. > >>>>>>>>> + * @param npkts > >>>>>>>>> + * Number of packets to free in list. > >>>>>>>>> + */ > >>>>>>>>> +static inline void rte_pktmbuf_free_bulk(struct rte_mbuf *m_li= st[], int16_t > >>>>>>>>> npkts) > >>>>>>>>> +{ > >>>>>>>>> + while(npkts--) > >>>>>>>>> + rte_pktmbuf_free(*m_list++); > >>>>>>>>> +} > >>>>>>>>> + > >>>>>>>>> #ifdef RTE_MBUF_REFCNT > >>>>>>>>> > >>>>>>>>> /** > >>>>>>>>> -- > >>>>>>>>> 2.1.0 > >>>>>>>> > >>>>>>> > >>>>>>> Keith Wiles, Principal Technologist with CTO office, Wind River m= obile 972-213-5533 > >>>> > >>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobi= le 972-213-5533 > >>> > >>> Keith Wiles, Principal Technologist with CTO office, Wind River mobil= e 972-213-5533 > >> > >> Keith Wiles, Principal Technologist with CTO office, Wind River mobile= 972-213-5533 >=20 > Keith Wiles, Principal Technologist with CTO office, Wind River mobile 97= 2-213-5533 >=20