From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id D27F67E2C for ; Tue, 7 Oct 2014 18:27:00 +0200 (CEST) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga102.fm.intel.com with ESMTP; 07 Oct 2014 09:34:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="396655278" Received: from irsmsx101.ger.corp.intel.com ([163.33.3.153]) by FMSMGA003.fm.intel.com with ESMTP; 07 Oct 2014 09:27:17 -0700 Received: from irsmsx109.ger.corp.intel.com (163.33.3.23) by IRSMSX101.ger.corp.intel.com (163.33.3.153) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 7 Oct 2014 17:34:00 +0100 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.174]) by IRSMSX109.ger.corp.intel.com ([169.254.13.253]) with mapi id 14.03.0195.001; Tue, 7 Oct 2014 17:33:59 +0100 From: "Ananyev, Konstantin" To: "Wiles, Roger Keith (Wind River)" Thread-Topic: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() Thread-Index: AQHP4Ch2aL1BNwlnX0CzjWDe7U+wDpwitckAgABi6wCAAADNAIAAF86A///+ewCAADtJgIAABkgAgADpxACAAEf5gIAAF4gwgAACtQCAABU3cA== Date: Tue, 7 Oct 2014 16:33:59 +0000 Message-ID: <2601191342CEEE43887BDE71AB977258213912B2@IRSMSX105.ger.corp.intel.com> References: <1412464229-125521-1-git-send-email-keith.wiles@windriver.com> <1412464229-125521-2-git-send-email-keith.wiles@windriver.com> <59AF69C657FD0841A61C55336867B5B03441BE9E@IRSMSX103.ger.corp.intel.com> <5DD5FF6E-C045-4764-A5B1-877C88B023F5@windriver.com> <20141006145330.GA2548@BRICHA3-MOBL> <2601191342CEEE43887BDE71AB97725821390E75@IRSMSX105.ger.corp.intel.com> <545592DF-3306-49F7-8685-10BD021B9854@windriver.com> <1AAECD5E-9A22-481D-9712-C75B8C1FAFC1@windriver.com> <844D44A2-27B2-47F9-BB6D-5A3A2F1757F6@windriver.com> <2601191342CEEE43887BDE71AB97725821391035@IRSMSX105.ger.corp.intel.com> <1CB2412C-DC76-438D-A35F-BB03BA6A2A3A@windriver.com> <2601191342CEEE43887BDE71AB97725821391240@IRSMSX105.ger.corp.intel.com> <388C030D-2E72-4B97-A909-F27DDB4055BE@windriver.com> In-Reply-To: <388C030D-2E72-4B97-A909-F27DDB4055BE@windriver.com> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 16:27:01 -0000 > -----Original Message----- > From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] > Sent: Tuesday, October 07, 2014 4:56 PM > To: Ananyev, Konstantin > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc= _bulk() and rte_pktmbuf_free_bulk() >=20 >=20 > On Oct 7, 2014, at 10:42 AM, Ananyev, Konstantin wrote: >=20 > > Hi Keith, > > > >> -----Original Message----- > >> From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] > >> Sent: Tuesday, October 07, 2014 3:22 PM > >> To: Ananyev, Konstantin > >> Cc: dev@dpdk.org > >> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_al= loc_bulk() and rte_pktmbuf_free_bulk() > >> > >> > >> On Oct 7, 2014, at 4:09 AM, Ananyev, Konstantin wrote: > >> > >>> > >>> > >>>> -----Original Message----- > >>>> From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] > >>>> Sent: Monday, October 06, 2014 9:08 PM > >>>> To: Ananyev, Konstantin > >>>> Cc: dev@dpdk.org > >>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_= alloc_bulk() and rte_pktmbuf_free_bulk() > >>>> > >>>> Attaching to the list does not work. If you want the code let me kno= w it is only about 5K in size. > >>>> > >>>> On Oct 6, 2014, at 2:45 PM, Wiles, Roger Keith wrote: > >>>> > >>>>> > >>>>> On Oct 6, 2014, at 11:13 AM, Wiles, Roger Keith wrote: > >>>>> > >>>>>> > >>>>>> On Oct 6, 2014, at 10:54 AM, Ananyev, Konstantin wrote: > >>>>>> > >>>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richa= rdson > >>>>>>>> Sent: Monday, October 06, 2014 3:54 PM > >>>>>>>> To: Wiles, Roger Keith (Wind River) > >>>>>>>> Cc: dev@dpdk.org > >>>>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktm= buf_alloc_bulk() and rte_pktmbuf_free_bulk() > >>>>>>>> > >>>>>>>> On Mon, Oct 06, 2014 at 03:50:38PM +0100, Wiles, Roger Keith wro= te: > >>>>>>>>> Hi Bruce, > >>>>>>>>> > >>>>>>>>> Do I need to reject the for the new routines or just make sure = the vector driver does not get updated to use those > routines? > >>>>>>>>> > >>>>>>>> > >>>>>>>> The new routines are probably useful in the general case. I see = no issue > >>>>>>>> with having them in the code, so long as the vector driver is no= t modified > >>>>>>>> to use them. > >>>>>>> > >>>>>>> I 'd say the same thing for non-vector RX/TX PMD code-paths too. > >>>>>>> > >>>>>>> BTW, are the new functions comments valid? > >>>>>>> > >>>>>>> + * @return > >>>>>>> + * - 0 if the number of mbufs allocated was ok > >>>>>>> + * - <0 is an ERROR. > >>>>>>> + */ > >>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk( > >>>>>>> > >>>>>>> Though, as I can see __rte_mbuf_raw_alloc_bulk() returns either: > >>>>>>> - number of allocated mbuf (cnt) > >>>>>>> - negative error code > >>>>>> > >>>>>> Let me fix up the comments. > >>>>>>> > >>>>>>> And: > >>>>>>> + * @return > >>>>>>> + * - The number of valid mbufs pointers in the m_list array. > >>>>>>> + * - Zero if the request cnt could not be allocated. > >>>>>>> + */ > >>>>>>> +static inline int __attribute__((always_inline)) > >>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *= m_list[], int16_t cnt) > >>>>>>> +{ > >>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); > >>>>>>> +} > >>>>>>> > >>>>>>> Shouldn't be "less than zero if the request cnt could not be allo= cated."? > >>>>>>> > >>>>>>> BTW, is there any point to have __rte_mbuf_raw_alloc_bulk() at al= l? > >>>>>>> After all, as you are calling rte_pktmbuf_reset() inside it, it d= oesn't look __raw__ any more. > >>>>>>> Might be just put its content into rte_pktmbuf_alloc_bulk() and g= et rid of it. > >>>>>>> > >>>>>> I was just following the non-bulk routine style __rte_mbuf_raw_all= oc(), but I can pull that into a single routine. > >>>>>> > >>>>>>> Also wonder, what is the advantage of having multiple counters in= side the same loop? > >>>>>>> i.e: > >>>>>>> + for(i =3D 0; i < cnt; i++) { > >>>>>>> + m =3D *m_list++; > >>>>>>> > >>>>>>> Why not just: > >>>>>>> > >>>>>>> for(i =3D 0; i < cnt; i++) { > >>>>>>> m =3D &m_list[i]; > >>>>>>> > >>>>>>> Same for free: > >>>>>>> + while(npkts--) > >>>>>>> + rte_pktmbuf_free(*m_list++); > >>>>>>> > >>>>>>> While not just: > >>>>>>> for (i =3D 0; i < npkts; i++) > >>>>>>> rte_pktmbuf_free(&m_list[i]); > >>>>>> > >>>>>> Maybe I have it wrong or the compilers are doing the right thing n= ow, but at one point the &m_list[i] would cause the > compiler > >> to > >>>> generate a shift or multiple of 'i' and then add it to the base of m= _list. If that is not the case anymore then I can update the code > as > >>>> you suggested. Using the *m_list++ just adds the size of a pointer t= o a register and continues. > >>>>> > >>>>> I compared the clang assembler (.s file) output from an example tes= t code I wrote to see if we have any differences in the code > >>>> using the two styles and I found no difference and the code looked t= he same. I am not a Intel assembler expert and I would > >> suggest > >>>> someone else determine if it generates different code. I tried to co= mpare the GCC outputs and it did look the same to me. > >>> > >>> That's was my question: > >>> Modern compilers are able to generate a good code for a simple loop a= s above. > >>> So what's the point to use 2 iterators inside the loop, when just one= is enough? > >>> Nothing wrong technically, but makes code a bit harder to follow. > >>> Plus, in general, it is a good practise to minimise number of iterato= rs inside the loop, when possible. > >>> > >>> Konstantin > >> > >> Hi Konstantin, > >> > >> I really do not understand the concern if the code is the same, as it = appears to me the current patch is very clean and simple. > Maybe > >> you have not seen the v2 patch and now v3 patch I sent this morning to= fix Bruce's comment suggestion. > >> > >> For the case of the free routine your suggestion would require an extr= a counter/variable a bit more code a 'for' loop instead of a > >> 'while' loop. > > > > My point was that just one iterator for both loops is enough. > > In general, it is a good practise to minimise number of iterators per l= oop if possible: > > in some cases compiler might get confused and wouldn't be able to elim= inate redundant iterators itself. >=20 > I learned a while back to not to be a compiler, but a programmer :-) Now = a days the compilers handle the basic cases we have here Yes, in most cases they would. That's why I don't insist. Konstantin > and for the special cases we need to be aware of how the compiler generat= es code. I agree having less iterators per loop is cleaner, > but in this case I do not think it matters. > > Though yes - technically there is nothing wrong with your approach. > > So if you prefer to keep it as it is - I wouldn't insist. > > > > Konstantin > > > >> +static inline void __attribute__((always_inline)) > >> +rte_pktmbuf_free_bulk(struct rte_mbuf *m_list[], int16_t npkts) > >> +{ > >> + while(npkts--) > >> + rte_pktmbuf_free(*m_list++); > >> +} > >> > >> For the case of the alloc routine I did remove the rte_mbuf * m variab= le and now I believe it is very clean and changing it to use > index > >> variables is just a personal preference. I personal preference of this= type is not useful IMO and does not cause any harm. Unless > you > >> can suggest a good technical reason to change I am going to leave the = patch as is. > >> > >> +static inline int __attribute__((always_inline)) > >> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_lis= t[], int16_t cnt) > >> +{ > >> + int ret; > >> + > >> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); > >> + if ( ret =3D=3D 0 ) { > >> + ret =3D cnt; > >> + while(cnt--) { > >> +#ifdef RTE_MBUF_REFCNT > >> + rte_mbuf_refcnt_set(*m_list, 1); > >> +#endif /* RTE_MBUF_REFCNT */ > >> + rte_pktmbuf_reset(*m_list++); > >> + } > >> + } > >> + return ret; > >> +} > >> > >>>>> > >>>>> I have attached the code and output, please let me know if I did so= mething wrong, but as it stands using the original style is > what I > >>>> want to go with. > >>>>> > >>>>>>> > >>>>>>> Konstantin > >>>>>>> > >>>>>>>> > >>>>>>>> /Bruce > >>>>>>>> > >>>>>>>>> Thanks > >>>>>>>>> ++Keith > >>>>>>>>> > >>>>>>>>> On Oct 6, 2014, at 3:56 AM, Richardson, Bruce wrote: > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> -----Original Message----- > >>>>>>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Keith Wi= les > >>>>>>>>>>> Sent: Sunday, October 05, 2014 12:10 AM > >>>>>>>>>>> To: dev@dpdk.org > >>>>>>>>>>> Subject: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmb= uf_alloc_bulk() > >>>>>>>>>>> and rte_pktmbuf_free_bulk() > >>>>>>>>>>> > >>>>>>>>>>> Minor helper routines to mirror the mempool routines and remo= ve the code > >>>>>>>>>>> from applications. The ixgbe_rxtx_vec.c routine could be chan= ged to use > >>>>>>>>>>> the ret_pktmbuf_alloc_bulk() routine inplace of rte_mempool_g= et_bulk(). > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> I believe such a change would cause a performance regression, = as the extra init code in the alloc_bulk() function would > >> take > >>>>>>>> additional cycles and is not needed. The vector routines use the= mempool function directly, so that there is no overhead of > >>>> mbuf > >>>>>>>> initialization, as the vector routines use their additional "kno= wledge" of what the mbufs will be used for to init them in a > faster > >>>> manner > >>>>>>>> than can be done inside the mbuf library. > >>>>>>>>>> > >>>>>>>>>> /Bruce > >>>>>>>>>> > >>>>>>>>>>> Signed-off-by: Keith Wiles > >>>>>>>>>>> --- > >>>>>>>>>>> lib/librte_mbuf/rte_mbuf.h | 77 > >>>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++ > >>>>>>>>>>> 1 file changed, 77 insertions(+) > >>>>>>>>>>> > >>>>>>>>>>> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte= _mbuf.h > >>>>>>>>>>> index 1c6e115..f298621 100644 > >>>>>>>>>>> --- a/lib/librte_mbuf/rte_mbuf.h > >>>>>>>>>>> +++ b/lib/librte_mbuf/rte_mbuf.h > >>>>>>>>>>> @@ -546,6 +546,41 @@ static inline void rte_pktmbuf_reset(str= uct rte_mbuf > >>>>>>>>>>> *m) > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> /** > >>>>>>>>>>> + * @internal Allocate a list of mbufs from mempool *mp*. > >>>>>>>>>>> + * The use of that function is reserved for RTE internal nee= ds. > >>>>>>>>>>> + * Please use rte_pktmbuf_alloc_bulk(). > >>>>>>>>>>> + * > >>>>>>>>>>> + * @param mp > >>>>>>>>>>> + * The mempool from which mbuf is allocated. > >>>>>>>>>>> + * @param m_list > >>>>>>>>>>> + * The array to place the allocated rte_mbufs pointers. > >>>>>>>>>>> + * @param cnt > >>>>>>>>>>> + * The number of mbufs to allocate > >>>>>>>>>>> + * @return > >>>>>>>>>>> + * - 0 if the number of mbufs allocated was ok > >>>>>>>>>>> + * - <0 is an ERROR. > >>>>>>>>>>> + */ > >>>>>>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk(struct rte_mempo= ol *mp, struct > >>>>>>>>>>> rte_mbuf *m_list[], int cnt) > >>>>>>>>>>> +{ > >>>>>>>>>>> + struct rte_mbuf *m; > >>>>>>>>>>> + int ret; > >>>>>>>>>>> + > >>>>>>>>>>> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); > >>>>>>>>>>> + if ( ret =3D=3D 0 ) { > >>>>>>>>>>> + int i; > >>>>>>>>>>> + for(i =3D 0; i < cnt; i++) { > >>>>>>>>>>> + m =3D *m_list++; > >>>>>>>>>>> +#ifdef RTE_MBUF_REFCNT > >>>>>>>>>>> + rte_mbuf_refcnt_set(m, 1); > >>>>>>>>>>> +#endif /* RTE_MBUF_REFCNT */ > >>>>>>>>>>> + rte_pktmbuf_reset(m); > >>>>>>>>>>> + } > >>>>>>>>>>> + ret =3D cnt; > >>>>>>>>>>> + } > >>>>>>>>>>> + return ret; > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +/** > >>>>>>>>>>> * Allocate a new mbuf from a mempool. > >>>>>>>>>>> * > >>>>>>>>>>> * This new mbuf contains one segment, which has a length of 0= . The pointer > >>>>>>>>>>> @@ -671,6 +706,32 @@ __rte_pktmbuf_prefree_seg(struct rte_mbu= f *m) > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> /** > >>>>>>>>>>> + * Allocate a list of mbufs from a mempool into a mbufs arra= y. > >>>>>>>>>>> + * > >>>>>>>>>>> + * This mbuf list contains one segment per mbuf, which has a= length of 0. The > >>>>>>>>>>> pointer > >>>>>>>>>>> + * to data is initialized to have some bytes of headroom in = the buffer > >>>>>>>>>>> + * (if buffer size allows). > >>>>>>>>>>> + * > >>>>>>>>>>> + * The routine is just a simple wrapper routine to reduce co= de in the application > >>>>>>>>>>> and > >>>>>>>>>>> + * provide a cleaner API for multiple mbuf requests. > >>>>>>>>>>> + * > >>>>>>>>>>> + * @param mp > >>>>>>>>>>> + * The mempool from which the mbuf is allocated. > >>>>>>>>>>> + * @param m_list > >>>>>>>>>>> + * An array of mbuf pointers, cnt must be less then or equ= al to the size of the > >>>>>>>>>>> list. > >>>>>>>>>>> + * @param cnt > >>>>>>>>>>> + * Number of slots in the m_list array to fill. > >>>>>>>>>>> + * @return > >>>>>>>>>>> + * - The number of valid mbufs pointers in the m_list arra= y. > >>>>>>>>>>> + * - Zero if the request cnt could not be allocated. > >>>>>>>>>>> + */ > >>>>>>>>>>> +static inline int __attribute__((always_inline)) > >>>>>>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mb= uf *m_list[], > >>>>>>>>>>> int16_t cnt) > >>>>>>>>>>> +{ > >>>>>>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> +/** > >>>>>>>>>>> * Free a segment of a packet mbuf into its original mempool. > >>>>>>>>>>> * > >>>>>>>>>>> * Free an mbuf, without parsing other segments in case of cha= ined > >>>>>>>>>>> @@ -708,6 +769,22 @@ static inline void rte_pktmbuf_free(stru= ct rte_mbuf > >>>>>>>>>>> *m) > >>>>>>>>>>> } > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> +/** > >>>>>>>>>>> + * Free a list of packet mbufs back into its original mempoo= l. > >>>>>>>>>>> + * > >>>>>>>>>>> + * Free a list of mbufs by calling rte_pktmbuf_free() in a l= oop as a wrapper > >>>>>>>>>>> function. > >>>>>>>>>>> + * > >>>>>>>>>>> + * @param m_list > >>>>>>>>>>> + * An array of rte_mbuf pointers to be freed. > >>>>>>>>>>> + * @param npkts > >>>>>>>>>>> + * Number of packets to free in list. > >>>>>>>>>>> + */ > >>>>>>>>>>> +static inline void rte_pktmbuf_free_bulk(struct rte_mbuf *m_= list[], int16_t > >>>>>>>>>>> npkts) > >>>>>>>>>>> +{ > >>>>>>>>>>> + while(npkts--) > >>>>>>>>>>> + rte_pktmbuf_free(*m_list++); > >>>>>>>>>>> +} > >>>>>>>>>>> + > >>>>>>>>>>> #ifdef RTE_MBUF_REFCNT > >>>>>>>>>>> > >>>>>>>>>>> /** > >>>>>>>>>>> -- > >>>>>>>>>>> 2.1.0 > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> Keith Wiles, Principal Technologist with CTO office, Wind River= mobile 972-213-5533 > >>>>>> > >>>>>> Keith Wiles, Principal Technologist with CTO office, Wind River mo= bile 972-213-5533 > >>>>> > >>>>> Keith Wiles, Principal Technologist with CTO office, Wind River mob= ile 972-213-5533 > >>>> > >>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobi= le 972-213-5533 > >> > >> Keith Wiles, Principal Technologist with CTO office, Wind River mobile= 972-213-5533 >=20 > Keith Wiles, Principal Technologist with CTO office, Wind River mobile 97= 2-213-5533 >=20