From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail1.windriver.com (mail1.windriver.com [147.11.146.13]) by dpdk.org (Postfix) with ESMTP id BC5D82E8A for ; Tue, 7 Oct 2014 17:48:57 +0200 (CEST) Received: from ALA-HCB.corp.ad.wrs.com (ala-hcb.corp.ad.wrs.com [147.11.189.41]) by mail1.windriver.com (8.14.9/8.14.5) with ESMTP id s97Fu9oi015236 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Tue, 7 Oct 2014 08:56:09 -0700 (PDT) Received: from ALA-MBB.corp.ad.wrs.com ([169.254.1.18]) by ALA-HCB.corp.ad.wrs.com ([147.11.189.41]) with mapi id 14.03.0174.001; Tue, 7 Oct 2014 08:56:09 -0700 From: "Wiles, Roger Keith" To: "ANANYEV, KONSTANTIN" Thread-Topic: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() Thread-Index: AQHP4UOohbitp8XoLEu9K6u9lPldRZwjnJoAgAAAzQCAABEUAIAABTUAgAA7SICAAAZJAIAA2lOAgABXaoCAABaCgIAAA7iA Date: Tue, 7 Oct 2014 15:56:08 +0000 Message-ID: <388C030D-2E72-4B97-A909-F27DDB4055BE@windriver.com> References: <1412464229-125521-1-git-send-email-keith.wiles@windriver.com> <1412464229-125521-2-git-send-email-keith.wiles@windriver.com> <59AF69C657FD0841A61C55336867B5B03441BE9E@IRSMSX103.ger.corp.intel.com> <5DD5FF6E-C045-4764-A5B1-877C88B023F5@windriver.com> <20141006145330.GA2548@BRICHA3-MOBL> <2601191342CEEE43887BDE71AB97725821390E75@IRSMSX105.ger.corp.intel.com> <545592DF-3306-49F7-8685-10BD021B9854@windriver.com> <1AAECD5E-9A22-481D-9712-C75B8C1FAFC1@windriver.com> <844D44A2-27B2-47F9-BB6D-5A3A2F1757F6@windriver.com> <2601191342CEEE43887BDE71AB97725821391035@IRSMSX105.ger.corp.intel.com> <1CB2412C-DC76-438D-A35F-BB03BA6A2A3A@windriver.com> <2601191342CEEE43887BDE71AB97725821391240@IRSMSX105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB97725821391240@IRSMSX105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.25.40.166] Content-Type: text/plain; charset="us-ascii" Content-ID: <8B58211D3FCDA94889DAD7539438C13E@local> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 15:48:58 -0000 On Oct 7, 2014, at 10:42 AM, Ananyev, Konstantin wrote: > Hi Keith, >=20 >> -----Original Message----- >> From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] >> Sent: Tuesday, October 07, 2014 3:22 PM >> To: Ananyev, Konstantin >> Cc: dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_allo= c_bulk() and rte_pktmbuf_free_bulk() >>=20 >>=20 >> On Oct 7, 2014, at 4:09 AM, Ananyev, Konstantin wrote: >>=20 >>>=20 >>>=20 >>>> -----Original Message----- >>>> From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] >>>> Sent: Monday, October 06, 2014 9:08 PM >>>> To: Ananyev, Konstantin >>>> Cc: dev@dpdk.org >>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_al= loc_bulk() and rte_pktmbuf_free_bulk() >>>>=20 >>>> Attaching to the list does not work. If you want the code let me know = it is only about 5K in size. >>>>=20 >>>> On Oct 6, 2014, at 2:45 PM, Wiles, Roger Keith wrote: >>>>=20 >>>>>=20 >>>>> On Oct 6, 2014, at 11:13 AM, Wiles, Roger Keith wrote: >>>>>=20 >>>>>>=20 >>>>>> On Oct 6, 2014, at 10:54 AM, Ananyev, Konstantin wrote: >>>>>>=20 >>>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richard= son >>>>>>>> Sent: Monday, October 06, 2014 3:54 PM >>>>>>>> To: Wiles, Roger Keith (Wind River) >>>>>>>> Cc: dev@dpdk.org >>>>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbu= f_alloc_bulk() and rte_pktmbuf_free_bulk() >>>>>>>>=20 >>>>>>>> On Mon, Oct 06, 2014 at 03:50:38PM +0100, Wiles, Roger Keith wrote= : >>>>>>>>> Hi Bruce, >>>>>>>>>=20 >>>>>>>>> Do I need to reject the for the new routines or just make sure th= e vector driver does not get updated to use those routines? >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>> The new routines are probably useful in the general case. I see no= issue >>>>>>>> with having them in the code, so long as the vector driver is not = modified >>>>>>>> to use them. >>>>>>>=20 >>>>>>> I 'd say the same thing for non-vector RX/TX PMD code-paths too. >>>>>>>=20 >>>>>>> BTW, are the new functions comments valid? >>>>>>>=20 >>>>>>> + * @return >>>>>>> + * - 0 if the number of mbufs allocated was ok >>>>>>> + * - <0 is an ERROR. >>>>>>> + */ >>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk( >>>>>>>=20 >>>>>>> Though, as I can see __rte_mbuf_raw_alloc_bulk() returns either: >>>>>>> - number of allocated mbuf (cnt) >>>>>>> - negative error code >>>>>>=20 >>>>>> Let me fix up the comments. >>>>>>>=20 >>>>>>> And: >>>>>>> + * @return >>>>>>> + * - The number of valid mbufs pointers in the m_list array. >>>>>>> + * - Zero if the request cnt could not be allocated. >>>>>>> + */ >>>>>>> +static inline int __attribute__((always_inline)) >>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_= list[], int16_t cnt) >>>>>>> +{ >>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); >>>>>>> +} >>>>>>>=20 >>>>>>> Shouldn't be "less than zero if the request cnt could not be alloca= ted."? >>>>>>>=20 >>>>>>> BTW, is there any point to have __rte_mbuf_raw_alloc_bulk() at all? >>>>>>> After all, as you are calling rte_pktmbuf_reset() inside it, it doe= sn't look __raw__ any more. >>>>>>> Might be just put its content into rte_pktmbuf_alloc_bulk() and get= rid of it. >>>>>>>=20 >>>>>> I was just following the non-bulk routine style __rte_mbuf_raw_alloc= (), but I can pull that into a single routine. >>>>>>=20 >>>>>>> Also wonder, what is the advantage of having multiple counters insi= de the same loop? >>>>>>> i.e: >>>>>>> + for(i =3D 0; i < cnt; i++) { >>>>>>> + m =3D *m_list++; >>>>>>>=20 >>>>>>> Why not just: >>>>>>>=20 >>>>>>> for(i =3D 0; i < cnt; i++) { >>>>>>> m =3D &m_list[i]; >>>>>>>=20 >>>>>>> Same for free: >>>>>>> + while(npkts--) >>>>>>> + rte_pktmbuf_free(*m_list++); >>>>>>>=20 >>>>>>> While not just: >>>>>>> for (i =3D 0; i < npkts; i++) >>>>>>> rte_pktmbuf_free(&m_list[i]); >>>>>>=20 >>>>>> Maybe I have it wrong or the compilers are doing the right thing now= , but at one point the &m_list[i] would cause the compiler >> to >>>> generate a shift or multiple of 'i' and then add it to the base of m_l= ist. If that is not the case anymore then I can update the code as >>>> you suggested. Using the *m_list++ just adds the size of a pointer to = a register and continues. >>>>>=20 >>>>> I compared the clang assembler (.s file) output from an example test = code I wrote to see if we have any differences in the code >>>> using the two styles and I found no difference and the code looked the= same. I am not a Intel assembler expert and I would >> suggest >>>> someone else determine if it generates different code. I tried to comp= are the GCC outputs and it did look the same to me. >>>=20 >>> That's was my question: >>> Modern compilers are able to generate a good code for a simple loop as = above. >>> So what's the point to use 2 iterators inside the loop, when just one i= s enough? >>> Nothing wrong technically, but makes code a bit harder to follow. >>> Plus, in general, it is a good practise to minimise number of iterators= inside the loop, when possible. >>>=20 >>> Konstantin >>=20 >> Hi Konstantin, >>=20 >> I really do not understand the concern if the code is the same, as it ap= pears to me the current patch is very clean and simple. Maybe >> you have not seen the v2 patch and now v3 patch I sent this morning to f= ix Bruce's comment suggestion. >>=20 >> For the case of the free routine your suggestion would require an extra = counter/variable a bit more code a 'for' loop instead of a >> 'while' loop. >=20 > My point was that just one iterator for both loops is enough. > In general, it is a good practise to minimise number of iterators per loo= p if possible: > in some cases compiler might get confused and wouldn't be able to elimin= ate redundant iterators itself. I learned a while back to not to be a compiler, but a programmer :-) Now a = days the compilers handle the basic cases we have here and for the special = cases we need to be aware of how the compiler generates code. I agree havin= g less iterators per loop is cleaner, but in this case I do not think it ma= tters. > Though yes - technically there is nothing wrong with your approach. > So if you prefer to keep it as it is - I wouldn't insist. >=20 > Konstantin >=20 >> +static inline void __attribute__((always_inline)) >> +rte_pktmbuf_free_bulk(struct rte_mbuf *m_list[], int16_t npkts) >> +{ >> + while(npkts--) >> + rte_pktmbuf_free(*m_list++); >> +} >>=20 >> For the case of the alloc routine I did remove the rte_mbuf * m variable= and now I believe it is very clean and changing it to use index >> variables is just a personal preference. I personal preference of this t= ype is not useful IMO and does not cause any harm. Unless you >> can suggest a good technical reason to change I am going to leave the pa= tch as is. >>=20 >> +static inline int __attribute__((always_inline)) >> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_list[= ], int16_t cnt) >> +{ >> + int ret; >> + >> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); >> + if ( ret =3D=3D 0 ) { >> + ret =3D cnt; >> + while(cnt--) { >> +#ifdef RTE_MBUF_REFCNT >> + rte_mbuf_refcnt_set(*m_list, 1); >> +#endif /* RTE_MBUF_REFCNT */ >> + rte_pktmbuf_reset(*m_list++); >> + } >> + } >> + return ret; >> +} >>=20 >>>>>=20 >>>>> I have attached the code and output, please let me know if I did some= thing wrong, but as it stands using the original style is what I >>>> want to go with. >>>>>=20 >>>>>>>=20 >>>>>>> Konstantin >>>>>>>=20 >>>>>>>>=20 >>>>>>>> /Bruce >>>>>>>>=20 >>>>>>>>> Thanks >>>>>>>>> ++Keith >>>>>>>>>=20 >>>>>>>>> On Oct 6, 2014, at 3:56 AM, Richardson, Bruce wrote: >>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Keith Wile= s >>>>>>>>>>> Sent: Sunday, October 05, 2014 12:10 AM >>>>>>>>>>> To: dev@dpdk.org >>>>>>>>>>> Subject: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf= _alloc_bulk() >>>>>>>>>>> and rte_pktmbuf_free_bulk() >>>>>>>>>>>=20 >>>>>>>>>>> Minor helper routines to mirror the mempool routines and remove= the code >>>>>>>>>>> from applications. The ixgbe_rxtx_vec.c routine could be change= d to use >>>>>>>>>>> the ret_pktmbuf_alloc_bulk() routine inplace of rte_mempool_get= _bulk(). >>>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> I believe such a change would cause a performance regression, as= the extra init code in the alloc_bulk() function would >> take >>>>>>>> additional cycles and is not needed. The vector routines use the m= empool function directly, so that there is no overhead of >>>> mbuf >>>>>>>> initialization, as the vector routines use their additional "knowl= edge" of what the mbufs will be used for to init them in a faster >>>> manner >>>>>>>> than can be done inside the mbuf library. >>>>>>>>>>=20 >>>>>>>>>> /Bruce >>>>>>>>>>=20 >>>>>>>>>>> Signed-off-by: Keith Wiles >>>>>>>>>>> --- >>>>>>>>>>> lib/librte_mbuf/rte_mbuf.h | 77 >>>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>>>>>> 1 file changed, 77 insertions(+) >>>>>>>>>>>=20 >>>>>>>>>>> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_m= buf.h >>>>>>>>>>> index 1c6e115..f298621 100644 >>>>>>>>>>> --- a/lib/librte_mbuf/rte_mbuf.h >>>>>>>>>>> +++ b/lib/librte_mbuf/rte_mbuf.h >>>>>>>>>>> @@ -546,6 +546,41 @@ static inline void rte_pktmbuf_reset(struc= t rte_mbuf >>>>>>>>>>> *m) >>>>>>>>>>> } >>>>>>>>>>>=20 >>>>>>>>>>> /** >>>>>>>>>>> + * @internal Allocate a list of mbufs from mempool *mp*. >>>>>>>>>>> + * The use of that function is reserved for RTE internal needs= . >>>>>>>>>>> + * Please use rte_pktmbuf_alloc_bulk(). >>>>>>>>>>> + * >>>>>>>>>>> + * @param mp >>>>>>>>>>> + * The mempool from which mbuf is allocated. >>>>>>>>>>> + * @param m_list >>>>>>>>>>> + * The array to place the allocated rte_mbufs pointers. >>>>>>>>>>> + * @param cnt >>>>>>>>>>> + * The number of mbufs to allocate >>>>>>>>>>> + * @return >>>>>>>>>>> + * - 0 if the number of mbufs allocated was ok >>>>>>>>>>> + * - <0 is an ERROR. >>>>>>>>>>> + */ >>>>>>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk(struct rte_mempool= *mp, struct >>>>>>>>>>> rte_mbuf *m_list[], int cnt) >>>>>>>>>>> +{ >>>>>>>>>>> + struct rte_mbuf *m; >>>>>>>>>>> + int ret; >>>>>>>>>>> + >>>>>>>>>>> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); >>>>>>>>>>> + if ( ret =3D=3D 0 ) { >>>>>>>>>>> + int i; >>>>>>>>>>> + for(i =3D 0; i < cnt; i++) { >>>>>>>>>>> + m =3D *m_list++; >>>>>>>>>>> +#ifdef RTE_MBUF_REFCNT >>>>>>>>>>> + rte_mbuf_refcnt_set(m, 1); >>>>>>>>>>> +#endif /* RTE_MBUF_REFCNT */ >>>>>>>>>>> + rte_pktmbuf_reset(m); >>>>>>>>>>> + } >>>>>>>>>>> + ret =3D cnt; >>>>>>>>>>> + } >>>>>>>>>>> + return ret; >>>>>>>>>>> +} >>>>>>>>>>> + >>>>>>>>>>> +/** >>>>>>>>>>> * Allocate a new mbuf from a mempool. >>>>>>>>>>> * >>>>>>>>>>> * This new mbuf contains one segment, which has a length of 0. = The pointer >>>>>>>>>>> @@ -671,6 +706,32 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf = *m) >>>>>>>>>>> } >>>>>>>>>>>=20 >>>>>>>>>>> /** >>>>>>>>>>> + * Allocate a list of mbufs from a mempool into a mbufs array. >>>>>>>>>>> + * >>>>>>>>>>> + * This mbuf list contains one segment per mbuf, which has a l= ength of 0. The >>>>>>>>>>> pointer >>>>>>>>>>> + * to data is initialized to have some bytes of headroom in th= e buffer >>>>>>>>>>> + * (if buffer size allows). >>>>>>>>>>> + * >>>>>>>>>>> + * The routine is just a simple wrapper routine to reduce code= in the application >>>>>>>>>>> and >>>>>>>>>>> + * provide a cleaner API for multiple mbuf requests. >>>>>>>>>>> + * >>>>>>>>>>> + * @param mp >>>>>>>>>>> + * The mempool from which the mbuf is allocated. >>>>>>>>>>> + * @param m_list >>>>>>>>>>> + * An array of mbuf pointers, cnt must be less then or equal= to the size of the >>>>>>>>>>> list. >>>>>>>>>>> + * @param cnt >>>>>>>>>>> + * Number of slots in the m_list array to fill. >>>>>>>>>>> + * @return >>>>>>>>>>> + * - The number of valid mbufs pointers in the m_list array. >>>>>>>>>>> + * - Zero if the request cnt could not be allocated. >>>>>>>>>>> + */ >>>>>>>>>>> +static inline int __attribute__((always_inline)) >>>>>>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf= *m_list[], >>>>>>>>>>> int16_t cnt) >>>>>>>>>>> +{ >>>>>>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); >>>>>>>>>>> +} >>>>>>>>>>> + >>>>>>>>>>> +/** >>>>>>>>>>> * Free a segment of a packet mbuf into its original mempool. >>>>>>>>>>> * >>>>>>>>>>> * Free an mbuf, without parsing other segments in case of chain= ed >>>>>>>>>>> @@ -708,6 +769,22 @@ static inline void rte_pktmbuf_free(struct= rte_mbuf >>>>>>>>>>> *m) >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>>=20 >>>>>>>>>>> +/** >>>>>>>>>>> + * Free a list of packet mbufs back into its original mempool. >>>>>>>>>>> + * >>>>>>>>>>> + * Free a list of mbufs by calling rte_pktmbuf_free() in a loo= p as a wrapper >>>>>>>>>>> function. >>>>>>>>>>> + * >>>>>>>>>>> + * @param m_list >>>>>>>>>>> + * An array of rte_mbuf pointers to be freed. >>>>>>>>>>> + * @param npkts >>>>>>>>>>> + * Number of packets to free in list. >>>>>>>>>>> + */ >>>>>>>>>>> +static inline void rte_pktmbuf_free_bulk(struct rte_mbuf *m_li= st[], int16_t >>>>>>>>>>> npkts) >>>>>>>>>>> +{ >>>>>>>>>>> + while(npkts--) >>>>>>>>>>> + rte_pktmbuf_free(*m_list++); >>>>>>>>>>> +} >>>>>>>>>>> + >>>>>>>>>>> #ifdef RTE_MBUF_REFCNT >>>>>>>>>>>=20 >>>>>>>>>>> /** >>>>>>>>>>> -- >>>>>>>>>>> 2.1.0 >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Keith Wiles, Principal Technologist with CTO office, Wind River m= obile 972-213-5533 >>>>>>=20 >>>>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobi= le 972-213-5533 >>>>>=20 >>>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobil= e 972-213-5533 >>>>=20 >>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobile= 972-213-5533 >>=20 >> Keith Wiles, Principal Technologist with CTO office, Wind River mobile 9= 72-213-5533 Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-= 213-5533