From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) by dpdk.org (Postfix) with ESMTP id 63DAF212 for ; Tue, 7 Oct 2014 16:15:04 +0200 (CEST) Received: from ALA-HCB.corp.ad.wrs.com (ala-hcb.corp.ad.wrs.com [147.11.189.41]) by mail.windriver.com (8.14.9/8.14.5) with ESMTP id s97EME5P013804 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Tue, 7 Oct 2014 07:22:14 -0700 (PDT) Received: from ALA-MBB.corp.ad.wrs.com ([169.254.1.18]) by ALA-HCB.corp.ad.wrs.com ([147.11.189.41]) with mapi id 14.03.0174.001; Tue, 7 Oct 2014 07:22:14 -0700 From: "Wiles, Roger Keith" To: "ANANYEV, KONSTANTIN" Thread-Topic: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() Thread-Index: AQHP4UOohbitp8XoLEu9K6u9lPldRZwjnJoAgAAAzQCAABEUAIAABTUAgAA7SICAAAZJAIAA2lOAgABXaoA= Date: Tue, 7 Oct 2014 14:22:13 +0000 Message-ID: <1CB2412C-DC76-438D-A35F-BB03BA6A2A3A@windriver.com> References: <1412464229-125521-1-git-send-email-keith.wiles@windriver.com> <1412464229-125521-2-git-send-email-keith.wiles@windriver.com> <59AF69C657FD0841A61C55336867B5B03441BE9E@IRSMSX103.ger.corp.intel.com> <5DD5FF6E-C045-4764-A5B1-877C88B023F5@windriver.com> <20141006145330.GA2548@BRICHA3-MOBL> <2601191342CEEE43887BDE71AB97725821390E75@IRSMSX105.ger.corp.intel.com> <545592DF-3306-49F7-8685-10BD021B9854@windriver.com> <1AAECD5E-9A22-481D-9712-C75B8C1FAFC1@windriver.com> <844D44A2-27B2-47F9-BB6D-5A3A2F1757F6@windriver.com> <2601191342CEEE43887BDE71AB97725821391035@IRSMSX105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB97725821391035@IRSMSX105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.25.40.166] Content-Type: text/plain; charset="Windows-1252" Content-ID: <14832AF91B9DAF45BB8B67B16A4C1B66@local> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk() X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 14:15:05 -0000 On Oct 7, 2014, at 4:09 AM, Ananyev, Konstantin wrote: >=20 >=20 >> -----Original Message----- >> From: Wiles, Roger Keith [mailto:keith.wiles@windriver.com] >> Sent: Monday, October 06, 2014 9:08 PM >> To: Ananyev, Konstantin >> Cc: dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_allo= c_bulk() and rte_pktmbuf_free_bulk() >>=20 >> Attaching to the list does not work. If you want the code let me know it= is only about 5K in size. >>=20 >> On Oct 6, 2014, at 2:45 PM, Wiles, Roger Keith wrote: >>=20 >>>=20 >>> On Oct 6, 2014, at 11:13 AM, Wiles, Roger Keith wrote: >>>=20 >>>>=20 >>>> On Oct 6, 2014, at 10:54 AM, Ananyev, Konstantin wrote: >>>>=20 >>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardso= n >>>>>> Sent: Monday, October 06, 2014 3:54 PM >>>>>> To: Wiles, Roger Keith (Wind River) >>>>>> Cc: dev@dpdk.org >>>>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_= alloc_bulk() and rte_pktmbuf_free_bulk() >>>>>>=20 >>>>>> On Mon, Oct 06, 2014 at 03:50:38PM +0100, Wiles, Roger Keith wrote: >>>>>>> Hi Bruce, >>>>>>>=20 >>>>>>> Do I need to reject the for the new routines or just make sure the = vector driver does not get updated to use those routines? >>>>>>>=20 >>>>>>=20 >>>>>> The new routines are probably useful in the general case. I see no i= ssue >>>>>> with having them in the code, so long as the vector driver is not mo= dified >>>>>> to use them. >>>>>=20 >>>>> I 'd say the same thing for non-vector RX/TX PMD code-paths too. >>>>>=20 >>>>> BTW, are the new functions comments valid? >>>>>=20 >>>>> + * @return >>>>> + * - 0 if the number of mbufs allocated was ok >>>>> + * - <0 is an ERROR. >>>>> + */ >>>>> +static inline int __rte_mbuf_raw_alloc_bulk( >>>>>=20 >>>>> Though, as I can see __rte_mbuf_raw_alloc_bulk() returns either: >>>>> - number of allocated mbuf (cnt) >>>>> - negative error code >>>>=20 >>>> Let me fix up the comments. >>>>>=20 >>>>> And: >>>>> + * @return >>>>> + * - The number of valid mbufs pointers in the m_list array. >>>>> + * - Zero if the request cnt could not be allocated. >>>>> + */ >>>>> +static inline int __attribute__((always_inline)) >>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_li= st[], int16_t cnt) >>>>> +{ >>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); >>>>> +} >>>>>=20 >>>>> Shouldn't be "less than zero if the request cnt could not be allocate= d."? >>>>>=20 >>>>> BTW, is there any point to have __rte_mbuf_raw_alloc_bulk() at all? >>>>> After all, as you are calling rte_pktmbuf_reset() inside it, it doesn= 't look __raw__ any more. >>>>> Might be just put its content into rte_pktmbuf_alloc_bulk() and get r= id of it. >>>>>=20 >>>> I was just following the non-bulk routine style __rte_mbuf_raw_alloc()= , but I can pull that into a single routine. >>>>=20 >>>>> Also wonder, what is the advantage of having multiple counters inside= the same loop? >>>>> i.e: >>>>> + for(i =3D 0; i < cnt; i++) { >>>>> + m =3D *m_list++; >>>>>=20 >>>>> Why not just: >>>>>=20 >>>>> for(i =3D 0; i < cnt; i++) { >>>>> m =3D &m_list[i]; >>>>>=20 >>>>> Same for free: >>>>> + while(npkts--) >>>>> + rte_pktmbuf_free(*m_list++); >>>>>=20 >>>>> While not just: >>>>> for (i =3D 0; i < npkts; i++) >>>>> rte_pktmbuf_free(&m_list[i]); >>>>=20 >>>> Maybe I have it wrong or the compilers are doing the right thing now, = but at one point the &m_list[i] would cause the compiler to >> generate a shift or multiple of 'i' and then add it to the base of m_lis= t. If that is not the case anymore then I can update the code as >> you suggested. Using the *m_list++ just adds the size of a pointer to a = register and continues. >>>=20 >>> I compared the clang assembler (.s file) output from an example test co= de I wrote to see if we have any differences in the code >> using the two styles and I found no difference and the code looked the s= ame. I am not a Intel assembler expert and I would suggest >> someone else determine if it generates different code. I tried to compar= e the GCC outputs and it did look the same to me. >=20 > That's was my question: > Modern compilers are able to generate a good code for a simple loop as ab= ove. > So what's the point to use 2 iterators inside the loop, when just one is = enough? > Nothing wrong technically, but makes code a bit harder to follow. > Plus, in general, it is a good practise to minimise number of iterators i= nside the loop, when possible. >=20 > Konstantin Hi Konstantin, I really do not understand the concern if the code is the same, as it appea= rs to me the current patch is very clean and simple. Maybe you have not see= n the v2 patch and now v3 patch I sent this morning to fix Bruce=92s commen= t suggestion. For the case of the free routine your suggestion would require an extra cou= nter/variable a bit more code a =91for=92 loop instead of a =91while=92 loo= p.=20 +static inline void __attribute__((always_inline)) +rte_pktmbuf_free_bulk(struct rte_mbuf *m_list[], int16_t npkts) +{ + while(npkts--) + rte_pktmbuf_free(*m_list++); +} For the case of the alloc routine I did remove the rte_mbuf * m variable an= d now I believe it is very clean and changing it to use index variables is = just a personal preference. I personal preference of this type is not usefu= l IMO and does not cause any harm. Unless you can suggest a good technical = reason to change I am going to leave the patch as is. +static inline int __attribute__((always_inline)) +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_list[], = int16_t cnt) +{ + int ret; + + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); + if ( ret =3D=3D 0 ) { + ret =3D cnt; + while(cnt--) { +#ifdef RTE_MBUF_REFCNT + rte_mbuf_refcnt_set(*m_list, 1); +#endif /* RTE_MBUF_REFCNT */ + rte_pktmbuf_reset(*m_list++); + } + } + return ret; +} >>>=20 >>> I have attached the code and output, please let me know if I did someth= ing wrong, but as it stands using the original style is what I >> want to go with. >>>=20 >>>>>=20 >>>>> Konstantin >>>>>=20 >>>>>>=20 >>>>>> /Bruce >>>>>>=20 >>>>>>> Thanks >>>>>>> ++Keith >>>>>>>=20 >>>>>>> On Oct 6, 2014, at 3:56 AM, Richardson, Bruce wrote: >>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>> -----Original Message----- >>>>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Keith Wiles >>>>>>>>> Sent: Sunday, October 05, 2014 12:10 AM >>>>>>>>> To: dev@dpdk.org >>>>>>>>> Subject: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_a= lloc_bulk() >>>>>>>>> and rte_pktmbuf_free_bulk() >>>>>>>>>=20 >>>>>>>>> Minor helper routines to mirror the mempool routines and remove t= he code >>>>>>>>> from applications. The ixgbe_rxtx_vec.c routine could be changed = to use >>>>>>>>> the ret_pktmbuf_alloc_bulk() routine inplace of rte_mempool_get_b= ulk(). >>>>>>>>>=20 >>>>>>>>=20 >>>>>>>> I believe such a change would cause a performance regression, as t= he extra init code in the alloc_bulk() function would take >>>>>> additional cycles and is not needed. The vector routines use the mem= pool function directly, so that there is no overhead of >> mbuf >>>>>> initialization, as the vector routines use their additional "knowled= ge" of what the mbufs will be used for to init them in a faster >> manner >>>>>> than can be done inside the mbuf library. >>>>>>>>=20 >>>>>>>> /Bruce >>>>>>>>=20 >>>>>>>>> Signed-off-by: Keith Wiles >>>>>>>>> --- >>>>>>>>> lib/librte_mbuf/rte_mbuf.h | 77 >>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>>>> 1 file changed, 77 insertions(+) >>>>>>>>>=20 >>>>>>>>> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbu= f.h >>>>>>>>> index 1c6e115..f298621 100644 >>>>>>>>> --- a/lib/librte_mbuf/rte_mbuf.h >>>>>>>>> +++ b/lib/librte_mbuf/rte_mbuf.h >>>>>>>>> @@ -546,6 +546,41 @@ static inline void rte_pktmbuf_reset(struct = rte_mbuf >>>>>>>>> *m) >>>>>>>>> } >>>>>>>>>=20 >>>>>>>>> /** >>>>>>>>> + * @internal Allocate a list of mbufs from mempool *mp*. >>>>>>>>> + * The use of that function is reserved for RTE internal needs. >>>>>>>>> + * Please use rte_pktmbuf_alloc_bulk(). >>>>>>>>> + * >>>>>>>>> + * @param mp >>>>>>>>> + * The mempool from which mbuf is allocated. >>>>>>>>> + * @param m_list >>>>>>>>> + * The array to place the allocated rte_mbufs pointers. >>>>>>>>> + * @param cnt >>>>>>>>> + * The number of mbufs to allocate >>>>>>>>> + * @return >>>>>>>>> + * - 0 if the number of mbufs allocated was ok >>>>>>>>> + * - <0 is an ERROR. >>>>>>>>> + */ >>>>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk(struct rte_mempool *= mp, struct >>>>>>>>> rte_mbuf *m_list[], int cnt) >>>>>>>>> +{ >>>>>>>>> + struct rte_mbuf *m; >>>>>>>>> + int ret; >>>>>>>>> + >>>>>>>>> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt); >>>>>>>>> + if ( ret =3D=3D 0 ) { >>>>>>>>> + int i; >>>>>>>>> + for(i =3D 0; i < cnt; i++) { >>>>>>>>> + m =3D *m_list++; >>>>>>>>> +#ifdef RTE_MBUF_REFCNT >>>>>>>>> + rte_mbuf_refcnt_set(m, 1); >>>>>>>>> +#endif /* RTE_MBUF_REFCNT */ >>>>>>>>> + rte_pktmbuf_reset(m); >>>>>>>>> + } >>>>>>>>> + ret =3D cnt; >>>>>>>>> + } >>>>>>>>> + return ret; >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> +/** >>>>>>>>> * Allocate a new mbuf from a mempool. >>>>>>>>> * >>>>>>>>> * This new mbuf contains one segment, which has a length of 0. Th= e pointer >>>>>>>>> @@ -671,6 +706,32 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m= ) >>>>>>>>> } >>>>>>>>>=20 >>>>>>>>> /** >>>>>>>>> + * Allocate a list of mbufs from a mempool into a mbufs array. >>>>>>>>> + * >>>>>>>>> + * This mbuf list contains one segment per mbuf, which has a len= gth of 0. The >>>>>>>>> pointer >>>>>>>>> + * to data is initialized to have some bytes of headroom in the = buffer >>>>>>>>> + * (if buffer size allows). >>>>>>>>> + * >>>>>>>>> + * The routine is just a simple wrapper routine to reduce code i= n the application >>>>>>>>> and >>>>>>>>> + * provide a cleaner API for multiple mbuf requests. >>>>>>>>> + * >>>>>>>>> + * @param mp >>>>>>>>> + * The mempool from which the mbuf is allocated. >>>>>>>>> + * @param m_list >>>>>>>>> + * An array of mbuf pointers, cnt must be less then or equal t= o the size of the >>>>>>>>> list. >>>>>>>>> + * @param cnt >>>>>>>>> + * Number of slots in the m_list array to fill. >>>>>>>>> + * @return >>>>>>>>> + * - The number of valid mbufs pointers in the m_list array. >>>>>>>>> + * - Zero if the request cnt could not be allocated. >>>>>>>>> + */ >>>>>>>>> +static inline int __attribute__((always_inline)) >>>>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *= m_list[], >>>>>>>>> int16_t cnt) >>>>>>>>> +{ >>>>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt); >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> +/** >>>>>>>>> * Free a segment of a packet mbuf into its original mempool. >>>>>>>>> * >>>>>>>>> * Free an mbuf, without parsing other segments in case of chained >>>>>>>>> @@ -708,6 +769,22 @@ static inline void rte_pktmbuf_free(struct r= te_mbuf >>>>>>>>> *m) >>>>>>>>> } >>>>>>>>> } >>>>>>>>>=20 >>>>>>>>> +/** >>>>>>>>> + * Free a list of packet mbufs back into its original mempool. >>>>>>>>> + * >>>>>>>>> + * Free a list of mbufs by calling rte_pktmbuf_free() in a loop = as a wrapper >>>>>>>>> function. >>>>>>>>> + * >>>>>>>>> + * @param m_list >>>>>>>>> + * An array of rte_mbuf pointers to be freed. >>>>>>>>> + * @param npkts >>>>>>>>> + * Number of packets to free in list. >>>>>>>>> + */ >>>>>>>>> +static inline void rte_pktmbuf_free_bulk(struct rte_mbuf *m_list= [], int16_t >>>>>>>>> npkts) >>>>>>>>> +{ >>>>>>>>> + while(npkts--) >>>>>>>>> + rte_pktmbuf_free(*m_list++); >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> #ifdef RTE_MBUF_REFCNT >>>>>>>>>=20 >>>>>>>>> /** >>>>>>>>> -- >>>>>>>>> 2.1.0 >>>>>>>>=20 >>>>>>>=20 >>>>>>> Keith Wiles, Principal Technologist with CTO office, Wind River mob= ile 972-213-5533 >>>>=20 >>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobile= 972-213-5533 >>>=20 >>> Keith Wiles, Principal Technologist with CTO office, Wind River mobile = 972-213-5533 >>=20 >> Keith Wiles, Principal Technologist with CTO office, Wind River mobile 9= 72-213-5533 Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-= 213-5533