From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <keith.wiles@windriver.com>
Received: from mail1.windriver.com (mail1.windriver.com [147.11.146.13])
by dpdk.org (Postfix) with ESMTP id F28DC3B5
for <dev@dpdk.org>; Mon, 6 Oct 2014 21:38:20 +0200 (CEST)
Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com
[147.11.189.40])
by mail1.windriver.com (8.14.9/8.14.5) with ESMTP id s96JjS8Q029307
(version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL)
for <dev@dpdk.org>; Mon, 6 Oct 2014 12:45:29 -0700 (PDT)
Received: from ALA-MBB.corp.ad.wrs.com ([169.254.1.18]) by
ALA-HCA.corp.ad.wrs.com ([147.11.189.40]) with mapi id 14.03.0174.001; Mon, 6
Oct 2014 12:45:28 -0700
From: "Wiles, Roger Keith" <keith.wiles@windriver.com>
To: "ANANYEV, KONSTANTIN" <konstantin.ananyev@intel.com>
Thread-Topic: [dpdk-dev] [PATCH 2/2] Adding the routines
rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk()
Thread-Index: AQHP4UOohbitp8XoLEu9K6u9lPldRZwjnJoAgAAAzQCAABEUAIAABTUAgAA7SIA=
Date: Mon, 6 Oct 2014 19:45:27 +0000
Message-ID: <1AAECD5E-9A22-481D-9712-C75B8C1FAFC1@windriver.com>
References: <1412464229-125521-1-git-send-email-keith.wiles@windriver.com>
<1412464229-125521-2-git-send-email-keith.wiles@windriver.com>
<59AF69C657FD0841A61C55336867B5B03441BE9E@IRSMSX103.ger.corp.intel.com>
<5DD5FF6E-C045-4764-A5B1-877C88B023F5@windriver.com>
<20141006145330.GA2548@BRICHA3-MOBL>
<2601191342CEEE43887BDE71AB97725821390E75@IRSMSX105.ger.corp.intel.com>
<545592DF-3306-49F7-8685-10BD021B9854@windriver.com>
In-Reply-To: <545592DF-3306-49F7-8685-10BD021B9854@windriver.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-originating-ip: [172.25.40.166]
MIME-Version: 1.0
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.15
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines
rte_pktmbuf_alloc_bulk() and rte_pktmbuf_free_bulk()
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Oct 2014 19:38:21 -0000
On Oct 6, 2014, at 11:13 AM, Wiles, Roger Keith <keith.wiles@windriver.com>=
wrote:
>
> On Oct 6, 2014, at 10:54 AM, Ananyev, Konstantin <konstantin.ananyev@inte=
l.com> wrote:
>
>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
>>> Sent: Monday, October 06, 2014 3:54 PM
>>> To: Wiles, Roger Keith (Wind River)
>>> Cc: dev@dpdk.org
>>> Subject: Re: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_all=
oc_bulk() and rte_pktmbuf_free_bulk()
>>>
>>> On Mon, Oct 06, 2014 at 03:50:38PM +0100, Wiles, Roger Keith wrote:
>>>> Hi Bruce,
>>>>
>>>> Do I need to reject the for the new routines or just make sure the vec=
tor driver does not get updated to use those routines?
>>>>
>>>
>>> The new routines are probably useful in the general case. I see no issu=
e
>>> with having them in the code, so long as the vector driver is not modif=
ied
>>> to use them.
>>
>> I 'd say the same thing for non-vector RX/TX PMD code-paths too.
>>
>> BTW, are the new functions comments valid?
>>
>> + * @return
>> + * - 0 if the number of mbufs allocated was ok
>> + * - <0 is an ERROR.
>> + */
>> +static inline int __rte_mbuf_raw_alloc_bulk(
>>
>> Though, as I can see __rte_mbuf_raw_alloc_bulk() returns either:
>> - number of allocated mbuf (cnt)
>> - negative error code
>
> Let me fix up the comments.
>>
>> And:
>> + * @return
>> + * - The number of valid mbufs pointers in the m_list array.
>> + * - Zero if the request cnt could not be allocated.
>> + */
>> +static inline int __attribute__((always_inline))
>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_list[=
], int16_t cnt)
>> +{
>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt);
>> +}
>>
>> Shouldn't be "less than zero if the request cnt could not be allocated."=
?
>>
>> BTW, is there any point to have __rte_mbuf_raw_alloc_bulk() at all?
>> After all, as you are calling rte_pktmbuf_reset() inside it, it doesn't =
look __raw__ any more.
>> Might be just put its content into rte_pktmbuf_alloc_bulk() and get rid =
of it.
>>
> I was just following the non-bulk routine style __rte_mbuf_raw_alloc(), b=
ut I can pull that into a single routine.
>
>> Also wonder, what is the advantage of having multiple counters inside th=
e same loop?
>> i.e:
>> + for(i =3D 0; i < cnt; i++) {
>> + m =3D *m_list++;
>>
>> Why not just:
>>
>> for(i =3D 0; i < cnt; i++) {
>> m =3D &m_list[i];
>>
>> Same for free:
>> + while(npkts--)
>> + rte_pktmbuf_free(*m_list++);
>>
>> While not just:
>> for (i =3D 0; i < npkts; i++)
>> rte_pktmbuf_free(&m_list[i]);
>
> Maybe I have it wrong or the compilers are doing the right thing now, but=
at one point the &m_list[i] would cause the compiler to generate a shift o=
r multiple of =91i=92 and then add it to the base of m_list. If that is not=
the case anymore then I can update the code as you suggested. Using the *m=
_list++ just adds the size of a pointer to a register and continues.
I compared the clang assembler (.s file) output from an example test code I=
wrote to see if we have any differences in the code using the two styles a=
nd I found no difference and the code looked the same. I am not a Intel ass=
embler expert and I would suggest someone else determine if it generates di=
fferent code. I tried to compare the GCC outputs and it did look the same t=
o me.
I have attached the code and output, please let me know if I did something =
wrong, but as it stands using the original style is what I want to go with.
>>
>> Konstantin
>>
>>>
>>> /Bruce
>>>
>>>> Thanks
>>>> ++Keith
>>>>
>>>> On Oct 6, 2014, at 3:56 AM, Richardson, Bruce <bruce.richardson@intel.=
com> wrote:
>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Keith Wiles
>>>>>> Sent: Sunday, October 05, 2014 12:10 AM
>>>>>> To: dev@dpdk.org
>>>>>> Subject: [dpdk-dev] [PATCH 2/2] Adding the routines rte_pktmbuf_allo=
c_bulk()
>>>>>> and rte_pktmbuf_free_bulk()
>>>>>>
>>>>>> Minor helper routines to mirror the mempool routines and remove the =
code
>>>>>> from applications. The ixgbe_rxtx_vec.c routine could be changed to =
use
>>>>>> the ret_pktmbuf_alloc_bulk() routine inplace of rte_mempool_get_bulk=
().
>>>>>>
>>>>>
>>>>> I believe such a change would cause a performance regression, as the =
extra init code in the alloc_bulk() function would take
>>> additional cycles and is not needed. The vector routines use the mempoo=
l function directly, so that there is no overhead of mbuf
>>> initialization, as the vector routines use their additional "knowledge"=
of what the mbufs will be used for to init them in a faster manner
>>> than can be done inside the mbuf library.
>>>>>
>>>>> /Bruce
>>>>>
>>>>>> Signed-off-by: Keith Wiles <keith.wiles@windriver.com>
>>>>>> ---
>>>>>> lib/librte_mbuf/rte_mbuf.h | 77
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++
>>>>>> 1 file changed, 77 insertions(+)
>>>>>>
>>>>>> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
>>>>>> index 1c6e115..f298621 100644
>>>>>> --- a/lib/librte_mbuf/rte_mbuf.h
>>>>>> +++ b/lib/librte_mbuf/rte_mbuf.h
>>>>>> @@ -546,6 +546,41 @@ static inline void rte_pktmbuf_reset(struct rte=
_mbuf
>>>>>> *m)
>>>>>> }
>>>>>>
>>>>>> /**
>>>>>> + * @internal Allocate a list of mbufs from mempool *mp*.
>>>>>> + * The use of that function is reserved for RTE internal needs.
>>>>>> + * Please use rte_pktmbuf_alloc_bulk().
>>>>>> + *
>>>>>> + * @param mp
>>>>>> + * The mempool from which mbuf is allocated.
>>>>>> + * @param m_list
>>>>>> + * The array to place the allocated rte_mbufs pointers.
>>>>>> + * @param cnt
>>>>>> + * The number of mbufs to allocate
>>>>>> + * @return
>>>>>> + * - 0 if the number of mbufs allocated was ok
>>>>>> + * - <0 is an ERROR.
>>>>>> + */
>>>>>> +static inline int __rte_mbuf_raw_alloc_bulk(struct rte_mempool *mp,=
struct
>>>>>> rte_mbuf *m_list[], int cnt)
>>>>>> +{
>>>>>> + struct rte_mbuf *m;
>>>>>> + int ret;
>>>>>> +
>>>>>> + ret =3D rte_mempool_get_bulk(mp, (void **)m_list, cnt);
>>>>>> + if ( ret =3D=3D 0 ) {
>>>>>> + int i;
>>>>>> + for(i =3D 0; i < cnt; i++) {
>>>>>> + m =3D *m_list++;
>>>>>> +#ifdef RTE_MBUF_REFCNT
>>>>>> + rte_mbuf_refcnt_set(m, 1);
>>>>>> +#endif /* RTE_MBUF_REFCNT */
>>>>>> + rte_pktmbuf_reset(m);
>>>>>> + }
>>>>>> + ret =3D cnt;
>>>>>> + }
>>>>>> + return ret;
>>>>>> +}
>>>>>> +
>>>>>> +/**
>>>>>> * Allocate a new mbuf from a mempool.
>>>>>> *
>>>>>> * This new mbuf contains one segment, which has a length of 0. The p=
ointer
>>>>>> @@ -671,6 +706,32 @@ __rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
>>>>>> }
>>>>>>
>>>>>> /**
>>>>>> + * Allocate a list of mbufs from a mempool into a mbufs array.
>>>>>> + *
>>>>>> + * This mbuf list contains one segment per mbuf, which has a length=
of 0. The
>>>>>> pointer
>>>>>> + * to data is initialized to have some bytes of headroom in the buf=
fer
>>>>>> + * (if buffer size allows).
>>>>>> + *
>>>>>> + * The routine is just a simple wrapper routine to reduce code in t=
he application
>>>>>> and
>>>>>> + * provide a cleaner API for multiple mbuf requests.
>>>>>> + *
>>>>>> + * @param mp
>>>>>> + * The mempool from which the mbuf is allocated.
>>>>>> + * @param m_list
>>>>>> + * An array of mbuf pointers, cnt must be less then or equal to t=
he size of the
>>>>>> list.
>>>>>> + * @param cnt
>>>>>> + * Number of slots in the m_list array to fill.
>>>>>> + * @return
>>>>>> + * - The number of valid mbufs pointers in the m_list array.
>>>>>> + * - Zero if the request cnt could not be allocated.
>>>>>> + */
>>>>>> +static inline int __attribute__((always_inline))
>>>>>> +rte_pktmbuf_alloc_bulk(struct rte_mempool *mp, struct rte_mbuf *m_l=
ist[],
>>>>>> int16_t cnt)
>>>>>> +{
>>>>>> + return __rte_mbuf_raw_alloc_bulk(mp, m_list, cnt);
>>>>>> +}
>>>>>> +
>>>>>> +/**
>>>>>> * Free a segment of a packet mbuf into its original mempool.
>>>>>> *
>>>>>> * Free an mbuf, without parsing other segments in case of chained
>>>>>> @@ -708,6 +769,22 @@ static inline void rte_pktmbuf_free(struct rte_=
mbuf
>>>>>> *m)
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> +/**
>>>>>> + * Free a list of packet mbufs back into its original mempool.
>>>>>> + *
>>>>>> + * Free a list of mbufs by calling rte_pktmbuf_free() in a loop as =
a wrapper
>>>>>> function.
>>>>>> + *
>>>>>> + * @param m_list
>>>>>> + * An array of rte_mbuf pointers to be freed.
>>>>>> + * @param npkts
>>>>>> + * Number of packets to free in list.
>>>>>> + */
>>>>>> +static inline void rte_pktmbuf_free_bulk(struct rte_mbuf *m_list[],=
int16_t
>>>>>> npkts)
>>>>>> +{
>>>>>> + while(npkts--)
>>>>>> + rte_pktmbuf_free(*m_list++);
>>>>>> +}
>>>>>> +
>>>>>> #ifdef RTE_MBUF_REFCNT
>>>>>>
>>>>>> /**
>>>>>> --
>>>>>> 2.1.0
>>>>>
>>>>
>>>> Keith Wiles, Principal Technologist with CTO office, Wind River mobile=
972-213-5533
>
> Keith Wiles, Principal Technologist with CTO office, Wind River mobile 97=
2-213-5533
Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-=
213-5533