[dpdk-dev] Aligned RX data.

[dpdk-dev] Aligned RX data.

[Alex Markuze]

Hi , I'm trying to receive aligned packets from the wire.
Meaning that for all received packets the pkt.data is always aligned to
(512 -H).

Looking at the pmds of ixgbe/vmxnet I see that the pmds call
__rte_mbuf_raw_alloc and set the rx descriptor with a
RTE_MBUF_DATA_DMA_ADDR_DEFAULT
Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.

Do I need to modify each pmd I'm using to be able to receive aligned data?
Or have I missed something?

Thanks

Re: [dpdk-dev] Aligned RX data.

[Ananyev, Konstantin]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> Sent: Tuesday, October 07, 2014 10:40 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Aligned RX data.
> 
> Hi , I'm trying to receive aligned packets from the wire.
> Meaning that for all received packets the pkt.data is always aligned to
> (512 -H).
> 
> Looking at the pmds of ixgbe/vmxnet I see that the pmds call
> __rte_mbuf_raw_alloc and set the rx descriptor with a
> RTE_MBUF_DATA_DMA_ADDR_DEFAULT
> Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
> 
> Do I need to modify each pmd I'm using to be able to receive aligned data?

Make sure that your all your mbufs are aligned by 512 and set in your config RTE_PKTMBUF_HEADROOM=512-H?


> Or have I missed something?
> 
> Thanks

Re: [dpdk-dev] Aligned RX data.

[Alex Markuze]

RTE_PKTMBUF_HEADROOM defines the headroom this would be true only if the
buff_start was aligned to 512 which is not.

On Tue, Oct 7, 2014 at 1:05 PM, Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> > Sent: Tuesday, October 07, 2014 10:40 AM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Aligned RX data.
> >
> > Hi , I'm trying to receive aligned packets from the wire.
> > Meaning that for all received packets the pkt.data is always aligned to
> > (512 -H).
> >
> > Looking at the pmds of ixgbe/vmxnet I see that the pmds call
> > __rte_mbuf_raw_alloc and set the rx descriptor with a
> > RTE_MBUF_DATA_DMA_ADDR_DEFAULT
> > Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
> >
> > Do I need to modify each pmd I'm using to be able to receive aligned
> data?
>
> Make sure that your all your mbufs are aligned by 512 and set in your
> config RTE_PKTMBUF_HEADROOM=512-H?
>
>
> > Or have I missed something?
> >
> > Thanks
>

Re: [dpdk-dev] Aligned RX data.

[Ananyev, Konstantin]

> -----Original Message-----
> From: Ananyev, Konstantin
> Sent: Tuesday, October 07, 2014 5:03 PM
> To: Ananyev, Konstantin
> Subject: FW: [dpdk-dev] Aligned RX data.
> 
> 
> 
> From: Alex Markuze [mailto:alex@weka.io]
> Sent: Tuesday, October 07, 2014 4:52 PM
> To: Ananyev, Konstantin
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Aligned RX data.
> 
> RTE_PKTMBUF_HEADROOM defines the headroom

Yes.

>this would be true only if the buff_start was aligned to 512 which is not.

As I said: " Make sure that your all your mbufs are aligned by 512".  

Konstantin

> 
> On Tue, Oct 7, 2014 at 1:05 PM, Ananyev, Konstantin <konstantin.ananyev@intel.com> wrote:
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> > Sent: Tuesday, October 07, 2014 10:40 AM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] Aligned RX data.
> >
> > Hi , I'm trying to receive aligned packets from the wire.
> > Meaning that for all received packets the pkt.data is always aligned to
> > (512 -H).
> >
> > Looking at the pmds of ixgbe/vmxnet I see that the pmds call
> > __rte_mbuf_raw_alloc and set the rx descriptor with a
> > RTE_MBUF_DATA_DMA_ADDR_DEFAULT
> > Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
> >
> > Do I need to modify each pmd I'm using to be able to receive aligned data?
> Make sure that your all your mbufs are aligned by 512 and set in your config RTE_PKTMBUF_HEADROOM=512-H?
> 
> 
> > Or have I missed something?
> >
> > Thanks

Re: [dpdk-dev] Aligned RX data.

[Alex Markuze]

O.k, And how would I do that?
I'm guessing there is something I can control in rte_pktmbuf_pool_init?
I would appreciate If you could spare a word or two in the matter.

On Tue, Oct 7, 2014 at 7:11 PM, Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

>
>
> > -----Original Message-----
> > From: Ananyev, Konstantin
> > Sent: Tuesday, October 07, 2014 5:03 PM
> > To: Ananyev, Konstantin
> > Subject: FW: [dpdk-dev] Aligned RX data.
> >
> >
> >
> > From: Alex Markuze [mailto:alex@weka.io]
> > Sent: Tuesday, October 07, 2014 4:52 PM
> > To: Ananyev, Konstantin
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] Aligned RX data.
> >
> > RTE_PKTMBUF_HEADROOM defines the headroom
>
> Yes.
>
> >this would be true only if the buff_start was aligned to 512 which is not.
>
> As I said: " Make sure that your all your mbufs are aligned by 512".
>
> Konstantin
>
> >
> > On Tue, Oct 7, 2014 at 1:05 PM, Ananyev, Konstantin <
> konstantin.ananyev@intel.com> wrote:
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> > > Sent: Tuesday, October 07, 2014 10:40 AM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Aligned RX data.
> > >
> > > Hi , I'm trying to receive aligned packets from the wire.
> > > Meaning that for all received packets the pkt.data is always aligned to
> > > (512 -H).
> > >
> > > Looking at the pmds of ixgbe/vmxnet I see that the pmds call
> > > __rte_mbuf_raw_alloc and set the rx descriptor with a
> > > RTE_MBUF_DATA_DMA_ADDR_DEFAULT
> > > Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
> > >
> > > Do I need to modify each pmd I'm using to be able to receive aligned
> data?
> > Make sure that your all your mbufs are aligned by 512 and set in your
> config RTE_PKTMBUF_HEADROOM=512-H?
> >
> >
> > > Or have I missed something?
> > >
> > > Thanks
>
>

Re: [dpdk-dev] Aligned RX data.

[Alex Markuze]

Hi All,
Is there a way to create a mempool such that all mbufs are aligned to X.
lets say X is 512.

Thanks.


On Sat, Oct 11, 2014 at 5:04 PM, Alex Markuze <alex@weka.io> wrote:

> O.k, And how would I do that?
> I'm guessing there is something I can control in rte_pktmbuf_pool_init?
> I would appreciate If you could spare a word or two in the matter.
>
> On Tue, Oct 7, 2014 at 7:11 PM, Ananyev, Konstantin <
> konstantin.ananyev@intel.com> wrote:
>
>>
>>
>> > -----Original Message-----
>> > From: Ananyev, Konstantin
>> > Sent: Tuesday, October 07, 2014 5:03 PM
>> > To: Ananyev, Konstantin
>> > Subject: FW: [dpdk-dev] Aligned RX data.
>> >
>> >
>> >
>> > From: Alex Markuze [mailto:alex@weka.io]
>> > Sent: Tuesday, October 07, 2014 4:52 PM
>> > To: Ananyev, Konstantin
>> > Cc: dev@dpdk.org
>> > Subject: Re: [dpdk-dev] Aligned RX data.
>> >
>> > RTE_PKTMBUF_HEADROOM defines the headroom
>>
>> Yes.
>>
>> >this would be true only if the buff_start was aligned to 512 which is
>> not.
>>
>> As I said: " Make sure that your all your mbufs are aligned by 512".
>>
>> Konstantin
>>
>> >
>> > On Tue, Oct 7, 2014 at 1:05 PM, Ananyev, Konstantin <
>> konstantin.ananyev@intel.com> wrote:
>> >
>> >
>> > > -----Original Message-----
>> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
>> > > Sent: Tuesday, October 07, 2014 10:40 AM
>> > > To: dev@dpdk.org
>> > > Subject: [dpdk-dev] Aligned RX data.
>> > >
>> > > Hi , I'm trying to receive aligned packets from the wire.
>> > > Meaning that for all received packets the pkt.data is always aligned
>> to
>> > > (512 -H).
>> > >
>> > > Looking at the pmds of ixgbe/vmxnet I see that the pmds call
>> > > __rte_mbuf_raw_alloc and set the rx descriptor with a
>> > > RTE_MBUF_DATA_DMA_ADDR_DEFAULT
>> > > Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
>> > >
>> > > Do I need to modify each pmd I'm using to be able to receive aligned
>> data?
>> > Make sure that your all your mbufs are aligned by 512 and set in your
>> config RTE_PKTMBUF_HEADROOM=512-H?
>> >
>> >
>> > > Or have I missed something?
>> > >
>> > > Thanks
>>
>>
>

Re: [dpdk-dev] Aligned RX data.

[Ananyev, Konstantin]

> -----Original Message-----
> From: Ananyev, Konstantin
> Sent: Monday, October 13, 2014 12:30 PM
> To: Ananyev, Konstantin
> Subject: FW: [dpdk-dev] Aligned RX data.
> 
> 
> 
> From: Alex Markuze [mailto:alex@weka.io]
> Sent: Monday, October 13, 2014 9:47 AM
> To: Ananyev, Konstantin
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] Aligned RX data.
> 
> Hi All,
> Is there a way to create a mempool such that all mbufs are aligned to X. lets say X is 512.
> 
> Thanks.
> 

For example something like that:

struct rte_mempool *
mempool_xz1_create(uint32_t elt_num, int32_t socket_id)
{
        struct rte_mempool *mp;
        const struct rte_memzone *mz;
        struct rte_mempool_objsz obj_sz;
        uint32_t flags, elt_size, total_size;
        size_t sz;
        phys_addr_t pa;
        void *va;

        /* mp element header_size==64B,  trailer_size==0. */
        flags = MEMPOOL_F_NO_SPREAD;

        /* to make total element size of mp 2K. */
        elt_size = 2048 - 64;

        total_size = rte_mempool_calc_obj_size(elt_size, flags, &obj_sz);
        sz = elt_num * total_size + 512;

        if ((mz = rte_memzone_reserve_aligned("xz1_obj", sz, socket_id,
                        0, 512)) == NULL)
                return (NULL);

        va = (char *)mz->addr + 512 - obj_sz.header_size;
        pa = mz->phys_addr + 512 - obj_sz.header_size;

        mp = rte_mempool_xmem_create("xz1", elt_num, elt_size,
                256, sizeof(struct rte_pktmbuf_pool_private),
                rte_pktmbuf_pool_init, NULL,
                rte_pktmbuf_init, NULL,
                socket_id, flags, va, &pa,
                MEMPOOL_PG_NUM_DEFAULT, MEMPOOL_PG_SHIFT_MAX);

        return (mp);
}

Each mbuf will be aligned on 512B boundary and  1856 (2K - 64B header - 128B mbuf).

Alternative way - is to provide your own element constructor instead of rte_pktmbuf_init() for mempool_create.
And inside it align buf_addr and buf_physaddr.
Though in that case you have to set RTE_MBUF_REFCNT=n in your config.
That's why I'd say it is a not recommended.

Konstantin

> 
> On Sat, Oct 11, 2014 at 5:04 PM, Alex Markuze <alex@weka.io> wrote:
> O.k, And how would I do that?
> I'm guessing there is something I can control in rte_pktmbuf_pool_init?
> I would appreciate If you could spare a word or two in the matter.
> 
> On Tue, Oct 7, 2014 at 7:11 PM, Ananyev, Konstantin <konstantin.ananyev@intel.com> wrote:
> 
> 
> > -----Original Message-----
> > From: Ananyev, Konstantin
> > Sent: Tuesday, October 07, 2014 5:03 PM
> > To: Ananyev, Konstantin
> > Subject: FW: [dpdk-dev] Aligned RX data.
> >
> >
> >
> > From: Alex Markuze [mailto:alex@weka.io]
> > Sent: Tuesday, October 07, 2014 4:52 PM
> > To: Ananyev, Konstantin
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] Aligned RX data.
> >
> > RTE_PKTMBUF_HEADROOM defines the headroom
> 
> Yes.
> 
> >this would be true only if the buff_start was aligned to 512 which is not.
> 
> As I said: " Make sure that your all your mbufs are aligned by 512".
> 
> Konstantin
> 
> >
> > On Tue, Oct 7, 2014 at 1:05 PM, Ananyev, Konstantin <konstantin.ananyev@intel.com> wrote:
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> > > Sent: Tuesday, October 07, 2014 10:40 AM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] Aligned RX data.
> > >
> > > Hi , I'm trying to receive aligned packets from the wire.
> > > Meaning that for all received packets the pkt.data is always aligned to
> > > (512 -H).
> > >
> > > Looking at the pmds of ixgbe/vmxnet I see that the pmds call
> > > __rte_mbuf_raw_alloc and set the rx descriptor with a
> > > RTE_MBUF_DATA_DMA_ADDR_DEFAULT
> > > Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
> > >
> > > Do I need to modify each pmd I'm using to be able to receive aligned data?
> > Make sure that your all your mbufs are aligned by 512 and set in your config RTE_PKTMBUF_HEADROOM=512-H?
> >
> >
> > > Or have I missed something?
> > >
> > > Thanks
> 

Re: [dpdk-dev] Aligned RX data.

[Alex Markuze]

Very helpful, thanks a lot.
It sure does seem to do the trick.

On Mon, Oct 13, 2014 at 2:43 PM, Ananyev, Konstantin <
konstantin.ananyev@intel.com> wrote:

>
>
> > -----Original Message-----
> > From: Ananyev, Konstantin
> > Sent: Monday, October 13, 2014 12:30 PM
> > To: Ananyev, Konstantin
> > Subject: FW: [dpdk-dev] Aligned RX data.
> >
> >
> >
> > From: Alex Markuze [mailto:alex@weka.io]
> > Sent: Monday, October 13, 2014 9:47 AM
> > To: Ananyev, Konstantin
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] Aligned RX data.
> >
> > Hi All,
> > Is there a way to create a mempool such that all mbufs are aligned to X.
> lets say X is 512.
> >
> > Thanks.
> >
>
> For example something like that:
>
> struct rte_mempool *
> mempool_xz1_create(uint32_t elt_num, int32_t socket_id)
> {
>         struct rte_mempool *mp;
>         const struct rte_memzone *mz;
>         struct rte_mempool_objsz obj_sz;
>         uint32_t flags, elt_size, total_size;
>         size_t sz;
>         phys_addr_t pa;
>         void *va;
>
>         /* mp element header_size==64B,  trailer_size==0. */
>         flags = MEMPOOL_F_NO_SPREAD;
>
>         /* to make total element size of mp 2K. */
>         elt_size = 2048 - 64;
>
>         total_size = rte_mempool_calc_obj_size(elt_size, flags, &obj_sz);
>         sz = elt_num * total_size + 512;
>
>         if ((mz = rte_memzone_reserve_aligned("xz1_obj", sz, socket_id,
>                         0, 512)) == NULL)
>                 return (NULL);
>
>         va = (char *)mz->addr + 512 - obj_sz.header_size;
>         pa = mz->phys_addr + 512 - obj_sz.header_size;
>
>         mp = rte_mempool_xmem_create("xz1", elt_num, elt_size,
>                 256, sizeof(struct rte_pktmbuf_pool_private),
>                 rte_pktmbuf_pool_init, NULL,
>                 rte_pktmbuf_init, NULL,
>                 socket_id, flags, va, &pa,
>                 MEMPOOL_PG_NUM_DEFAULT, MEMPOOL_PG_SHIFT_MAX);
>
>         return (mp);
> }
>
> Each mbuf will be aligned on 512B boundary and  1856 (2K - 64B header -
> 128B mbuf).
>
> Alternative way - is to provide your own element constructor instead of
> rte_pktmbuf_init() for mempool_create.
> And inside it align buf_addr and buf_physaddr.
> Though in that case you have to set RTE_MBUF_REFCNT=n in your config.
> That's why I'd say it is a not recommended.
>
> Konstantin
>
> >
> > On Sat, Oct 11, 2014 at 5:04 PM, Alex Markuze <alex@weka.io> wrote:
> > O.k, And how would I do that?
> > I'm guessing there is something I can control in rte_pktmbuf_pool_init?
> > I would appreciate If you could spare a word or two in the matter.
> >
> > On Tue, Oct 7, 2014 at 7:11 PM, Ananyev, Konstantin <
> konstantin.ananyev@intel.com> wrote:
> >
> >
> > > -----Original Message-----
> > > From: Ananyev, Konstantin
> > > Sent: Tuesday, October 07, 2014 5:03 PM
> > > To: Ananyev, Konstantin
> > > Subject: FW: [dpdk-dev] Aligned RX data.
> > >
> > >
> > >
> > > From: Alex Markuze [mailto:alex@weka.io]
> > > Sent: Tuesday, October 07, 2014 4:52 PM
> > > To: Ananyev, Konstantin
> > > Cc: dev@dpdk.org
> > > Subject: Re: [dpdk-dev] Aligned RX data.
> > >
> > > RTE_PKTMBUF_HEADROOM defines the headroom
> >
> > Yes.
> >
> > >this would be true only if the buff_start was aligned to 512 which is
> not.
> >
> > As I said: " Make sure that your all your mbufs are aligned by 512".
> >
> > Konstantin
> >
> > >
> > > On Tue, Oct 7, 2014 at 1:05 PM, Ananyev, Konstantin <
> konstantin.ananyev@intel.com> wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> > > > Sent: Tuesday, October 07, 2014 10:40 AM
> > > > To: dev@dpdk.org
> > > > Subject: [dpdk-dev] Aligned RX data.
> > > >
> > > > Hi , I'm trying to receive aligned packets from the wire.
> > > > Meaning that for all received packets the pkt.data is always aligned
> to
> > > > (512 -H).
> > > >
> > > > Looking at the pmds of ixgbe/vmxnet I see that the pmds call
> > > > __rte_mbuf_raw_alloc and set the rx descriptor with a
> > > > RTE_MBUF_DATA_DMA_ADDR_DEFAULT
> > > > Instead of the more appropriate RTE_MBUF_DATA_DMA_ADDR.
> > > >
> > > > Do I need to modify each pmd I'm using to be able to receive aligned
> data?
> > > Make sure that your all your mbufs are aligned by 512 and set in your
> config RTE_PKTMBUF_HEADROOM=512-H?
> > >
> > >
> > > > Or have I missed something?
> > > >
> > > > Thanks
> >
>
>