[dpdk-users] A question about rte_pktmbuf_init() and alike functions

[dpdk-users] A question about rte_pktmbuf_init() and alike functions

[Александр Самойлов]

 Hi all. I'm sorry if the question is naive or stupid, but still I couldn't find the answer in the manual.

The question is: are the element init functions called just once (with the mempool creation) or are they called each time we retrieve a new element from a pool? I'm asking because I use a mempool pool with pkt_mbuf structures for generating new Ethernet frames (with IP and TCP data) and it works just fine for a while (I reckon just as long as I'm getting a new pktmbuf with rte_pktmbuf_alloc and not a recycled one) and at some point my generated frames start to have the wrong TCP ckecksum (even though TCP checksum it totally OK at first). And the strange thing is that if, for instance, mempool has 2048 elements -  my program works for 5 seconds (rough estimate), and if mempool has 4096 elements - my program works for 10 seconds. The connection is obvious. I used to believe that init() functions are called each time we retrieve an element from a mempool, but now I'm not so sure.


Alex Samoylov

Re: [dpdk-users] A question about rte_pktmbuf_init() and alike functions

[Wiles, Keith]

On 11/3/15, 11:49 AM, "users on behalf of Александр Самойлов" <users-bounces@dpdk.org on behalf of cidjey1991@mail.ru> wrote:

> Hi all. I'm sorry if the question is naive or stupid, but still I couldn't find the answer in the manual.

The only naive or stupid question is one you do not ask or so goes the statement :-)

>
>The question is: are the element init functions called just once (with the mempool creation) or are they called each time we retrieve a new element from a pool? I'm asking because I use a mempool pool with pkt_mbuf structures for generating new Ethernet frames (with IP and TCP data) and it works just fine for a while (I reckon just as long as I'm getting a new pktmbuf with rte_pktmbuf_alloc and not a recycled one) and at some point my generated frames start to have the wrong TCP ckecksum (even though TCP checksum it totally OK at first). And the strange thing is that if, for instance, mempool has 2048 elements -  my program works for 5 seconds (rough estimate), and if mempool has 4096 elements - my program works for 10 seconds. The connection is obvious. I used to believe that init() functions are called each time we retrieve an element from a mempool, but now I'm not so sure.

When you call rte_pktmbuf_pool_create() it passes rte_pktmbuf_pool_init and rte_pktmbuf_init to the rte_mempool_create() routine. The rte_pktmbuf_pool_init() is called once for the pool and the rte_pktmbuf_init() is called for each mbuf it creates from the pool at create time only.

On rte_pktbuf_alloc() it will calls __rte_mbuf_raw_alloc() and then rte_pktmbuf_reset() to reset the mbuf into a sane state. If you are trying to save mbuf state across frees then it will not work and you have to find another way. For Pktgen I wanted to keep the mbufs intact across frees and it is not super easy and you can look in Pktgen code, but I would suggest it only if really required.

I hope that helps.

>
>
>Alex Samoylov


Regards,
Keith

Re: [dpdk-users] A question about rte_pktmbuf_init() and alike functions

[Matt Laswell]

The init functions are called once per object at initialization time, not
at allocation time.  From a performance perspective, this is preferable,
though it leaves a dependency on the application to do subsequent
initialization when the object is allocated or when it is freed.

--
Matt Laswell
Principal Software Engineer, infinite io
laswell@infiniteio.com

On Tue, Nov 3, 2015 at 11:49 AM, Александр Самойлов <cidjey1991@mail.ru>
wrote:

>  Hi all. I'm sorry if the question is naive or stupid, but still I
> couldn't find the answer in the manual.
>
> The question is: are the element init functions called just once (with the
> mempool creation) or are they called each time we retrieve a new element
> from a pool? I'm asking because I use a mempool pool with pkt_mbuf
> structures for generating new Ethernet frames (with IP and TCP data) and it
> works just fine for a while (I reckon just as long as I'm getting a new
> pktmbuf with rte_pktmbuf_alloc and not a recycled one) and at some point my
> generated frames start to have the wrong TCP ckecksum (even though TCP
> checksum it totally OK at first). And the strange thing is that if, for
> instance, mempool has 2048 elements -  my program works for 5 seconds
> (rough estimate), and if mempool has 4096 elements - my program works for
> 10 seconds. The connection is obvious. I used to believe that init()
> functions are called each time we retrieve an element from a mempool, but
> now I'm not so sure.
>
>
> Alex Samoylov

[dpdk-users] A question about rte_pktmbuf_init() and alike functions

[Александр Самойлов]

 An update. I decided to try this:

m = rte_pktmbuf_alloc(mp);
if(unlikely(m == NULL))
{
    RTE_LOG(INFO, USER1, "%s: Couldn't allocate pkt_mbuf\n", __func__);
    return NULL;
}
rte_pktmbuf_init(mp, NULL, m, 0);
rte_mbuf_refcnt_set(m, 1);
Well, it worked out. My memory is not dirty anymore and TCP checksum is always OK. Thank you for the input.
But this raises another question: You said that rte_pktmbuf_alloc() implies resetting the buffer to a "sane state". 
So why this reset doesn't make my new "m" buffer "sane enough" to keep a new frame while rte_pktmbuf_init does?


Alex Samoylov

Re: [dpdk-users] A question about rte_pktmbuf_init() and alike functions

[Wiles, Keith]

On 11/3/15, 1:23 PM, "users on behalf of Александр Самойлов" <users-bounces@dpdk.org on behalf of cidjey1991@mail.ru> wrote:

> An update. I decided to try this:
>
>m = rte_pktmbuf_alloc(mp);
>if(unlikely(m == NULL))
>{
>    RTE_LOG(INFO, USER1, "%s: Couldn't allocate pkt_mbuf\n", __func__);
>    return NULL;
>}
>rte_pktmbuf_init(mp, NULL, m, 0);
>rte_mbuf_refcnt_set(m, 1);
>Well, it worked out. My memory is not dirty anymore and TCP checksum is always OK. Thank you for the input.
>But this raises another question: You said that rte_pktmbuf_alloc() implies resetting the buffer to a "sane state". 
>So why this reset doesn't make my new "m" buffer "sane enough" to keep a new frame while rte_pktmbuf_init does?

You can look at the code, but the reset tries to do the least amount work in touching the mbuf header for performance.
>
>
>Alex Samoylov
>
>
>


Regards,
Keith

Re: [dpdk-users] A question about rte_pktmbuf_init() and alike functions

[Александр Самойлов]

 Oh, that's emberrasing. After another manual reading I found this one (in mbuf section of Programmer's Guide): 

Freeing a mbuf means returning it into its original mempool. The content of an mbuf is not modified when it is stored in a pool (as a free mbuf). Fields initialized by the constructor do not need to be re-initialized at mbuf allocation.

Should've guessed then. But still, maybe it's reasonable to highlight this part a little? Smth like "So you can't safely use a mbuf repeatedly without memzeroing its content" or smth? :)

P.S. Sorry for disturbance


>
>
>On 11/3/15, 1:23 PM, "users on behalf of Александр Самойлов" < users-bounces@dpdk.org on behalf of  cidjey1991@mail.ru > wrote:
>
>> An update. I decided to try this:
>>
>>m = rte_pktmbuf_alloc(mp);
>>if(unlikely(m == NULL))
>>{
>>    RTE_LOG(INFO, USER1, "%s: Couldn't allocate pkt_mbuf\n", __func__);
>>    return NULL;
>>}
>>rte_pktmbuf_init(mp, NULL, m, 0);
>>rte_mbuf_refcnt_set(m, 1);
>>Well, it worked out. My memory is not dirty anymore and TCP checksum is always OK. Thank you for the input.
>>But this raises another question: You said that rte_pktmbuf_alloc() implies resetting the buffer to a "sane state". 
>>So why this reset doesn't make my new "m" buffer "sane enough" to keep a new frame while rte_pktmbuf_init does?
>
>You can look at the code, but the reset tries to do the least amount work in touching the mbuf header for performance.
>>
>>
>>Alex Samoylov
>>
>>
>>
>
>
>Regards,
>Keith
>
>
>
>