DPDK Memory Management Help

DPDK Memory Management Help

[Alan Beadle]

Hi everyone,

I'm working with DPDK and could use some advice about memory
management of the DPDK heap.

I need to prevent DPDK from freeing the rte_mbuf buffers once the
corresponding packets have been sent with rte_eth_tx_burst(). My
application has local readers as well, and so I want to keep that data
around for them to read via shared memory. I would prefer to rely on
explicit calls to free any of these buffers. I can find plenty of
sources about rte_eth_tx_burst() freeing messages once they are sent,
but cannot find a way to override this behavior, or any mention of an
alternate call to send without freeing.

This would let me avoid copying the data into a separate buffer just
for local readers and to let them read from the same buffers that DPDK
already requires., leading to a significant performance gain (less
memcpy). This seems like something that should be possible *without*
modifying DPDK, but I haven't figured out how yet. I know that I would
need to take care that the buffers are not freed prior to sending.

Thanks for any advice,
-Alan

Re: DPDK Memory Management Help

[David Marchand]

On Tue, Oct 1, 2024 at 2:26 PM Alan Beadle <ab.beadle@gmail.com> wrote:
>
> Hi everyone,
>
> I'm working with DPDK and could use some advice about memory
> management of the DPDK heap.
>
> I need to prevent DPDK from freeing the rte_mbuf buffers once the
> corresponding packets have been sent with rte_eth_tx_burst(). My
> application has local readers as well, and so I want to keep that data
> around for them to read via shared memory. I would prefer to rely on
> explicit calls to free any of these buffers. I can find plenty of
> sources about rte_eth_tx_burst() freeing messages once they are sent,
> but cannot find a way to override this behavior, or any mention of an
> alternate call to send without freeing.
>
> This would let me avoid copying the data into a separate buffer just
> for local readers and to let them read from the same buffers that DPDK
> already requires., leading to a significant performance gain (less
> memcpy). This seems like something that should be possible *without*
> modifying DPDK, but I haven't figured out how yet. I know that I would
> need to take care that the buffers are not freed prior to sending.

This is a frequent question, so maybe we should update the doc.

You can increment the refcnt on the mbuf in your application before
calling rte_eth_tx_burst().
See rte_mbuf_refcnt_update(m, 1).
It is then your application responsibility to call rte_pktmbuf_free()
once it is done with the mbuf.


-- 
David Marchand