mbuf refcnt issue

["Lombardo, Ed" <Ed.Lombardo@netscout.com>]

[-- Attachment #1: Type: text/plain, Size: 1914 bytes --]

Hi,
I have an application where we receive packets and transmit them.  The packet data is inspected and later mbuf is freed to mempool.

The pipeline is such that the Rx packet mbuf is saved to rx worker ring, then the application threads process the packets and decides if to transmit the packet and if true then increments the mbuf to a value of 2.  The batch of mbufs to transmit are put in a Tx ring queue for the Tx thread to pull from and call the DPDK rte_eth_tx_burst() with the batch of mbufs (limited to 400 mbufs).  In theory the transmit operation will decrement the mbuf refcnt.  In our application we could see the tx of the mbuf followed by another application thread that calls to free the mbufs, or vice versa.  We have no way to synchronize these threads.

Is the mbuf refcnt updates thread safe to allow un-deterministic handling of the mbufs among multiple threads?  The decision to transmit the mbuf and increment the mbuf refcnt and load in the tx ring is completed before the application says it is finished and frees the mbufs.

I am seeing in my error checking code the mbuf refcnt contains large values like 65520, 65529, 65530, 65534, 65535 in the early pipeline stage refcnt checks.

I read online and in the DPDK code that the mbuf refcnt update is atomic, and is thread safe; so, this is good.

Now this part is unclear to me and that is when the rte_eth_tx_burst() is called and returns the number of packets transmitted , does this  mean that transmit of the packets are completed and mbuf refcnt is decremented by 1 on return, or maybe the Tx engine queue is populated and mbuf refcnt is not decremented until it is actually transmitted, or much worse later in time.
Is the DPDK Tx operation intended to be the last stage of any pipeline that will free the mbuf if successfully transmitted?


Any help is appreciated to help me resolve my issue.

Thanks,
Ed



[-- Attachment #2: Type: text/html, Size: 4149 bytes --]