rte_eth_tx_burst return 0 after some time

[Alan Arondel <arondel.alan@gmail.com>]

Hello Everyone,

I'm trying to add an export module to my application. I use DPDK
version 20.11, a 82599ES 10-Gigabit SFI/SFP+ Network Connection 10fb
and the vfio-pci driver.
port configuration is the following:

port_conf->txmode.mq_mode = ETH_MQ_TX_NONE;
port_conf->txmode.offloads = DEV_TX_OFFLOAD_MULTI_SEGS;

I use only 1 queue on the card and the configuration is the following :
    nb_desc : 4096
    rs_threshold : 32
    free_threshold : 32

I have an issue where at some point during the lifetime of the
application, the rte_eth_tx_burst return only 0 and nothing can be
done once it is stuck after several seconds of the beginning of
transmission.

I have checked couple of things :
- i still get burst of packets to send
- i still have lot of mbuf in my mempool
- i use t-rex to generate traffic to forward and one of them have lot
of small packet and i can support up to 2,8Mpps (roughly 7,8Gbps)
sadly with another traffic i can't go up more than 200Kpps at best
(roughly 2Gbps if not less)

once the application is stuck, I tried to stop the traffic and restart
it with a very low bandwidth like 40Mbps but it changes nothing

here is a code snippet of what i'm trying to do :

nb_rx = rte_ring_sc_dequeue_burst(od_h->rx_rings[i], (void **) pkts_burst,

OUTPUT_DPDK_BURST_SIZE, NULL);

for (j = 0; j < nb_rx; j++) {

  if (!rte_pktmbuf_is_contiguous(pkts_burst[j])) {
      stats->jumbo_frame ++; /* i got some jumbo frame in both traffic
which i want to track, mbuf size is 1512 + RTE_PKTMBUF_HEADROOM */
   }

   ret = rte_eth_tx_burst(port_id, 0, &pkts_burst[j], 1); /* i know
this is weird but i have another issue with the order of the packets
when sending more than 1 packet even if the mbuf inside pkts_burst are
ordered correctly */
        if (unlikely(! ret)) {
              /* packet not sent, just drop it */
              rte_pktmbuf_free(pkts_burst[j]);
              stats->freed_mbuf ++;

               int flushed = rte_eth_tx_done_cleanup(port_id, 0, 0);
/* this is my last test since i saw we could flush the mbuf stuck in
the cache of the driver */
               if (flushed >= 0) {
                    stats->mbuf_flushed += flushed; /* i can see some
of them but not moving once the application do not send anymore
packets */
                } else {
                   stats->mbuf_flushed = flushed;
                }
          } else {
                stats->nb_sent ++;
                 sent = 1;
          }
     }
  i ++;
}

As far as I understand, the logic of the code seems to be ok since it
works on another type of traffic.
However i have no clue why it is stuck at some point. If you got some
insight to dig further on this i'm all ears.
Also, I would like to understand the free_threshold. As far as i
understand, mbuf will "stay" in the driver into there is not enough
descriptor available regarding the threshold hence the mbuf return to
the mempool. Is that right ?

Thank you for any help i will have on this.

Regards,
Alan.