[dpdk-users] rte_kni_tx_burst() hangs because of no free descriptors

[dpdk-users] rte_kni_tx_burst() hangs because of no free descriptors

[=?gb18030?B?eWluZ3poaQ==?=]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb18030", Size: 2513 bytes --]

¡ª                                
Hi Experts,
 
Background:
 
We are using DPDK to develop a LoadBalancer following below logic: When a new packet is received:
 1. if the dst_addr is management IP, forward to KNI. 2. if the dst_addr is in VIP list, select backend and forward(modify dst mac address). 3. otherwise drop the packet. 
 
At this stage, we use one single thread for KNI forwarding and another for VIP forwarding(forward to eth).
 
DPDK version: 16.07
 NIC: 82599ES 10-Gigabit SFI/SFP+ Network Connection
 Linux: 14.04.1-Ubuntu x64  
 
Promblem description:
 
The program runs correctly for sometime(around 2 hours for 400Mb traffic). But it it will hang. When problem happens, rte_eth_tx_burst() will not able to send out any packets(always returns 0). We tracked into that function and noticed it is actually calling ixgbe driver's ixgbe_xmit_pkts_vec() function in our environment, because we use default tx queue configuration, after printing some info, we found if the free function works fine:
 tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 31 
 
it will trigger free and make 32 more free descriptors:
 tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 62 
 
but when something going wrong, it will no longer free anything:
 tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 0 tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 0 
 
It may related with the DD flag of the descriptor but we are not quite sure.
 
Our program logic:
  
create two mbuf pools on socket 0, one for rx_queue and one for kni. (all lcore threads runs on socket0)
 
init kni interface with rte_kni_alloc()
 
 
init one NIC interface with
 rte_eth_dev_configure(); rte_eth_rx_queue_setup(); rte_eth_tx_queue_setup(); rte_eth_dev_start(); 
 
 
 
in the eth main loop: (code is simplified)
 while(1) {     n = rte_eth_rx_burst(packets);     for (i = 0; i < n; ++i)      {         if (SEND_TO_KNI) {             m = rte_kni_tx_burst(packets[i]);             if (m != 1)) {                 rte_pktmbuf_free(packets[i]);             }         }         if (SEND_TO_ETH) {             // after modify the packet             m = rte_eth_tx_burst(packets[i]);             if (m != 1)) {                 rte_pktmbuf_free(packets[i]);             }         }         // otherwise drop the packet         rte_pktmbuf_free(packets[i]);     } } 
 
  
Please advise if I'm using DPDK in a wrong way. Sorry if I missed something basic, I'm new to DPDK.
 
Thanks in advance
 Best regards