[dpdk-dev] [PATCH] enic: fix TX hang when number of packets > queue size

[dpdk-dev] [PATCH] enic: fix TX hang when number of packets > queue size

[John Daley]

If the nb_pkts parameter to rte_eth_tx_burst() was greater than
the TX descriptor count, a completion was not being requested
from the NIC, so descriptors would not be released back to the
host causing a lock-up.

Introduce a limit of how many TX descriptors can be used in a single
call to the enic PMD burst TX function before requesting a completion.

Fixes: d739ba4c6abf ("enic: improve Tx packet rate")

Signed-off-by: John Daley <johndale@cisco.com>
---
 drivers/net/enic/enic_ethdev.c | 20 ++++++++++++++++----
 drivers/net/enic/enic_res.h    |  1 +
 2 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 4969476..6bea940 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -523,7 +523,7 @@ static void enicpmd_remove_mac_addr(struct rte_eth_dev *eth_dev, __rte_unused ui
 static uint16_t enicpmd_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 	uint16_t nb_pkts)
 {
-	unsigned int index;
+	uint16_t index;
 	unsigned int frags;
 	unsigned int pkt_len;
 	unsigned int seg_len;
@@ -535,6 +535,7 @@ static uint16_t enicpmd_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 	unsigned short vlan_id;
 	unsigned short ol_flags;
 	uint8_t last_seg, eop;
+	unsigned int host_tx_descs = 0;
 
 	for (index = 0; index < nb_pkts; index++) {
 		tx_pkt = *tx_pkts++;
@@ -550,6 +551,7 @@ static uint16_t enicpmd_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 				return index;
 			}
 		}
+
 		pkt_len = tx_pkt->pkt_len;
 		vlan_id = tx_pkt->vlan_tci;
 		ol_flags = tx_pkt->ol_flags;
@@ -559,9 +561,19 @@ static uint16_t enicpmd_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 			next_tx_pkt = tx_pkt->next;
 			seg_len = tx_pkt->data_len;
 			inc_len += seg_len;
-			eop = (pkt_len == inc_len) || (!next_tx_pkt);
-			last_seg = eop &&
-				(index == ((unsigned int)nb_pkts - 1));
+
+			host_tx_descs++;
+			last_seg = 0;
+			eop = 0;
+			if ((pkt_len == inc_len) || !next_tx_pkt) {
+				eop = 1;
+				/* post if last packet in batch or > thresh */
+				if ((index == (nb_pkts - 1)) ||
+				   (host_tx_descs > ENIC_TX_POST_THRESH)) {
+					last_seg = 1;
+					host_tx_descs = 0;
+				}
+			}
 			enic_send_pkt(enic, wq, tx_pkt, (unsigned short)seg_len,
 				      !frags, eop, last_seg, ol_flags, vlan_id);
 			tx_pkt = next_tx_pkt;
diff --git a/drivers/net/enic/enic_res.h b/drivers/net/enic/enic_res.h
index 33f2e84..00fa71d 100644
--- a/drivers/net/enic/enic_res.h
+++ b/drivers/net/enic/enic_res.h
@@ -53,6 +53,7 @@
 
 #define ENIC_NON_TSO_MAX_DESC		16
 #define ENIC_DEFAULT_RX_FREE_THRESH	32
+#define ENIC_TX_POST_THRESH		(ENIC_MIN_WQ_DESCS / 2)
 
 #define ENIC_SETTING(enic, f) ((enic->config.flags & VENETF_##f) ? 1 : 0)
 
-- 
2.7.0

Re: [dpdk-dev] [PATCH] enic: fix TX hang when number of packets > queue size

[Thomas Monjalon]

2016-03-25 17:43, John Daley:
> If the nb_pkts parameter to rte_eth_tx_burst() was greater than
> the TX descriptor count, a completion was not being requested
> from the NIC, so descriptors would not be released back to the
> host causing a lock-up.
> 
> Introduce a limit of how many TX descriptors can be used in a single
> call to the enic PMD burst TX function before requesting a completion.
> 
> Fixes: d739ba4c6abf ("enic: improve Tx packet rate")
> 
> Signed-off-by: John Daley <johndale@cisco.com>

Applied, thanks