[PATCH 00/15] DPAA and DPAA2 driver changes

[PATCH 00/15] DPAA and DPAA2 driver changes

[Gagandeep Singh]

This series have list of patch for bug fixes and
some enhancements to DPAA1 and DPAA2 net drivers.

Apeksha Gupta (2):
  net/enetfec: fix restart issue
  net/enetfec: fix buffer leak issue

Gagandeep Singh (7):
  net/dpaa: support ESP packet type in packet parsing
  net/dpaa2: use internal mempool for SG table
  net/dpaa2: fix buffer free on transmit SG packets
  net/dpaa: use internal mempool for SG table
  bus/dpaa: mempool ops registration change
  net/dpaa: fix buffer free on transmit SG packets
  net/dpaa: fix buffer free in slow path

Rohit Raj (3):
  bus/fslmc: add timeout in MC send command API
  net/dpaa: fix Jumbo packet Rx in case of VSP
  bus/dpaa: pass interface name as a string instead of pointer

Vanshika Shukla (2):
  bus/dpaa: use non-block mode for FD open
  net/dpaa2: fix dpdmux configuration for error behaviour

brick (1):
  net/dpaa2: check free enqueue descriptors before Tx

 drivers/bus/dpaa/base/qbman/process.c     |   8 +-
 drivers/bus/dpaa/base/qbman/qman_driver.c |   6 +-
 drivers/bus/dpaa/dpaa_bus.c               |  10 +-
 drivers/bus/fslmc/mc/mc_sys.c             |  18 ++-
 drivers/event/dpaa2/dpaa2_eventdev.c      |   8 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  20 ++-
 drivers/net/dpaa/dpaa_ethdev.h            |  19 +++
 drivers/net/dpaa/dpaa_flow.c              |  13 +-
 drivers/net/dpaa/dpaa_flow.h              |   5 +-
 drivers/net/dpaa/dpaa_rxtx.c              |  97 +++++++-----
 drivers/net/dpaa/dpaa_rxtx.h              |   6 +
 drivers/net/dpaa2/dpaa2_ethdev.c          |  14 ++
 drivers/net/dpaa2/dpaa2_ethdev.h          |  21 +++
 drivers/net/dpaa2/dpaa2_mux.c             |   2 +-
 drivers/net/dpaa2/dpaa2_rxtx.c            | 174 +++++++++++++++-------
 drivers/net/enetfec/enet_ethdev.c         |   4 +
 drivers/net/enetfec/enet_rxtx.c           |  29 ++--
 17 files changed, 315 insertions(+), 139 deletions(-)

-- 
2.25.1

[PATCH 01/15] bus/dpaa: use non-block mode for FD open

[Gagandeep Singh]

From: Vanshika Shukla <vanshika.shukla@nxp.com>

This patch sets qman portal file descriptors used for
interrupts IO processing in non-blocking mode to avoid
any unwanted blocks while IO operations over the FD.

Signed-off-by: Vanshika Shukla <vanshika.shukla@nxp.com>
---
 drivers/bus/dpaa/base/qbman/qman_driver.c | 6 +++---
 drivers/net/dpaa/dpaa_ethdev.c            | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/bus/dpaa/base/qbman/qman_driver.c b/drivers/bus/dpaa/base/qbman/qman_driver.c
index dfbafe581a..7a129a2d86 100644
--- a/drivers/bus/dpaa/base/qbman/qman_driver.c
+++ b/drivers/bus/dpaa/base/qbman/qman_driver.c
@@ -1,7 +1,7 @@
 /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
  *
  * Copyright 2008-2016 Freescale Semiconductor Inc.
- * Copyright 2017,2019 NXP
+ * Copyright 2017-2022 NXP
  *
  */
 
@@ -64,7 +64,7 @@ static int fsl_qman_portal_init(uint32_t index, int is_shared)
 	qpcfg.addr_virt[DPAA_PORTAL_CE] = map.addr.cena;
 	qpcfg.addr_virt[DPAA_PORTAL_CI] = map.addr.cinh;
 
-	qmfd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY);
+	qmfd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY | O_NONBLOCK);
 	if (qmfd == -1) {
 		pr_err("QMan irq init failed\n");
 		process_portal_unmap(&map.addr);
@@ -174,7 +174,7 @@ struct qman_portal *fsl_qman_fq_portal_create(int *fd)
 	q_pcfg->addr_virt[DPAA_PORTAL_CE] = q_map.addr.cena;
 	q_pcfg->addr_virt[DPAA_PORTAL_CI] = q_map.addr.cinh;
 
-	q_fd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY);
+	q_fd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY | O_NONBLOCK);
 	if (q_fd == -1) {
 		pr_err("QMan irq init failed\n");
 		goto err;
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index f21cbba0d9..abcb1bc9ec 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -1496,7 +1496,7 @@ static int dpaa_dev_queue_intr_disable(struct rte_eth_dev *dev,
 
 	temp1 = read(rxq->q_fd, &temp, sizeof(temp));
 	if (temp1 != sizeof(temp))
-		DPAA_PMD_ERR("irq read error");
+		DPAA_PMD_DEBUG("read did not return anything");
 
 	qman_fq_portal_thread_irq(rxq->qp);
 
-- 
2.25.1

[PATCH 02/15] net/enetfec: fix restart issue

[Gagandeep Singh]

From: Apeksha Gupta <apeksha.gupta@nxp.com>

Queue reset is missing in restart because of which
IO cannot work on device restart.

This patch fixes the issue by resetting the queues on
device restart.

Fixes: b84fdd39638b ("net/enetfec: support UIO")
Cc: stable@dpdk.org

Signed-off-by: Apeksha Gupta <apeksha.gupta@nxp.com>
Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
---
 drivers/net/enetfec/enet_ethdev.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/net/enetfec/enet_ethdev.c b/drivers/net/enetfec/enet_ethdev.c
index c938e58204..898aad1c37 100644
--- a/drivers/net/enetfec/enet_ethdev.c
+++ b/drivers/net/enetfec/enet_ethdev.c
@@ -54,6 +54,7 @@ enetfec_restart(struct rte_eth_dev *dev)
 	uint32_t rcntl = OPT_FRAME_SIZE | 0x04;
 	uint32_t ecntl = ENETFEC_ETHEREN;
 	uint32_t val;
+	int i;
 
 	/* Clear any outstanding interrupt. */
 	writel(0xffffffff, (uint8_t *)fep->hw_baseaddr_v + ENETFEC_EIR);
@@ -149,6 +150,9 @@ enetfec_restart(struct rte_eth_dev *dev)
 	/* And last, enable the transmit and receive processing */
 	rte_write32(rte_cpu_to_le_32(ecntl),
 		(uint8_t *)fep->hw_baseaddr_v + ENETFEC_ECR);
+
+	for (i = 0; i < fep->max_rx_queues; i++)
+		rte_write32(0, fep->rx_queues[i]->bd.active_reg_desc);
 	rte_delay_us(10);
 }
 
-- 
2.25.1

[PATCH 03/15] net/enetfec: fix buffer leak issue

[Gagandeep Singh]

From: Apeksha Gupta <apeksha.gupta@nxp.com>

Driver has no proper handling to free unused
allocated mbufs in case of error or when the rx
processing complete because of which mempool
can be empty after some time.

This patch fixes this issue by moving the buffer
allocation code to the right place in driver.

Fixes: ecae71571b0d ("net/enetfec: support Rx/Tx")
Cc: stable@dpdk.org

Signed-off-by: Apeksha Gupta <apeksha.gupta@nxp.com>
Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
---
 drivers/net/enetfec/enet_rxtx.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/drivers/net/enetfec/enet_rxtx.c b/drivers/net/enetfec/enet_rxtx.c
index 49b326315d..0aea8b240d 100644
--- a/drivers/net/enetfec/enet_rxtx.c
+++ b/drivers/net/enetfec/enet_rxtx.c
@@ -39,11 +39,6 @@ enetfec_recv_pkts(void *rxq1, struct rte_mbuf **rx_pkts,
 		if (pkt_received >= nb_pkts)
 			break;
 
-		new_mbuf = rte_pktmbuf_alloc(pool);
-		if (unlikely(new_mbuf == NULL)) {
-			stats->rx_nombuf++;
-			break;
-		}
 		/* Check for errors. */
 		status ^= RX_BD_LAST;
 		if (status & (RX_BD_LG | RX_BD_SH | RX_BD_NO |
@@ -72,6 +67,12 @@ enetfec_recv_pkts(void *rxq1, struct rte_mbuf **rx_pkts,
 			goto rx_processing_done;
 		}
 
+		new_mbuf = rte_pktmbuf_alloc(pool);
+		if (unlikely(new_mbuf == NULL)) {
+			stats->rx_nombuf++;
+			break;
+		}
+
 		/* Process the incoming frame. */
 		stats->ipackets++;
 		pkt_len = rte_le_to_cpu_16(rte_read16(&bdp->bd_datlen));
@@ -193,7 +194,16 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			tx_st = 0;
 			break;
 		}
+
+		mbuf = *(tx_pkts);
+		if (mbuf->nb_segs > 1) {
+			ENETFEC_DP_LOG(DEBUG, "SG not supported");
+			return pkt_transmitted;
+		}
+
+		tx_pkts++;
 		bdp = txq->bd.cur;
+
 		/* First clean the ring */
 		index = enet_get_bd_index(bdp, &txq->bd);
 		status = rte_le_to_cpu_16(rte_read16(&bdp->bd_sc));
@@ -207,9 +217,6 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			txq->tx_mbuf[index] = NULL;
 		}
 
-		mbuf = *(tx_pkts);
-		tx_pkts++;
-
 		/* Fill in a Tx ring entry */
 		last_bdp = bdp;
 		status &= ~TX_BD_STATS;
@@ -219,10 +226,6 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		stats->opackets++;
 		stats->obytes += buflen;
 
-		if (mbuf->nb_segs > 1) {
-			ENETFEC_DP_LOG(DEBUG, "SG not supported");
-			return -1;
-		}
 		status |= (TX_BD_LAST);
 		data = rte_pktmbuf_mtod(mbuf, void *);
 		for (i = 0; i <= buflen; i += RTE_CACHE_LINE_SIZE)
@@ -268,5 +271,5 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		 */
 		txq->bd.cur = bdp;
 	}
-	return nb_pkts;
+	return pkt_transmitted;
 }
-- 
2.25.1

[PATCH 04/15] net/dpaa2: fix dpdmux configuration for error behaviour

[Gagandeep Singh]

From: Vanshika Shukla <vanshika.shukla@nxp.com>

Driver is giving the wrong interface ID while setting the
error behaviour.

This patch fixes the issue by passing the correct MAC interface
index value to the API.

Fixes: 3d43972b1b42 ("net/dpaa2: do not drop parse error packets by dpdmux")
Cc: stable@dpdk.org

Signed-off-by: Vanshika Shukla <vanshika.shukla@nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.h | 3 +++
 drivers/net/dpaa2/dpaa2_mux.c    | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index f69df95253..32ae762e4a 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -66,6 +66,9 @@
 /* Tx confirmation enabled */
 #define DPAA2_TX_CONF_ENABLE	0x06
 
+/* DPDMUX index for DPMAC */
+#define DPAA2_DPDMUX_DPMAC_IDX 0
+
 /* HW loopback the egress traffic to self ingress*/
 #define DPAA2_TX_MAC_LOOPBACK_MODE 0x20
 
diff --git a/drivers/net/dpaa2/dpaa2_mux.c b/drivers/net/dpaa2/dpaa2_mux.c
index 3289f388e1..7456f43f42 100644
--- a/drivers/net/dpaa2/dpaa2_mux.c
+++ b/drivers/net/dpaa2/dpaa2_mux.c
@@ -336,7 +336,7 @@ dpaa2_create_dpdmux_device(int vdev_fd __rte_unused,
 
 		ret = dpdmux_if_set_errors_behavior(&dpdmux_dev->dpdmux,
 				CMD_PRI_LOW,
-				dpdmux_dev->token, dpdmux_id,
+				dpdmux_dev->token, DPAA2_DPDMUX_DPMAC_IDX,
 				&mux_err_cfg);
 		if (ret) {
 			DPAA2_PMD_ERR("dpdmux_if_set_errors_behavior %s err %d",
-- 
2.25.1

[PATCH 05/15] net/dpaa2: check free enqueue descriptors before Tx

[Gagandeep Singh]

From: brick <brick.yang@nxp.com>

Check if there exists free enqueue descriptors before enqueuing Tx
packet. Also try to free enqueue descriptors in case they are not
free.

Fixes: ed1cdbed6a15 ("net/dpaa2: support multiple Tx queues enqueue for ordered")
Cc: stable@dpdk.org

Signed-off-by: brick <brick.yang@nxp.com>
Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
---
 drivers/event/dpaa2/dpaa2_eventdev.c |  8 ++---
 drivers/net/dpaa2/dpaa2_rxtx.c       | 50 +++++++++++++++++++---------
 2 files changed, 38 insertions(+), 20 deletions(-)

diff --git a/drivers/event/dpaa2/dpaa2_eventdev.c b/drivers/event/dpaa2/dpaa2_eventdev.c
index 1001297cda..d09c5b8778 100644
--- a/drivers/event/dpaa2/dpaa2_eventdev.c
+++ b/drivers/event/dpaa2/dpaa2_eventdev.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2017,2019-2021 NXP
+ * Copyright 2017,2019-2022 NXP
  */
 
 #include <assert.h>
@@ -175,7 +175,7 @@ dpaa2_eventdev_enqueue_burst(void *port, const struct rte_event ev[],
 				if (retry_count > DPAA2_EV_TX_RETRY_COUNT) {
 					num_tx += loop;
 					nb_events -= loop;
-					return num_tx + loop;
+					return num_tx;
 				}
 			} else {
 				loop += ret;
@@ -1015,9 +1015,7 @@ dpaa2_eventdev_txa_enqueue(void *port,
 		txq[i] = rte_eth_devices[m[i]->port].data->tx_queues[qid];
 	}
 
-	dpaa2_dev_tx_multi_txq_ordered(txq, m, nb_events);
-
-	return nb_events;
+	return dpaa2_dev_tx_multi_txq_ordered(txq, m, nb_events);
 }
 
 static struct eventdev_ops dpaa2_eventdev_ops = {
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 9436a95ac8..bc0e49b0d4 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -1525,7 +1525,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 	uint32_t loop, retry_count;
 	int32_t ret;
 	struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
-	uint32_t frames_to_send;
+	uint32_t frames_to_send, num_free_eq_desc = 0;
 	struct rte_mempool *mp;
 	struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS];
 	struct dpaa2_queue *dpaa2_q[MAX_TX_RING_SLOTS];
@@ -1547,16 +1547,44 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 	}
 	swp = DPAA2_PER_LCORE_PORTAL;
 
-	for (loop = 0; loop < nb_pkts; loop++) {
+	frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
+		dpaa2_eqcr_size : nb_pkts;
+
+	for (loop = 0; loop < frames_to_send; loop++) {
 		dpaa2_q[loop] = (struct dpaa2_queue *)queue[loop];
 		eth_data = dpaa2_q[loop]->eth_data;
 		priv = eth_data->dev_private;
+		if (!priv->en_loose_ordered) {
+			if (*dpaa2_seqn(*bufs) & DPAA2_ENQUEUE_FLAG_ORP) {
+				if (!num_free_eq_desc) {
+					num_free_eq_desc = dpaa2_free_eq_descriptors();
+					if (!num_free_eq_desc)
+						goto send_frames;
+				}
+				num_free_eq_desc--;
+			}
+		}
+
+		DPAA2_PMD_DP_DEBUG("===> eth_data =%p, fqid =%d\n",
+				   eth_data, dpaa2_q[loop]->fqid);
+
+		/*Check if the queue is congested*/
+		retry_count = 0;
+		while (qbman_result_SCN_state(dpaa2_q[loop]->cscn)) {
+			retry_count++;
+			/* Retry for some time before giving up */
+			if (retry_count > CONG_RETRY_COUNT)
+				goto send_frames;
+		}
+
+		/*Prepare enqueue descriptor*/
 		qbman_eq_desc_clear(&eqdesc[loop]);
+
 		if (*dpaa2_seqn(*bufs) && priv->en_ordered) {
 			order_sendq = (struct dpaa2_queue *)priv->tx_vq[0];
 			dpaa2_set_enqueue_descriptor(order_sendq,
-							     (*bufs),
-							     &eqdesc[loop]);
+						     (*bufs),
+						     &eqdesc[loop]);
 		} else {
 			qbman_eq_desc_set_no_orp(&eqdesc[loop],
 							 DPAA2_EQ_RESP_ERR_FQ);
@@ -1564,14 +1592,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 						     dpaa2_q[loop]->fqid);
 		}
 
-		retry_count = 0;
-		while (qbman_result_SCN_state(dpaa2_q[loop]->cscn)) {
-			retry_count++;
-			/* Retry for some time before giving up */
-			if (retry_count > CONG_RETRY_COUNT)
-				goto send_frames;
-		}
-
 		if (likely(RTE_MBUF_DIRECT(*bufs))) {
 			mp = (*bufs)->pool;
 			/* Check the basic scenario and set
@@ -1591,7 +1611,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 					&fd_arr[loop],
 					mempool_to_bpid(mp));
 				bufs++;
-				dpaa2_q[loop]++;
 				continue;
 			}
 		} else {
@@ -1637,18 +1656,19 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 		}
 
 		bufs++;
-		dpaa2_q[loop]++;
 	}
 
 send_frames:
 	frames_to_send = loop;
 	loop = 0;
+	retry_count = 0;
 	while (loop < frames_to_send) {
 		ret = qbman_swp_enqueue_multiple_desc(swp, &eqdesc[loop],
 				&fd_arr[loop],
 				frames_to_send - loop);
 		if (likely(ret > 0)) {
 			loop += ret;
+			retry_count = 0;
 		} else {
 			retry_count++;
 			if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
@@ -1834,7 +1854,7 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 		retry_count = 0;
 		while (i < loop) {
 			ret = qbman_swp_enqueue_multiple_desc(swp,
-				       &eqdesc[loop], &fd_arr[i], loop - i);
+				       &eqdesc[i], &fd_arr[i], loop - i);
 			if (unlikely(ret < 0)) {
 				retry_count++;
 				if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
-- 
2.25.1

Re: [PATCH 05/15] net/dpaa2: check free enqueue descriptors before Tx

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> From: brick <brick.yang@nxp.com>
> 
> Check if there exists free enqueue descriptors before enqueuing Tx
> packet. Also try to free enqueue descriptors in case they are not
> free.
> 
> Fixes: ed1cdbed6a15 ("net/dpaa2: support multiple Tx queues enqueue for ordered")
> Cc: stable@dpdk.org
> 
> Signed-off-by: brick <brick.yang@nxp.com>

Can you please use name tag as "Name Surname <email lower case>", like
  Signed-off-by: Brick Yang <brick.yang@nxp.com>

<...>

> +		DPAA2_PMD_DP_DEBUG("===> eth_data =%p, fqid =%d\n",
> +				   eth_data, dpaa2_q[loop]->fqid);
> +
> +		/*Check if the queue is congested*/

syntax, more common to put space before/after '/* ' & ' */'

> +		retry_count = 0;
> +		while (qbman_result_SCN_state(dpaa2_q[loop]->cscn)) {
> +			retry_count++;
> +			/* Retry for some time before giving up */
> +			if (retry_count > CONG_RETRY_COUNT)
> +				goto send_frames;
> +		}
> +
> +		/*Prepare enqueue descriptor*/

ditto

[PATCH 06/15] net/dpaa: support ESP packet type in packet parsing

[Gagandeep Singh]

Add support of ESP packet type in packet receive path.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/net/dpaa/dpaa_rxtx.c | 10 ++++++++++
 drivers/net/dpaa/dpaa_rxtx.h |  6 ++++++
 2 files changed, 16 insertions(+)

diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index 20b75efb63..22205cec30 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -178,6 +178,16 @@ static inline void dpaa_eth_packet_info(struct rte_mbuf *m, void *fd_virt_addr)
 		m->packet_type = RTE_PTYPE_L2_ETHER |
 			RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_UDP;
 		break;
+	case DPAA_PKT_TYPE_IPSEC_IPV4:
+		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
+			m->packet_type = RTE_PTYPE_L2_ETHER |
+				RTE_PTYPE_L3_IPV4 | RTE_PTYPE_TUNNEL_ESP;
+		break;
+	case DPAA_PKT_TYPE_IPSEC_IPV6:
+		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
+			m->packet_type = RTE_PTYPE_L2_ETHER |
+				RTE_PTYPE_L3_IPV6 | RTE_PTYPE_TUNNEL_ESP;
+		break;
 	case DPAA_PKT_TYPE_IPV4_EXT_UDP:
 		m->packet_type = RTE_PTYPE_L2_ETHER |
 			RTE_PTYPE_L3_IPV4_EXT | RTE_PTYPE_L4_UDP;
diff --git a/drivers/net/dpaa/dpaa_rxtx.h b/drivers/net/dpaa/dpaa_rxtx.h
index 99e09196e9..b2d7c0f2a3 100644
--- a/drivers/net/dpaa/dpaa_rxtx.h
+++ b/drivers/net/dpaa/dpaa_rxtx.h
@@ -47,6 +47,7 @@
  *	L4R 0xE0 -
  *		0x20 - TCP
  *		0x40 - UDP
+ *		0x60 - IPsec
  *		0x80 - SCTP
  *	L3R 0xEDC4 (in Big Endian) -
  *		0x8000 - IPv4
@@ -63,6 +64,7 @@
  */
 #define DPAA_PARSE_MASK			0x00F044EF00800000
 #define DPAA_PARSE_VLAN_MASK		0x0000000000700000
+#define DPAA_PARSE_ESP_MASK		0x0008000000000000
 
 /* Parsed values (Little Endian) */
 #define DPAA_PKT_TYPE_NONE		0x0000000000000000
@@ -137,6 +139,10 @@
 			(0x0020000000000000 | DPAA_PKT_TYPE_TUNNEL_4_6)
 #define DPAA_PKT_TYPE_TUNNEL_6_4_TCP \
 			(0x0020000000000000 | DPAA_PKT_TYPE_TUNNEL_6_4)
+#define DPAA_PKT_TYPE_IPSEC_IPV4 \
+			(0x0060000000000000 | DPAA_PKT_TYPE_IPV4)
+#define DPAA_PKT_TYPE_IPSEC_IPV6 \
+			(0x0060000000000000 | DPAA_PKT_TYPE_IPV6)
 
 /* Checksum Errors */
 #define DPAA_PKT_IP_CSUM_ERR		0x0000400200000000
-- 
2.25.1

Re: [PATCH 06/15] net/dpaa: support ESP packet type in packet parsing

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> Add support of ESP packet type in packet receive path.
> 
> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> ---
>   drivers/net/dpaa/dpaa_rxtx.c | 10 ++++++++++
>   drivers/net/dpaa/dpaa_rxtx.h |  6 ++++++
>   2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
> index 20b75efb63..22205cec30 100644
> --- a/drivers/net/dpaa/dpaa_rxtx.c
> +++ b/drivers/net/dpaa/dpaa_rxtx.c
> @@ -178,6 +178,16 @@ static inline void dpaa_eth_packet_info(struct rte_mbuf *m, void *fd_virt_addr)
>   		m->packet_type = RTE_PTYPE_L2_ETHER |
>   			RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_UDP;
>   		break;
> +	case DPAA_PKT_TYPE_IPSEC_IPV4:
> +		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
> +			m->packet_type = RTE_PTYPE_L2_ETHER |
> +				RTE_PTYPE_L3_IPV4 | RTE_PTYPE_TUNNEL_ESP;
> +		break;
> +	case DPAA_PKT_TYPE_IPSEC_IPV6:
> +		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
> +			m->packet_type = RTE_PTYPE_L2_ETHER |
> +				RTE_PTYPE_L3_IPV6 | RTE_PTYPE_TUNNEL_ESP;
> +		break;
>   	case DPAA_PKT_TYPE_IPV4_EXT_UDP:
>   		m->packet_type = RTE_PTYPE_L2_ETHER |
>   			RTE_PTYPE_L3_IPV4_EXT | RTE_PTYPE_L4_UDP;

Shouldn't 'dpaa_supported_ptypes_get()' needs to be updated to notify 
host on the capability to report ESP types.

> diff --git a/drivers/net/dpaa/dpaa_rxtx.h b/drivers/net/dpaa/dpaa_rxtx.h
> index 99e09196e9..b2d7c0f2a3 100644
> --- a/drivers/net/dpaa/dpaa_rxtx.h
> +++ b/drivers/net/dpaa/dpaa_rxtx.h
> @@ -47,6 +47,7 @@
>    *	L4R 0xE0 -
>    *		0x20 - TCP
>    *		0x40 - UDP
> + *		0x60 - IPsec
>    *		0x80 - SCTP
>    *	L3R 0xEDC4 (in Big Endian) -
>    *		0x8000 - IPv4
> @@ -63,6 +64,7 @@
>    */
>   #define DPAA_PARSE_MASK			0x00F044EF00800000
>   #define DPAA_PARSE_VLAN_MASK		0x0000000000700000
> +#define DPAA_PARSE_ESP_MASK		0x0008000000000000
>   
>   /* Parsed values (Little Endian) */
>   #define DPAA_PKT_TYPE_NONE		0x0000000000000000
> @@ -137,6 +139,10 @@
>   			(0x0020000000000000 | DPAA_PKT_TYPE_TUNNEL_4_6)
>   #define DPAA_PKT_TYPE_TUNNEL_6_4_TCP \
>   			(0x0020000000000000 | DPAA_PKT_TYPE_TUNNEL_6_4)
> +#define DPAA_PKT_TYPE_IPSEC_IPV4 \
> +			(0x0060000000000000 | DPAA_PKT_TYPE_IPV4)
> +#define DPAA_PKT_TYPE_IPSEC_IPV6 \
> +			(0x0060000000000000 | DPAA_PKT_TYPE_IPV6)
>   
>   /* Checksum Errors */
>   #define DPAA_PKT_IP_CSUM_ERR		0x0000400200000000

RE: [PATCH 06/15] net/dpaa: support ESP packet type in packet parsing

[Gagandeep Singh]

Hi

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Wednesday, October 5, 2022 7:51 PM
> To: Gagandeep Singh <G.Singh@nxp.com>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH 06/15] net/dpaa: support ESP packet type in packet
> parsing
> 
> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> > Add support of ESP packet type in packet receive path.
> >
> > Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> > ---
> >   drivers/net/dpaa/dpaa_rxtx.c | 10 ++++++++++
> >   drivers/net/dpaa/dpaa_rxtx.h |  6 ++++++
> >   2 files changed, 16 insertions(+)
> >
> > diff --git a/drivers/net/dpaa/dpaa_rxtx.c
> > b/drivers/net/dpaa/dpaa_rxtx.c index 20b75efb63..22205cec30 100644
> > --- a/drivers/net/dpaa/dpaa_rxtx.c
> > +++ b/drivers/net/dpaa/dpaa_rxtx.c
> > @@ -178,6 +178,16 @@ static inline void dpaa_eth_packet_info(struct
> rte_mbuf *m, void *fd_virt_addr)
> >   		m->packet_type = RTE_PTYPE_L2_ETHER |
> >   			RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_UDP;
> >   		break;
> > +	case DPAA_PKT_TYPE_IPSEC_IPV4:
> > +		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
> > +			m->packet_type = RTE_PTYPE_L2_ETHER |
> > +				RTE_PTYPE_L3_IPV4 |
> RTE_PTYPE_TUNNEL_ESP;
> > +		break;
> > +	case DPAA_PKT_TYPE_IPSEC_IPV6:
> > +		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
> > +			m->packet_type = RTE_PTYPE_L2_ETHER |
> > +				RTE_PTYPE_L3_IPV6 |
> RTE_PTYPE_TUNNEL_ESP;
> > +		break;
> >   	case DPAA_PKT_TYPE_IPV4_EXT_UDP:
> >   		m->packet_type = RTE_PTYPE_L2_ETHER |
> >   			RTE_PTYPE_L3_IPV4_EXT | RTE_PTYPE_L4_UDP;
> 
> Shouldn't 'dpaa_supported_ptypes_get()' needs to be updated to notify host
> on the capability to report ESP types.

Yes, I will update it in next version.

> 
> > diff --git a/drivers/net/dpaa/dpaa_rxtx.h
> > b/drivers/net/dpaa/dpaa_rxtx.h index 99e09196e9..b2d7c0f2a3 100644
> > --- a/drivers/net/dpaa/dpaa_rxtx.h
> > +++ b/drivers/net/dpaa/dpaa_rxtx.h
> > @@ -47,6 +47,7 @@
> >    *	L4R 0xE0 -
> >    *		0x20 - TCP
> >    *		0x40 - UDP
> > + *		0x60 - IPsec
> >    *		0x80 - SCTP
> >    *	L3R 0xEDC4 (in Big Endian) -
> >    *		0x8000 - IPv4
> > @@ -63,6 +64,7 @@
> >    */
> >   #define DPAA_PARSE_MASK			0x00F044EF00800000
> >   #define DPAA_PARSE_VLAN_MASK		0x0000000000700000
> > +#define DPAA_PARSE_ESP_MASK		0x0008000000000000
> >
> >   /* Parsed values (Little Endian) */
> >   #define DPAA_PKT_TYPE_NONE		0x0000000000000000
> > @@ -137,6 +139,10 @@
> >   			(0x0020000000000000 |
> DPAA_PKT_TYPE_TUNNEL_4_6)
> >   #define DPAA_PKT_TYPE_TUNNEL_6_4_TCP \
> >   			(0x0020000000000000 |
> DPAA_PKT_TYPE_TUNNEL_6_4)
> > +#define DPAA_PKT_TYPE_IPSEC_IPV4 \
> > +			(0x0060000000000000 | DPAA_PKT_TYPE_IPV4)
> #define
> > +DPAA_PKT_TYPE_IPSEC_IPV6 \
> > +			(0x0060000000000000 | DPAA_PKT_TYPE_IPV6)
> >
> >   /* Checksum Errors */
> >   #define DPAA_PKT_IP_CSUM_ERR		0x0000400200000000

[PATCH 07/15] net/dpaa2: use internal mempool for SG table

[Gagandeep Singh]

Creating and using driver's mempool for
allocating the SG table memory required for
FD creation instead of relying on user mempool.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.c | 14 ++++++++++++++
 drivers/net/dpaa2/dpaa2_ethdev.h |  9 +++++++++
 drivers/net/dpaa2/dpaa2_rxtx.c   | 13 ++++++-------
 3 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 37a8b43114..c7aae70300 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -78,6 +78,8 @@ bool dpaa2_enable_err_queue;
 #define MAX_NB_RX_DESC		11264
 int total_nb_rx_desc;
 
+struct rte_mempool *dpaa2_tx_sg_pool;
+
 struct rte_dpaa2_xstats_name_off {
 	char name[RTE_ETH_XSTATS_NAME_SIZE];
 	uint8_t page_id; /* dpni statistics page id */
@@ -2907,6 +2909,18 @@ rte_dpaa2_probe(struct rte_dpaa2_driver *dpaa2_drv,
 	/* Invoke PMD device initialization function */
 	diag = dpaa2_dev_init(eth_dev);
 	if (diag == 0) {
+		if (!dpaa2_tx_sg_pool) {
+			dpaa2_tx_sg_pool =
+				rte_pktmbuf_pool_create("dpaa2_mbuf_tx_sg_pool",
+				DPAA2_POOL_SIZE,
+				DPAA2_POOL_CACHE_SIZE, 0,
+				DPAA2_MAX_SGS * sizeof(struct qbman_sge),
+				rte_socket_id());
+			if (dpaa2_tx_sg_pool == NULL) {
+				DPAA2_PMD_ERR("SG pool creation failed\n");
+				return -ENOMEM;
+			}
+		}
 		rte_eth_dev_probing_finish(eth_dev);
 		return 0;
 	}
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 32ae762e4a..872dced517 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -121,6 +121,15 @@
 #define DPAA2_PKT_TYPE_VLAN_1		0x0160
 #define DPAA2_PKT_TYPE_VLAN_2		0x0260
 
+/* Global pool used by driver for SG list TX */
+extern struct rte_mempool *dpaa2_tx_sg_pool;
+/* Maximum SG segments */
+#define DPAA2_MAX_SGS 128
+/* SG pool size */
+#define DPAA2_POOL_SIZE 2048
+/* SG pool cache size */
+#define DPAA2_POOL_CACHE_SIZE 256
+
 /* enable timestamp in mbuf*/
 extern bool dpaa2_enable_ts[];
 extern uint64_t dpaa2_timestamp_rx_dynflag;
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index bc0e49b0d4..dcd86c4056 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -403,7 +403,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
 static int __rte_noinline __rte_hot
 eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 		  struct qbman_fd *fd,
-		  struct rte_mempool *mp, uint16_t bpid)
+		  uint16_t bpid)
 {
 	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
 	struct qbman_sge *sgt, *sge = NULL;
@@ -433,12 +433,12 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 		}
 		DPAA2_SET_FD_OFFSET(fd, offset);
 	} else {
-		temp = rte_pktmbuf_alloc(mp);
+		temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
 		if (temp == NULL) {
 			DPAA2_PMD_DP_DEBUG("No memory to allocate S/G table\n");
 			return -ENOMEM;
 		}
-		DPAA2_SET_ONLY_FD_BPID(fd, bpid);
+		DPAA2_SET_ONLY_FD_BPID(fd, mempool_to_bpid(dpaa2_tx_sg_pool));
 		DPAA2_SET_FD_OFFSET(fd, temp->data_off);
 #ifdef RTE_LIBRTE_MEMPOOL_DEBUG
 		rte_mempool_check_cookies(rte_mempool_from_obj((void *)temp),
@@ -1321,9 +1321,10 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 
 			if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
 				if (unlikely((*bufs)->nb_segs > 1)) {
+					mp = (*bufs)->pool;
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
-							      mp, 0))
+							      mempool_to_bpid(mp)))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
@@ -1372,7 +1373,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							&fd_arr[loop],
-							mp, bpid))
+							bpid))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
@@ -1646,7 +1647,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 			if (unlikely((*bufs)->nb_segs > 1)) {
 				if (eth_mbuf_to_sg_fd(*bufs,
 						      &fd_arr[loop],
-						      mp,
 						      bpid))
 					goto send_frames;
 			} else {
@@ -1810,7 +1810,6 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
-							      mp,
 							      bpid))
 						goto send_n_return;
 				} else {
-- 
2.25.1

Re: [PATCH 07/15] net/dpaa2: use internal mempool for SG table

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> Creating and using driver's mempool for
> allocating the SG table memory required for
> FD creation instead of relying on user mempool.
> 

As far as I can see this is in the Tx path, can you please explain why 
driver need an internal pktmbuf pool?

And shouldn't mempool needs to be freed on driver close and release.

Same comment applies for dpaa version of the patch (12/15).

> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> ---
>   drivers/net/dpaa2/dpaa2_ethdev.c | 14 ++++++++++++++
>   drivers/net/dpaa2/dpaa2_ethdev.h |  9 +++++++++
>   drivers/net/dpaa2/dpaa2_rxtx.c   | 13 ++++++-------
>   3 files changed, 29 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
> index 37a8b43114..c7aae70300 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -78,6 +78,8 @@ bool dpaa2_enable_err_queue;
>   #define MAX_NB_RX_DESC		11264
>   int total_nb_rx_desc;
>   
> +struct rte_mempool *dpaa2_tx_sg_pool;
> +
>   struct rte_dpaa2_xstats_name_off {
>   	char name[RTE_ETH_XSTATS_NAME_SIZE];
>   	uint8_t page_id; /* dpni statistics page id */
> @@ -2907,6 +2909,18 @@ rte_dpaa2_probe(struct rte_dpaa2_driver *dpaa2_drv,
>   	/* Invoke PMD device initialization function */
>   	diag = dpaa2_dev_init(eth_dev);
>   	if (diag == 0) {
> +		if (!dpaa2_tx_sg_pool) {
> +			dpaa2_tx_sg_pool =
> +				rte_pktmbuf_pool_create("dpaa2_mbuf_tx_sg_pool",
> +				DPAA2_POOL_SIZE,
> +				DPAA2_POOL_CACHE_SIZE, 0,
> +				DPAA2_MAX_SGS * sizeof(struct qbman_sge),
> +				rte_socket_id());
> +			if (dpaa2_tx_sg_pool == NULL) {
> +				DPAA2_PMD_ERR("SG pool creation failed\n");
> +				return -ENOMEM;
> +			}
> +		}
>   		rte_eth_dev_probing_finish(eth_dev);
>   		return 0;
>   	}
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
> index 32ae762e4a..872dced517 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.h
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.h
> @@ -121,6 +121,15 @@
>   #define DPAA2_PKT_TYPE_VLAN_1		0x0160
>   #define DPAA2_PKT_TYPE_VLAN_2		0x0260
>   
> +/* Global pool used by driver for SG list TX */
> +extern struct rte_mempool *dpaa2_tx_sg_pool;
> +/* Maximum SG segments */
> +#define DPAA2_MAX_SGS 128
> +/* SG pool size */
> +#define DPAA2_POOL_SIZE 2048
> +/* SG pool cache size */
> +#define DPAA2_POOL_CACHE_SIZE 256
> +
>   /* enable timestamp in mbuf*/
>   extern bool dpaa2_enable_ts[];
>   extern uint64_t dpaa2_timestamp_rx_dynflag;
> diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
> index bc0e49b0d4..dcd86c4056 100644
> --- a/drivers/net/dpaa2/dpaa2_rxtx.c
> +++ b/drivers/net/dpaa2/dpaa2_rxtx.c
> @@ -403,7 +403,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
>   static int __rte_noinline __rte_hot
>   eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
>   		  struct qbman_fd *fd,
> -		  struct rte_mempool *mp, uint16_t bpid)
> +		  uint16_t bpid)
>   {
>   	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
>   	struct qbman_sge *sgt, *sge = NULL;
> @@ -433,12 +433,12 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
>   		}
>   		DPAA2_SET_FD_OFFSET(fd, offset);
>   	} else {
> -		temp = rte_pktmbuf_alloc(mp);
> +		temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
>   		if (temp == NULL) {
>   			DPAA2_PMD_DP_DEBUG("No memory to allocate S/G table\n");
>   			return -ENOMEM;
>   		}
> -		DPAA2_SET_ONLY_FD_BPID(fd, bpid);
> +		DPAA2_SET_ONLY_FD_BPID(fd, mempool_to_bpid(dpaa2_tx_sg_pool));
>   		DPAA2_SET_FD_OFFSET(fd, temp->data_off);
>   #ifdef RTE_LIBRTE_MEMPOOL_DEBUG
>   		rte_mempool_check_cookies(rte_mempool_from_obj((void *)temp),
> @@ -1321,9 +1321,10 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
>   
>   			if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
>   				if (unlikely((*bufs)->nb_segs > 1)) {
> +					mp = (*bufs)->pool;
>   					if (eth_mbuf_to_sg_fd(*bufs,
>   							      &fd_arr[loop],
> -							      mp, 0))
> +							      mempool_to_bpid(mp)))
>   						goto send_n_return;
>   				} else {
>   					eth_mbuf_to_fd(*bufs,
> @@ -1372,7 +1373,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
>   				if (unlikely((*bufs)->nb_segs > 1)) {
>   					if (eth_mbuf_to_sg_fd(*bufs,
>   							&fd_arr[loop],
> -							mp, bpid))
> +							bpid))
>   						goto send_n_return;
>   				} else {
>   					eth_mbuf_to_fd(*bufs,
> @@ -1646,7 +1647,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
>   			if (unlikely((*bufs)->nb_segs > 1)) {
>   				if (eth_mbuf_to_sg_fd(*bufs,
>   						      &fd_arr[loop],
> -						      mp,
>   						      bpid))
>   					goto send_frames;
>   			} else {
> @@ -1810,7 +1810,6 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
>   				if (unlikely((*bufs)->nb_segs > 1)) {
>   					if (eth_mbuf_to_sg_fd(*bufs,
>   							      &fd_arr[loop],
> -							      mp,
>   							      bpid))
>   						goto send_n_return;
>   				} else {

RE: [PATCH 07/15] net/dpaa2: use internal mempool for SG table

[Gagandeep Singh]

Hi

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Wednesday, October 5, 2022 7:51 PM
> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> Subject: Re: [PATCH 07/15] net/dpaa2: use internal mempool for SG table
> 
> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> > Creating and using driver's mempool for allocating the SG table memory
> > required for FD creation instead of relying on user mempool.
> >
> 
> As far as I can see this is in the Tx path, can you please explain why
> driver need an internal pktmbuf pool?
> 
> And shouldn't mempool needs to be freed on driver close and release.
> 
> Same comment applies for dpaa version of the patch (12/15).
> 

In TX path, in case of SG packets, driver need additional SG table memory to store the data
of all segments to pass to HW. Earlier it was dependent on the user's pool to allocate memory
which is wrong. so allocating the new pool in the driver for this purpose.

We are creating only one pool for all the DPAA net devices on first DPAA net device probe, so it can
be free only on last device probe. So will it be worth to free the pool on last device probe as
memory will be released on process kill as well?

> > Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> > ---
> >   drivers/net/dpaa2/dpaa2_ethdev.c | 14 ++++++++++++++
> >   drivers/net/dpaa2/dpaa2_ethdev.h |  9 +++++++++
> >   drivers/net/dpaa2/dpaa2_rxtx.c   | 13 ++++++-------
> >   3 files changed, 29 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
> b/drivers/net/dpaa2/dpaa2_ethdev.c
> > index 37a8b43114..c7aae70300 100644
> > --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> > +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> > @@ -78,6 +78,8 @@ bool dpaa2_enable_err_queue;
> >   #define MAX_NB_RX_DESC		11264
> >   int total_nb_rx_desc;
> >
> > +struct rte_mempool *dpaa2_tx_sg_pool;
> > +
> >   struct rte_dpaa2_xstats_name_off {
> >   	char name[RTE_ETH_XSTATS_NAME_SIZE];
> >   	uint8_t page_id; /* dpni statistics page id */
> > @@ -2907,6 +2909,18 @@ rte_dpaa2_probe(struct rte_dpaa2_driver
> *dpaa2_drv,
> >   	/* Invoke PMD device initialization function */
> >   	diag = dpaa2_dev_init(eth_dev);
> >   	if (diag == 0) {
> > +		if (!dpaa2_tx_sg_pool) {
> > +			dpaa2_tx_sg_pool =
> > +
> 	rte_pktmbuf_pool_create("dpaa2_mbuf_tx_sg_pool",
> > +				DPAA2_POOL_SIZE,
> > +				DPAA2_POOL_CACHE_SIZE, 0,
> > +				DPAA2_MAX_SGS * sizeof(struct qbman_sge),
> > +				rte_socket_id());
> > +			if (dpaa2_tx_sg_pool == NULL) {
> > +				DPAA2_PMD_ERR("SG pool creation
> failed\n");
> > +				return -ENOMEM;
> > +			}
> > +		}
> >   		rte_eth_dev_probing_finish(eth_dev);
> >   		return 0;
> >   	}
> > diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h
> b/drivers/net/dpaa2/dpaa2_ethdev.h
> > index 32ae762e4a..872dced517 100644
> > --- a/drivers/net/dpaa2/dpaa2_ethdev.h
> > +++ b/drivers/net/dpaa2/dpaa2_ethdev.h
> > @@ -121,6 +121,15 @@
> >   #define DPAA2_PKT_TYPE_VLAN_1		0x0160
> >   #define DPAA2_PKT_TYPE_VLAN_2		0x0260
> >
> > +/* Global pool used by driver for SG list TX */
> > +extern struct rte_mempool *dpaa2_tx_sg_pool;
> > +/* Maximum SG segments */
> > +#define DPAA2_MAX_SGS 128
> > +/* SG pool size */
> > +#define DPAA2_POOL_SIZE 2048
> > +/* SG pool cache size */
> > +#define DPAA2_POOL_CACHE_SIZE 256
> > +
> >   /* enable timestamp in mbuf*/
> >   extern bool dpaa2_enable_ts[];
> >   extern uint64_t dpaa2_timestamp_rx_dynflag;
> > diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c
> b/drivers/net/dpaa2/dpaa2_rxtx.c
> > index bc0e49b0d4..dcd86c4056 100644
> > --- a/drivers/net/dpaa2/dpaa2_rxtx.c
> > +++ b/drivers/net/dpaa2/dpaa2_rxtx.c
> > @@ -403,7 +403,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
> >   static int __rte_noinline __rte_hot
> >   eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
> >   		  struct qbman_fd *fd,
> > -		  struct rte_mempool *mp, uint16_t bpid)
> > +		  uint16_t bpid)
> >   {
> >   	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
> >   	struct qbman_sge *sgt, *sge = NULL;
> > @@ -433,12 +433,12 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
> >   		}
> >   		DPAA2_SET_FD_OFFSET(fd, offset);
> >   	} else {
> > -		temp = rte_pktmbuf_alloc(mp);
> > +		temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
> >   		if (temp == NULL) {
> >   			DPAA2_PMD_DP_DEBUG("No memory to allocate
> S/G table\n");
> >   			return -ENOMEM;
> >   		}
> > -		DPAA2_SET_ONLY_FD_BPID(fd, bpid);
> > +		DPAA2_SET_ONLY_FD_BPID(fd,
> mempool_to_bpid(dpaa2_tx_sg_pool));
> >   		DPAA2_SET_FD_OFFSET(fd, temp->data_off);
> >   #ifdef RTE_LIBRTE_MEMPOOL_DEBUG
> >   		rte_mempool_check_cookies(rte_mempool_from_obj((void
> *)temp),
> > @@ -1321,9 +1321,10 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf
> **bufs, uint16_t nb_pkts)
> >
> >   			if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
> >   				if (unlikely((*bufs)->nb_segs > 1)) {
> > +					mp = (*bufs)->pool;
> >   					if (eth_mbuf_to_sg_fd(*bufs,
> >   							      &fd_arr[loop],
> > -							      mp, 0))
> > +
> mempool_to_bpid(mp)))
> >   						goto send_n_return;
> >   				} else {
> >   					eth_mbuf_to_fd(*bufs,
> > @@ -1372,7 +1373,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf
> **bufs, uint16_t nb_pkts)
> >   				if (unlikely((*bufs)->nb_segs > 1)) {
> >   					if (eth_mbuf_to_sg_fd(*bufs,
> >   							&fd_arr[loop],
> > -							mp, bpid))
> > +							bpid))
> >   						goto send_n_return;
> >   				} else {
> >   					eth_mbuf_to_fd(*bufs,
> > @@ -1646,7 +1647,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
> >   			if (unlikely((*bufs)->nb_segs > 1)) {
> >   				if (eth_mbuf_to_sg_fd(*bufs,
> >   						      &fd_arr[loop],
> > -						      mp,
> >   						      bpid))
> >   					goto send_frames;
> >   			} else {
> > @@ -1810,7 +1810,6 @@ dpaa2_dev_tx_ordered(void *queue, struct
> rte_mbuf **bufs, uint16_t nb_pkts)
> >   				if (unlikely((*bufs)->nb_segs > 1)) {
> >   					if (eth_mbuf_to_sg_fd(*bufs,
> >   							      &fd_arr[loop],
> > -							      mp,
> >   							      bpid))
> >   						goto send_n_return;
> >   				} else {

Re: [PATCH 07/15] net/dpaa2: use internal mempool for SG table

[Ferruh Yigit]

On 10/6/2022 9:49 AM, Gagandeep Singh wrote:
> Hi
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>> Sent: Wednesday, October 5, 2022 7:51 PM
>> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
>> Subject: Re: [PATCH 07/15] net/dpaa2: use internal mempool for SG table
>>
>> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
>>> Creating and using driver's mempool for allocating the SG table memory
>>> required for FD creation instead of relying on user mempool.
>>>
>>
>> As far as I can see this is in the Tx path, can you please explain why
>> driver need an internal pktmbuf pool?
>>
>> And shouldn't mempool needs to be freed on driver close and release.
>>
>> Same comment applies for dpaa version of the patch (12/15).
>>
> 
> In TX path, in case of SG packets, driver need additional SG table memory to store the data
> of all segments to pass to HW. Earlier it was dependent on the user's pool to allocate memory
> which is wrong. so allocating the new pool in the driver for this purpose.
> 

Agree to have driver internal mempool for driver internal requirements.

> We are creating only one pool for all the DPAA net devices on first DPAA net device probe, so it can
> be free only on last device probe. So will it be worth to free the pool on last device probe as
> memory will be released on process kill as well?
> 

Yes, it will be cleared but it is good discipline to free memory on 
quit/close.
What do you think to allocate one per-port, and store it in device 
private data, it makes easy to free it when port closed or released?

>>> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
>>> ---
>>>    drivers/net/dpaa2/dpaa2_ethdev.c | 14 ++++++++++++++
>>>    drivers/net/dpaa2/dpaa2_ethdev.h |  9 +++++++++
>>>    drivers/net/dpaa2/dpaa2_rxtx.c   | 13 ++++++-------
>>>    3 files changed, 29 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
>> b/drivers/net/dpaa2/dpaa2_ethdev.c
>>> index 37a8b43114..c7aae70300 100644
>>> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
>>> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
>>> @@ -78,6 +78,8 @@ bool dpaa2_enable_err_queue;
>>>    #define MAX_NB_RX_DESC		11264
>>>    int total_nb_rx_desc;
>>>
>>> +struct rte_mempool *dpaa2_tx_sg_pool;
>>> +
>>>    struct rte_dpaa2_xstats_name_off {
>>>    	char name[RTE_ETH_XSTATS_NAME_SIZE];
>>>    	uint8_t page_id; /* dpni statistics page id */
>>> @@ -2907,6 +2909,18 @@ rte_dpaa2_probe(struct rte_dpaa2_driver
>> *dpaa2_drv,
>>>    	/* Invoke PMD device initialization function */
>>>    	diag = dpaa2_dev_init(eth_dev);
>>>    	if (diag == 0) {
>>> +		if (!dpaa2_tx_sg_pool) {
>>> +			dpaa2_tx_sg_pool =
>>> +
>> 	rte_pktmbuf_pool_create("dpaa2_mbuf_tx_sg_pool",
>>> +				DPAA2_POOL_SIZE,
>>> +				DPAA2_POOL_CACHE_SIZE, 0,
>>> +				DPAA2_MAX_SGS * sizeof(struct qbman_sge),
>>> +				rte_socket_id());
>>> +			if (dpaa2_tx_sg_pool == NULL) {
>>> +				DPAA2_PMD_ERR("SG pool creation
>> failed\n");
>>> +				return -ENOMEM;
>>> +			}
>>> +		}
>>>    		rte_eth_dev_probing_finish(eth_dev);
>>>    		return 0;
>>>    	}
>>> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h
>> b/drivers/net/dpaa2/dpaa2_ethdev.h
>>> index 32ae762e4a..872dced517 100644
>>> --- a/drivers/net/dpaa2/dpaa2_ethdev.h
>>> +++ b/drivers/net/dpaa2/dpaa2_ethdev.h
>>> @@ -121,6 +121,15 @@
>>>    #define DPAA2_PKT_TYPE_VLAN_1		0x0160
>>>    #define DPAA2_PKT_TYPE_VLAN_2		0x0260
>>>
>>> +/* Global pool used by driver for SG list TX */
>>> +extern struct rte_mempool *dpaa2_tx_sg_pool;
>>> +/* Maximum SG segments */
>>> +#define DPAA2_MAX_SGS 128
>>> +/* SG pool size */
>>> +#define DPAA2_POOL_SIZE 2048
>>> +/* SG pool cache size */
>>> +#define DPAA2_POOL_CACHE_SIZE 256
>>> +
>>>    /* enable timestamp in mbuf*/
>>>    extern bool dpaa2_enable_ts[];
>>>    extern uint64_t dpaa2_timestamp_rx_dynflag;
>>> diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c
>> b/drivers/net/dpaa2/dpaa2_rxtx.c
>>> index bc0e49b0d4..dcd86c4056 100644
>>> --- a/drivers/net/dpaa2/dpaa2_rxtx.c
>>> +++ b/drivers/net/dpaa2/dpaa2_rxtx.c
>>> @@ -403,7 +403,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
>>>    static int __rte_noinline __rte_hot
>>>    eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
>>>    		  struct qbman_fd *fd,
>>> -		  struct rte_mempool *mp, uint16_t bpid)
>>> +		  uint16_t bpid)
>>>    {
>>>    	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
>>>    	struct qbman_sge *sgt, *sge = NULL;
>>> @@ -433,12 +433,12 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
>>>    		}
>>>    		DPAA2_SET_FD_OFFSET(fd, offset);
>>>    	} else {
>>> -		temp = rte_pktmbuf_alloc(mp);
>>> +		temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
>>>    		if (temp == NULL) {
>>>    			DPAA2_PMD_DP_DEBUG("No memory to allocate
>> S/G table\n");
>>>    			return -ENOMEM;
>>>    		}
>>> -		DPAA2_SET_ONLY_FD_BPID(fd, bpid);
>>> +		DPAA2_SET_ONLY_FD_BPID(fd,
>> mempool_to_bpid(dpaa2_tx_sg_pool));
>>>    		DPAA2_SET_FD_OFFSET(fd, temp->data_off);
>>>    #ifdef RTE_LIBRTE_MEMPOOL_DEBUG
>>>    		rte_mempool_check_cookies(rte_mempool_from_obj((void
>> *)temp),
>>> @@ -1321,9 +1321,10 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf
>> **bufs, uint16_t nb_pkts)
>>>
>>>    			if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
>>>    				if (unlikely((*bufs)->nb_segs > 1)) {
>>> +					mp = (*bufs)->pool;
>>>    					if (eth_mbuf_to_sg_fd(*bufs,
>>>    							      &fd_arr[loop],
>>> -							      mp, 0))
>>> +
>> mempool_to_bpid(mp)))
>>>    						goto send_n_return;
>>>    				} else {
>>>    					eth_mbuf_to_fd(*bufs,
>>> @@ -1372,7 +1373,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf
>> **bufs, uint16_t nb_pkts)
>>>    				if (unlikely((*bufs)->nb_segs > 1)) {
>>>    					if (eth_mbuf_to_sg_fd(*bufs,
>>>    							&fd_arr[loop],
>>> -							mp, bpid))
>>> +							bpid))
>>>    						goto send_n_return;
>>>    				} else {
>>>    					eth_mbuf_to_fd(*bufs,
>>> @@ -1646,7 +1647,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
>>>    			if (unlikely((*bufs)->nb_segs > 1)) {
>>>    				if (eth_mbuf_to_sg_fd(*bufs,
>>>    						      &fd_arr[loop],
>>> -						      mp,
>>>    						      bpid))
>>>    					goto send_frames;
>>>    			} else {
>>> @@ -1810,7 +1810,6 @@ dpaa2_dev_tx_ordered(void *queue, struct
>> rte_mbuf **bufs, uint16_t nb_pkts)
>>>    				if (unlikely((*bufs)->nb_segs > 1)) {
>>>    					if (eth_mbuf_to_sg_fd(*bufs,
>>>    							      &fd_arr[loop],
>>> -							      mp,
>>>    							      bpid))
>>>    						goto send_n_return;
>>>    				} else {
> 

RE: [PATCH 07/15] net/dpaa2: use internal mempool for SG table

[Gagandeep Singh]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Thursday, October 6, 2022 3:08 PM
> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> Subject: Re: [PATCH 07/15] net/dpaa2: use internal mempool for SG table
> 
> On 10/6/2022 9:49 AM, Gagandeep Singh wrote:
> > Hi
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@amd.com>
> >> Sent: Wednesday, October 5, 2022 7:51 PM
> >> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> >> Subject: Re: [PATCH 07/15] net/dpaa2: use internal mempool for SG
> >> table
> >>
> >> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> >>> Creating and using driver's mempool for allocating the SG table
> >>> memory required for FD creation instead of relying on user mempool.
> >>>
> >>
> >> As far as I can see this is in the Tx path, can you please explain
> >> why driver need an internal pktmbuf pool?
> >>
> >> And shouldn't mempool needs to be freed on driver close and release.
> >>
> >> Same comment applies for dpaa version of the patch (12/15).
> >>
> >
> > In TX path, in case of SG packets, driver need additional SG table
> > memory to store the data of all segments to pass to HW. Earlier it was
> > dependent on the user's pool to allocate memory which is wrong. so
> allocating the new pool in the driver for this purpose.
> >
> 
> Agree to have driver internal mempool for driver internal requirements.
> 
> > We are creating only one pool for all the DPAA net devices on first
> > DPAA net device probe, so it can be free only on last device probe. So
> > will it be worth to free the pool on last device probe as memory will be
> released on process kill as well?
> >
> 
> Yes, it will be cleared but it is good discipline to free memory on quit/close.
> What do you think to allocate one per-port, and store it in device private
> data, it makes easy to free it when port closed or released?
> 
Yes It will be easy to have per device mempool, But due to limited number of
HW pools we cannot allocate as many as eth devices number of pool for internal
Uses.

Ok, I will free the pool on last eth device remove in next version.

> >>> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> >>> ---
> >>>    drivers/net/dpaa2/dpaa2_ethdev.c | 14 ++++++++++++++
> >>>    drivers/net/dpaa2/dpaa2_ethdev.h |  9 +++++++++
> >>>    drivers/net/dpaa2/dpaa2_rxtx.c   | 13 ++++++-------
> >>>    3 files changed, 29 insertions(+), 7 deletions(-)
> >>>
> >>> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
> >> b/drivers/net/dpaa2/dpaa2_ethdev.c
> >>> index 37a8b43114..c7aae70300 100644
> >>> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> >>> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> >>> @@ -78,6 +78,8 @@ bool dpaa2_enable_err_queue;
> >>>    #define MAX_NB_RX_DESC		11264
> >>>    int total_nb_rx_desc;
> >>>
> >>> +struct rte_mempool *dpaa2_tx_sg_pool;
> >>> +
> >>>    struct rte_dpaa2_xstats_name_off {
> >>>    	char name[RTE_ETH_XSTATS_NAME_SIZE];
> >>>    	uint8_t page_id; /* dpni statistics page id */ @@ -2907,6
> >>> +2909,18 @@ rte_dpaa2_probe(struct rte_dpaa2_driver
> >> *dpaa2_drv,
> >>>    	/* Invoke PMD device initialization function */
> >>>    	diag = dpaa2_dev_init(eth_dev);
> >>>    	if (diag == 0) {
> >>> +		if (!dpaa2_tx_sg_pool) {
> >>> +			dpaa2_tx_sg_pool =
> >>> +
> >> 	rte_pktmbuf_pool_create("dpaa2_mbuf_tx_sg_pool",
> >>> +				DPAA2_POOL_SIZE,
> >>> +				DPAA2_POOL_CACHE_SIZE, 0,
> >>> +				DPAA2_MAX_SGS * sizeof(struct qbman_sge),
> >>> +				rte_socket_id());
> >>> +			if (dpaa2_tx_sg_pool == NULL) {
> >>> +				DPAA2_PMD_ERR("SG pool creation
> >> failed\n");
> >>> +				return -ENOMEM;
> >>> +			}
> >>> +		}
> >>>    		rte_eth_dev_probing_finish(eth_dev);
> >>>    		return 0;
> >>>    	}
> >>> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h
> >> b/drivers/net/dpaa2/dpaa2_ethdev.h
> >>> index 32ae762e4a..872dced517 100644
> >>> --- a/drivers/net/dpaa2/dpaa2_ethdev.h
> >>> +++ b/drivers/net/dpaa2/dpaa2_ethdev.h
> >>> @@ -121,6 +121,15 @@
> >>>    #define DPAA2_PKT_TYPE_VLAN_1		0x0160
> >>>    #define DPAA2_PKT_TYPE_VLAN_2		0x0260
> >>>
> >>> +/* Global pool used by driver for SG list TX */ extern struct
> >>> +rte_mempool *dpaa2_tx_sg_pool;
> >>> +/* Maximum SG segments */
> >>> +#define DPAA2_MAX_SGS 128
> >>> +/* SG pool size */
> >>> +#define DPAA2_POOL_SIZE 2048
> >>> +/* SG pool cache size */
> >>> +#define DPAA2_POOL_CACHE_SIZE 256
> >>> +
> >>>    /* enable timestamp in mbuf*/
> >>>    extern bool dpaa2_enable_ts[];
> >>>    extern uint64_t dpaa2_timestamp_rx_dynflag; diff --git
> >>> a/drivers/net/dpaa2/dpaa2_rxtx.c
> >> b/drivers/net/dpaa2/dpaa2_rxtx.c
> >>> index bc0e49b0d4..dcd86c4056 100644
> >>> --- a/drivers/net/dpaa2/dpaa2_rxtx.c
> >>> +++ b/drivers/net/dpaa2/dpaa2_rxtx.c
> >>> @@ -403,7 +403,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
> >>>    static int __rte_noinline __rte_hot
> >>>    eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
> >>>    		  struct qbman_fd *fd,
> >>> -		  struct rte_mempool *mp, uint16_t bpid)
> >>> +		  uint16_t bpid)
> >>>    {
> >>>    	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
> >>>    	struct qbman_sge *sgt, *sge = NULL; @@ -433,12 +433,12 @@
> >>> eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
> >>>    		}
> >>>    		DPAA2_SET_FD_OFFSET(fd, offset);
> >>>    	} else {
> >>> -		temp = rte_pktmbuf_alloc(mp);
> >>> +		temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
> >>>    		if (temp == NULL) {
> >>>    			DPAA2_PMD_DP_DEBUG("No memory to allocate
> >> S/G table\n");
> >>>    			return -ENOMEM;
> >>>    		}
> >>> -		DPAA2_SET_ONLY_FD_BPID(fd, bpid);
> >>> +		DPAA2_SET_ONLY_FD_BPID(fd,
> >> mempool_to_bpid(dpaa2_tx_sg_pool));
> >>>    		DPAA2_SET_FD_OFFSET(fd, temp->data_off);
> >>>    #ifdef RTE_LIBRTE_MEMPOOL_DEBUG
> >>>    		rte_mempool_check_cookies(rte_mempool_from_obj((void
> >> *)temp),
> >>> @@ -1321,9 +1321,10 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf
> >> **bufs, uint16_t nb_pkts)
> >>>
> >>>    			if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
> >>>    				if (unlikely((*bufs)->nb_segs > 1)) {
> >>> +					mp = (*bufs)->pool;
> >>>    					if (eth_mbuf_to_sg_fd(*bufs,
> >>>    							      &fd_arr[loop],
> >>> -							      mp, 0))
> >>> +
> >> mempool_to_bpid(mp)))
> >>>    						goto send_n_return;
> >>>    				} else {
> >>>    					eth_mbuf_to_fd(*bufs,
> >>> @@ -1372,7 +1373,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf
> >> **bufs, uint16_t nb_pkts)
> >>>    				if (unlikely((*bufs)->nb_segs > 1)) {
> >>>    					if (eth_mbuf_to_sg_fd(*bufs,
> >>>    							&fd_arr[loop],
> >>> -							mp, bpid))
> >>> +							bpid))
> >>>    						goto send_n_return;
> >>>    				} else {
> >>>    					eth_mbuf_to_fd(*bufs,
> >>> @@ -1646,7 +1647,6 @@ dpaa2_dev_tx_multi_txq_ordered(void
> **queue,
> >>>    			if (unlikely((*bufs)->nb_segs > 1)) {
> >>>    				if (eth_mbuf_to_sg_fd(*bufs,
> >>>    						      &fd_arr[loop],
> >>> -						      mp,
> >>>    						      bpid))
> >>>    					goto send_frames;
> >>>    			} else {
> >>> @@ -1810,7 +1810,6 @@ dpaa2_dev_tx_ordered(void *queue, struct
> >> rte_mbuf **bufs, uint16_t nb_pkts)
> >>>    				if (unlikely((*bufs)->nb_segs > 1)) {
> >>>    					if (eth_mbuf_to_sg_fd(*bufs,
> >>>    							      &fd_arr[loop],
> >>> -							      mp,
> >>>    							      bpid))
> >>>    						goto send_n_return;
> >>>    				} else {
> >

[PATCH 08/15] net/dpaa2: fix buffer free on transmit SG packets

[Gagandeep Singh]

When using SG list to TX with external and direct buffers,
HW free the direct buffers and driver free the external buffers.

Software scans the complete SG mbuf list to find the external
buffers to free, but this is wrong as hardware can free the
direct buffers if any present in the list and same can be
re-allocated for other purpose in multi thread or high spead
running traffic environment with new data in it. So the software
which is scanning the SG mbuf list, if that list has any direct
buffer present then that direct buffer's next pointor can give
wrong pointer value, if already freed by hardware which
can do the mempool corruption or memory leak.

In this patch instead of relying on user given SG mbuf list
we are storing the buffers in an internal list which will
be scanned by driver after transmit to free non-direct
buffers.

This patch also fixes 2 more memory leak issues.

Driver is freeing complete SG list by checking external buffer
flag in first segment only, but external buffer can be attached
to any of the segment. Because of which driver either can double
free buffers or there can be memory leak.

In case of indirect buffers, driver is modifying the original
buffer list to free the indirect buffers but this orginal buffer
list is being used even after transmit packets for software
buffer cleanup. This can cause the buffer leak issue.

Fixes: 6bfbafe18d15 ("net/dpaa2: support external buffers in Tx")
Cc: stable@dpdk.org

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.h |   9 +++
 drivers/net/dpaa2/dpaa2_rxtx.c   | 111 +++++++++++++++++++++++--------
 2 files changed, 92 insertions(+), 28 deletions(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 872dced517..c88c8146dc 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -129,6 +129,15 @@ extern struct rte_mempool *dpaa2_tx_sg_pool;
 #define DPAA2_POOL_SIZE 2048
 /* SG pool cache size */
 #define DPAA2_POOL_CACHE_SIZE 256
+/* structure to free external and indirect
+ * buffers.
+ */
+struct sw_buf_free {
+	/* To which packet this segment belongs */
+	uint16_t pkt_id;
+	/* The actual segment */
+	struct rte_mbuf *seg;
+};
 
 /* enable timestamp in mbuf*/
 extern bool dpaa2_enable_ts[];
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index dcd86c4056..94815485b8 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -403,9 +403,12 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
 static int __rte_noinline __rte_hot
 eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 		  struct qbman_fd *fd,
+		  struct sw_buf_free *free_buf,
+		  uint32_t *free_count,
+		  uint32_t pkt_id,
 		  uint16_t bpid)
 {
-	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
+	struct rte_mbuf *cur_seg = mbuf, *mi, *temp;
 	struct qbman_sge *sgt, *sge = NULL;
 	int i, offset = 0;
 
@@ -486,10 +489,11 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 #endif
 				}
 			}
-			cur_seg = cur_seg->next;
 		} else if (RTE_MBUF_HAS_EXTBUF(cur_seg)) {
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 			DPAA2_SET_FLE_IVP(sge);
-			cur_seg = cur_seg->next;
 		} else {
 			/* Get owner MBUF from indirect buffer */
 			mi = rte_mbuf_from_indirect(cur_seg);
@@ -503,11 +507,11 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 						   mempool_to_bpid(mi->pool));
 				rte_mbuf_refcnt_update(mi, 1);
 			}
-			prev_seg = cur_seg;
-			cur_seg = cur_seg->next;
-			prev_seg->next = NULL;
-			rte_pktmbuf_free(prev_seg);
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 		}
+		cur_seg = cur_seg->next;
 	}
 	DPAA2_SG_SET_FINAL(sge, true);
 	return 0;
@@ -515,11 +519,19 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 
 static void
 eth_mbuf_to_fd(struct rte_mbuf *mbuf,
-	       struct qbman_fd *fd, uint16_t bpid) __rte_unused;
+	       struct qbman_fd *fd,
+	       struct sw_buf_free *buf_to_free,
+	       uint32_t *free_count,
+	       uint32_t pkt_id,
+	       uint16_t bpid) __rte_unused;
 
 static void __rte_noinline __rte_hot
 eth_mbuf_to_fd(struct rte_mbuf *mbuf,
-	       struct qbman_fd *fd, uint16_t bpid)
+	       struct qbman_fd *fd,
+	       struct sw_buf_free *buf_to_free,
+	       uint32_t *free_count,
+	       uint32_t pkt_id,
+	       uint16_t bpid)
 {
 	DPAA2_MBUF_TO_CONTIG_FD(mbuf, fd, bpid);
 
@@ -540,6 +552,9 @@ eth_mbuf_to_fd(struct rte_mbuf *mbuf,
 				(void **)&mbuf, 1, 0);
 #endif
 	} else if (RTE_MBUF_HAS_EXTBUF(mbuf)) {
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 		DPAA2_SET_FD_IVP(fd);
 	} else {
 		struct rte_mbuf *mi;
@@ -549,7 +564,10 @@ eth_mbuf_to_fd(struct rte_mbuf *mbuf,
 			DPAA2_SET_FD_IVP(fd);
 		else
 			rte_mbuf_refcnt_update(mi, 1);
-		rte_pktmbuf_free(mbuf);
+
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 	}
 }
 
@@ -1226,7 +1244,8 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
 	struct dpaa2_dev_priv *priv = eth_data->dev_private;
 	uint32_t flags[MAX_TX_RING_SLOTS] = {0};
-	struct rte_mbuf **orig_bufs = bufs;
+	struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
 		ret = dpaa2_affine_qbman_swp();
@@ -1324,11 +1343,17 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 					mp = (*bufs)->pool;
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
+							      buf_to_free,
+							      &free_count,
+							      loop,
 							      mempool_to_bpid(mp)))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
-						       &fd_arr[loop], 0);
+							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop, 0);
 				}
 				bufs++;
 #ifdef RTE_LIBRTE_IEEE1588
@@ -1373,11 +1398,17 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop,
 							bpid))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
-						       &fd_arr[loop], bpid);
+							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop, bpid);
 				}
 			}
 #ifdef RTE_LIBRTE_IEEE1588
@@ -1410,12 +1441,9 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	}
 	dpaa2_q->tx_pkts += num_tx;
 
-	loop = 0;
-	while (loop < num_tx) {
-		if (unlikely(RTE_MBUF_HAS_EXTBUF(*orig_bufs)))
-			rte_pktmbuf_free(*orig_bufs);
-		orig_bufs++;
-		loop++;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
 	}
 
 	return num_tx;
@@ -1445,12 +1473,9 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 skip_tx:
 	dpaa2_q->tx_pkts += num_tx;
 
-	loop = 0;
-	while (loop < num_tx) {
-		if (unlikely(RTE_MBUF_HAS_EXTBUF(*orig_bufs)))
-			rte_pktmbuf_free(*orig_bufs);
-		orig_bufs++;
-		loop++;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
 	}
 
 	return num_tx;
@@ -1523,7 +1548,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 		struct rte_mbuf **bufs, uint16_t nb_pkts)
 {
 	/* Function to transmit the frames to multiple queues respectively.*/
-	uint32_t loop, retry_count;
+	uint32_t loop, i, retry_count;
 	int32_t ret;
 	struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
 	uint32_t frames_to_send, num_free_eq_desc = 0;
@@ -1536,6 +1561,8 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 	struct rte_eth_dev_data *eth_data;
 	struct dpaa2_dev_priv *priv;
 	struct dpaa2_queue *order_sendq;
+	struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
 		ret = dpaa2_affine_qbman_swp();
@@ -1647,11 +1674,17 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 			if (unlikely((*bufs)->nb_segs > 1)) {
 				if (eth_mbuf_to_sg_fd(*bufs,
 						      &fd_arr[loop],
+						      buf_to_free,
+						      &free_count,
+						      loop,
 						      bpid))
 					goto send_frames;
 			} else {
 				eth_mbuf_to_fd(*bufs,
-					       &fd_arr[loop], bpid);
+						&fd_arr[loop],
+						buf_to_free,
+						&free_count,
+						loop, bpid);
 			}
 		}
 
@@ -1676,6 +1709,10 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 		}
 	}
 
+	for (i = 0; i < free_count; i++) {
+		if (buf_to_free[i].pkt_id < loop)
+			rte_pktmbuf_free_seg(buf_to_free[i].seg);
+	}
 	return loop;
 }
 
@@ -1698,6 +1735,8 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	int32_t ret;
 	uint16_t num_tx = 0;
 	uint16_t bpid;
+	struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
 		ret = dpaa2_affine_qbman_swp();
@@ -1810,11 +1849,17 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
+							      buf_to_free,
+							      &free_count,
+							      loop,
 							      bpid))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
-						       &fd_arr[loop], bpid);
+							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop, bpid);
 				}
 			}
 			bufs++;
@@ -1843,6 +1888,11 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 		nb_pkts -= loop;
 	}
 	dpaa2_q->tx_pkts += num_tx;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
+	}
+
 	return num_tx;
 
 send_n_return:
@@ -1867,6 +1917,11 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	}
 skip_tx:
 	dpaa2_q->tx_pkts += num_tx;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
+	}
+
 	return num_tx;
 }
 
-- 
2.25.1

Re: [PATCH 08/15] net/dpaa2: fix buffer free on transmit SG packets

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> When using SG list to TX with external and direct buffers,
> HW free the direct buffers and driver free the external buffers.
> 
> Software scans the complete SG mbuf list to find the external
> buffers to free, but this is wrong as hardware can free the
> direct buffers if any present in the list and same can be
> re-allocated for other purpose in multi thread or high spead

s/spead/speed/

> running traffic environment with new data in it. So the software
> which is scanning the SG mbuf list, if that list has any direct
> buffer present then that direct buffer's next pointor can give

s/pointor/pointer/

> wrong pointer value, if already freed by hardware which
> can do the mempool corruption or memory leak.
> 
> In this patch instead of relying on user given SG mbuf list
> we are storing the buffers in an internal list which will
> be scanned by driver after transmit to free non-direct
> buffers.
> 
> This patch also fixes 2 more memory leak issues.
> 
> Driver is freeing complete SG list by checking external buffer
> flag in first segment only, but external buffer can be attached
> to any of the segment. Because of which driver either can double
> free buffers or there can be memory leak.
> 
> In case of indirect buffers, driver is modifying the original
> buffer list to free the indirect buffers but this orginal buffer

s/orginal/original/

same fixes needed for dpaa version of this patch, 14/15.

> list is being used even after transmit packets for software
> buffer cleanup. This can cause the buffer leak issue.
> 
> Fixes: 6bfbafe18d15 ("net/dpaa2: support external buffers in Tx")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>

<...>

[PATCH 09/15] bus/fslmc: add timeout in MC send command API

[Gagandeep Singh]

From: Rohit Raj <rohit.raj@nxp.com>

Adding one second timeout in MC send command API to ensure it doesn't
gets stuck in case of failure.

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
---
 drivers/bus/fslmc/mc/mc_sys.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/drivers/bus/fslmc/mc/mc_sys.c b/drivers/bus/fslmc/mc/mc_sys.c
index efafdc3101..ab9a074835 100644
--- a/drivers/bus/fslmc/mc/mc_sys.c
+++ b/drivers/bus/fslmc/mc/mc_sys.c
@@ -1,13 +1,14 @@
 /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
  *
  * Copyright 2013-2015 Freescale Semiconductor Inc.
- * Copyright 2017 NXP
+ * Copyright 2017,2022 NXP
  *
  */
 #include <fsl_mc_sys.h>
 #include <fsl_mc_cmd.h>
 
 #include <rte_spinlock.h>
+#include <rte_cycles.h>
 
 /** User space framework uses MC Portal in shared mode. Following change
  * introduces lock in MC FLIB
@@ -54,7 +55,7 @@ static int mc_status_to_error(enum mc_cmd_status status)
 int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	enum mc_cmd_status status;
-	uint64_t response;
+	uint64_t response, start_time, total_time, time_to_wait;
 
 	if (!mc_io || !mc_io->regs)
 		return -EACCES;
@@ -64,15 +65,20 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 
 	mc_write_command(mc_io->regs, cmd);
 
+	/* Wait for one second. rte_get_timer_hz() returns frequency of CPU */
+	time_to_wait = rte_get_timer_hz();
+	total_time = 0;
+	start_time = rte_get_timer_cycles();
+
 	/* Spin until status changes */
 	do {
 		response = ioread64(mc_io->regs);
 		status = mc_cmd_read_status((struct mc_command *)&response);
+		total_time = rte_get_timer_cycles() - start_time;
+	} while (status == MC_CMD_STATUS_READY && total_time <= time_to_wait);
 
-		/* --- Call wait function here to prevent blocking ---
-		 * Change the loop condition accordingly to exit on timeout.
-		 */
-	} while (status == MC_CMD_STATUS_READY);
+	if (status == MC_CMD_STATUS_READY)
+		return mc_status_to_error(MC_CMD_STATUS_TIMEOUT);
 
 	/* Read the response back into the command buffer */
 	mc_read_response(mc_io->regs, cmd);
-- 
2.25.1

[PATCH 10/15] net/dpaa: fix Jumbo packet Rx in case of VSP

[Gagandeep Singh]

From: Rohit Raj <rohit.raj@nxp.com>

For packet length of size more than 2K bytes, segmented packets were
being received in DPDK even if mbuf size was greater than packet
length. This is due to the configuration in VSP.

This patch fixes the issue by configurating the VSP according to the
mbuf size configured during mempool configuration.

Fixes: e4abd4ff183c ("net/dpaa: support virtual storage profile")
Cc: stable@dpdk.org

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.c |  5 ++---
 drivers/net/dpaa/dpaa_flow.c   | 13 ++++++-------
 drivers/net/dpaa/dpaa_flow.h   |  5 +++--
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index abcb1bc9ec..9b281ba8cb 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -988,8 +988,7 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
 	} else {
 		DPAA_PMD_WARN("The requested maximum Rx packet size (%u) is"
 		     " larger than a single mbuf (%u) and scattered"
-		     " mode has not been requested",
-		     max_rx_pktlen, buffsz - RTE_PKTMBUF_HEADROOM);
+		     " mode has not been requested", max_rx_pktlen, buffsz);
 	}
 
 	dpaa_intf->bp_info = DPAA_MEMPOOL_TO_POOL_INFO(mp);
@@ -1004,7 +1003,7 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
 		if (vsp_id >= 0) {
 			ret = dpaa_port_vsp_update(dpaa_intf, fmc_q, vsp_id,
 					DPAA_MEMPOOL_TO_POOL_INFO(mp)->bpid,
-					fif);
+					fif, buffsz + RTE_PKTMBUF_HEADROOM);
 			if (ret) {
 				DPAA_PMD_ERR("dpaa_port_vsp_update failed");
 				return ret;
diff --git a/drivers/net/dpaa/dpaa_flow.c b/drivers/net/dpaa/dpaa_flow.c
index 1ccd036027..690ba6bcb3 100644
--- a/drivers/net/dpaa/dpaa_flow.c
+++ b/drivers/net/dpaa/dpaa_flow.c
@@ -939,7 +939,7 @@ int dpaa_fm_term(void)
 
 static int dpaa_port_vsp_configure(struct dpaa_if *dpaa_intf,
 		uint8_t vsp_id, t_handle fman_handle,
-		struct fman_if *fif)
+		struct fman_if *fif, u32 mbuf_data_room_size)
 {
 	t_fm_vsp_params vsp_params;
 	t_fm_buffer_prefix_content buf_prefix_cont;
@@ -976,10 +976,8 @@ static int dpaa_port_vsp_configure(struct dpaa_if *dpaa_intf,
 		return -1;
 	}
 	vsp_params.ext_buf_pools.num_of_pools_used = 1;
-	vsp_params.ext_buf_pools.ext_buf_pool[0].id =
-		dpaa_intf->vsp_bpid[vsp_id];
-	vsp_params.ext_buf_pools.ext_buf_pool[0].size =
-		RTE_MBUF_DEFAULT_BUF_SIZE;
+	vsp_params.ext_buf_pools.ext_buf_pool[0].id = dpaa_intf->vsp_bpid[vsp_id];
+	vsp_params.ext_buf_pools.ext_buf_pool[0].size = mbuf_data_room_size;
 
 	dpaa_intf->vsp_handle[vsp_id] = fm_vsp_config(&vsp_params);
 	if (!dpaa_intf->vsp_handle[vsp_id]) {
@@ -1023,7 +1021,7 @@ static int dpaa_port_vsp_configure(struct dpaa_if *dpaa_intf,
 
 int dpaa_port_vsp_update(struct dpaa_if *dpaa_intf,
 		bool fmc_mode, uint8_t vsp_id, uint32_t bpid,
-		struct fman_if *fif)
+		struct fman_if *fif, u32 mbuf_data_room_size)
 {
 	int ret = 0;
 	t_handle fman_handle;
@@ -1054,7 +1052,8 @@ int dpaa_port_vsp_update(struct dpaa_if *dpaa_intf,
 
 	dpaa_intf->vsp_bpid[vsp_id] = bpid;
 
-	return dpaa_port_vsp_configure(dpaa_intf, vsp_id, fman_handle, fif);
+	return dpaa_port_vsp_configure(dpaa_intf, vsp_id, fman_handle, fif,
+				       mbuf_data_room_size);
 }
 
 int dpaa_port_vsp_cleanup(struct dpaa_if *dpaa_intf, struct fman_if *fif)
diff --git a/drivers/net/dpaa/dpaa_flow.h b/drivers/net/dpaa/dpaa_flow.h
index f5e131acfa..4742b8dd0a 100644
--- a/drivers/net/dpaa/dpaa_flow.h
+++ b/drivers/net/dpaa/dpaa_flow.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2017,2019 NXP
+ * Copyright 2017,2019,2022 NXP
  */
 
 #ifndef __DPAA_FLOW_H__
@@ -11,7 +11,8 @@ int dpaa_fm_config(struct rte_eth_dev *dev, uint64_t req_dist_set);
 int dpaa_fm_deconfig(struct dpaa_if *dpaa_intf, struct fman_if *fif);
 void dpaa_write_fm_config_to_file(void);
 int dpaa_port_vsp_update(struct dpaa_if *dpaa_intf,
-	bool fmc_mode, uint8_t vsp_id, uint32_t bpid, struct fman_if *fif);
+	bool fmc_mode, uint8_t vsp_id, uint32_t bpid, struct fman_if *fif,
+	u32 mbuf_data_room_size);
 int dpaa_port_vsp_cleanup(struct dpaa_if *dpaa_intf, struct fman_if *fif);
 int dpaa_port_fmc_init(struct fman_if *fif,
 		       uint32_t *fqids, int8_t *vspids, int max_nb_rxq);
-- 
2.25.1

[PATCH 11/15] bus/dpaa: pass interface name as a string instead of pointer

[Gagandeep Singh]

From: Rohit Raj <rohit.raj@nxp.com>

Due to change in latest kernel, passing the interface name to
kernel through IOCTL as string instead of character pointer.

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
---
 drivers/bus/dpaa/base/qbman/process.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/bus/dpaa/base/qbman/process.c b/drivers/bus/dpaa/base/qbman/process.c
index 9bc92681cd..3504ec97db 100644
--- a/drivers/bus/dpaa/base/qbman/process.c
+++ b/drivers/bus/dpaa/base/qbman/process.c
@@ -302,7 +302,7 @@ int bman_free_raw_portal(struct dpaa_raw_portal *portal)
 	_IOW(DPAA_IOCTL_MAGIC, 0x0E, struct usdpaa_ioctl_link_status)
 
 #define DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT \
-	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char*)
+	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char[IF_NAME_MAX_LEN])
 
 int dpaa_intr_enable(char *if_name, int efd)
 {
@@ -330,7 +330,7 @@ int dpaa_intr_disable(char *if_name)
 	if (ret)
 		return ret;
 
-	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT, &if_name);
+	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT, if_name);
 	if (ret) {
 		if (errno == EINVAL)
 			printf("Failed to disable interrupt: Not Supported\n");
@@ -472,7 +472,7 @@ int dpaa_update_link_speed(char *if_name, int link_speed, int link_duplex)
 }
 
 #define DPAA_IOCTL_RESTART_LINK_AUTONEG \
-	_IOW(DPAA_IOCTL_MAGIC, 0x13, char *)
+	_IOW(DPAA_IOCTL_MAGIC, 0x13, char[IF_NAME_MAX_LEN])
 
 int dpaa_restart_link_autoneg(char *if_name)
 {
@@ -481,7 +481,7 @@ int dpaa_restart_link_autoneg(char *if_name)
 	if (ret)
 		return ret;
 
-	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, &if_name);
+	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, if_name);
 	if (ret) {
 		if (errno == EINVAL)
 			printf("Failed to restart autoneg: Not Supported\n");
-- 
2.25.1

Re: [PATCH 11/15] bus/dpaa: pass interface name as a string instead of pointer

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> From: Rohit Raj <rohit.raj@nxp.com>
> 
> Due to change in latest kernel, passing the interface name to
> kernel through IOCTL as string instead of character pointer.
> 

This kernel component is the one that is delivered part of SDK I assume, 
instead of an upstreamed one.

What is the way for user match kernel code and DPDK driver, like is 
there any matching version information documented?

> Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
> ---
>   drivers/bus/dpaa/base/qbman/process.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/bus/dpaa/base/qbman/process.c b/drivers/bus/dpaa/base/qbman/process.c
> index 9bc92681cd..3504ec97db 100644
> --- a/drivers/bus/dpaa/base/qbman/process.c
> +++ b/drivers/bus/dpaa/base/qbman/process.c
> @@ -302,7 +302,7 @@ int bman_free_raw_portal(struct dpaa_raw_portal *portal)
>   	_IOW(DPAA_IOCTL_MAGIC, 0x0E, struct usdpaa_ioctl_link_status)
>   
>   #define DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT \
> -	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char*)
> +	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char[IF_NAME_MAX_LEN])
>   
>   int dpaa_intr_enable(char *if_name, int efd)
>   {
> @@ -330,7 +330,7 @@ int dpaa_intr_disable(char *if_name)
>   	if (ret)
>   		return ret;
>   
> -	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT, &if_name);
> +	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT, if_name);
>   	if (ret) {
>   		if (errno == EINVAL)
>   			printf("Failed to disable interrupt: Not Supported\n");
> @@ -472,7 +472,7 @@ int dpaa_update_link_speed(char *if_name, int link_speed, int link_duplex)
>   }
>   
>   #define DPAA_IOCTL_RESTART_LINK_AUTONEG \
> -	_IOW(DPAA_IOCTL_MAGIC, 0x13, char *)
> +	_IOW(DPAA_IOCTL_MAGIC, 0x13, char[IF_NAME_MAX_LEN])
>   
>   int dpaa_restart_link_autoneg(char *if_name)
>   {
> @@ -481,7 +481,7 @@ int dpaa_restart_link_autoneg(char *if_name)
>   	if (ret)
>   		return ret;
>   
> -	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, &if_name);
> +	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, if_name);
>   	if (ret) {
>   		if (errno == EINVAL)
>   			printf("Failed to restart autoneg: Not Supported\n");

RE: [PATCH 11/15] bus/dpaa: pass interface name as a string instead of pointer

[Gagandeep Singh]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Wednesday, October 5, 2022 7:52 PM
> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> Cc: Rohit Raj <rohit.raj@nxp.com>
> Subject: Re: [PATCH 11/15] bus/dpaa: pass interface name as a string instead
> of pointer
> 
> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> > From: Rohit Raj <rohit.raj@nxp.com>
> >
> > Due to change in latest kernel, passing the interface name to kernel
> > through IOCTL as string instead of character pointer.
> >
> 
> This kernel component is the one that is delivered part of SDK I assume,
> instead of an upstreamed one.
> 
> What is the way for user match kernel code and DPDK driver, like is there any
> matching version information documented?
> 
We share the version compatible information in our internal SDK release document.

> > Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
> > ---
> >   drivers/bus/dpaa/base/qbman/process.c | 8 ++++----
> >   1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/bus/dpaa/base/qbman/process.c
> > b/drivers/bus/dpaa/base/qbman/process.c
> > index 9bc92681cd..3504ec97db 100644
> > --- a/drivers/bus/dpaa/base/qbman/process.c
> > +++ b/drivers/bus/dpaa/base/qbman/process.c
> > @@ -302,7 +302,7 @@ int bman_free_raw_portal(struct dpaa_raw_portal
> *portal)
> >   	_IOW(DPAA_IOCTL_MAGIC, 0x0E, struct usdpaa_ioctl_link_status)
> >
> >   #define DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT \
> > -	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char*)
> > +	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char[IF_NAME_MAX_LEN])
> >
> >   int dpaa_intr_enable(char *if_name, int efd)
> >   {
> > @@ -330,7 +330,7 @@ int dpaa_intr_disable(char *if_name)
> >   	if (ret)
> >   		return ret;
> >
> > -	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT,
> &if_name);
> > +	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT,
> if_name);
> >   	if (ret) {
> >   		if (errno == EINVAL)
> >   			printf("Failed to disable interrupt: Not Supported\n");
> @@ -472,7
> > +472,7 @@ int dpaa_update_link_speed(char *if_name, int link_speed, int
> link_duplex)
> >   }
> >
> >   #define DPAA_IOCTL_RESTART_LINK_AUTONEG \
> > -	_IOW(DPAA_IOCTL_MAGIC, 0x13, char *)
> > +	_IOW(DPAA_IOCTL_MAGIC, 0x13, char[IF_NAME_MAX_LEN])
> >
> >   int dpaa_restart_link_autoneg(char *if_name)
> >   {
> > @@ -481,7 +481,7 @@ int dpaa_restart_link_autoneg(char *if_name)
> >   	if (ret)
> >   		return ret;
> >
> > -	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, &if_name);
> > +	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, if_name);
> >   	if (ret) {
> >   		if (errno == EINVAL)
> >   			printf("Failed to restart autoneg: Not Supported\n");

Re: [PATCH 11/15] bus/dpaa: pass interface name as a string instead of pointer

[Ferruh Yigit]

On 10/6/2022 9:51 AM, Gagandeep Singh wrote:
> 
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>> Sent: Wednesday, October 5, 2022 7:52 PM
>> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
>> Cc: Rohit Raj <rohit.raj@nxp.com>
>> Subject: Re: [PATCH 11/15] bus/dpaa: pass interface name as a string instead
>> of pointer
>>
>> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
>>> From: Rohit Raj <rohit.raj@nxp.com>
>>>
>>> Due to change in latest kernel, passing the interface name to kernel
>>> through IOCTL as string instead of character pointer.
>>>
>>
>> This kernel component is the one that is delivered part of SDK I assume,
>> instead of an upstreamed one.
>>
>> What is the way for user match kernel code and DPDK driver, like is there any
>> matching version information documented?
>>
> We share the version compatible information in our internal SDK release document.
> 

That is what I thought, does it make sense to link to that information 
from DPDK documentation?

>>> Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
>>> ---
>>>    drivers/bus/dpaa/base/qbman/process.c | 8 ++++----
>>>    1 file changed, 4 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/bus/dpaa/base/qbman/process.c
>>> b/drivers/bus/dpaa/base/qbman/process.c
>>> index 9bc92681cd..3504ec97db 100644
>>> --- a/drivers/bus/dpaa/base/qbman/process.c
>>> +++ b/drivers/bus/dpaa/base/qbman/process.c
>>> @@ -302,7 +302,7 @@ int bman_free_raw_portal(struct dpaa_raw_portal
>> *portal)
>>>    	_IOW(DPAA_IOCTL_MAGIC, 0x0E, struct usdpaa_ioctl_link_status)
>>>
>>>    #define DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT \
>>> -	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char*)
>>> +	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char[IF_NAME_MAX_LEN])
>>>
>>>    int dpaa_intr_enable(char *if_name, int efd)
>>>    {
>>> @@ -330,7 +330,7 @@ int dpaa_intr_disable(char *if_name)
>>>    	if (ret)
>>>    		return ret;
>>>
>>> -	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT,
>> &if_name);
>>> +	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT,
>> if_name);
>>>    	if (ret) {
>>>    		if (errno == EINVAL)
>>>    			printf("Failed to disable interrupt: Not Supported\n");
>> @@ -472,7
>>> +472,7 @@ int dpaa_update_link_speed(char *if_name, int link_speed, int
>> link_duplex)
>>>    }
>>>
>>>    #define DPAA_IOCTL_RESTART_LINK_AUTONEG \
>>> -	_IOW(DPAA_IOCTL_MAGIC, 0x13, char *)
>>> +	_IOW(DPAA_IOCTL_MAGIC, 0x13, char[IF_NAME_MAX_LEN])
>>>
>>>    int dpaa_restart_link_autoneg(char *if_name)
>>>    {
>>> @@ -481,7 +481,7 @@ int dpaa_restart_link_autoneg(char *if_name)
>>>    	if (ret)
>>>    		return ret;
>>>
>>> -	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, &if_name);
>>> +	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, if_name);
>>>    	if (ret) {
>>>    		if (errno == EINVAL)
>>>    			printf("Failed to restart autoneg: Not Supported\n");
> 

RE: [PATCH 11/15] bus/dpaa: pass interface name as a string instead of pointer

[Gagandeep Singh]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Thursday, October 6, 2022 3:10 PM
> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> Cc: Rohit Raj <rohit.raj@nxp.com>
> Subject: Re: [PATCH 11/15] bus/dpaa: pass interface name as a string instead
> of pointer
> 
> On 10/6/2022 9:51 AM, Gagandeep Singh wrote:
> >
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@amd.com>
> >> Sent: Wednesday, October 5, 2022 7:52 PM
> >> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> >> Cc: Rohit Raj <rohit.raj@nxp.com>
> >> Subject: Re: [PATCH 11/15] bus/dpaa: pass interface name as a string
> >> instead of pointer
> >>
> >> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> >>> From: Rohit Raj <rohit.raj@nxp.com>
> >>>
> >>> Due to change in latest kernel, passing the interface name to kernel
> >>> through IOCTL as string instead of character pointer.
> >>>
> >>
> >> This kernel component is the one that is delivered part of SDK I
> >> assume, instead of an upstreamed one.
> >>
> >> What is the way for user match kernel code and DPDK driver, like is
> >> there any matching version information documented?
> >>
> > We share the version compatible information in our internal SDK release
> document.
> >
> 
> That is what I thought, does it make sense to link to that information from
> DPDK documentation?
> 
Sure, I will update the DPAA NIC guide for this.


> >>> Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
> >>> ---
> >>>    drivers/bus/dpaa/base/qbman/process.c | 8 ++++----
> >>>    1 file changed, 4 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/drivers/bus/dpaa/base/qbman/process.c
> >>> b/drivers/bus/dpaa/base/qbman/process.c
> >>> index 9bc92681cd..3504ec97db 100644
> >>> --- a/drivers/bus/dpaa/base/qbman/process.c
> >>> +++ b/drivers/bus/dpaa/base/qbman/process.c
> >>> @@ -302,7 +302,7 @@ int bman_free_raw_portal(struct
> dpaa_raw_portal
> >> *portal)
> >>>    	_IOW(DPAA_IOCTL_MAGIC, 0x0E, struct usdpaa_ioctl_link_status)
> >>>
> >>>    #define DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT \
> >>> -	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char*)
> >>> +	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char[IF_NAME_MAX_LEN])
> >>>
> >>>    int dpaa_intr_enable(char *if_name, int efd)
> >>>    {
> >>> @@ -330,7 +330,7 @@ int dpaa_intr_disable(char *if_name)
> >>>    	if (ret)
> >>>    		return ret;
> >>>
> >>> -	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT,
> >> &if_name);
> >>> +	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT,
> >> if_name);
> >>>    	if (ret) {
> >>>    		if (errno == EINVAL)
> >>>    			printf("Failed to disable interrupt: Not Supported\n");
> >> @@ -472,7
> >>> +472,7 @@ int dpaa_update_link_speed(char *if_name, int link_speed,
> >>> +int
> >> link_duplex)
> >>>    }
> >>>
> >>>    #define DPAA_IOCTL_RESTART_LINK_AUTONEG \
> >>> -	_IOW(DPAA_IOCTL_MAGIC, 0x13, char *)
> >>> +	_IOW(DPAA_IOCTL_MAGIC, 0x13, char[IF_NAME_MAX_LEN])
> >>>
> >>>    int dpaa_restart_link_autoneg(char *if_name)
> >>>    {
> >>> @@ -481,7 +481,7 @@ int dpaa_restart_link_autoneg(char *if_name)
> >>>    	if (ret)
> >>>    		return ret;
> >>>
> >>> -	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, &if_name);
> >>> +	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, if_name);
> >>>    	if (ret) {
> >>>    		if (errno == EINVAL)
> >>>    			printf("Failed to restart autoneg: Not Supported\n");
> >

[PATCH 12/15] net/dpaa: use internal mempool for SG table

[Gagandeep Singh]

Creating and using driver's mempool for
allocating the SG table memory required for
FD creation.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.c | 13 +++++++++++++
 drivers/net/dpaa/dpaa_ethdev.h |  9 +++++++++
 drivers/net/dpaa/dpaa_rxtx.c   |  9 ++++-----
 3 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 9b281ba8cb..1e804e1462 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -133,6 +133,7 @@ static const struct rte_dpaa_xstats_name_off dpaa_xstats_strings[] = {
 };
 
 static struct rte_dpaa_driver rte_dpaa_pmd;
+struct rte_mempool *dpaa_tx_sg_pool;
 
 static int
 dpaa_eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info);
@@ -2222,6 +2223,18 @@ rte_dpaa_probe(struct rte_dpaa_driver *dpaa_drv,
 	/* Invoke PMD device initialization function */
 	diag = dpaa_dev_init(eth_dev);
 	if (diag == 0) {
+		if (!dpaa_tx_sg_pool) {
+			dpaa_tx_sg_pool =
+				rte_pktmbuf_pool_create("dpaa_mbuf_tx_sg_pool",
+				DPAA_POOL_SIZE,
+				DPAA_POOL_CACHE_SIZE, 0,
+				DPAA_MAX_SGS * sizeof(struct qm_sg_entry),
+				rte_socket_id());
+			if (dpaa_tx_sg_pool == NULL) {
+				DPAA_PMD_ERR("SG pool creation failed\n");
+				return -ENOMEM;
+			}
+		}
 		rte_eth_dev_probing_finish(eth_dev);
 		return 0;
 	}
diff --git a/drivers/net/dpaa/dpaa_ethdev.h b/drivers/net/dpaa/dpaa_ethdev.h
index 6fdd57dbc3..f9c0554530 100644
--- a/drivers/net/dpaa/dpaa_ethdev.h
+++ b/drivers/net/dpaa/dpaa_ethdev.h
@@ -33,6 +33,13 @@
 
 #define DPAA_SGT_MAX_ENTRIES 16 /* maximum number of entries in SG Table */
 
+/* Maximum SG segments supported on all cores*/
+#define DPAA_MAX_SGS 128
+/* SG pool size */
+#define DPAA_POOL_SIZE 2048
+/* SG pool cache size */
+#define DPAA_POOL_CACHE_SIZE 256
+
 /* RX queue tail drop threshold (CGR Based) in frame count */
 #define CGR_RX_PERFQ_THRESH 256
 #define CGR_TX_CGR_THRESH 512
@@ -103,6 +110,8 @@
 
 #define FMC_FILE "/tmp/fmc.bin"
 
+extern struct rte_mempool *dpaa_tx_sg_pool;
+
 /* Each network interface is represented by one of these */
 struct dpaa_if {
 	int valid;
diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index 22205cec30..e23206bf5c 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -803,8 +803,7 @@ uint16_t dpaa_eth_queue_rx(void *q,
 
 static int
 dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
-		struct qm_fd *fd,
-		struct dpaa_bp_info *bp_info)
+		struct qm_fd *fd)
 {
 	struct rte_mbuf *cur_seg = mbuf, *prev_seg = NULL;
 	struct rte_mbuf *temp, *mi;
@@ -813,7 +812,7 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 
 	DPAA_DP_LOG(DEBUG, "Creating SG FD to transmit");
 
-	temp = rte_pktmbuf_alloc(bp_info->mp);
+	temp = rte_pktmbuf_alloc(dpaa_tx_sg_pool);
 	if (!temp) {
 		DPAA_PMD_ERR("Failure in allocation of mbuf");
 		return -1;
@@ -849,7 +848,7 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 	fd->format = QM_FD_SG;
 	fd->addr = temp->buf_iova;
 	fd->offset = temp->data_off;
-	fd->bpid = bp_info ? bp_info->bpid : 0xff;
+	fd->bpid = DPAA_MEMPOOL_TO_BPID(dpaa_tx_sg_pool);
 	fd->length20 = mbuf->pkt_len;
 
 	while (i < DPAA_SGT_MAX_ENTRIES) {
@@ -967,7 +966,7 @@ tx_on_dpaa_pool(struct rte_mbuf *mbuf,
 		tx_on_dpaa_pool_unsegmented(mbuf, bp_info, fd_arr);
 	} else if (mbuf->nb_segs > 1 &&
 		   mbuf->nb_segs <= DPAA_SGT_MAX_ENTRIES) {
-		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr, bp_info)) {
+		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr)) {
 			DPAA_PMD_DEBUG("Unable to create Scatter Gather FD");
 			return 1;
 		}
-- 
2.25.1

[PATCH 13/15] bus/dpaa: mempool ops registration change

[Gagandeep Singh]

moving the mempool ops registration before DPAA
devices probe so that device probe functions can
also be able to use mempool operations.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/bus/dpaa/dpaa_bus.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c
index 682427ba2c..3ca5147761 100644
--- a/drivers/bus/dpaa/dpaa_bus.c
+++ b/drivers/bus/dpaa/dpaa_bus.c
@@ -655,6 +655,11 @@ rte_dpaa_bus_probe(void)
 	if (TAILQ_EMPTY(&rte_dpaa_bus.device_list))
 		return 0;
 
+	/* Register DPAA mempool ops only if any DPAA device has
+	 * been detected.
+	 */
+	rte_mbuf_set_platform_mempool_ops(DPAA_MEMPOOL_OPS_NAME);
+
 	svr_file = fopen(DPAA_SOC_ID_FILE, "r");
 	if (svr_file) {
 		if (fscanf(svr_file, "svr:%x", &svr_ver) > 0)
@@ -704,11 +709,6 @@ rte_dpaa_bus_probe(void)
 		}
 	}
 
-	/* Register DPAA mempool ops only if any DPAA device has
-	 * been detected.
-	 */
-	rte_mbuf_set_platform_mempool_ops(DPAA_MEMPOOL_OPS_NAME);
-
 	return 0;
 }
 
-- 
2.25.1

[PATCH 14/15] net/dpaa: fix buffer free on transmit SG packets

[Gagandeep Singh]

When using SG list to TX with external and direct buffers,
HW free direct buffers and driver free external buffers.

Software scans the complete SG mbuf list to find the external
buffers to free, but this is wrong as hardware can free the
direct buffers if any present in the list and same can be
re-allocated for other purpose in multi thread or high spead
running traffic environment with new data in it. So the software
which is scanning the SG mbuf list, if that list has any direct
buffer present then that direct buffer's next pointor can give
wrong pointer value, if already freed by hardware which
can do the mempool corruption or memory leak.

In this patch instead of relying on user given SG mbuf list
we are storing the buffers in an internal list which will
be scanned by driver after transmit to free non-direct
buffers.

This patch also fixes below issues.

Driver is freeing complete SG list by checking external buffer
flag in first segment only, but external buffer can be attached
to any of the segment. Because of this, driver either can double
free buffers or there can be memory leak.

In case of indirect buffers, driver is modifying the original
buffer list to free the indirect buffers but this orginal buffer
list is being used by driver even after transmit packets for
non-direct buffer cleanup. This can cause the buffer leak issue.

Fixes: f191d5abda54 ("net/dpaa: support external buffers in Tx")
Cc: stable@dpdk.org

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.h | 10 ++++++
 drivers/net/dpaa/dpaa_rxtx.c   | 61 ++++++++++++++++++++++------------
 2 files changed, 49 insertions(+), 22 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.h b/drivers/net/dpaa/dpaa_ethdev.h
index f9c0554530..502c1c88b8 100644
--- a/drivers/net/dpaa/dpaa_ethdev.h
+++ b/drivers/net/dpaa/dpaa_ethdev.h
@@ -112,6 +112,16 @@
 
 extern struct rte_mempool *dpaa_tx_sg_pool;
 
+/* structure to free external and indirect
+ * buffers.
+ */
+struct dpaa_sw_buf_free {
+	/* To which packet this segment belongs */
+	uint16_t pkt_id;
+	/* The actual segment */
+	struct rte_mbuf *seg;
+};
+
 /* Each network interface is represented by one of these */
 struct dpaa_if {
 	int valid;
diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index e23206bf5c..4d285b4f38 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -803,9 +803,12 @@ uint16_t dpaa_eth_queue_rx(void *q,
 
 static int
 dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
-		struct qm_fd *fd)
+		struct qm_fd *fd,
+		struct dpaa_sw_buf_free *free_buf,
+		uint32_t *free_count,
+		uint32_t pkt_id)
 {
-	struct rte_mbuf *cur_seg = mbuf, *prev_seg = NULL;
+	struct rte_mbuf *cur_seg = mbuf;
 	struct rte_mbuf *temp, *mi;
 	struct qm_sg_entry *sg_temp, *sgt;
 	int i = 0;
@@ -869,10 +872,11 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 				sg_temp->bpid =
 					DPAA_MEMPOOL_TO_BPID(cur_seg->pool);
 			}
-			cur_seg = cur_seg->next;
 		} else if (RTE_MBUF_HAS_EXTBUF(cur_seg)) {
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 			sg_temp->bpid = 0xff;
-			cur_seg = cur_seg->next;
 		} else {
 			/* Get owner MBUF from indirect buffer */
 			mi = rte_mbuf_from_indirect(cur_seg);
@@ -885,11 +889,11 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 				sg_temp->bpid = DPAA_MEMPOOL_TO_BPID(mi->pool);
 				rte_mbuf_refcnt_update(mi, 1);
 			}
-			prev_seg = cur_seg;
-			cur_seg = cur_seg->next;
-			prev_seg->next = NULL;
-			rte_pktmbuf_free(prev_seg);
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 		}
+		cur_seg = cur_seg->next;
 		if (cur_seg == NULL) {
 			sg_temp->final = 1;
 			cpu_to_hw_sg(sg_temp);
@@ -904,7 +908,10 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 static inline void
 tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 			    struct dpaa_bp_info *bp_info,
-			    struct qm_fd *fd_arr)
+			    struct qm_fd *fd_arr,
+			    struct dpaa_sw_buf_free *buf_to_free,
+			    uint32_t *free_count,
+			    uint32_t pkt_id)
 {
 	struct rte_mbuf *mi = NULL;
 
@@ -923,6 +930,9 @@ tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 			DPAA_MBUF_TO_CONTIG_FD(mbuf, fd_arr, bp_info->bpid);
 		}
 	} else if (RTE_MBUF_HAS_EXTBUF(mbuf)) {
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 		DPAA_MBUF_TO_CONTIG_FD(mbuf, fd_arr,
 				bp_info ? bp_info->bpid : 0xff);
 	} else {
@@ -946,7 +956,9 @@ tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 			DPAA_MBUF_TO_CONTIG_FD(mbuf, fd_arr,
 						bp_info ? bp_info->bpid : 0xff);
 		}
-		rte_pktmbuf_free(mbuf);
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 	}
 
 	if (mbuf->ol_flags & DPAA_TX_CKSUM_OFFLOAD_MASK)
@@ -957,16 +969,21 @@ tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 static inline uint16_t
 tx_on_dpaa_pool(struct rte_mbuf *mbuf,
 		struct dpaa_bp_info *bp_info,
-		struct qm_fd *fd_arr)
+		struct qm_fd *fd_arr,
+		struct dpaa_sw_buf_free *buf_to_free,
+		uint32_t *free_count,
+		uint32_t pkt_id)
 {
 	DPAA_DP_LOG(DEBUG, "BMAN offloaded buffer, mbuf: %p", mbuf);
 
 	if (mbuf->nb_segs == 1) {
 		/* Case for non-segmented buffers */
-		tx_on_dpaa_pool_unsegmented(mbuf, bp_info, fd_arr);
+		tx_on_dpaa_pool_unsegmented(mbuf, bp_info, fd_arr,
+				buf_to_free, free_count, pkt_id);
 	} else if (mbuf->nb_segs > 1 &&
 		   mbuf->nb_segs <= DPAA_SGT_MAX_ENTRIES) {
-		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr)) {
+		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr, buf_to_free,
+					   free_count, pkt_id)) {
 			DPAA_PMD_DEBUG("Unable to create Scatter Gather FD");
 			return 1;
 		}
@@ -1070,7 +1087,8 @@ dpaa_eth_queue_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 	uint16_t state;
 	int ret, realloc_mbuf = 0;
 	uint32_t seqn, index, flags[DPAA_TX_BURST_SIZE] = {0};
-	struct rte_mbuf **orig_bufs = bufs;
+	struct dpaa_sw_buf_free buf_to_free[DPAA_MAX_SGS * DPAA_MAX_DEQUEUE_NUM_FRAMES];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA_PER_LCORE_PORTAL)) {
 		ret = rte_dpaa_portal_init((void *)0);
@@ -1153,7 +1171,10 @@ dpaa_eth_queue_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 			}
 indirect_buf:
 			state = tx_on_dpaa_pool(mbuf, bp_info,
-						&fd_arr[loop]);
+						&fd_arr[loop],
+						buf_to_free,
+						&free_count,
+						loop);
 			if (unlikely(state)) {
 				/* Set frames_to_send & nb_bufs so
 				 * that packets are transmitted till
@@ -1178,13 +1199,9 @@ dpaa_eth_queue_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 
 	DPAA_DP_LOG(DEBUG, "Transmitted %d buffers on queue: %p", sent, q);
 
-
-	loop = 0;
-	while (loop < sent) {
-		if (unlikely(RTE_MBUF_HAS_EXTBUF(*orig_bufs)))
-			rte_pktmbuf_free(*orig_bufs);
-		orig_bufs++;
-		loop++;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < sent)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
 	}
 
 	return sent;
-- 
2.25.1

[PATCH 15/15] net/dpaa: fix buffer free in slow path

[Gagandeep Singh]

Adding a check in slow path to free those buffers
which are not external.

Fixes: 9124e65dd3eb ("net/dpaa: enable Tx queue taildrop")
Cc: stable@dpdk.org

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 drivers/net/dpaa/dpaa_rxtx.c | 23 ++++++++---------------
 1 file changed, 8 insertions(+), 15 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index 4d285b4f38..ce4f3d6c85 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -455,7 +455,7 @@ dpaa_free_mbuf(const struct qm_fd *fd)
 	bp_info = DPAA_BPID_TO_POOL_INFO(fd->bpid);
 	format = (fd->opaque & DPAA_FD_FORMAT_MASK) >> DPAA_FD_FORMAT_SHIFT;
 	if (unlikely(format == qm_fd_sg)) {
-		struct rte_mbuf *first_seg, *prev_seg, *cur_seg, *temp;
+		struct rte_mbuf *first_seg, *cur_seg;
 		struct qm_sg_entry *sgt, *sg_temp;
 		void *vaddr, *sg_vaddr;
 		int i = 0;
@@ -469,32 +469,25 @@ dpaa_free_mbuf(const struct qm_fd *fd)
 		sgt = vaddr + fd_offset;
 		sg_temp = &sgt[i++];
 		hw_sg_to_cpu(sg_temp);
-		temp = (struct rte_mbuf *)
-			((char *)vaddr - bp_info->meta_data_size);
 		sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
 						qm_sg_entry_get64(sg_temp));
-
 		first_seg = (struct rte_mbuf *)((char *)sg_vaddr -
 						bp_info->meta_data_size);
 		first_seg->nb_segs = 1;
-		prev_seg = first_seg;
 		while (i < DPAA_SGT_MAX_ENTRIES) {
 			sg_temp = &sgt[i++];
 			hw_sg_to_cpu(sg_temp);
-			sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
+			if (sg_temp->bpid != 0xFF) {
+				bp_info = DPAA_BPID_TO_POOL_INFO(sg_temp->bpid);
+				sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
 						qm_sg_entry_get64(sg_temp));
-			cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
+				cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
 						      bp_info->meta_data_size);
-			first_seg->nb_segs += 1;
-			prev_seg->next = cur_seg;
-			if (sg_temp->final) {
-				cur_seg->next = NULL;
-				break;
+				rte_pktmbuf_free_seg(cur_seg);
 			}
-			prev_seg = cur_seg;
+			if (sg_temp->final)
+				break;
 		}
-
-		rte_pktmbuf_free_seg(temp);
 		rte_pktmbuf_free_seg(first_seg);
 		return 0;
 	}
-- 
2.25.1

Re: [PATCH 15/15] net/dpaa: fix buffer free in slow path

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> Adding a check in slow path to free those buffers
> which are not external.
> 

Can you please explain what was the error before fix, what was happening 
when you try to free all mbufs?

Also it seems previous logic was different, with 'prev_seg' etc, can you 
explain what/why changed there?

> Fixes: 9124e65dd3eb ("net/dpaa: enable Tx queue taildrop")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> ---
>   drivers/net/dpaa/dpaa_rxtx.c | 23 ++++++++---------------
>   1 file changed, 8 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
> index 4d285b4f38..ce4f3d6c85 100644
> --- a/drivers/net/dpaa/dpaa_rxtx.c
> +++ b/drivers/net/dpaa/dpaa_rxtx.c
> @@ -455,7 +455,7 @@ dpaa_free_mbuf(const struct qm_fd *fd)
>   	bp_info = DPAA_BPID_TO_POOL_INFO(fd->bpid);
>   	format = (fd->opaque & DPAA_FD_FORMAT_MASK) >> DPAA_FD_FORMAT_SHIFT;
>   	if (unlikely(format == qm_fd_sg)) {
> -		struct rte_mbuf *first_seg, *prev_seg, *cur_seg, *temp;
> +		struct rte_mbuf *first_seg, *cur_seg;
>   		struct qm_sg_entry *sgt, *sg_temp;
>   		void *vaddr, *sg_vaddr;
>   		int i = 0;
> @@ -469,32 +469,25 @@ dpaa_free_mbuf(const struct qm_fd *fd)
>   		sgt = vaddr + fd_offset;
>   		sg_temp = &sgt[i++];
>   		hw_sg_to_cpu(sg_temp);
> -		temp = (struct rte_mbuf *)
> -			((char *)vaddr - bp_info->meta_data_size);
>   		sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
>   						qm_sg_entry_get64(sg_temp));
> -
>   		first_seg = (struct rte_mbuf *)((char *)sg_vaddr -
>   						bp_info->meta_data_size);
>   		first_seg->nb_segs = 1;
> -		prev_seg = first_seg;
>   		while (i < DPAA_SGT_MAX_ENTRIES) {
>   			sg_temp = &sgt[i++];
>   			hw_sg_to_cpu(sg_temp);
> -			sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> +			if (sg_temp->bpid != 0xFF) {
> +				bp_info = DPAA_BPID_TO_POOL_INFO(sg_temp->bpid);
> +				sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
>   						qm_sg_entry_get64(sg_temp));
> -			cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
> +				cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
>   						      bp_info->meta_data_size);
> -			first_seg->nb_segs += 1;
> -			prev_seg->next = cur_seg;
> -			if (sg_temp->final) {
> -				cur_seg->next = NULL;
> -				break;
> +				rte_pktmbuf_free_seg(cur_seg);
>   			}
> -			prev_seg = cur_seg;
> +			if (sg_temp->final)
> +				break;
>   		}
> -
> -		rte_pktmbuf_free_seg(temp);
>   		rte_pktmbuf_free_seg(first_seg);
>   		return 0;
>   	}

RE: [PATCH 15/15] net/dpaa: fix buffer free in slow path

[Gagandeep Singh]

Hi,

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Wednesday, October 5, 2022 7:52 PM
> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> Cc: stable@dpdk.org
> Subject: Re: [PATCH 15/15] net/dpaa: fix buffer free in slow path
> 
> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> > Adding a check in slow path to free those buffers which are not
> > external.
> >
> 
> Can you please explain what was the error before fix, what was happening
> when you try to free all mbufs?
> 
> Also it seems previous logic was different, with 'prev_seg' etc, can you
> explain what/why changed there?
> 
Actually, there were two issues, this function was converting all the segments present in HW frame
descriptor to mbuf SG list by doing while on segments in FD (HW descriptor) and in the end
it frees only one segment by calling the API rte_pktmbuf_free_seg(), so for other segments
memory will be leaked.

Now in this change, doing the loop on each segment in FD and if the segment has a valid
buffer pool id (HW pool id), freeing that segment in the loop itself without converting to a mbuf list.
if we free all the buffers even those with invalid HW bpid (which will only be the external buffer case),
then there can be double free because all the external buffer free handling is being done by the
Xmit function.

> > Fixes: 9124e65dd3eb ("net/dpaa: enable Tx queue taildrop")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> > ---
> >   drivers/net/dpaa/dpaa_rxtx.c | 23 ++++++++---------------
> >   1 file changed, 8 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/net/dpaa/dpaa_rxtx.c
> > b/drivers/net/dpaa/dpaa_rxtx.c index 4d285b4f38..ce4f3d6c85 100644
> > --- a/drivers/net/dpaa/dpaa_rxtx.c
> > +++ b/drivers/net/dpaa/dpaa_rxtx.c
> > @@ -455,7 +455,7 @@ dpaa_free_mbuf(const struct qm_fd *fd)
> >   	bp_info = DPAA_BPID_TO_POOL_INFO(fd->bpid);
> >   	format = (fd->opaque & DPAA_FD_FORMAT_MASK) >>
> DPAA_FD_FORMAT_SHIFT;
> >   	if (unlikely(format == qm_fd_sg)) {
> > -		struct rte_mbuf *first_seg, *prev_seg, *cur_seg, *temp;
> > +		struct rte_mbuf *first_seg, *cur_seg;
> >   		struct qm_sg_entry *sgt, *sg_temp;
> >   		void *vaddr, *sg_vaddr;
> >   		int i = 0;
> > @@ -469,32 +469,25 @@ dpaa_free_mbuf(const struct qm_fd *fd)
> >   		sgt = vaddr + fd_offset;
> >   		sg_temp = &sgt[i++];
> >   		hw_sg_to_cpu(sg_temp);
> > -		temp = (struct rte_mbuf *)
> > -			((char *)vaddr - bp_info->meta_data_size);
> >   		sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> >
> 	qm_sg_entry_get64(sg_temp));
> > -
> >   		first_seg = (struct rte_mbuf *)((char *)sg_vaddr -
> >   						bp_info->meta_data_size);
> >   		first_seg->nb_segs = 1;
> > -		prev_seg = first_seg;
> >   		while (i < DPAA_SGT_MAX_ENTRIES) {
> >   			sg_temp = &sgt[i++];
> >   			hw_sg_to_cpu(sg_temp);
> > -			sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> > +			if (sg_temp->bpid != 0xFF) {
> > +				bp_info =
> DPAA_BPID_TO_POOL_INFO(sg_temp->bpid);
> > +				sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> >
> 	qm_sg_entry_get64(sg_temp));
> > -			cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
> > +				cur_seg = (struct rte_mbuf *)((char
> *)sg_vaddr -
> >   						      bp_info-
> >meta_data_size);
> > -			first_seg->nb_segs += 1;
> > -			prev_seg->next = cur_seg;
> > -			if (sg_temp->final) {
> > -				cur_seg->next = NULL;
> > -				break;
> > +				rte_pktmbuf_free_seg(cur_seg);
> >   			}
> > -			prev_seg = cur_seg;
> > +			if (sg_temp->final)
> > +				break;
> >   		}
> > -
> > -		rte_pktmbuf_free_seg(temp);
> >   		rte_pktmbuf_free_seg(first_seg);
> >   		return 0;
> >   	}

Re: [PATCH 15/15] net/dpaa: fix buffer free in slow path

[Ferruh Yigit]

On 10/6/2022 9:51 AM, Gagandeep Singh wrote:
> Hi,
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@amd.com>
>> Sent: Wednesday, October 5, 2022 7:52 PM
>> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
>> Cc: stable@dpdk.org
>> Subject: Re: [PATCH 15/15] net/dpaa: fix buffer free in slow path
>>
>> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
>>> Adding a check in slow path to free those buffers which are not
>>> external.
>>>
>>
>> Can you please explain what was the error before fix, what was happening
>> when you try to free all mbufs?
>>
>> Also it seems previous logic was different, with 'prev_seg' etc, can you
>> explain what/why changed there?
>>
> Actually, there were two issues, this function was converting all the segments present in HW frame
> descriptor to mbuf SG list by doing while on segments in FD (HW descriptor) and in the end
> it frees only one segment by calling the API rte_pktmbuf_free_seg(), so for other segments
> memory will be leaked.
> 

ack

> Now in this change, doing the loop on each segment in FD and if the segment has a valid
> buffer pool id (HW pool id), freeing that segment in the loop itself without converting to a mbuf list.
> if we free all the buffers even those with invalid HW bpid (which will only be the external buffer case),
> then there can be double free because all the external buffer free handling is being done by the
> Xmit function.
> 

Got it, can you please give more information in the commit log as above, 
and can you please elaborate impact of possible double free, will it 
crash etc?

>>> Fixes: 9124e65dd3eb ("net/dpaa: enable Tx queue taildrop")
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
>>> ---
>>>    drivers/net/dpaa/dpaa_rxtx.c | 23 ++++++++---------------
>>>    1 file changed, 8 insertions(+), 15 deletions(-)
>>>
>>> diff --git a/drivers/net/dpaa/dpaa_rxtx.c
>>> b/drivers/net/dpaa/dpaa_rxtx.c index 4d285b4f38..ce4f3d6c85 100644
>>> --- a/drivers/net/dpaa/dpaa_rxtx.c
>>> +++ b/drivers/net/dpaa/dpaa_rxtx.c
>>> @@ -455,7 +455,7 @@ dpaa_free_mbuf(const struct qm_fd *fd)
>>>    	bp_info = DPAA_BPID_TO_POOL_INFO(fd->bpid);
>>>    	format = (fd->opaque & DPAA_FD_FORMAT_MASK) >>
>> DPAA_FD_FORMAT_SHIFT;
>>>    	if (unlikely(format == qm_fd_sg)) {
>>> -		struct rte_mbuf *first_seg, *prev_seg, *cur_seg, *temp;
>>> +		struct rte_mbuf *first_seg, *cur_seg;
>>>    		struct qm_sg_entry *sgt, *sg_temp;
>>>    		void *vaddr, *sg_vaddr;
>>>    		int i = 0;
>>> @@ -469,32 +469,25 @@ dpaa_free_mbuf(const struct qm_fd *fd)
>>>    		sgt = vaddr + fd_offset;
>>>    		sg_temp = &sgt[i++];
>>>    		hw_sg_to_cpu(sg_temp);
>>> -		temp = (struct rte_mbuf *)
>>> -			((char *)vaddr - bp_info->meta_data_size);
>>>    		sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
>>>
>> 	qm_sg_entry_get64(sg_temp));
>>> -
>>>    		first_seg = (struct rte_mbuf *)((char *)sg_vaddr -
>>>    						bp_info->meta_data_size);
>>>    		first_seg->nb_segs = 1;
>>> -		prev_seg = first_seg;
>>>    		while (i < DPAA_SGT_MAX_ENTRIES) {
>>>    			sg_temp = &sgt[i++];
>>>    			hw_sg_to_cpu(sg_temp);
>>> -			sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
>>> +			if (sg_temp->bpid != 0xFF) {
>>> +				bp_info =
>> DPAA_BPID_TO_POOL_INFO(sg_temp->bpid);
>>> +				sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
>>>
>> 	qm_sg_entry_get64(sg_temp));
>>> -			cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
>>> +				cur_seg = (struct rte_mbuf *)((char
>> *)sg_vaddr -
>>>    						      bp_info-
>>> meta_data_size);
>>> -			first_seg->nb_segs += 1;
>>> -			prev_seg->next = cur_seg;
>>> -			if (sg_temp->final) {
>>> -				cur_seg->next = NULL;
>>> -				break;
>>> +				rte_pktmbuf_free_seg(cur_seg);
>>>    			}
>>> -			prev_seg = cur_seg;
>>> +			if (sg_temp->final)
>>> +				break;
>>>    		}
>>> -
>>> -		rte_pktmbuf_free_seg(temp);
>>>    		rte_pktmbuf_free_seg(first_seg);
>>>    		return 0;
>>>    	}
> 

RE: [PATCH 15/15] net/dpaa: fix buffer free in slow path

[Gagandeep Singh]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@amd.com>
> Sent: Thursday, October 6, 2022 3:12 PM
> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> Cc: stable@dpdk.org
> Subject: Re: [PATCH 15/15] net/dpaa: fix buffer free in slow path
> 
> On 10/6/2022 9:51 AM, Gagandeep Singh wrote:
> > Hi,
> >
> >> -----Original Message-----
> >> From: Ferruh Yigit <ferruh.yigit@amd.com>
> >> Sent: Wednesday, October 5, 2022 7:52 PM
> >> To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org
> >> Cc: stable@dpdk.org
> >> Subject: Re: [PATCH 15/15] net/dpaa: fix buffer free in slow path
> >>
> >> On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> >>> Adding a check in slow path to free those buffers which are not
> >>> external.
> >>>
> >>
> >> Can you please explain what was the error before fix, what was
> >> happening when you try to free all mbufs?
> >>
> >> Also it seems previous logic was different, with 'prev_seg' etc, can
> >> you explain what/why changed there?
> >>
> > Actually, there were two issues, this function was converting all the
> > segments present in HW frame descriptor to mbuf SG list by doing while
> > on segments in FD (HW descriptor) and in the end it frees only one
> > segment by calling the API rte_pktmbuf_free_seg(), so for other segments
> memory will be leaked.
> >
> 
> ack
> 
> > Now in this change, doing the loop on each segment in FD and if the
> > segment has a valid buffer pool id (HW pool id), freeing that segment in the
> loop itself without converting to a mbuf list.
> > if we free all the buffers even those with invalid HW bpid (which will
> > only be the external buffer case), then there can be double free
> > because all the external buffer free handling is being done by the Xmit
> function.
> >
> 
> Got it, can you please give more information in the commit log as above, and
> can you please elaborate impact of possible double free, will it crash etc?
> 
Ok. I will update the commit message.

> >>> Fixes: 9124e65dd3eb ("net/dpaa: enable Tx queue taildrop")
> >>> Cc: stable@dpdk.org
> >>>
> >>> Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> >>> ---
> >>>    drivers/net/dpaa/dpaa_rxtx.c | 23 ++++++++---------------
> >>>    1 file changed, 8 insertions(+), 15 deletions(-)
> >>>
> >>> diff --git a/drivers/net/dpaa/dpaa_rxtx.c
> >>> b/drivers/net/dpaa/dpaa_rxtx.c index 4d285b4f38..ce4f3d6c85 100644
> >>> --- a/drivers/net/dpaa/dpaa_rxtx.c
> >>> +++ b/drivers/net/dpaa/dpaa_rxtx.c
> >>> @@ -455,7 +455,7 @@ dpaa_free_mbuf(const struct qm_fd *fd)
> >>>    	bp_info = DPAA_BPID_TO_POOL_INFO(fd->bpid);
> >>>    	format = (fd->opaque & DPAA_FD_FORMAT_MASK) >>
> >> DPAA_FD_FORMAT_SHIFT;
> >>>    	if (unlikely(format == qm_fd_sg)) {
> >>> -		struct rte_mbuf *first_seg, *prev_seg, *cur_seg, *temp;
> >>> +		struct rte_mbuf *first_seg, *cur_seg;
> >>>    		struct qm_sg_entry *sgt, *sg_temp;
> >>>    		void *vaddr, *sg_vaddr;
> >>>    		int i = 0;
> >>> @@ -469,32 +469,25 @@ dpaa_free_mbuf(const struct qm_fd *fd)
> >>>    		sgt = vaddr + fd_offset;
> >>>    		sg_temp = &sgt[i++];
> >>>    		hw_sg_to_cpu(sg_temp);
> >>> -		temp = (struct rte_mbuf *)
> >>> -			((char *)vaddr - bp_info->meta_data_size);
> >>>    		sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> >>>
> >> 	qm_sg_entry_get64(sg_temp));
> >>> -
> >>>    		first_seg = (struct rte_mbuf *)((char *)sg_vaddr -
> >>>    						bp_info->meta_data_size);
> >>>    		first_seg->nb_segs = 1;
> >>> -		prev_seg = first_seg;
> >>>    		while (i < DPAA_SGT_MAX_ENTRIES) {
> >>>    			sg_temp = &sgt[i++];
> >>>    			hw_sg_to_cpu(sg_temp);
> >>> -			sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> >>> +			if (sg_temp->bpid != 0xFF) {
> >>> +				bp_info =
> >> DPAA_BPID_TO_POOL_INFO(sg_temp->bpid);
> >>> +				sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
> >>>
> >> 	qm_sg_entry_get64(sg_temp));
> >>> -			cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
> >>> +				cur_seg = (struct rte_mbuf *)((char
> >> *)sg_vaddr -
> >>>    						      bp_info-
> >>> meta_data_size);
> >>> -			first_seg->nb_segs += 1;
> >>> -			prev_seg->next = cur_seg;
> >>> -			if (sg_temp->final) {
> >>> -				cur_seg->next = NULL;
> >>> -				break;
> >>> +				rte_pktmbuf_free_seg(cur_seg);
> >>>    			}
> >>> -			prev_seg = cur_seg;
> >>> +			if (sg_temp->final)
> >>> +				break;
> >>>    		}
> >>> -
> >>> -		rte_pktmbuf_free_seg(temp);
> >>>    		rte_pktmbuf_free_seg(first_seg);
> >>>    		return 0;
> >>>    	}
> >

Re: [PATCH 00/15] DPAA and DPAA2 driver changes

[Ferruh Yigit]

On 9/28/2022 6:25 AM, Gagandeep Singh wrote:
> This series have list of patch for bug fixes and
> some enhancements to DPAA1 and DPAA2 net drivers.
> 
> Apeksha Gupta (2):
>    net/enetfec: fix restart issue
>    net/enetfec: fix buffer leak issue
> 
> Gagandeep Singh (7):
>    net/dpaa: support ESP packet type in packet parsing
>    net/dpaa2: use internal mempool for SG table
>    net/dpaa2: fix buffer free on transmit SG packets
>    net/dpaa: use internal mempool for SG table
>    bus/dpaa: mempool ops registration change
>    net/dpaa: fix buffer free on transmit SG packets
>    net/dpaa: fix buffer free in slow path
> 
> Rohit Raj (3):
>    bus/fslmc: add timeout in MC send command API
>    net/dpaa: fix Jumbo packet Rx in case of VSP
>    bus/dpaa: pass interface name as a string instead of pointer
> 
> Vanshika Shukla (2):
>    bus/dpaa: use non-block mode for FD open
>    net/dpaa2: fix dpdmux configuration for error behaviour
> 
> brick (1):
>    net/dpaa2: check free enqueue descriptors before Tx
> 

Hi Gagandeep,

I put some comments/questions to some patches, can you please check?

Also patches requires ack/review tag from their maintainer to proceed.

RE: [PATCH 00/15] DPAA and DPAA2 driver changes

[Hemant Agrawal]

Series-
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

> -----Original Message-----
> From: Gagandeep Singh <G.Singh@nxp.com>
> Sent: Wednesday, September 28, 2022 10:55 AM
> To: ferruh.yigit@amd.com; dev@dpdk.org
> Cc: Gagandeep Singh <G.Singh@nxp.com>
> Subject: [PATCH 00/15] DPAA and DPAA2 driver changes
> 
> This series have list of patch for bug fixes and some enhancements to DPAA1
> and DPAA2 net drivers.
> 
> Apeksha Gupta (2):
>   net/enetfec: fix restart issue
>   net/enetfec: fix buffer leak issue
> 
> Gagandeep Singh (7):
>   net/dpaa: support ESP packet type in packet parsing
>   net/dpaa2: use internal mempool for SG table
>   net/dpaa2: fix buffer free on transmit SG packets
>   net/dpaa: use internal mempool for SG table
>   bus/dpaa: mempool ops registration change
>   net/dpaa: fix buffer free on transmit SG packets
>   net/dpaa: fix buffer free in slow path
> 
> Rohit Raj (3):
>   bus/fslmc: add timeout in MC send command API
>   net/dpaa: fix Jumbo packet Rx in case of VSP
>   bus/dpaa: pass interface name as a string instead of pointer
> 
> Vanshika Shukla (2):
>   bus/dpaa: use non-block mode for FD open
>   net/dpaa2: fix dpdmux configuration for error behaviour
> 
> brick (1):
>   net/dpaa2: check free enqueue descriptors before Tx
> 
>  drivers/bus/dpaa/base/qbman/process.c     |   8 +-
>  drivers/bus/dpaa/base/qbman/qman_driver.c |   6 +-
>  drivers/bus/dpaa/dpaa_bus.c               |  10 +-
>  drivers/bus/fslmc/mc/mc_sys.c             |  18 ++-
>  drivers/event/dpaa2/dpaa2_eventdev.c      |   8 +-
>  drivers/net/dpaa/dpaa_ethdev.c            |  20 ++-
>  drivers/net/dpaa/dpaa_ethdev.h            |  19 +++
>  drivers/net/dpaa/dpaa_flow.c              |  13 +-
>  drivers/net/dpaa/dpaa_flow.h              |   5 +-
>  drivers/net/dpaa/dpaa_rxtx.c              |  97 +++++++-----
>  drivers/net/dpaa/dpaa_rxtx.h              |   6 +
>  drivers/net/dpaa2/dpaa2_ethdev.c          |  14 ++
>  drivers/net/dpaa2/dpaa2_ethdev.h          |  21 +++
>  drivers/net/dpaa2/dpaa2_mux.c             |   2 +-
>  drivers/net/dpaa2/dpaa2_rxtx.c            | 174 +++++++++++++++-------
>  drivers/net/enetfec/enet_ethdev.c         |   4 +
>  drivers/net/enetfec/enet_rxtx.c           |  29 ++--
>  17 files changed, 315 insertions(+), 139 deletions(-)
> 
> --
> 2.25.1

[PATCH v2 00/16] DPAA and DPAA2 driver changes

[Gagandeep Singh]

This series have list of patch for bug fixes and
some enhancements to DPAA1 and DPAA2 net drivers.

v2-change-log:
* update commit messages
* update DPAA NIC doc for kernel version matching
* update DPAA supported packet type info
* Freeing the internal pool on last device remove
* fixes syntax for some comments

Apeksha Gupta (2):
  net/enetfec: fix restart issue
  net/enetfec: fix buffer leak issue

Brick Yang (1):
  net/dpaa2: check free enqueue descriptors before Tx

Gagandeep Singh (8):
  net/dpaa: support ESP packet type in packet parsing
  net/dpaa2: use internal mempool for SG table
  net/dpaa2: fix buffer free on transmit SG packets
  doc: add kernel version compatible information
  net/dpaa: use internal mempool for SG table
  bus/dpaa: mempool ops registration change
  net/dpaa: fix buffer free on transmit SG packets
  net/dpaa: fix buffer free in slow path

Rohit Raj (3):
  bus/fslmc: add timeout in MC send command API
  net/dpaa: fix Jumbo packet Rx in case of VSP
  bus/dpaa: pass interface name as a string instead of pointer

Vanshika Shukla (2):
  bus/dpaa: use non-block mode for FD open
  net/dpaa2: fix dpdmux configuration for error behaviour

 doc/guides/nics/dpaa.rst                  |   4 +
 drivers/bus/dpaa/base/qbman/process.c     |   8 +-
 drivers/bus/dpaa/base/qbman/qman_driver.c |   6 +-
 drivers/bus/dpaa/dpaa_bus.c               |  10 +-
 drivers/bus/fslmc/mc/mc_sys.c             |  18 ++-
 drivers/event/dpaa2/dpaa2_eventdev.c      |   8 +-
 drivers/net/dpaa/dpaa_ethdev.c            |  28 +++-
 drivers/net/dpaa/dpaa_ethdev.h            |  19 +++
 drivers/net/dpaa/dpaa_flow.c              |  13 +-
 drivers/net/dpaa/dpaa_flow.h              |   5 +-
 drivers/net/dpaa/dpaa_rxtx.c              |  97 +++++++-----
 drivers/net/dpaa/dpaa_rxtx.h              |   6 +
 drivers/net/dpaa2/dpaa2_ethdev.c          |  19 +++
 drivers/net/dpaa2/dpaa2_ethdev.h          |  21 +++
 drivers/net/dpaa2/dpaa2_mux.c             |   2 +-
 drivers/net/dpaa2/dpaa2_rxtx.c            | 174 +++++++++++++++-------
 drivers/net/enetfec/enet_ethdev.c         |   4 +
 drivers/net/enetfec/enet_rxtx.c           |  29 ++--
 18 files changed, 331 insertions(+), 140 deletions(-)

-- 
2.25.1

[PATCH v2 01/16] bus/dpaa: use non-block mode for FD open

[Gagandeep Singh]

From: Vanshika Shukla <vanshika.shukla@nxp.com>

This patch sets qman portal file descriptors used for
interrupts IO processing in non-blocking mode to avoid
any unwanted blocks while IO operations over the FD.

Signed-off-by: Vanshika Shukla <vanshika.shukla@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/bus/dpaa/base/qbman/qman_driver.c | 6 +++---
 drivers/net/dpaa/dpaa_ethdev.c            | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/bus/dpaa/base/qbman/qman_driver.c b/drivers/bus/dpaa/base/qbman/qman_driver.c
index dfbafe581a..7a129a2d86 100644
--- a/drivers/bus/dpaa/base/qbman/qman_driver.c
+++ b/drivers/bus/dpaa/base/qbman/qman_driver.c
@@ -1,7 +1,7 @@
 /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
  *
  * Copyright 2008-2016 Freescale Semiconductor Inc.
- * Copyright 2017,2019 NXP
+ * Copyright 2017-2022 NXP
  *
  */
 
@@ -64,7 +64,7 @@ static int fsl_qman_portal_init(uint32_t index, int is_shared)
 	qpcfg.addr_virt[DPAA_PORTAL_CE] = map.addr.cena;
 	qpcfg.addr_virt[DPAA_PORTAL_CI] = map.addr.cinh;
 
-	qmfd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY);
+	qmfd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY | O_NONBLOCK);
 	if (qmfd == -1) {
 		pr_err("QMan irq init failed\n");
 		process_portal_unmap(&map.addr);
@@ -174,7 +174,7 @@ struct qman_portal *fsl_qman_fq_portal_create(int *fd)
 	q_pcfg->addr_virt[DPAA_PORTAL_CE] = q_map.addr.cena;
 	q_pcfg->addr_virt[DPAA_PORTAL_CI] = q_map.addr.cinh;
 
-	q_fd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY);
+	q_fd = open(QMAN_PORTAL_IRQ_PATH, O_RDONLY | O_NONBLOCK);
 	if (q_fd == -1) {
 		pr_err("QMan irq init failed\n");
 		goto err;
diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index f21cbba0d9..abcb1bc9ec 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -1496,7 +1496,7 @@ static int dpaa_dev_queue_intr_disable(struct rte_eth_dev *dev,
 
 	temp1 = read(rxq->q_fd, &temp, sizeof(temp));
 	if (temp1 != sizeof(temp))
-		DPAA_PMD_ERR("irq read error");
+		DPAA_PMD_DEBUG("read did not return anything");
 
 	qman_fq_portal_thread_irq(rxq->qp);
 
-- 
2.25.1

[PATCH v2 02/16] net/enetfec: fix restart issue

[Gagandeep Singh]

From: Apeksha Gupta <apeksha.gupta@nxp.com>

Queue reset is missing in restart because of which
IO cannot work on device restart.

This patch fixes the issue by resetting the queues on
device restart.

Fixes: b84fdd39638b ("net/enetfec: support UIO")
Cc: stable@dpdk.org

Signed-off-by: Apeksha Gupta <apeksha.gupta@nxp.com>
Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/enetfec/enet_ethdev.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/net/enetfec/enet_ethdev.c b/drivers/net/enetfec/enet_ethdev.c
index c938e58204..898aad1c37 100644
--- a/drivers/net/enetfec/enet_ethdev.c
+++ b/drivers/net/enetfec/enet_ethdev.c
@@ -54,6 +54,7 @@ enetfec_restart(struct rte_eth_dev *dev)
 	uint32_t rcntl = OPT_FRAME_SIZE | 0x04;
 	uint32_t ecntl = ENETFEC_ETHEREN;
 	uint32_t val;
+	int i;
 
 	/* Clear any outstanding interrupt. */
 	writel(0xffffffff, (uint8_t *)fep->hw_baseaddr_v + ENETFEC_EIR);
@@ -149,6 +150,9 @@ enetfec_restart(struct rte_eth_dev *dev)
 	/* And last, enable the transmit and receive processing */
 	rte_write32(rte_cpu_to_le_32(ecntl),
 		(uint8_t *)fep->hw_baseaddr_v + ENETFEC_ECR);
+
+	for (i = 0; i < fep->max_rx_queues; i++)
+		rte_write32(0, fep->rx_queues[i]->bd.active_reg_desc);
 	rte_delay_us(10);
 }
 
-- 
2.25.1

[PATCH v2 03/16] net/enetfec: fix buffer leak issue

[Gagandeep Singh]

From: Apeksha Gupta <apeksha.gupta@nxp.com>

Driver has no proper handling to free unused
allocated mbufs in case of error or when the rx
processing complete because of which mempool
can be empty after some time.

This patch fixes this issue by moving the buffer
allocation code to the right place in driver.

Fixes: ecae71571b0d ("net/enetfec: support Rx/Tx")
Cc: stable@dpdk.org

Signed-off-by: Apeksha Gupta <apeksha.gupta@nxp.com>
Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/enetfec/enet_rxtx.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/drivers/net/enetfec/enet_rxtx.c b/drivers/net/enetfec/enet_rxtx.c
index 49b326315d..0aea8b240d 100644
--- a/drivers/net/enetfec/enet_rxtx.c
+++ b/drivers/net/enetfec/enet_rxtx.c
@@ -39,11 +39,6 @@ enetfec_recv_pkts(void *rxq1, struct rte_mbuf **rx_pkts,
 		if (pkt_received >= nb_pkts)
 			break;
 
-		new_mbuf = rte_pktmbuf_alloc(pool);
-		if (unlikely(new_mbuf == NULL)) {
-			stats->rx_nombuf++;
-			break;
-		}
 		/* Check for errors. */
 		status ^= RX_BD_LAST;
 		if (status & (RX_BD_LG | RX_BD_SH | RX_BD_NO |
@@ -72,6 +67,12 @@ enetfec_recv_pkts(void *rxq1, struct rte_mbuf **rx_pkts,
 			goto rx_processing_done;
 		}
 
+		new_mbuf = rte_pktmbuf_alloc(pool);
+		if (unlikely(new_mbuf == NULL)) {
+			stats->rx_nombuf++;
+			break;
+		}
+
 		/* Process the incoming frame. */
 		stats->ipackets++;
 		pkt_len = rte_le_to_cpu_16(rte_read16(&bdp->bd_datlen));
@@ -193,7 +194,16 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			tx_st = 0;
 			break;
 		}
+
+		mbuf = *(tx_pkts);
+		if (mbuf->nb_segs > 1) {
+			ENETFEC_DP_LOG(DEBUG, "SG not supported");
+			return pkt_transmitted;
+		}
+
+		tx_pkts++;
 		bdp = txq->bd.cur;
+
 		/* First clean the ring */
 		index = enet_get_bd_index(bdp, &txq->bd);
 		status = rte_le_to_cpu_16(rte_read16(&bdp->bd_sc));
@@ -207,9 +217,6 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			txq->tx_mbuf[index] = NULL;
 		}
 
-		mbuf = *(tx_pkts);
-		tx_pkts++;
-
 		/* Fill in a Tx ring entry */
 		last_bdp = bdp;
 		status &= ~TX_BD_STATS;
@@ -219,10 +226,6 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		stats->opackets++;
 		stats->obytes += buflen;
 
-		if (mbuf->nb_segs > 1) {
-			ENETFEC_DP_LOG(DEBUG, "SG not supported");
-			return -1;
-		}
 		status |= (TX_BD_LAST);
 		data = rte_pktmbuf_mtod(mbuf, void *);
 		for (i = 0; i <= buflen; i += RTE_CACHE_LINE_SIZE)
@@ -268,5 +271,5 @@ enetfec_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		 */
 		txq->bd.cur = bdp;
 	}
-	return nb_pkts;
+	return pkt_transmitted;
 }
-- 
2.25.1

[PATCH v2 04/16] net/dpaa2: fix dpdmux configuration for error behaviour

[Gagandeep Singh]

From: Vanshika Shukla <vanshika.shukla@nxp.com>

Driver is giving the wrong interface ID while setting the
error behaviour.

This patch fixes the issue by passing the correct MAC interface
index value to the API.

Fixes: 3d43972b1b42 ("net/dpaa2: do not drop parse error packets by dpdmux")
Cc: stable@dpdk.org

Signed-off-by: Vanshika Shukla <vanshika.shukla@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.h | 3 +++
 drivers/net/dpaa2/dpaa2_mux.c    | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index f69df95253..32ae762e4a 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -66,6 +66,9 @@
 /* Tx confirmation enabled */
 #define DPAA2_TX_CONF_ENABLE	0x06
 
+/* DPDMUX index for DPMAC */
+#define DPAA2_DPDMUX_DPMAC_IDX 0
+
 /* HW loopback the egress traffic to self ingress*/
 #define DPAA2_TX_MAC_LOOPBACK_MODE 0x20
 
diff --git a/drivers/net/dpaa2/dpaa2_mux.c b/drivers/net/dpaa2/dpaa2_mux.c
index 3289f388e1..7456f43f42 100644
--- a/drivers/net/dpaa2/dpaa2_mux.c
+++ b/drivers/net/dpaa2/dpaa2_mux.c
@@ -336,7 +336,7 @@ dpaa2_create_dpdmux_device(int vdev_fd __rte_unused,
 
 		ret = dpdmux_if_set_errors_behavior(&dpdmux_dev->dpdmux,
 				CMD_PRI_LOW,
-				dpdmux_dev->token, dpdmux_id,
+				dpdmux_dev->token, DPAA2_DPDMUX_DPMAC_IDX,
 				&mux_err_cfg);
 		if (ret) {
 			DPAA2_PMD_ERR("dpdmux_if_set_errors_behavior %s err %d",
-- 
2.25.1

[PATCH v2 05/16] net/dpaa2: check free enqueue descriptors before Tx

[Gagandeep Singh]

From: Brick Yang <brick.yang@nxp.com>

Check if there exists free enqueue descriptors before enqueuing Tx
packet. Also try to free enqueue descriptors in case they are not
free.

Fixes: ed1cdbed6a15 ("net/dpaa2: support multiple Tx queues enqueue for ordered")
Cc: stable@dpdk.org

Signed-off-by: Brick Yang <brick.yang@nxp.com>
Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/event/dpaa2/dpaa2_eventdev.c |  8 ++---
 drivers/net/dpaa2/dpaa2_rxtx.c       | 50 +++++++++++++++++++---------
 2 files changed, 38 insertions(+), 20 deletions(-)

diff --git a/drivers/event/dpaa2/dpaa2_eventdev.c b/drivers/event/dpaa2/dpaa2_eventdev.c
index f499d0d015..fa1a1ade80 100644
--- a/drivers/event/dpaa2/dpaa2_eventdev.c
+++ b/drivers/event/dpaa2/dpaa2_eventdev.c
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2017,2019-2021 NXP
+ * Copyright 2017,2019-2022 NXP
  */
 
 #include <assert.h>
@@ -176,7 +176,7 @@ dpaa2_eventdev_enqueue_burst(void *port, const struct rte_event ev[],
 				if (retry_count > DPAA2_EV_TX_RETRY_COUNT) {
 					num_tx += loop;
 					nb_events -= loop;
-					return num_tx + loop;
+					return num_tx;
 				}
 			} else {
 				loop += ret;
@@ -1016,9 +1016,7 @@ dpaa2_eventdev_txa_enqueue(void *port,
 		txq[i] = rte_eth_devices[m[i]->port].data->tx_queues[qid];
 	}
 
-	dpaa2_dev_tx_multi_txq_ordered(txq, m, nb_events);
-
-	return nb_events;
+	return dpaa2_dev_tx_multi_txq_ordered(txq, m, nb_events);
 }
 
 static struct eventdev_ops dpaa2_eventdev_ops = {
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 9436a95ac8..571ea6d16d 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -1525,7 +1525,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 	uint32_t loop, retry_count;
 	int32_t ret;
 	struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
-	uint32_t frames_to_send;
+	uint32_t frames_to_send, num_free_eq_desc = 0;
 	struct rte_mempool *mp;
 	struct qbman_eq_desc eqdesc[MAX_TX_RING_SLOTS];
 	struct dpaa2_queue *dpaa2_q[MAX_TX_RING_SLOTS];
@@ -1547,16 +1547,44 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 	}
 	swp = DPAA2_PER_LCORE_PORTAL;
 
-	for (loop = 0; loop < nb_pkts; loop++) {
+	frames_to_send = (nb_pkts > dpaa2_eqcr_size) ?
+		dpaa2_eqcr_size : nb_pkts;
+
+	for (loop = 0; loop < frames_to_send; loop++) {
 		dpaa2_q[loop] = (struct dpaa2_queue *)queue[loop];
 		eth_data = dpaa2_q[loop]->eth_data;
 		priv = eth_data->dev_private;
+		if (!priv->en_loose_ordered) {
+			if (*dpaa2_seqn(*bufs) & DPAA2_ENQUEUE_FLAG_ORP) {
+				if (!num_free_eq_desc) {
+					num_free_eq_desc = dpaa2_free_eq_descriptors();
+					if (!num_free_eq_desc)
+						goto send_frames;
+				}
+				num_free_eq_desc--;
+			}
+		}
+
+		DPAA2_PMD_DP_DEBUG("===> eth_data =%p, fqid =%d\n",
+				   eth_data, dpaa2_q[loop]->fqid);
+
+		/* Check if the queue is congested */
+		retry_count = 0;
+		while (qbman_result_SCN_state(dpaa2_q[loop]->cscn)) {
+			retry_count++;
+			/* Retry for some time before giving up */
+			if (retry_count > CONG_RETRY_COUNT)
+				goto send_frames;
+		}
+
+		/* Prepare enqueue descriptor */
 		qbman_eq_desc_clear(&eqdesc[loop]);
+
 		if (*dpaa2_seqn(*bufs) && priv->en_ordered) {
 			order_sendq = (struct dpaa2_queue *)priv->tx_vq[0];
 			dpaa2_set_enqueue_descriptor(order_sendq,
-							     (*bufs),
-							     &eqdesc[loop]);
+						     (*bufs),
+						     &eqdesc[loop]);
 		} else {
 			qbman_eq_desc_set_no_orp(&eqdesc[loop],
 							 DPAA2_EQ_RESP_ERR_FQ);
@@ -1564,14 +1592,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 						     dpaa2_q[loop]->fqid);
 		}
 
-		retry_count = 0;
-		while (qbman_result_SCN_state(dpaa2_q[loop]->cscn)) {
-			retry_count++;
-			/* Retry for some time before giving up */
-			if (retry_count > CONG_RETRY_COUNT)
-				goto send_frames;
-		}
-
 		if (likely(RTE_MBUF_DIRECT(*bufs))) {
 			mp = (*bufs)->pool;
 			/* Check the basic scenario and set
@@ -1591,7 +1611,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 					&fd_arr[loop],
 					mempool_to_bpid(mp));
 				bufs++;
-				dpaa2_q[loop]++;
 				continue;
 			}
 		} else {
@@ -1637,18 +1656,19 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 		}
 
 		bufs++;
-		dpaa2_q[loop]++;
 	}
 
 send_frames:
 	frames_to_send = loop;
 	loop = 0;
+	retry_count = 0;
 	while (loop < frames_to_send) {
 		ret = qbman_swp_enqueue_multiple_desc(swp, &eqdesc[loop],
 				&fd_arr[loop],
 				frames_to_send - loop);
 		if (likely(ret > 0)) {
 			loop += ret;
+			retry_count = 0;
 		} else {
 			retry_count++;
 			if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
@@ -1834,7 +1854,7 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 		retry_count = 0;
 		while (i < loop) {
 			ret = qbman_swp_enqueue_multiple_desc(swp,
-				       &eqdesc[loop], &fd_arr[i], loop - i);
+				       &eqdesc[i], &fd_arr[i], loop - i);
 			if (unlikely(ret < 0)) {
 				retry_count++;
 				if (retry_count > DPAA2_MAX_TX_RETRY_COUNT)
-- 
2.25.1

[PATCH v2 06/16] net/dpaa: support ESP packet type in packet parsing

[Gagandeep Singh]

Add support of ESP packet type in packet receive path.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.c |  3 ++-
 drivers/net/dpaa/dpaa_rxtx.c   | 10 ++++++++++
 drivers/net/dpaa/dpaa_rxtx.h   |  6 ++++++
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index abcb1bc9ec..c4aac424b4 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -360,7 +360,8 @@ dpaa_supported_ptypes_get(struct rte_eth_dev *dev)
 		RTE_PTYPE_L4_FRAG,
 		RTE_PTYPE_L4_TCP,
 		RTE_PTYPE_L4_UDP,
-		RTE_PTYPE_L4_SCTP
+		RTE_PTYPE_L4_SCTP,
+		RTE_PTYPE_TUNNEL_ESP
 	};
 
 	PMD_INIT_FUNC_TRACE();
diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index 20b75efb63..22205cec30 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -178,6 +178,16 @@ static inline void dpaa_eth_packet_info(struct rte_mbuf *m, void *fd_virt_addr)
 		m->packet_type = RTE_PTYPE_L2_ETHER |
 			RTE_PTYPE_L3_IPV6 | RTE_PTYPE_L4_UDP;
 		break;
+	case DPAA_PKT_TYPE_IPSEC_IPV4:
+		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
+			m->packet_type = RTE_PTYPE_L2_ETHER |
+				RTE_PTYPE_L3_IPV4 | RTE_PTYPE_TUNNEL_ESP;
+		break;
+	case DPAA_PKT_TYPE_IPSEC_IPV6:
+		if (*((uintptr_t *)&annot->parse) & DPAA_PARSE_ESP_MASK)
+			m->packet_type = RTE_PTYPE_L2_ETHER |
+				RTE_PTYPE_L3_IPV6 | RTE_PTYPE_TUNNEL_ESP;
+		break;
 	case DPAA_PKT_TYPE_IPV4_EXT_UDP:
 		m->packet_type = RTE_PTYPE_L2_ETHER |
 			RTE_PTYPE_L3_IPV4_EXT | RTE_PTYPE_L4_UDP;
diff --git a/drivers/net/dpaa/dpaa_rxtx.h b/drivers/net/dpaa/dpaa_rxtx.h
index 99e09196e9..b2d7c0f2a3 100644
--- a/drivers/net/dpaa/dpaa_rxtx.h
+++ b/drivers/net/dpaa/dpaa_rxtx.h
@@ -47,6 +47,7 @@
  *	L4R 0xE0 -
  *		0x20 - TCP
  *		0x40 - UDP
+ *		0x60 - IPsec
  *		0x80 - SCTP
  *	L3R 0xEDC4 (in Big Endian) -
  *		0x8000 - IPv4
@@ -63,6 +64,7 @@
  */
 #define DPAA_PARSE_MASK			0x00F044EF00800000
 #define DPAA_PARSE_VLAN_MASK		0x0000000000700000
+#define DPAA_PARSE_ESP_MASK		0x0008000000000000
 
 /* Parsed values (Little Endian) */
 #define DPAA_PKT_TYPE_NONE		0x0000000000000000
@@ -137,6 +139,10 @@
 			(0x0020000000000000 | DPAA_PKT_TYPE_TUNNEL_4_6)
 #define DPAA_PKT_TYPE_TUNNEL_6_4_TCP \
 			(0x0020000000000000 | DPAA_PKT_TYPE_TUNNEL_6_4)
+#define DPAA_PKT_TYPE_IPSEC_IPV4 \
+			(0x0060000000000000 | DPAA_PKT_TYPE_IPV4)
+#define DPAA_PKT_TYPE_IPSEC_IPV6 \
+			(0x0060000000000000 | DPAA_PKT_TYPE_IPV6)
 
 /* Checksum Errors */
 #define DPAA_PKT_IP_CSUM_ERR		0x0000400200000000
-- 
2.25.1

[PATCH v2 07/16] net/dpaa2: use internal mempool for SG table

[Gagandeep Singh]

Creating and using driver's mempool for
allocating the SG table memory required for
FD creation instead of relying on user mempool.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.c | 19 +++++++++++++++++++
 drivers/net/dpaa2/dpaa2_ethdev.h |  9 +++++++++
 drivers/net/dpaa2/dpaa2_rxtx.c   | 13 ++++++-------
 3 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 37a8b43114..679f33ae1a 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -78,6 +78,9 @@ bool dpaa2_enable_err_queue;
 #define MAX_NB_RX_DESC		11264
 int total_nb_rx_desc;
 
+int dpaa2_valid_dev;
+struct rte_mempool *dpaa2_tx_sg_pool;
+
 struct rte_dpaa2_xstats_name_off {
 	char name[RTE_ETH_XSTATS_NAME_SIZE];
 	uint8_t page_id; /* dpni statistics page id */
@@ -2907,7 +2910,20 @@ rte_dpaa2_probe(struct rte_dpaa2_driver *dpaa2_drv,
 	/* Invoke PMD device initialization function */
 	diag = dpaa2_dev_init(eth_dev);
 	if (diag == 0) {
+		if (!dpaa2_tx_sg_pool) {
+			dpaa2_tx_sg_pool =
+				rte_pktmbuf_pool_create("dpaa2_mbuf_tx_sg_pool",
+				DPAA2_POOL_SIZE,
+				DPAA2_POOL_CACHE_SIZE, 0,
+				DPAA2_MAX_SGS * sizeof(struct qbman_sge),
+				rte_socket_id());
+			if (dpaa2_tx_sg_pool == NULL) {
+				DPAA2_PMD_ERR("SG pool creation failed\n");
+				return -ENOMEM;
+			}
+		}
 		rte_eth_dev_probing_finish(eth_dev);
+		dpaa2_valid_dev++;
 		return 0;
 	}
 
@@ -2923,6 +2939,9 @@ rte_dpaa2_remove(struct rte_dpaa2_device *dpaa2_dev)
 
 	eth_dev = dpaa2_dev->eth_dev;
 	dpaa2_dev_close(eth_dev);
+	dpaa2_valid_dev--;
+	if (!dpaa2_valid_dev)
+		rte_mempool_free(dpaa2_tx_sg_pool);
 	ret = rte_eth_dev_release_port(eth_dev);
 
 	return ret;
diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 32ae762e4a..872dced517 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -121,6 +121,15 @@
 #define DPAA2_PKT_TYPE_VLAN_1		0x0160
 #define DPAA2_PKT_TYPE_VLAN_2		0x0260
 
+/* Global pool used by driver for SG list TX */
+extern struct rte_mempool *dpaa2_tx_sg_pool;
+/* Maximum SG segments */
+#define DPAA2_MAX_SGS 128
+/* SG pool size */
+#define DPAA2_POOL_SIZE 2048
+/* SG pool cache size */
+#define DPAA2_POOL_CACHE_SIZE 256
+
 /* enable timestamp in mbuf*/
 extern bool dpaa2_enable_ts[];
 extern uint64_t dpaa2_timestamp_rx_dynflag;
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 571ea6d16d..53e06b3884 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -403,7 +403,7 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
 static int __rte_noinline __rte_hot
 eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 		  struct qbman_fd *fd,
-		  struct rte_mempool *mp, uint16_t bpid)
+		  uint16_t bpid)
 {
 	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
 	struct qbman_sge *sgt, *sge = NULL;
@@ -433,12 +433,12 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 		}
 		DPAA2_SET_FD_OFFSET(fd, offset);
 	} else {
-		temp = rte_pktmbuf_alloc(mp);
+		temp = rte_pktmbuf_alloc(dpaa2_tx_sg_pool);
 		if (temp == NULL) {
 			DPAA2_PMD_DP_DEBUG("No memory to allocate S/G table\n");
 			return -ENOMEM;
 		}
-		DPAA2_SET_ONLY_FD_BPID(fd, bpid);
+		DPAA2_SET_ONLY_FD_BPID(fd, mempool_to_bpid(dpaa2_tx_sg_pool));
 		DPAA2_SET_FD_OFFSET(fd, temp->data_off);
 #ifdef RTE_LIBRTE_MEMPOOL_DEBUG
 		rte_mempool_check_cookies(rte_mempool_from_obj((void *)temp),
@@ -1321,9 +1321,10 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 
 			if (unlikely(RTE_MBUF_HAS_EXTBUF(*bufs))) {
 				if (unlikely((*bufs)->nb_segs > 1)) {
+					mp = (*bufs)->pool;
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
-							      mp, 0))
+							      mempool_to_bpid(mp)))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
@@ -1372,7 +1373,7 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							&fd_arr[loop],
-							mp, bpid))
+							bpid))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
@@ -1646,7 +1647,6 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 			if (unlikely((*bufs)->nb_segs > 1)) {
 				if (eth_mbuf_to_sg_fd(*bufs,
 						      &fd_arr[loop],
-						      mp,
 						      bpid))
 					goto send_frames;
 			} else {
@@ -1810,7 +1810,6 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
-							      mp,
 							      bpid))
 						goto send_n_return;
 				} else {
-- 
2.25.1

[PATCH v2 08/16] net/dpaa2: fix buffer free on transmit SG packets

[Gagandeep Singh]

When using SG list to TX with external and direct buffers,
HW free the direct buffers and driver free the external buffers.

Software scans the complete SG mbuf list to find the external
buffers to free, but this is wrong as hardware can free the
direct buffers if any present in the list and same can be
re-allocated for other purpose in multi thread or high speed
running traffic environment with new data in it. So the software
which is scanning the SG mbuf list, if that list has any direct
buffer present then that direct buffer's next pointer can give
wrong pointer value, if already freed by hardware which
can do the mempool corruption or memory leak.

In this patch instead of relying on user given SG mbuf list
we are storing the buffers in an internal list which will
be scanned by driver after transmit to free non-direct
buffers.

This patch also fixes 2 more memory leak issues.

Driver is freeing complete SG list by checking external buffer
flag in first segment only, but external buffer can be attached
to any of the segment. Because of which driver either can double
free buffers or there can be memory leak.

In case of indirect buffers, driver is modifying the original
buffer list to free the indirect buffers but this original buffer
list is being used even after transmit packets for software
buffer cleanup. This can cause the buffer leak issue.

Fixes: 6bfbafe18d15 ("net/dpaa2: support external buffers in Tx")
Cc: stable@dpdk.org

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.h |   9 +++
 drivers/net/dpaa2/dpaa2_rxtx.c   | 111 +++++++++++++++++++++++--------
 2 files changed, 92 insertions(+), 28 deletions(-)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.h b/drivers/net/dpaa2/dpaa2_ethdev.h
index 872dced517..c88c8146dc 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.h
+++ b/drivers/net/dpaa2/dpaa2_ethdev.h
@@ -129,6 +129,15 @@ extern struct rte_mempool *dpaa2_tx_sg_pool;
 #define DPAA2_POOL_SIZE 2048
 /* SG pool cache size */
 #define DPAA2_POOL_CACHE_SIZE 256
+/* structure to free external and indirect
+ * buffers.
+ */
+struct sw_buf_free {
+	/* To which packet this segment belongs */
+	uint16_t pkt_id;
+	/* The actual segment */
+	struct rte_mbuf *seg;
+};
 
 /* enable timestamp in mbuf*/
 extern bool dpaa2_enable_ts[];
diff --git a/drivers/net/dpaa2/dpaa2_rxtx.c b/drivers/net/dpaa2/dpaa2_rxtx.c
index 53e06b3884..b0ee58fc9f 100644
--- a/drivers/net/dpaa2/dpaa2_rxtx.c
+++ b/drivers/net/dpaa2/dpaa2_rxtx.c
@@ -403,9 +403,12 @@ eth_fd_to_mbuf(const struct qbman_fd *fd,
 static int __rte_noinline __rte_hot
 eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 		  struct qbman_fd *fd,
+		  struct sw_buf_free *free_buf,
+		  uint32_t *free_count,
+		  uint32_t pkt_id,
 		  uint16_t bpid)
 {
-	struct rte_mbuf *cur_seg = mbuf, *prev_seg, *mi, *temp;
+	struct rte_mbuf *cur_seg = mbuf, *mi, *temp;
 	struct qbman_sge *sgt, *sge = NULL;
 	int i, offset = 0;
 
@@ -486,10 +489,11 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 #endif
 				}
 			}
-			cur_seg = cur_seg->next;
 		} else if (RTE_MBUF_HAS_EXTBUF(cur_seg)) {
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 			DPAA2_SET_FLE_IVP(sge);
-			cur_seg = cur_seg->next;
 		} else {
 			/* Get owner MBUF from indirect buffer */
 			mi = rte_mbuf_from_indirect(cur_seg);
@@ -503,11 +507,11 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 						   mempool_to_bpid(mi->pool));
 				rte_mbuf_refcnt_update(mi, 1);
 			}
-			prev_seg = cur_seg;
-			cur_seg = cur_seg->next;
-			prev_seg->next = NULL;
-			rte_pktmbuf_free(prev_seg);
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 		}
+		cur_seg = cur_seg->next;
 	}
 	DPAA2_SG_SET_FINAL(sge, true);
 	return 0;
@@ -515,11 +519,19 @@ eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 
 static void
 eth_mbuf_to_fd(struct rte_mbuf *mbuf,
-	       struct qbman_fd *fd, uint16_t bpid) __rte_unused;
+	       struct qbman_fd *fd,
+	       struct sw_buf_free *buf_to_free,
+	       uint32_t *free_count,
+	       uint32_t pkt_id,
+	       uint16_t bpid) __rte_unused;
 
 static void __rte_noinline __rte_hot
 eth_mbuf_to_fd(struct rte_mbuf *mbuf,
-	       struct qbman_fd *fd, uint16_t bpid)
+	       struct qbman_fd *fd,
+	       struct sw_buf_free *buf_to_free,
+	       uint32_t *free_count,
+	       uint32_t pkt_id,
+	       uint16_t bpid)
 {
 	DPAA2_MBUF_TO_CONTIG_FD(mbuf, fd, bpid);
 
@@ -540,6 +552,9 @@ eth_mbuf_to_fd(struct rte_mbuf *mbuf,
 				(void **)&mbuf, 1, 0);
 #endif
 	} else if (RTE_MBUF_HAS_EXTBUF(mbuf)) {
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 		DPAA2_SET_FD_IVP(fd);
 	} else {
 		struct rte_mbuf *mi;
@@ -549,7 +564,10 @@ eth_mbuf_to_fd(struct rte_mbuf *mbuf,
 			DPAA2_SET_FD_IVP(fd);
 		else
 			rte_mbuf_refcnt_update(mi, 1);
-		rte_pktmbuf_free(mbuf);
+
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 	}
 }
 
@@ -1226,7 +1244,8 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	struct rte_eth_dev_data *eth_data = dpaa2_q->eth_data;
 	struct dpaa2_dev_priv *priv = eth_data->dev_private;
 	uint32_t flags[MAX_TX_RING_SLOTS] = {0};
-	struct rte_mbuf **orig_bufs = bufs;
+	struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
 		ret = dpaa2_affine_qbman_swp();
@@ -1324,11 +1343,17 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 					mp = (*bufs)->pool;
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
+							      buf_to_free,
+							      &free_count,
+							      loop,
 							      mempool_to_bpid(mp)))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
-						       &fd_arr[loop], 0);
+							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop, 0);
 				}
 				bufs++;
 #ifdef RTE_LIBRTE_IEEE1588
@@ -1373,11 +1398,17 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop,
 							bpid))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
-						       &fd_arr[loop], bpid);
+							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop, bpid);
 				}
 			}
 #ifdef RTE_LIBRTE_IEEE1588
@@ -1410,12 +1441,9 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	}
 	dpaa2_q->tx_pkts += num_tx;
 
-	loop = 0;
-	while (loop < num_tx) {
-		if (unlikely(RTE_MBUF_HAS_EXTBUF(*orig_bufs)))
-			rte_pktmbuf_free(*orig_bufs);
-		orig_bufs++;
-		loop++;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
 	}
 
 	return num_tx;
@@ -1445,12 +1473,9 @@ dpaa2_dev_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 skip_tx:
 	dpaa2_q->tx_pkts += num_tx;
 
-	loop = 0;
-	while (loop < num_tx) {
-		if (unlikely(RTE_MBUF_HAS_EXTBUF(*orig_bufs)))
-			rte_pktmbuf_free(*orig_bufs);
-		orig_bufs++;
-		loop++;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
 	}
 
 	return num_tx;
@@ -1523,7 +1548,7 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 		struct rte_mbuf **bufs, uint16_t nb_pkts)
 {
 	/* Function to transmit the frames to multiple queues respectively.*/
-	uint32_t loop, retry_count;
+	uint32_t loop, i, retry_count;
 	int32_t ret;
 	struct qbman_fd fd_arr[MAX_TX_RING_SLOTS];
 	uint32_t frames_to_send, num_free_eq_desc = 0;
@@ -1536,6 +1561,8 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 	struct rte_eth_dev_data *eth_data;
 	struct dpaa2_dev_priv *priv;
 	struct dpaa2_queue *order_sendq;
+	struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
 		ret = dpaa2_affine_qbman_swp();
@@ -1647,11 +1674,17 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 			if (unlikely((*bufs)->nb_segs > 1)) {
 				if (eth_mbuf_to_sg_fd(*bufs,
 						      &fd_arr[loop],
+						      buf_to_free,
+						      &free_count,
+						      loop,
 						      bpid))
 					goto send_frames;
 			} else {
 				eth_mbuf_to_fd(*bufs,
-					       &fd_arr[loop], bpid);
+						&fd_arr[loop],
+						buf_to_free,
+						&free_count,
+						loop, bpid);
 			}
 		}
 
@@ -1676,6 +1709,10 @@ dpaa2_dev_tx_multi_txq_ordered(void **queue,
 		}
 	}
 
+	for (i = 0; i < free_count; i++) {
+		if (buf_to_free[i].pkt_id < loop)
+			rte_pktmbuf_free_seg(buf_to_free[i].seg);
+	}
 	return loop;
 }
 
@@ -1698,6 +1735,8 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	int32_t ret;
 	uint16_t num_tx = 0;
 	uint16_t bpid;
+	struct sw_buf_free buf_to_free[DPAA2_MAX_SGS * dpaa2_dqrr_size];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA2_PER_LCORE_DPIO)) {
 		ret = dpaa2_affine_qbman_swp();
@@ -1810,11 +1849,17 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 				if (unlikely((*bufs)->nb_segs > 1)) {
 					if (eth_mbuf_to_sg_fd(*bufs,
 							      &fd_arr[loop],
+							      buf_to_free,
+							      &free_count,
+							      loop,
 							      bpid))
 						goto send_n_return;
 				} else {
 					eth_mbuf_to_fd(*bufs,
-						       &fd_arr[loop], bpid);
+							&fd_arr[loop],
+							buf_to_free,
+							&free_count,
+							loop, bpid);
 				}
 			}
 			bufs++;
@@ -1843,6 +1888,11 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 		nb_pkts -= loop;
 	}
 	dpaa2_q->tx_pkts += num_tx;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
+	}
+
 	return num_tx;
 
 send_n_return:
@@ -1867,6 +1917,11 @@ dpaa2_dev_tx_ordered(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 	}
 skip_tx:
 	dpaa2_q->tx_pkts += num_tx;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < num_tx)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
+	}
+
 	return num_tx;
 }
 
-- 
2.25.1

[PATCH v2 09/16] bus/fslmc: add timeout in MC send command API

[Gagandeep Singh]

From: Rohit Raj <rohit.raj@nxp.com>

Adding one second timeout in MC send command API to ensure it doesn't
gets stuck in case of failure.

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/bus/fslmc/mc/mc_sys.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/drivers/bus/fslmc/mc/mc_sys.c b/drivers/bus/fslmc/mc/mc_sys.c
index efafdc3101..ab9a074835 100644
--- a/drivers/bus/fslmc/mc/mc_sys.c
+++ b/drivers/bus/fslmc/mc/mc_sys.c
@@ -1,13 +1,14 @@
 /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0)
  *
  * Copyright 2013-2015 Freescale Semiconductor Inc.
- * Copyright 2017 NXP
+ * Copyright 2017,2022 NXP
  *
  */
 #include <fsl_mc_sys.h>
 #include <fsl_mc_cmd.h>
 
 #include <rte_spinlock.h>
+#include <rte_cycles.h>
 
 /** User space framework uses MC Portal in shared mode. Following change
  * introduces lock in MC FLIB
@@ -54,7 +55,7 @@ static int mc_status_to_error(enum mc_cmd_status status)
 int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 {
 	enum mc_cmd_status status;
-	uint64_t response;
+	uint64_t response, start_time, total_time, time_to_wait;
 
 	if (!mc_io || !mc_io->regs)
 		return -EACCES;
@@ -64,15 +65,20 @@ int mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd)
 
 	mc_write_command(mc_io->regs, cmd);
 
+	/* Wait for one second. rte_get_timer_hz() returns frequency of CPU */
+	time_to_wait = rte_get_timer_hz();
+	total_time = 0;
+	start_time = rte_get_timer_cycles();
+
 	/* Spin until status changes */
 	do {
 		response = ioread64(mc_io->regs);
 		status = mc_cmd_read_status((struct mc_command *)&response);
+		total_time = rte_get_timer_cycles() - start_time;
+	} while (status == MC_CMD_STATUS_READY && total_time <= time_to_wait);
 
-		/* --- Call wait function here to prevent blocking ---
-		 * Change the loop condition accordingly to exit on timeout.
-		 */
-	} while (status == MC_CMD_STATUS_READY);
+	if (status == MC_CMD_STATUS_READY)
+		return mc_status_to_error(MC_CMD_STATUS_TIMEOUT);
 
 	/* Read the response back into the command buffer */
 	mc_read_response(mc_io->regs, cmd);
-- 
2.25.1

[PATCH v2 10/16] net/dpaa: fix Jumbo packet Rx in case of VSP

[Gagandeep Singh]

From: Rohit Raj <rohit.raj@nxp.com>

For packet length of size more than 2K bytes, segmented packets were
being received in DPDK even if mbuf size was greater than packet
length. This is due to the configuration in VSP.

This patch fixes the issue by configuring the VSP according to the
mbuf size configured during mempool configuration.

Fixes: e4abd4ff183c ("net/dpaa: support virtual storage profile")
Cc: stable@dpdk.org

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.c |  5 ++---
 drivers/net/dpaa/dpaa_flow.c   | 13 ++++++-------
 drivers/net/dpaa/dpaa_flow.h   |  5 +++--
 3 files changed, 11 insertions(+), 12 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index c4aac424b4..3b4d6575c9 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -989,8 +989,7 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
 	} else {
 		DPAA_PMD_WARN("The requested maximum Rx packet size (%u) is"
 		     " larger than a single mbuf (%u) and scattered"
-		     " mode has not been requested",
-		     max_rx_pktlen, buffsz - RTE_PKTMBUF_HEADROOM);
+		     " mode has not been requested", max_rx_pktlen, buffsz);
 	}
 
 	dpaa_intf->bp_info = DPAA_MEMPOOL_TO_POOL_INFO(mp);
@@ -1005,7 +1004,7 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
 		if (vsp_id >= 0) {
 			ret = dpaa_port_vsp_update(dpaa_intf, fmc_q, vsp_id,
 					DPAA_MEMPOOL_TO_POOL_INFO(mp)->bpid,
-					fif);
+					fif, buffsz + RTE_PKTMBUF_HEADROOM);
 			if (ret) {
 				DPAA_PMD_ERR("dpaa_port_vsp_update failed");
 				return ret;
diff --git a/drivers/net/dpaa/dpaa_flow.c b/drivers/net/dpaa/dpaa_flow.c
index 1ccd036027..690ba6bcb3 100644
--- a/drivers/net/dpaa/dpaa_flow.c
+++ b/drivers/net/dpaa/dpaa_flow.c
@@ -939,7 +939,7 @@ int dpaa_fm_term(void)
 
 static int dpaa_port_vsp_configure(struct dpaa_if *dpaa_intf,
 		uint8_t vsp_id, t_handle fman_handle,
-		struct fman_if *fif)
+		struct fman_if *fif, u32 mbuf_data_room_size)
 {
 	t_fm_vsp_params vsp_params;
 	t_fm_buffer_prefix_content buf_prefix_cont;
@@ -976,10 +976,8 @@ static int dpaa_port_vsp_configure(struct dpaa_if *dpaa_intf,
 		return -1;
 	}
 	vsp_params.ext_buf_pools.num_of_pools_used = 1;
-	vsp_params.ext_buf_pools.ext_buf_pool[0].id =
-		dpaa_intf->vsp_bpid[vsp_id];
-	vsp_params.ext_buf_pools.ext_buf_pool[0].size =
-		RTE_MBUF_DEFAULT_BUF_SIZE;
+	vsp_params.ext_buf_pools.ext_buf_pool[0].id = dpaa_intf->vsp_bpid[vsp_id];
+	vsp_params.ext_buf_pools.ext_buf_pool[0].size = mbuf_data_room_size;
 
 	dpaa_intf->vsp_handle[vsp_id] = fm_vsp_config(&vsp_params);
 	if (!dpaa_intf->vsp_handle[vsp_id]) {
@@ -1023,7 +1021,7 @@ static int dpaa_port_vsp_configure(struct dpaa_if *dpaa_intf,
 
 int dpaa_port_vsp_update(struct dpaa_if *dpaa_intf,
 		bool fmc_mode, uint8_t vsp_id, uint32_t bpid,
-		struct fman_if *fif)
+		struct fman_if *fif, u32 mbuf_data_room_size)
 {
 	int ret = 0;
 	t_handle fman_handle;
@@ -1054,7 +1052,8 @@ int dpaa_port_vsp_update(struct dpaa_if *dpaa_intf,
 
 	dpaa_intf->vsp_bpid[vsp_id] = bpid;
 
-	return dpaa_port_vsp_configure(dpaa_intf, vsp_id, fman_handle, fif);
+	return dpaa_port_vsp_configure(dpaa_intf, vsp_id, fman_handle, fif,
+				       mbuf_data_room_size);
 }
 
 int dpaa_port_vsp_cleanup(struct dpaa_if *dpaa_intf, struct fman_if *fif)
diff --git a/drivers/net/dpaa/dpaa_flow.h b/drivers/net/dpaa/dpaa_flow.h
index f5e131acfa..4742b8dd0a 100644
--- a/drivers/net/dpaa/dpaa_flow.h
+++ b/drivers/net/dpaa/dpaa_flow.h
@@ -1,5 +1,5 @@
 /* SPDX-License-Identifier: BSD-3-Clause
- * Copyright 2017,2019 NXP
+ * Copyright 2017,2019,2022 NXP
  */
 
 #ifndef __DPAA_FLOW_H__
@@ -11,7 +11,8 @@ int dpaa_fm_config(struct rte_eth_dev *dev, uint64_t req_dist_set);
 int dpaa_fm_deconfig(struct dpaa_if *dpaa_intf, struct fman_if *fif);
 void dpaa_write_fm_config_to_file(void);
 int dpaa_port_vsp_update(struct dpaa_if *dpaa_intf,
-	bool fmc_mode, uint8_t vsp_id, uint32_t bpid, struct fman_if *fif);
+	bool fmc_mode, uint8_t vsp_id, uint32_t bpid, struct fman_if *fif,
+	u32 mbuf_data_room_size);
 int dpaa_port_vsp_cleanup(struct dpaa_if *dpaa_intf, struct fman_if *fif);
 int dpaa_port_fmc_init(struct fman_if *fif,
 		       uint32_t *fqids, int8_t *vspids, int max_nb_rxq);
-- 
2.25.1

[PATCH v2 11/16] doc: add kernel version compatible information

[Gagandeep Singh]

DPAA driver has dependency on kernel to perform various functionalities.
So kernel and DPDK version should be compatible for proper working.

This patch updates the DPAA guide with the information that user can
refer to find the compatible kernel version.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
---
 doc/guides/nics/dpaa.rst | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/guides/nics/dpaa.rst b/doc/guides/nics/dpaa.rst
index 9dad612058..5e05538ffb 100644
--- a/doc/guides/nics/dpaa.rst
+++ b/doc/guides/nics/dpaa.rst
@@ -187,6 +187,10 @@ See :doc:`../platform/dpaa` for setup information
 
 - Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>`
   to setup the basic DPDK environment.
+- DPAA driver has dependency on kernel to perform various functionalities.
+  So kernel and DPDK version should be compatible for proper working.
+  Refer release notes of NXP SDK guide to match the versions `NXP LSDK GUIDE
+  <https://www.nxp.com/design/software/embedded-software/linux-software-and-development-tools/layerscape-software-development-kit-v21-08:LAYERSCAPE-SDK>`_.
 
 .. note::
 
-- 
2.25.1

[PATCH v2 12/16] bus/dpaa: pass interface name as a string instead of pointer

[Gagandeep Singh]

From: Rohit Raj <rohit.raj@nxp.com>

Due to change in latest kernel, passing the interface name to
kernel through IOCTL as string instead of character pointer.

Signed-off-by: Rohit Raj <rohit.raj@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/bus/dpaa/base/qbman/process.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/bus/dpaa/base/qbman/process.c b/drivers/bus/dpaa/base/qbman/process.c
index 9bc92681cd..3504ec97db 100644
--- a/drivers/bus/dpaa/base/qbman/process.c
+++ b/drivers/bus/dpaa/base/qbman/process.c
@@ -302,7 +302,7 @@ int bman_free_raw_portal(struct dpaa_raw_portal *portal)
 	_IOW(DPAA_IOCTL_MAGIC, 0x0E, struct usdpaa_ioctl_link_status)
 
 #define DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT \
-	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char*)
+	_IOW(DPAA_IOCTL_MAGIC, 0x0F, char[IF_NAME_MAX_LEN])
 
 int dpaa_intr_enable(char *if_name, int efd)
 {
@@ -330,7 +330,7 @@ int dpaa_intr_disable(char *if_name)
 	if (ret)
 		return ret;
 
-	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT, &if_name);
+	ret = ioctl(fd, DPAA_IOCTL_DISABLE_LINK_STATUS_INTERRUPT, if_name);
 	if (ret) {
 		if (errno == EINVAL)
 			printf("Failed to disable interrupt: Not Supported\n");
@@ -472,7 +472,7 @@ int dpaa_update_link_speed(char *if_name, int link_speed, int link_duplex)
 }
 
 #define DPAA_IOCTL_RESTART_LINK_AUTONEG \
-	_IOW(DPAA_IOCTL_MAGIC, 0x13, char *)
+	_IOW(DPAA_IOCTL_MAGIC, 0x13, char[IF_NAME_MAX_LEN])
 
 int dpaa_restart_link_autoneg(char *if_name)
 {
@@ -481,7 +481,7 @@ int dpaa_restart_link_autoneg(char *if_name)
 	if (ret)
 		return ret;
 
-	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, &if_name);
+	ret = ioctl(fd, DPAA_IOCTL_RESTART_LINK_AUTONEG, if_name);
 	if (ret) {
 		if (errno == EINVAL)
 			printf("Failed to restart autoneg: Not Supported\n");
-- 
2.25.1

[PATCH v2 13/16] net/dpaa: use internal mempool for SG table

[Gagandeep Singh]

Creating and using driver's mempool for
allocating the SG table memory required for
FD creation.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.c | 18 ++++++++++++++++++
 drivers/net/dpaa/dpaa_ethdev.h |  9 +++++++++
 drivers/net/dpaa/dpaa_rxtx.c   |  9 ++++-----
 3 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index 3b4d6575c9..a6c86113d1 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -133,6 +133,8 @@ static const struct rte_dpaa_xstats_name_off dpaa_xstats_strings[] = {
 };
 
 static struct rte_dpaa_driver rte_dpaa_pmd;
+int dpaa_valid_dev;
+struct rte_mempool *dpaa_tx_sg_pool;
 
 static int
 dpaa_eth_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info);
@@ -2223,7 +2225,20 @@ rte_dpaa_probe(struct rte_dpaa_driver *dpaa_drv,
 	/* Invoke PMD device initialization function */
 	diag = dpaa_dev_init(eth_dev);
 	if (diag == 0) {
+		if (!dpaa_tx_sg_pool) {
+			dpaa_tx_sg_pool =
+				rte_pktmbuf_pool_create("dpaa_mbuf_tx_sg_pool",
+				DPAA_POOL_SIZE,
+				DPAA_POOL_CACHE_SIZE, 0,
+				DPAA_MAX_SGS * sizeof(struct qm_sg_entry),
+				rte_socket_id());
+			if (dpaa_tx_sg_pool == NULL) {
+				DPAA_PMD_ERR("SG pool creation failed\n");
+				return -ENOMEM;
+			}
+		}
 		rte_eth_dev_probing_finish(eth_dev);
+		dpaa_valid_dev++;
 		return 0;
 	}
 
@@ -2241,6 +2256,9 @@ rte_dpaa_remove(struct rte_dpaa_device *dpaa_dev)
 
 	eth_dev = dpaa_dev->eth_dev;
 	dpaa_eth_dev_close(eth_dev);
+	dpaa_valid_dev--;
+	if (!dpaa_valid_dev)
+		rte_mempool_free(dpaa_tx_sg_pool);
 	ret = rte_eth_dev_release_port(eth_dev);
 
 	return ret;
diff --git a/drivers/net/dpaa/dpaa_ethdev.h b/drivers/net/dpaa/dpaa_ethdev.h
index 6fdd57dbc3..f9c0554530 100644
--- a/drivers/net/dpaa/dpaa_ethdev.h
+++ b/drivers/net/dpaa/dpaa_ethdev.h
@@ -33,6 +33,13 @@
 
 #define DPAA_SGT_MAX_ENTRIES 16 /* maximum number of entries in SG Table */
 
+/* Maximum SG segments supported on all cores*/
+#define DPAA_MAX_SGS 128
+/* SG pool size */
+#define DPAA_POOL_SIZE 2048
+/* SG pool cache size */
+#define DPAA_POOL_CACHE_SIZE 256
+
 /* RX queue tail drop threshold (CGR Based) in frame count */
 #define CGR_RX_PERFQ_THRESH 256
 #define CGR_TX_CGR_THRESH 512
@@ -103,6 +110,8 @@
 
 #define FMC_FILE "/tmp/fmc.bin"
 
+extern struct rte_mempool *dpaa_tx_sg_pool;
+
 /* Each network interface is represented by one of these */
 struct dpaa_if {
 	int valid;
diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index 22205cec30..e23206bf5c 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -803,8 +803,7 @@ uint16_t dpaa_eth_queue_rx(void *q,
 
 static int
 dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
-		struct qm_fd *fd,
-		struct dpaa_bp_info *bp_info)
+		struct qm_fd *fd)
 {
 	struct rte_mbuf *cur_seg = mbuf, *prev_seg = NULL;
 	struct rte_mbuf *temp, *mi;
@@ -813,7 +812,7 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 
 	DPAA_DP_LOG(DEBUG, "Creating SG FD to transmit");
 
-	temp = rte_pktmbuf_alloc(bp_info->mp);
+	temp = rte_pktmbuf_alloc(dpaa_tx_sg_pool);
 	if (!temp) {
 		DPAA_PMD_ERR("Failure in allocation of mbuf");
 		return -1;
@@ -849,7 +848,7 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 	fd->format = QM_FD_SG;
 	fd->addr = temp->buf_iova;
 	fd->offset = temp->data_off;
-	fd->bpid = bp_info ? bp_info->bpid : 0xff;
+	fd->bpid = DPAA_MEMPOOL_TO_BPID(dpaa_tx_sg_pool);
 	fd->length20 = mbuf->pkt_len;
 
 	while (i < DPAA_SGT_MAX_ENTRIES) {
@@ -967,7 +966,7 @@ tx_on_dpaa_pool(struct rte_mbuf *mbuf,
 		tx_on_dpaa_pool_unsegmented(mbuf, bp_info, fd_arr);
 	} else if (mbuf->nb_segs > 1 &&
 		   mbuf->nb_segs <= DPAA_SGT_MAX_ENTRIES) {
-		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr, bp_info)) {
+		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr)) {
 			DPAA_PMD_DEBUG("Unable to create Scatter Gather FD");
 			return 1;
 		}
-- 
2.25.1

[PATCH v2 14/16] bus/dpaa: mempool ops registration change

[Gagandeep Singh]

moving the mempool ops registration before DPAA
devices probe so that device probe functions can
also be able to use mempool operations.

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/bus/dpaa/dpaa_bus.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c
index 682427ba2c..3ca5147761 100644
--- a/drivers/bus/dpaa/dpaa_bus.c
+++ b/drivers/bus/dpaa/dpaa_bus.c
@@ -655,6 +655,11 @@ rte_dpaa_bus_probe(void)
 	if (TAILQ_EMPTY(&rte_dpaa_bus.device_list))
 		return 0;
 
+	/* Register DPAA mempool ops only if any DPAA device has
+	 * been detected.
+	 */
+	rte_mbuf_set_platform_mempool_ops(DPAA_MEMPOOL_OPS_NAME);
+
 	svr_file = fopen(DPAA_SOC_ID_FILE, "r");
 	if (svr_file) {
 		if (fscanf(svr_file, "svr:%x", &svr_ver) > 0)
@@ -704,11 +709,6 @@ rte_dpaa_bus_probe(void)
 		}
 	}
 
-	/* Register DPAA mempool ops only if any DPAA device has
-	 * been detected.
-	 */
-	rte_mbuf_set_platform_mempool_ops(DPAA_MEMPOOL_OPS_NAME);
-
 	return 0;
 }
 
-- 
2.25.1

[PATCH v2 15/16] net/dpaa: fix buffer free on transmit SG packets

[Gagandeep Singh]

When using SG list to TX with external and direct buffers,
HW free direct buffers and driver free external buffers.

Software scans the complete SG mbuf list to find the external
buffers to free, but this is wrong as hardware can free the
direct buffers if any present in the list and same can be
re-allocated for other purpose in multi thread or high speed
running traffic environment with new data in it. So the software
which is scanning the SG mbuf list, if that list has any direct
buffer present then that direct buffer's next pointer can give
wrong pointer value, if already freed by hardware which
can do the mempool corruption or memory leak.

In this patch instead of relying on user given SG mbuf list
we are storing the buffers in an internal list which will
be scanned by driver after transmit to free non-direct
buffers.

This patch also fixes below issues.

Driver is freeing complete SG list by checking external buffer
flag in first segment only, but external buffer can be attached
to any of the segment. Because of this, driver either can double
free buffers or there can be memory leak.

In case of indirect buffers, driver is modifying the original
buffer list to free the indirect buffers but this original buffer
list is being used by driver even after transmit packets for
non-direct buffer cleanup. This can cause the buffer leak issue.

Fixes: f191d5abda54 ("net/dpaa: support external buffers in Tx")
Cc: stable@dpdk.org

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa/dpaa_ethdev.h | 10 ++++++
 drivers/net/dpaa/dpaa_rxtx.c   | 61 ++++++++++++++++++++++------------
 2 files changed, 49 insertions(+), 22 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_ethdev.h b/drivers/net/dpaa/dpaa_ethdev.h
index f9c0554530..502c1c88b8 100644
--- a/drivers/net/dpaa/dpaa_ethdev.h
+++ b/drivers/net/dpaa/dpaa_ethdev.h
@@ -112,6 +112,16 @@
 
 extern struct rte_mempool *dpaa_tx_sg_pool;
 
+/* structure to free external and indirect
+ * buffers.
+ */
+struct dpaa_sw_buf_free {
+	/* To which packet this segment belongs */
+	uint16_t pkt_id;
+	/* The actual segment */
+	struct rte_mbuf *seg;
+};
+
 /* Each network interface is represented by one of these */
 struct dpaa_if {
 	int valid;
diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index e23206bf5c..4d285b4f38 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -803,9 +803,12 @@ uint16_t dpaa_eth_queue_rx(void *q,
 
 static int
 dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
-		struct qm_fd *fd)
+		struct qm_fd *fd,
+		struct dpaa_sw_buf_free *free_buf,
+		uint32_t *free_count,
+		uint32_t pkt_id)
 {
-	struct rte_mbuf *cur_seg = mbuf, *prev_seg = NULL;
+	struct rte_mbuf *cur_seg = mbuf;
 	struct rte_mbuf *temp, *mi;
 	struct qm_sg_entry *sg_temp, *sgt;
 	int i = 0;
@@ -869,10 +872,11 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 				sg_temp->bpid =
 					DPAA_MEMPOOL_TO_BPID(cur_seg->pool);
 			}
-			cur_seg = cur_seg->next;
 		} else if (RTE_MBUF_HAS_EXTBUF(cur_seg)) {
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 			sg_temp->bpid = 0xff;
-			cur_seg = cur_seg->next;
 		} else {
 			/* Get owner MBUF from indirect buffer */
 			mi = rte_mbuf_from_indirect(cur_seg);
@@ -885,11 +889,11 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 				sg_temp->bpid = DPAA_MEMPOOL_TO_BPID(mi->pool);
 				rte_mbuf_refcnt_update(mi, 1);
 			}
-			prev_seg = cur_seg;
-			cur_seg = cur_seg->next;
-			prev_seg->next = NULL;
-			rte_pktmbuf_free(prev_seg);
+			free_buf[*free_count].seg = cur_seg;
+			free_buf[*free_count].pkt_id = pkt_id;
+			++*free_count;
 		}
+		cur_seg = cur_seg->next;
 		if (cur_seg == NULL) {
 			sg_temp->final = 1;
 			cpu_to_hw_sg(sg_temp);
@@ -904,7 +908,10 @@ dpaa_eth_mbuf_to_sg_fd(struct rte_mbuf *mbuf,
 static inline void
 tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 			    struct dpaa_bp_info *bp_info,
-			    struct qm_fd *fd_arr)
+			    struct qm_fd *fd_arr,
+			    struct dpaa_sw_buf_free *buf_to_free,
+			    uint32_t *free_count,
+			    uint32_t pkt_id)
 {
 	struct rte_mbuf *mi = NULL;
 
@@ -923,6 +930,9 @@ tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 			DPAA_MBUF_TO_CONTIG_FD(mbuf, fd_arr, bp_info->bpid);
 		}
 	} else if (RTE_MBUF_HAS_EXTBUF(mbuf)) {
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 		DPAA_MBUF_TO_CONTIG_FD(mbuf, fd_arr,
 				bp_info ? bp_info->bpid : 0xff);
 	} else {
@@ -946,7 +956,9 @@ tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 			DPAA_MBUF_TO_CONTIG_FD(mbuf, fd_arr,
 						bp_info ? bp_info->bpid : 0xff);
 		}
-		rte_pktmbuf_free(mbuf);
+		buf_to_free[*free_count].seg = mbuf;
+		buf_to_free[*free_count].pkt_id = pkt_id;
+		++*free_count;
 	}
 
 	if (mbuf->ol_flags & DPAA_TX_CKSUM_OFFLOAD_MASK)
@@ -957,16 +969,21 @@ tx_on_dpaa_pool_unsegmented(struct rte_mbuf *mbuf,
 static inline uint16_t
 tx_on_dpaa_pool(struct rte_mbuf *mbuf,
 		struct dpaa_bp_info *bp_info,
-		struct qm_fd *fd_arr)
+		struct qm_fd *fd_arr,
+		struct dpaa_sw_buf_free *buf_to_free,
+		uint32_t *free_count,
+		uint32_t pkt_id)
 {
 	DPAA_DP_LOG(DEBUG, "BMAN offloaded buffer, mbuf: %p", mbuf);
 
 	if (mbuf->nb_segs == 1) {
 		/* Case for non-segmented buffers */
-		tx_on_dpaa_pool_unsegmented(mbuf, bp_info, fd_arr);
+		tx_on_dpaa_pool_unsegmented(mbuf, bp_info, fd_arr,
+				buf_to_free, free_count, pkt_id);
 	} else if (mbuf->nb_segs > 1 &&
 		   mbuf->nb_segs <= DPAA_SGT_MAX_ENTRIES) {
-		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr)) {
+		if (dpaa_eth_mbuf_to_sg_fd(mbuf, fd_arr, buf_to_free,
+					   free_count, pkt_id)) {
 			DPAA_PMD_DEBUG("Unable to create Scatter Gather FD");
 			return 1;
 		}
@@ -1070,7 +1087,8 @@ dpaa_eth_queue_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 	uint16_t state;
 	int ret, realloc_mbuf = 0;
 	uint32_t seqn, index, flags[DPAA_TX_BURST_SIZE] = {0};
-	struct rte_mbuf **orig_bufs = bufs;
+	struct dpaa_sw_buf_free buf_to_free[DPAA_MAX_SGS * DPAA_MAX_DEQUEUE_NUM_FRAMES];
+	uint32_t free_count = 0;
 
 	if (unlikely(!DPAA_PER_LCORE_PORTAL)) {
 		ret = rte_dpaa_portal_init((void *)0);
@@ -1153,7 +1171,10 @@ dpaa_eth_queue_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 			}
 indirect_buf:
 			state = tx_on_dpaa_pool(mbuf, bp_info,
-						&fd_arr[loop]);
+						&fd_arr[loop],
+						buf_to_free,
+						&free_count,
+						loop);
 			if (unlikely(state)) {
 				/* Set frames_to_send & nb_bufs so
 				 * that packets are transmitted till
@@ -1178,13 +1199,9 @@ dpaa_eth_queue_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
 
 	DPAA_DP_LOG(DEBUG, "Transmitted %d buffers on queue: %p", sent, q);
 
-
-	loop = 0;
-	while (loop < sent) {
-		if (unlikely(RTE_MBUF_HAS_EXTBUF(*orig_bufs)))
-			rte_pktmbuf_free(*orig_bufs);
-		orig_bufs++;
-		loop++;
+	for (loop = 0; loop < free_count; loop++) {
+		if (buf_to_free[loop].pkt_id < sent)
+			rte_pktmbuf_free_seg(buf_to_free[loop].seg);
 	}
 
 	return sent;
-- 
2.25.1

[PATCH v2 16/16] net/dpaa: fix buffer free in slow path

[Gagandeep Singh]

If there is any error in packet or taildrop feature is enabled,
HW can reject those packets and put them in error queue. Driver
poll this error queue to free the buffers.
DPAA driver has an issue while freeing these rejected buffers. In
case of scatter gather packets, it is preparing the mbuf SG list
by scanning the HW descriptors and once the mbuf SG list prepared,
it free only first segment of the mbuf SG list by calling the
API rte_pktmbuf_free_seg(), This will leak the memory of other
segments and mempool can be empty.

Also there is one more issue, external buffer's memory may not
belong to mempool so driver itself free the external buffer
after successfully send the packet to HW to transmit instead of
let the HW to free it. So transmit function free all the external
buffers. But driver has no check for external buffers
while freeing the rejected buffers and this can do double free the
memory which can corrupt the user pool and crashes and undefined
behaviour of system can be seen.

This patch fixes the above mentioned issue by checking each
and every segment and freeing all the segments except external.

Fixes: 9124e65dd3eb ("net/dpaa: enable Tx queue taildrop")
Cc: stable@dpdk.org

Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/net/dpaa/dpaa_rxtx.c | 23 ++++++++---------------
 1 file changed, 8 insertions(+), 15 deletions(-)

diff --git a/drivers/net/dpaa/dpaa_rxtx.c b/drivers/net/dpaa/dpaa_rxtx.c
index 4d285b4f38..ce4f3d6c85 100644
--- a/drivers/net/dpaa/dpaa_rxtx.c
+++ b/drivers/net/dpaa/dpaa_rxtx.c
@@ -455,7 +455,7 @@ dpaa_free_mbuf(const struct qm_fd *fd)
 	bp_info = DPAA_BPID_TO_POOL_INFO(fd->bpid);
 	format = (fd->opaque & DPAA_FD_FORMAT_MASK) >> DPAA_FD_FORMAT_SHIFT;
 	if (unlikely(format == qm_fd_sg)) {
-		struct rte_mbuf *first_seg, *prev_seg, *cur_seg, *temp;
+		struct rte_mbuf *first_seg, *cur_seg;
 		struct qm_sg_entry *sgt, *sg_temp;
 		void *vaddr, *sg_vaddr;
 		int i = 0;
@@ -469,32 +469,25 @@ dpaa_free_mbuf(const struct qm_fd *fd)
 		sgt = vaddr + fd_offset;
 		sg_temp = &sgt[i++];
 		hw_sg_to_cpu(sg_temp);
-		temp = (struct rte_mbuf *)
-			((char *)vaddr - bp_info->meta_data_size);
 		sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
 						qm_sg_entry_get64(sg_temp));
-
 		first_seg = (struct rte_mbuf *)((char *)sg_vaddr -
 						bp_info->meta_data_size);
 		first_seg->nb_segs = 1;
-		prev_seg = first_seg;
 		while (i < DPAA_SGT_MAX_ENTRIES) {
 			sg_temp = &sgt[i++];
 			hw_sg_to_cpu(sg_temp);
-			sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
+			if (sg_temp->bpid != 0xFF) {
+				bp_info = DPAA_BPID_TO_POOL_INFO(sg_temp->bpid);
+				sg_vaddr = DPAA_MEMPOOL_PTOV(bp_info,
 						qm_sg_entry_get64(sg_temp));
-			cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
+				cur_seg = (struct rte_mbuf *)((char *)sg_vaddr -
 						      bp_info->meta_data_size);
-			first_seg->nb_segs += 1;
-			prev_seg->next = cur_seg;
-			if (sg_temp->final) {
-				cur_seg->next = NULL;
-				break;
+				rte_pktmbuf_free_seg(cur_seg);
 			}
-			prev_seg = cur_seg;
+			if (sg_temp->final)
+				break;
 		}
-
-		rte_pktmbuf_free_seg(temp);
 		rte_pktmbuf_free_seg(first_seg);
 		return 0;
 	}
-- 
2.25.1

Re: [PATCH v2 00/16] DPAA and DPAA2 driver changes

[Ferruh Yigit]

On 10/7/2022 4:27 AM, Gagandeep Singh wrote:
> This series have list of patch for bug fixes and
> some enhancements to DPAA1 and DPAA2 net drivers.
> 
> v2-change-log:
> * update commit messages
> * update DPAA NIC doc for kernel version matching
> * update DPAA supported packet type info
> * Freeing the internal pool on last device remove
> * fixes syntax for some comments
> 
> Apeksha Gupta (2):
>    net/enetfec: fix restart issue
>    net/enetfec: fix buffer leak issue
> 
> Brick Yang (1):
>    net/dpaa2: check free enqueue descriptors before Tx
> 
> Gagandeep Singh (8):
>    net/dpaa: support ESP packet type in packet parsing
>    net/dpaa2: use internal mempool for SG table
>    net/dpaa2: fix buffer free on transmit SG packets
>    doc: add kernel version compatible information
>    net/dpaa: use internal mempool for SG table
>    bus/dpaa: mempool ops registration change
>    net/dpaa: fix buffer free on transmit SG packets
>    net/dpaa: fix buffer free in slow path
> 
> Rohit Raj (3):
>    bus/fslmc: add timeout in MC send command API
>    net/dpaa: fix Jumbo packet Rx in case of VSP
>    bus/dpaa: pass interface name as a string instead of pointer
> 
> Vanshika Shukla (2):
>    bus/dpaa: use non-block mode for FD open
>    net/dpaa2: fix dpdmux configuration for error behaviour
> 

Series applied to dpdk-next-net/main, thanks.