[dpdk-dev] [PATCH 0/4] net/bnxt: rxq stop/start fixes

[dpdk-dev] [PATCH 0/4] net/bnxt: rxq stop/start fixes

[Lance Richardson]

This patch set fixes several problems involving stopping/starting
receive queues in the bnxt PMD.

Lance Richardson (4):
  net/bnxt: fix RSS table update for start/stop rxq
  net/bnxt: keep consistent rxq start/stop state
  net/bnxt: release hwrm lock before returning
  net/bnxt: fix rxq start/stop for Thor based NICs

 drivers/net/bnxt/bnxt_hwrm.c | 31 +++++++++++++++++----
 drivers/net/bnxt/bnxt_rxq.c  | 52 +++++++++++++++++++++++++++++-------
 drivers/net/bnxt/bnxt_vnic.c |  1 +
 3 files changed, 69 insertions(+), 15 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH 1/4] net/bnxt: fix RSS table update for start/stop rxq

[Lance Richardson]

A previous commit made updating of the RSS table when a receive
queue is stopped/started conditional on vnic->rx_queue_cnt being
nonzero. This count is only nonzero for dynamically created VNICs,
so the RSS table was not being updated in the normal path.
Fix by restoring the original logic.

Also ensure that vnic->rx_queue_cnt is initialized to zero when
reinitializing the VNIC array.

Fixes: 36024b2e7fe5 ("net/bnxt: allow dynamic creation of VNIC")

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_rxq.c  | 6 ++----
 drivers/net/bnxt/bnxt_vnic.c | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index 6420281d3..ab889c10e 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -466,8 +466,7 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		}
 
 		PMD_DRV_LOG(DEBUG, "Rx Queue Count %d\n", vnic->rx_queue_cnt);
-		if (vnic->rx_queue_cnt > 1)
-			rc = bnxt_vnic_rss_configure(bp, vnic);
+		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
 	if (rc == 0)
@@ -522,8 +521,7 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 			vnic->fw_grp_ids[rx_queue_id] = INVALID_HW_RING_ID;
 
 		PMD_DRV_LOG(DEBUG, "Rx Queue Count %d\n", vnic->rx_queue_cnt);
-		if (vnic->rx_queue_cnt > 1)
-			rc = bnxt_vnic_rss_configure(bp, vnic);
+		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
 	if (rc == 0)
diff --git a/drivers/net/bnxt/bnxt_vnic.c b/drivers/net/bnxt/bnxt_vnic.c
index 412a53dc0..52a4badfc 100644
--- a/drivers/net/bnxt/bnxt_vnic.c
+++ b/drivers/net/bnxt/bnxt_vnic.c
@@ -51,6 +51,7 @@ static void bnxt_init_vnics(struct bnxt *bp)
 		vnic->lb_rule = (uint16_t)HWRM_NA_SIGNATURE;
 		vnic->hash_mode =
 			HWRM_VNIC_RSS_CFG_INPUT_HASH_MODE_FLAGS_DEFAULT;
+		vnic->rx_queue_cnt = 0;
 
 		STAILQ_INIT(&vnic->filter);
 		STAILQ_INIT(&vnic->flow_list);
-- 
2.17.1

[dpdk-dev] [PATCH 2/4] net/bnxt: keep consistent rxq start/stop state

[Lance Richardson]

Receive queue state needs to reflect "started" state when rebuilding
the RSS table for Thor-based NICs. Move state update so that receive
queues being started are included in the RSS table.

Fixes: 38412304b50a ("net/bnxt: enable RSS for thor-based controllers")
Cc: stable@dpdk.org

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_rxq.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index ab889c10e..e82eda9d7 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -444,6 +444,8 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	 * If queue is it started, we do not post buffers for Rx.
 	 */
 	rxq->rx_started = true;
+	dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	bnxt_free_hwrm_rx_ring(bp, rx_queue_id);
 	rc = bnxt_alloc_hwrm_rx_ring(bp, rx_queue_id);
 	if (rc)
@@ -469,11 +471,11 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
-	if (rc == 0)
+	if (rc != 0) {
 		dev->data->rx_queue_state[rx_queue_id] =
-				RTE_ETH_QUEUE_STATE_STARTED;
-	else
+				RTE_ETH_QUEUE_STATE_STOPPED;
 		rxq->rx_started = false;
+	}
 
 	PMD_DRV_LOG(INFO,
 		    "queue %d, rx_deferred_start %d, state %d!\n",
-- 
2.17.1

[dpdk-dev] [PATCH 3/4] net/bnxt: release hwrm lock before returning

[Lance Richardson]

The function bnxt_vnic_rss_configure_thor() returns early when
all receive queues are stopped without releasing the hwrm
spinlock, which causes subsequent HWRM operations to hang. Fix
by ensuring that the lock is released before returning from
this function.

Fixes: 38412304b50a ("net/bnxt: enable RSS for thor-based controllers")
Cc: stable@dpdk.org

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_hwrm.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index ab6cb1dc3..bb4dcf4b2 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -4337,8 +4337,10 @@ bnxt_vnic_rss_configure_thor(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 			}
 
 			/* Return if no rings are active. */
-			if (cnt == max_rings)
+			if (cnt == max_rings) {
+				HWRM_UNLOCK();
 				return 0;
+			}
 
 			/* Add rx/cp ring pair to RSS table. */
 			rxr = rxqs[k]->rx_ring;
-- 
2.17.1

[dpdk-dev] [PATCH 4/4] net/bnxt: fix rxq start/stop for Thor based NICs

[Lance Richardson]

Controller-specific handling is required for Thor-based NICs when
stopping or starting a receive queue, otherwise packet reception
may not be reliably resumed when a stopped receive queue is
restarted:
  - The VNIC default receive ring needs to be recomputed when a
    receive queue is stopped or started.
  - When stopping the last (or only) receive queue for a given
    VNIC, ensure that no packets can reach the default receive
    ring by temporarily setting the VNIC MRU to zero.

Fixes: f8168ca0e690 ("net/bnxt: support thor controller")
Cc: stable@dpdk.org

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Kumar Khaparde <ajit.khaparde@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_hwrm.c | 27 +++++++++++++++++++++----
 drivers/net/bnxt/bnxt_rxq.c  | 38 +++++++++++++++++++++++++++++++++---
 2 files changed, 58 insertions(+), 7 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index bb4dcf4b2..2cba007ea 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -1696,10 +1696,29 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 	HWRM_PREP(req, VNIC_CFG, BNXT_USE_CHIMP_MB);
 
 	if (BNXT_CHIP_THOR(bp)) {
-		struct bnxt_rx_queue *rxq =
-			bp->eth_dev->data->rx_queues[vnic->start_grp_id];
-		struct bnxt_rx_ring_info *rxr = rxq->rx_ring;
-		struct bnxt_cp_ring_info *cpr = rxq->cp_ring;
+		int dflt_rxq = vnic->start_grp_id;
+		struct bnxt_rx_ring_info *rxr;
+		struct bnxt_cp_ring_info *cpr;
+		struct bnxt_rx_queue *rxq;
+		int i;
+
+		/*
+		 * The first active receive ring is used as the VNIC
+		 * default receive ring. If there are no active receive
+		 * rings (all corresponding receive queues are stopped),
+		 * the first receive ring is used.
+		 */
+		for (i = vnic->start_grp_id; i < vnic->end_grp_id; i++) {
+			rxq = bp->eth_dev->data->rx_queues[i];
+			if (rxq->rx_started) {
+				dflt_rxq = i;
+				break;
+			}
+		}
+
+		rxq = bp->eth_dev->data->rx_queues[dflt_rxq];
+		rxr = rxq->rx_ring;
+		cpr = rxq->cp_ring;
 
 		req.default_rx_ring_id =
 			rte_cpu_to_le_16(rxr->rx_ring_struct->fw_ring_id);
diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index e82eda9d7..e7c012f68 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -451,6 +451,10 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	if (rc)
 		return rc;
 
+	if (BNXT_CHIP_THOR(bp)) {
+		/* Reconfigure default receive ring and MRU. */
+		bnxt_hwrm_vnic_cfg(bp, rxq->vnic);
+	}
 	PMD_DRV_LOG(INFO, "Rx queue started %d\n", rx_queue_id);
 
 	if (dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) {
@@ -491,7 +495,8 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	struct rte_eth_conf *dev_conf = &bp->eth_dev->data->dev_conf;
 	struct bnxt_vnic_info *vnic = NULL;
 	struct bnxt_rx_queue *rxq = NULL;
-	int rc = 0;
+	int active_queue_cnt = 0;
+	int i, rc = 0;
 
 	rc = is_bnxt_in_error(bp);
 	if (rc)
@@ -507,8 +512,9 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	}
 
 	rxq = bp->rx_queues[rx_queue_id];
+	vnic = rxq->vnic;
 
-	if (rxq == NULL) {
+	if (!rxq || !vnic) {
 		PMD_DRV_LOG(ERR, "Invalid Rx queue %d\n", rx_queue_id);
 		return -EINVAL;
 	}
@@ -518,7 +524,6 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	PMD_DRV_LOG(DEBUG, "Rx queue stopped\n");
 
 	if (dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) {
-		vnic = rxq->vnic;
 		if (BNXT_HAS_RING_GRPS(bp))
 			vnic->fw_grp_ids[rx_queue_id] = INVALID_HW_RING_ID;
 
@@ -526,6 +531,33 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
+	if (BNXT_CHIP_THOR(bp)) {
+		/* Compute current number of active receive queues. */
+		for (i = vnic->start_grp_id; i < vnic->end_grp_id; i++)
+			if (bp->rx_queues[i]->rx_started)
+				active_queue_cnt++;
+
+		/*
+		 * For Thor, we need to ensure that the VNIC default receive
+		 * ring corresponds to an active receive queue. When no queue
+		 * is active, we need to temporarily set the MRU to zero so
+		 * that packets are dropped early in the receive pipeline in
+		 * order to prevent the VNIC default receive ring from being
+		 * accessed.
+		 */
+		if (active_queue_cnt == 0) {
+			uint16_t saved_mru = vnic->mru;
+
+			vnic->mru = 0;
+			/* Reconfigure default receive ring and MRU. */
+			bnxt_hwrm_vnic_cfg(bp, vnic);
+			vnic->mru = saved_mru;
+		} else {
+			/* Reconfigure default receive ring. */
+			bnxt_hwrm_vnic_cfg(bp, vnic);
+		}
+	}
+
 	if (rc == 0)
 		bnxt_rx_queue_release_mbufs(rxq);
 
-- 
2.17.1

Re: [dpdk-dev] [PATCH 0/4] net/bnxt: rxq stop/start fixes

[Lance Richardson]

This patch set depends on the "bnxt patchset with bug fixes" series:
    http://patchwork.dpdk.org/project/dpdk/list/?series=7216

It is based on current dpdk-next-net-brcm.

   Lance


On Mon, Nov 4, 2019 at 3:27 PM Lance Richardson
<lance.richardson@broadcom.com> wrote:
>
>
> This patch set fixes several problems involving stopping/starting
> receive queues in the bnxt PMD.
>
> Lance Richardson (4):
>   net/bnxt: fix RSS table update for start/stop rxq
>   net/bnxt: keep consistent rxq start/stop state
>   net/bnxt: release hwrm lock before returning
>   net/bnxt: fix rxq start/stop for Thor based NICs
>
>  drivers/net/bnxt/bnxt_hwrm.c | 31 +++++++++++++++++----
>  drivers/net/bnxt/bnxt_rxq.c  | 52 +++++++++++++++++++++++++++++-------
>  drivers/net/bnxt/bnxt_vnic.c |  1 +
>  3 files changed, 69 insertions(+), 15 deletions(-)
>
> --
> 2.17.1
>

[dpdk-dev] [PATCH v2 0/4] net/bnxt: rxq stop/start fixes

[Ajit Khaparde]

This patch set fixes several problems involving stopping/starting
receive queues in the bnxt PMD.

--
v1->v2: Updated commit logs for consistency.

Lance Richardson (4):
  net/bnxt: fix RSS table update for start/stop rxq
  net/bnxt: keep consistent rxq start/stop state
  net/bnxt: release hwrm lock before returning
  net/bnxt: fix rxq start/stop for Thor based NICs

 drivers/net/bnxt/bnxt_hwrm.c | 31 +++++++++++++++++----
 drivers/net/bnxt/bnxt_rxq.c  | 52 +++++++++++++++++++++++++++++-------
 drivers/net/bnxt/bnxt_vnic.c |  1 +
 3 files changed, 69 insertions(+), 15 deletions(-)

-- 
2.21.0 (Apple Git-122)

[dpdk-dev] [PATCH v2 1/4] net/bnxt: fix RSS table update for start/stop rxq

[Ajit Khaparde]

From: Lance Richardson <lance.richardson@broadcom.com>

A previous commit made updating of the RSS table when a receive
queue is stopped/started conditional on vnic->rx_queue_cnt being
nonzero. This count is only nonzero for dynamically created VNICs,
so the RSS table was not being updated in the normal path.
Fix by restoring the original logic.

Also ensure that vnic->rx_queue_cnt is initialized to zero when
reinitializing the VNIC array.

Fixes: 36024b2e7fe5 ("net/bnxt: allow dynamic creation of VNIC")

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_rxq.c  | 6 ++----
 drivers/net/bnxt/bnxt_vnic.c | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index 6420281d3..ab889c10e 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -466,8 +466,7 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		}
 
 		PMD_DRV_LOG(DEBUG, "Rx Queue Count %d\n", vnic->rx_queue_cnt);
-		if (vnic->rx_queue_cnt > 1)
-			rc = bnxt_vnic_rss_configure(bp, vnic);
+		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
 	if (rc == 0)
@@ -522,8 +521,7 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 			vnic->fw_grp_ids[rx_queue_id] = INVALID_HW_RING_ID;
 
 		PMD_DRV_LOG(DEBUG, "Rx Queue Count %d\n", vnic->rx_queue_cnt);
-		if (vnic->rx_queue_cnt > 1)
-			rc = bnxt_vnic_rss_configure(bp, vnic);
+		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
 	if (rc == 0)
diff --git a/drivers/net/bnxt/bnxt_vnic.c b/drivers/net/bnxt/bnxt_vnic.c
index 412a53dc0..52a4badfc 100644
--- a/drivers/net/bnxt/bnxt_vnic.c
+++ b/drivers/net/bnxt/bnxt_vnic.c
@@ -51,6 +51,7 @@ static void bnxt_init_vnics(struct bnxt *bp)
 		vnic->lb_rule = (uint16_t)HWRM_NA_SIGNATURE;
 		vnic->hash_mode =
 			HWRM_VNIC_RSS_CFG_INPUT_HASH_MODE_FLAGS_DEFAULT;
+		vnic->rx_queue_cnt = 0;
 
 		STAILQ_INIT(&vnic->filter);
 		STAILQ_INIT(&vnic->flow_list);
-- 
2.21.0 (Apple Git-122)

[dpdk-dev] [PATCH v2 2/4] net/bnxt: keep consistent rxq start/stop state

[Ajit Khaparde]

From: Lance Richardson <lance.richardson@broadcom.com>

Receive queue state needs to reflect "started" state when rebuilding
the RSS table for Thor-based NICs. Move state update so that receive
queues being started are included in the RSS table.

Fixes: 38412304b50a ("net/bnxt: enable RSS for thor-based controllers")
Cc: stable@dpdk.org

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_rxq.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index ab889c10e..e82eda9d7 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -444,6 +444,8 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	 * If queue is it started, we do not post buffers for Rx.
 	 */
 	rxq->rx_started = true;
+	dev->data->rx_queue_state[rx_queue_id] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	bnxt_free_hwrm_rx_ring(bp, rx_queue_id);
 	rc = bnxt_alloc_hwrm_rx_ring(bp, rx_queue_id);
 	if (rc)
@@ -469,11 +471,11 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
-	if (rc == 0)
+	if (rc != 0) {
 		dev->data->rx_queue_state[rx_queue_id] =
-				RTE_ETH_QUEUE_STATE_STARTED;
-	else
+				RTE_ETH_QUEUE_STATE_STOPPED;
 		rxq->rx_started = false;
+	}
 
 	PMD_DRV_LOG(INFO,
 		    "queue %d, rx_deferred_start %d, state %d!\n",
-- 
2.21.0 (Apple Git-122)

[dpdk-dev] [PATCH v2 3/4] net/bnxt: release hwrm lock before returning

[Ajit Khaparde]

From: Lance Richardson <lance.richardson@broadcom.com>

The function bnxt_vnic_rss_configure_thor() returns early when
all receive queues are stopped without releasing the hwrm
spinlock, which causes subsequent HWRM operations to hang. Fix
by ensuring that the lock is released before returning from
this function.

Fixes: 38412304b50a ("net/bnxt: enable RSS for thor-based controllers")
Cc: stable@dpdk.org

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_hwrm.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index ab6cb1dc3..bb4dcf4b2 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -4337,8 +4337,10 @@ bnxt_vnic_rss_configure_thor(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 			}
 
 			/* Return if no rings are active. */
-			if (cnt == max_rings)
+			if (cnt == max_rings) {
+				HWRM_UNLOCK();
 				return 0;
+			}
 
 			/* Add rx/cp ring pair to RSS table. */
 			rxr = rxqs[k]->rx_ring;
-- 
2.21.0 (Apple Git-122)

[dpdk-dev] [PATCH v2 4/4] net/bnxt: fix rxq start/stop for Thor based NICs

[Ajit Khaparde]

From: Lance Richardson <lance.richardson@broadcom.com>

Controller-specific handling is required for Thor-based NICs when
stopping or starting a receive queue, otherwise packet reception
may not be reliably resumed when a stopped receive queue is
restarted:
  - The VNIC default receive ring needs to be recomputed when a
    receive queue is stopped or started.
  - When stopping the last (or only) receive queue for a given
    VNIC, ensure that no packets can reach the default receive
    ring by temporarily setting the VNIC MRU to zero.

Fixes: f8168ca0e690 ("net/bnxt: support thor controller")
Cc: stable@dpdk.org

Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
---
 drivers/net/bnxt/bnxt_hwrm.c | 27 +++++++++++++++++++++----
 drivers/net/bnxt/bnxt_rxq.c  | 38 +++++++++++++++++++++++++++++++++---
 2 files changed, 58 insertions(+), 7 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_hwrm.c b/drivers/net/bnxt/bnxt_hwrm.c
index bb4dcf4b2..2cba007ea 100644
--- a/drivers/net/bnxt/bnxt_hwrm.c
+++ b/drivers/net/bnxt/bnxt_hwrm.c
@@ -1696,10 +1696,29 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 	HWRM_PREP(req, VNIC_CFG, BNXT_USE_CHIMP_MB);
 
 	if (BNXT_CHIP_THOR(bp)) {
-		struct bnxt_rx_queue *rxq =
-			bp->eth_dev->data->rx_queues[vnic->start_grp_id];
-		struct bnxt_rx_ring_info *rxr = rxq->rx_ring;
-		struct bnxt_cp_ring_info *cpr = rxq->cp_ring;
+		int dflt_rxq = vnic->start_grp_id;
+		struct bnxt_rx_ring_info *rxr;
+		struct bnxt_cp_ring_info *cpr;
+		struct bnxt_rx_queue *rxq;
+		int i;
+
+		/*
+		 * The first active receive ring is used as the VNIC
+		 * default receive ring. If there are no active receive
+		 * rings (all corresponding receive queues are stopped),
+		 * the first receive ring is used.
+		 */
+		for (i = vnic->start_grp_id; i < vnic->end_grp_id; i++) {
+			rxq = bp->eth_dev->data->rx_queues[i];
+			if (rxq->rx_started) {
+				dflt_rxq = i;
+				break;
+			}
+		}
+
+		rxq = bp->eth_dev->data->rx_queues[dflt_rxq];
+		rxr = rxq->rx_ring;
+		cpr = rxq->cp_ring;
 
 		req.default_rx_ring_id =
 			rte_cpu_to_le_16(rxr->rx_ring_struct->fw_ring_id);
diff --git a/drivers/net/bnxt/bnxt_rxq.c b/drivers/net/bnxt/bnxt_rxq.c
index e82eda9d7..e7c012f68 100644
--- a/drivers/net/bnxt/bnxt_rxq.c
+++ b/drivers/net/bnxt/bnxt_rxq.c
@@ -451,6 +451,10 @@ int bnxt_rx_queue_start(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	if (rc)
 		return rc;
 
+	if (BNXT_CHIP_THOR(bp)) {
+		/* Reconfigure default receive ring and MRU. */
+		bnxt_hwrm_vnic_cfg(bp, rxq->vnic);
+	}
 	PMD_DRV_LOG(INFO, "Rx queue started %d\n", rx_queue_id);
 
 	if (dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) {
@@ -491,7 +495,8 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	struct rte_eth_conf *dev_conf = &bp->eth_dev->data->dev_conf;
 	struct bnxt_vnic_info *vnic = NULL;
 	struct bnxt_rx_queue *rxq = NULL;
-	int rc = 0;
+	int active_queue_cnt = 0;
+	int i, rc = 0;
 
 	rc = is_bnxt_in_error(bp);
 	if (rc)
@@ -507,8 +512,9 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	}
 
 	rxq = bp->rx_queues[rx_queue_id];
+	vnic = rxq->vnic;
 
-	if (rxq == NULL) {
+	if (!rxq || !vnic) {
 		PMD_DRV_LOG(ERR, "Invalid Rx queue %d\n", rx_queue_id);
 		return -EINVAL;
 	}
@@ -518,7 +524,6 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	PMD_DRV_LOG(DEBUG, "Rx queue stopped\n");
 
 	if (dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) {
-		vnic = rxq->vnic;
 		if (BNXT_HAS_RING_GRPS(bp))
 			vnic->fw_grp_ids[rx_queue_id] = INVALID_HW_RING_ID;
 
@@ -526,6 +531,33 @@ int bnxt_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 		rc = bnxt_vnic_rss_configure(bp, vnic);
 	}
 
+	if (BNXT_CHIP_THOR(bp)) {
+		/* Compute current number of active receive queues. */
+		for (i = vnic->start_grp_id; i < vnic->end_grp_id; i++)
+			if (bp->rx_queues[i]->rx_started)
+				active_queue_cnt++;
+
+		/*
+		 * For Thor, we need to ensure that the VNIC default receive
+		 * ring corresponds to an active receive queue. When no queue
+		 * is active, we need to temporarily set the MRU to zero so
+		 * that packets are dropped early in the receive pipeline in
+		 * order to prevent the VNIC default receive ring from being
+		 * accessed.
+		 */
+		if (active_queue_cnt == 0) {
+			uint16_t saved_mru = vnic->mru;
+
+			vnic->mru = 0;
+			/* Reconfigure default receive ring and MRU. */
+			bnxt_hwrm_vnic_cfg(bp, vnic);
+			vnic->mru = saved_mru;
+		} else {
+			/* Reconfigure default receive ring. */
+			bnxt_hwrm_vnic_cfg(bp, vnic);
+		}
+	}
+
 	if (rc == 0)
 		bnxt_rx_queue_release_mbufs(rxq);
 
-- 
2.21.0 (Apple Git-122)

Re: [dpdk-dev] [PATCH v2 0/4] net/bnxt: rxq stop/start fixes

[Ajit Khaparde]

On Mon, Nov 4, 2019 at 3:02 PM Ajit Khaparde <ajit.khaparde@broadcom.com>
wrote:

> This patch set fixes several problems involving stopping/starting
> receive queues in the bnxt PMD.
>
> --
> v1->v2: Updated commit logs for consistency.
>
> Lance Richardson (4):
>   net/bnxt: fix RSS table update for start/stop rxq
>   net/bnxt: keep consistent rxq start/stop state
>   net/bnxt: release hwrm lock before returning
>   net/bnxt: fix rxq start/stop for Thor based NICs
>

Patchset applied to dpdk-next-net-brcm.

>
>  drivers/net/bnxt/bnxt_hwrm.c | 31 +++++++++++++++++----
>  drivers/net/bnxt/bnxt_rxq.c  | 52 +++++++++++++++++++++++++++++-------
>  drivers/net/bnxt/bnxt_vnic.c |  1 +
>  3 files changed, 69 insertions(+), 15 deletions(-)
>
> --
> 2.21.0 (Apple Git-122)
>
>