[PATCH 00/36] fix Rx and Tx queue state

[PATCH 00/36] fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state,
for example,

[1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
[2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")

Therefore, the drivers need to modify the queue state in time
according to the actual situation, especially when dev_start
and dev_stop are called. see [3] for more information.

[3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/

This patchset also resubmit the patch [2] and makes some fixes on the patch.

Jie Hai (36):
  net/axgbe: fix Rx and Tx queue state
  net/af_packet: fix Rx and Tx queue state
  net/af_xdp: fix Rx and Tx queue state
  net/avp: fix Rx and Tx queue state
  net/bnx2x: fix Rx and Tx queue state
  net/bnxt: fix Rx and Tx queue state
  net/bonding: fix Rx and Tx queue state
  net/cxgbe: fix Rx and Tx queue state
  net/dpaa: fix Rx and Tx queue state
  net/dpaa2: fix Rx and Tx queue state
  net/e1000: fix Rx and Tx queue state
  net/ena: fix Rx and Tx queue state
  net/enetc: fix Rx and Tx queue state
  net/enic: fix Rx and Tx queue state
  net/hinic: fix Rx and Tx queue state
  net/ipn3ke: fix Rx and Tx queue state
  net/memif: fix Rx and Tx queue state
  net/mana: fix Rx and Tx queue state
  net/mlx4: fix Rx and Tx queue state
  net/mvneta: fix Rx and Tx queue state
  net/mvpp2: fix Rx and Tx queue state
  net/netvsc: fix Rx and Tx queue state
  net/nfp: fix Rx and Tx queue state
  net/ngbe: fix Rx and Tx queue state
  net/null: fix Rx and Tx queue state
  net/octeon_ep: fix Rx and Tx queue state
  net/octeontx: fix Rx and Tx queue state
  net/pfe: fix Rx and Tx queue state
  net/ring: fix Rx and Tx queue state
  net/sfc: fix Rx and Tx queue state
  net/softnic: fix Rx and Tx queue state
  net/txgbe: fix Rx and Tx queue state
  net/vhost: fix Rx and Tx queue state
  net/virtio: fix Rx and Tx queue state
  net/vmxnet3: fix Rx and Tx queue state
  app/testpmd: fix primary process not polling all queues

 app/test-pmd/testpmd.c                        | 25 +++++++++++++++----
 drivers/net/af_packet/rte_eth_af_packet.c     |  9 +++++++
 drivers/net/af_xdp/rte_eth_af_xdp.c           | 13 ++++++++++
 drivers/net/avp/avp_ethdev.c                  | 12 +++++++++
 drivers/net/axgbe/axgbe_ethdev.c              |  7 ++++++
 drivers/net/axgbe/axgbe_rxtx.c                |  2 ++
 drivers/net/bnx2x/bnx2x_ethdev.c              | 12 +++++++++
 drivers/net/bnxt/bnxt_ethdev.c                |  6 +++++
 drivers/net/bonding/rte_eth_bond_pmd.c        | 10 ++++++++
 drivers/net/cxgbe/cxgbe_ethdev.c              |  6 +++++
 drivers/net/dpaa/dpaa_ethdev.c                | 12 +++++++++
 drivers/net/dpaa2/dpaa2_ethdev.c              | 11 ++++++++
 drivers/net/e1000/em_rxtx.c                   |  8 ++++++
 drivers/net/e1000/igb_rxtx.c                  |  4 +++
 drivers/net/ena/ena_ethdev.c                  | 12 +++++++++
 drivers/net/enetc/enetc_ethdev.c              | 12 +++++++++
 drivers/net/enic/enic_ethdev.c                |  6 +++++
 drivers/net/hinic/hinic_pmd_ethdev.c          | 12 +++++++++
 drivers/net/ipn3ke/ipn3ke_representor.c       | 12 +++++++++
 drivers/net/mana/rx.c                         |  3 +++
 drivers/net/mana/tx.c                         |  4 +++
 drivers/net/memif/rte_eth_memif.c             | 16 ++++++++++++
 drivers/net/mlx4/mlx4.c                       | 13 ++++++++++
 drivers/net/mvneta/mvneta_ethdev.c            | 12 +++++++++
 drivers/net/mvpp2/mrvl_ethdev.c               | 10 ++++++++
 drivers/net/netvsc/hn_ethdev.c                | 12 +++++++++
 drivers/net/nfp/flower/nfp_flower.c           |  8 ++++++
 .../net/nfp/flower/nfp_flower_representor.c   | 12 +++++++++
 drivers/net/nfp/nfp_common.c                  |  2 ++
 drivers/net/nfp/nfp_ethdev.c                  |  6 +++++
 drivers/net/nfp/nfp_ethdev_vf.c               |  6 +++++
 drivers/net/ngbe/ngbe_rxtx.c                  |  2 ++
 drivers/net/null/rte_eth_null.c               | 15 +++++++++++
 drivers/net/octeon_ep/otx_ep_ethdev.c         | 11 ++++++++
 drivers/net/octeontx/octeontx_ethdev.c        | 11 ++++++++
 drivers/net/pfe/pfe_ethdev.c                  | 11 ++++++++
 drivers/net/ring/rte_eth_ring.c               | 15 +++++++++++
 drivers/net/sfc/sfc_repr.c                    | 12 +++++++++
 drivers/net/softnic/rte_eth_softnic.c         | 12 +++++++++
 drivers/net/txgbe/txgbe_rxtx.c                |  8 ++++++
 drivers/net/vhost/rte_eth_vhost.c             | 12 +++++++++
 drivers/net/virtio/virtio_ethdev.c            | 11 ++++++++
 drivers/net/vmxnet3/vmxnet3_ethdev.c          | 12 +++++++++
 43 files changed, 422 insertions(+), 5 deletions(-)

-- 
2.30.0

[PATCH 01/36] net/axgbe: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/axgbe/axgbe_ethdev.c | 7 +++++++
 drivers/net/axgbe/axgbe_rxtx.c   | 2 ++
 2 files changed, 9 insertions(+)

diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index 48714eebe699..f378fd570e3a 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -352,6 +352,7 @@ static int
 axgbe_dev_start(struct rte_eth_dev *dev)
 {
 	struct axgbe_port *pdata = dev->data->dev_private;
+	uint16_t i;
 	int ret;
 
 	dev->dev_ops = &axgbe_eth_dev_ops;
@@ -388,6 +389,12 @@ axgbe_dev_start(struct rte_eth_dev *dev)
 
 	axgbe_set_rx_function(dev);
 	axgbe_set_tx_function(dev);
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
diff --git a/drivers/net/axgbe/axgbe_rxtx.c b/drivers/net/axgbe/axgbe_rxtx.c
index 625a92109baf..a9ff291cef3d 100644
--- a/drivers/net/axgbe/axgbe_rxtx.c
+++ b/drivers/net/axgbe/axgbe_rxtx.c
@@ -1124,6 +1124,7 @@ void axgbe_dev_clear_queues(struct rte_eth_dev *dev)
 			axgbe_rx_queue_release(rxq);
 			dev->data->rx_queues[i] = NULL;
 		}
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	for (i = 0; i < dev->data->nb_tx_queues; i++) {
@@ -1133,6 +1134,7 @@ void axgbe_dev_clear_queues(struct rte_eth_dev *dev)
 			axgbe_tx_queue_release(txq);
 			dev->data->tx_queues[i] = NULL;
 		}
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 }
 
-- 
2.30.0

[PATCH 02/36] net/af_packet: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/af_packet/rte_eth_af_packet.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c
index c13a0942aa1d..397a32db5886 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -313,7 +313,14 @@ eth_af_packet_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 static int
 eth_dev_start(struct rte_eth_dev *dev)
 {
+	struct pmd_internals *internals = dev->data->dev_private;
+	uint16_t i;
+
 	dev->data->dev_link.link_status = RTE_ETH_LINK_UP;
+	for (i = 0; i < internals->nb_queues; i++) {
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	}
 	return 0;
 }
 
@@ -341,6 +348,8 @@ eth_dev_stop(struct rte_eth_dev *dev)
 
 		internals->rx_queue[i].sockfd = -1;
 		internals->tx_queue[i].sockfd = -1;
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
-- 
2.30.0

[PATCH 03/36] net/af_xdp: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/af_xdp/rte_eth_af_xdp.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index c7786cc53a5e..0cc51223ba32 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -694,7 +694,13 @@ eth_af_xdp_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 static int
 eth_dev_start(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	dev->data->dev_link.link_status = RTE_ETH_LINK_UP;
+	for (i = 0; i < dev->data->nb_rx_queues; i++) {
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	}
 
 	return 0;
 }
@@ -703,7 +709,14 @@ eth_dev_start(struct rte_eth_dev *dev)
 static int
 eth_dev_stop(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
+	for (i = 0; i < dev->data->nb_rx_queues; i++) {
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	}
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 04/36] net/avp: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/avp/avp_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/avp/avp_ethdev.c b/drivers/net/avp/avp_ethdev.c
index b2a08f563542..53d9e38c939b 100644
--- a/drivers/net/avp/avp_ethdev.c
+++ b/drivers/net/avp/avp_ethdev.c
@@ -2036,6 +2036,7 @@ static int
 avp_dev_start(struct rte_eth_dev *eth_dev)
 {
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
+	uint16_t i;
 	int ret;
 
 	rte_spinlock_lock(&avp->lock);
@@ -2056,6 +2057,11 @@ avp_dev_start(struct rte_eth_dev *eth_dev)
 	/* remember current link state */
 	avp->flags |= AVP_F_LINKUP;
 
+	for (i = 0; i < avp->num_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < avp->num_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	ret = 0;
 
 unlock:
@@ -2067,6 +2073,7 @@ static int
 avp_dev_stop(struct rte_eth_dev *eth_dev)
 {
 	struct avp_dev *avp = AVP_DEV_PRIVATE_TO_HW(eth_dev->data->dev_private);
+	uint16_t i;
 	int ret;
 
 	rte_spinlock_lock(&avp->lock);
@@ -2086,6 +2093,11 @@ avp_dev_stop(struct rte_eth_dev *eth_dev)
 			    ret);
 	}
 
+	for (i = 0; i < avp->num_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < avp->num_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 unlock:
 	rte_spinlock_unlock(&avp->lock);
 	return ret;
-- 
2.30.0

[PATCH 05/36] net/bnx2x: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/bnx2x/bnx2x_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/bnx2x/bnx2x_ethdev.c b/drivers/net/bnx2x/bnx2x_ethdev.c
index 4448cf2de2d7..1327cbe912ae 100644
--- a/drivers/net/bnx2x/bnx2x_ethdev.c
+++ b/drivers/net/bnx2x/bnx2x_ethdev.c
@@ -211,6 +211,7 @@ bnx2x_dev_start(struct rte_eth_dev *dev)
 {
 	struct bnx2x_softc *sc = dev->data->dev_private;
 	int ret = 0;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE(sc);
 
@@ -244,6 +245,11 @@ bnx2x_dev_start(struct rte_eth_dev *dev)
 
 	bnx2x_print_device_info(sc);
 
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return ret;
 }
 
@@ -252,6 +258,7 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
 {
 	struct bnx2x_softc *sc = dev->data->dev_private;
 	int ret = 0;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE(sc);
 
@@ -277,6 +284,11 @@ bnx2x_dev_stop(struct rte_eth_dev *dev)
 		return ret;
 	}
 
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 06/36] net/bnxt: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/bnxt/bnxt_ethdev.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index ee1552452a11..7120d54d382e 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -1507,6 +1507,7 @@ static int bnxt_dev_stop(struct rte_eth_dev *eth_dev)
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
 	struct rte_intr_handle *intr_handle = pci_dev->intr_handle;
 	struct rte_eth_link link;
+	uint16_t i;
 	int ret;
 
 	eth_dev->data->dev_started = 0;
@@ -1567,6 +1568,11 @@ static int bnxt_dev_stop(struct rte_eth_dev *eth_dev)
 
 	eth_dev->data->scattered_rx = 0;
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 07/36] net/bonding: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/bonding/rte_eth_bond_pmd.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 73205f78f4e2..ede953c06301 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -2089,6 +2089,11 @@ bond_ethdev_start(struct rte_eth_dev *eth_dev)
 			internals->mode == BONDING_MODE_ALB)
 		bond_tlb_enable(internals);
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 
 out_err:
@@ -2174,6 +2179,11 @@ bond_ethdev_stop(struct rte_eth_dev *eth_dev)
 			deactivate_slave(eth_dev, slave_id);
 	}
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 08/36] net/cxgbe: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/cxgbe/cxgbe_ethdev.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/net/cxgbe/cxgbe_ethdev.c b/drivers/net/cxgbe/cxgbe_ethdev.c
index 45bbeaef0ceb..8cc3d9f257a0 100644
--- a/drivers/net/cxgbe/cxgbe_ethdev.c
+++ b/drivers/net/cxgbe/cxgbe_ethdev.c
@@ -414,6 +414,7 @@ int cxgbe_dev_stop(struct rte_eth_dev *eth_dev)
 {
 	struct port_info *pi = eth_dev->data->dev_private;
 	struct adapter *adapter = pi->adapter;
+	uint16_t i;
 
 	CXGBE_FUNC_TRACE();
 
@@ -429,6 +430,11 @@ int cxgbe_dev_stop(struct rte_eth_dev *eth_dev)
 	t4_sge_eth_clear_queues(pi);
 	eth_dev->data->scattered_rx = 0;
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 09/36] net/dpaa: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/dpaa/dpaa_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/dpaa/dpaa_ethdev.c b/drivers/net/dpaa/dpaa_ethdev.c
index a6c86113d125..ef4c06db6a4d 100644
--- a/drivers/net/dpaa/dpaa_ethdev.c
+++ b/drivers/net/dpaa/dpaa_ethdev.c
@@ -399,6 +399,7 @@ static void dpaa_interrupt_handler(void *param)
 static int dpaa_eth_dev_start(struct rte_eth_dev *dev)
 {
 	struct dpaa_if *dpaa_intf = dev->data->dev_private;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -413,12 +414,18 @@ static int dpaa_eth_dev_start(struct rte_eth_dev *dev)
 
 	fman_if_enable_rx(dev->process_private);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
 static int dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 {
 	struct fman_if *fif = dev->process_private;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
 	dev->data->dev_started = 0;
@@ -427,6 +434,11 @@ static int dpaa_eth_dev_stop(struct rte_eth_dev *dev)
 		fman_if_disable_rx(fif);
 	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 10/36] net/dpaa2: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/dpaa2/dpaa2_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c b/drivers/net/dpaa2/dpaa2_ethdev.c
index 679f33ae1a08..8e610b6bba30 100644
--- a/drivers/net/dpaa2/dpaa2_ethdev.c
+++ b/drivers/net/dpaa2/dpaa2_ethdev.c
@@ -1278,6 +1278,11 @@ dpaa2_dev_start(struct rte_eth_dev *dev)
 	if (priv->en_ordered)
 		dev->tx_pkt_burst = dpaa2_dev_tx_ordered;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -1295,6 +1300,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
 	struct rte_device *rdev = dev->device;
 	struct rte_intr_handle *intr_handle;
 	struct rte_dpaa2_device *dpaa2_dev;
+	uint16_t i;
 
 	dpaa2_dev = container_of(rdev, struct rte_dpaa2_device, device);
 	intr_handle = dpaa2_dev->intr_handle;
@@ -1329,6 +1335,11 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(dev, &link);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 11/36] net/e1000: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/e1000/em_rxtx.c  | 8 ++++++++
 drivers/net/e1000/igb_rxtx.c | 4 ++++
 2 files changed, 12 insertions(+)

diff --git a/drivers/net/e1000/em_rxtx.c b/drivers/net/e1000/em_rxtx.c
index cb5ce2307b3b..df5fbb782361 100644
--- a/drivers/net/e1000/em_rxtx.c
+++ b/drivers/net/e1000/em_rxtx.c
@@ -1576,6 +1576,8 @@ em_dev_clear_queues(struct rte_eth_dev *dev)
 			em_tx_queue_release_mbufs(txq);
 			em_reset_tx_queue(txq);
 		}
+
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
@@ -1584,6 +1586,8 @@ em_dev_clear_queues(struct rte_eth_dev *dev)
 			em_rx_queue_release_mbufs(rxq);
 			em_reset_rx_queue(rxq);
 		}
+
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 }
 
@@ -1812,6 +1816,8 @@ eth_em_rx_init(struct rte_eth_dev *dev)
 		rxdctl |= E1000_RXDCTL_GRAN;
 		E1000_WRITE_REG(hw, E1000_RXDCTL(i), rxdctl);
 
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 		/*
 		 * Due to EM devices not having any sort of hardware
 		 * limit for packet length, jumbo frame of any size
@@ -1946,6 +1952,8 @@ eth_em_tx_init(struct rte_eth_dev *dev)
 		txdctl |= (txq->wthresh & 0x3F) << 16;
 		txdctl |= E1000_TXDCTL_GRAN;
 		E1000_WRITE_REG(hw, E1000_TXDCTL(i), txdctl);
+
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	}
 
 	/* Program the Transmit Control Register. */
diff --git a/drivers/net/e1000/igb_rxtx.c b/drivers/net/e1000/igb_rxtx.c
index 61c6394310b3..448c4b7d9d0f 100644
--- a/drivers/net/e1000/igb_rxtx.c
+++ b/drivers/net/e1000/igb_rxtx.c
@@ -2745,6 +2745,8 @@ eth_igbvf_rx_init(struct rte_eth_dev *dev)
 		else
 			rxdctl |= ((rxq->wthresh & 0x1F) << 16);
 		E1000_WRITE_REG(hw, E1000_RXDCTL(i), rxdctl);
+
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	}
 
 	if (dev->data->dev_conf.rxmode.offloads & RTE_ETH_RX_OFFLOAD_SCATTER) {
@@ -2816,6 +2818,8 @@ eth_igbvf_tx_init(struct rte_eth_dev *dev)
 			txdctl |= ((txq->wthresh & 0x1F) << 16);
 		txdctl |= E1000_TXDCTL_QUEUE_ENABLE;
 		E1000_WRITE_REG(hw, E1000_TXDCTL(i), txdctl);
+
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	}
 
 }
-- 
2.30.0

[PATCH 12/36] net/ena: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/ena/ena_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c
index efcb163027c8..7345e480f880 100644
--- a/drivers/net/ena/ena_ethdev.c
+++ b/drivers/net/ena/ena_ethdev.c
@@ -1171,6 +1171,7 @@ static int ena_start(struct rte_eth_dev *dev)
 	struct ena_adapter *adapter = dev->data->dev_private;
 	uint64_t ticks;
 	int rc = 0;
+	uint16_t i;
 
 	/* Cannot allocate memory in secondary process */
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
@@ -1208,6 +1209,11 @@ static int ena_start(struct rte_eth_dev *dev)
 	++adapter->dev_stats.dev_start;
 	adapter->state = ENA_ADAPTER_STATE_RUNNING;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 
 err_rss_init:
@@ -1223,6 +1229,7 @@ static int ena_stop(struct rte_eth_dev *dev)
 	struct ena_com_dev *ena_dev = &adapter->ena_dev;
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = pci_dev->intr_handle;
+	uint16_t i;
 	int rc;
 
 	/* Cannot free memory in secondary process */
@@ -1254,6 +1261,11 @@ static int ena_stop(struct rte_eth_dev *dev)
 	adapter->state = ENA_ADAPTER_STATE_STOPPED;
 	dev->data->dev_started = 0;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 13/36] net/enetc: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/enetc/enetc_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index 1b4337bc488c..c9352f074683 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -17,6 +17,7 @@ enetc_dev_start(struct rte_eth_dev *dev)
 		ENETC_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct enetc_hw *enetc_hw = &hw->hw;
 	uint32_t val;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
 	if (hw->device_id == ENETC_DEV_ID_VF)
@@ -45,6 +46,11 @@ enetc_dev_start(struct rte_eth_dev *dev)
 			      ENETC_PM0_IFM_XGMII);
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -55,6 +61,7 @@ enetc_dev_stop(struct rte_eth_dev *dev)
 		ENETC_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct enetc_hw *enetc_hw = &hw->hw;
 	uint32_t val;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
 	dev->data->dev_started = 0;
@@ -69,6 +76,11 @@ enetc_dev_stop(struct rte_eth_dev *dev)
 	enetc_port_wr(enetc_hw, ENETC_PM0_CMD_CFG,
 		      val & (~(ENETC_PM0_TX_EN | ENETC_PM0_RX_EN)));
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 14/36] net/enic: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/enic/enic_ethdev.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index cdf091559196..a487256fa154 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -368,6 +368,7 @@ static int enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
 {
 	struct rte_eth_link link;
 	struct enic *enic = pmd_priv(eth_dev);
+	uint16_t i;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
 		return 0;
@@ -378,6 +379,11 @@ static int enicpmd_dev_stop(struct rte_eth_dev *eth_dev)
 	memset(&link, 0, sizeof(link));
 	rte_eth_linkstatus_set(eth_dev, &link);
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 15/36] net/hinic: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/hinic/hinic_pmd_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/hinic/hinic_pmd_ethdev.c b/drivers/net/hinic/hinic_pmd_ethdev.c
index 7aa5e7d8e929..adc9f75c81aa 100644
--- a/drivers/net/hinic/hinic_pmd_ethdev.c
+++ b/drivers/net/hinic/hinic_pmd_ethdev.c
@@ -980,6 +980,7 @@ static int hinic_dev_start(struct rte_eth_dev *dev)
 	int rc;
 	char *name;
 	struct hinic_nic_dev *nic_dev;
+	uint16_t i;
 
 	nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 	name = dev->data->name;
@@ -1047,6 +1048,11 @@ static int hinic_dev_start(struct rte_eth_dev *dev)
 
 	rte_bit_relaxed_set32(HINIC_DEV_START, &nic_dev->dev_status);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 
 en_port_fail:
@@ -1169,6 +1175,7 @@ static int hinic_dev_stop(struct rte_eth_dev *dev)
 	uint16_t port_id;
 	struct hinic_nic_dev *nic_dev;
 	struct rte_eth_link link;
+	uint16_t i;
 
 	nic_dev = HINIC_ETH_DEV_TO_PRIVATE_NIC_DEV(dev);
 	name = dev->data->name;
@@ -1215,6 +1222,11 @@ static int hinic_dev_stop(struct rte_eth_dev *dev)
 	hinic_free_all_rx_mbuf(dev);
 	hinic_free_all_tx_mbuf(dev);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 16/36] net/ipn3ke: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/ipn3ke/ipn3ke_representor.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c
index c82f8b533370..d904d3f251e3 100644
--- a/drivers/net/ipn3ke/ipn3ke_representor.c
+++ b/drivers/net/ipn3ke/ipn3ke_representor.c
@@ -120,6 +120,7 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
 	uint64_t base_mac;
 	uint32_t val;
 	char attr_name[IPN3KE_RAWDEV_ATTR_LEN_MAX];
+	uint16_t i;
 
 	rawdev = hw->rawdev;
 
@@ -190,6 +191,11 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
 
 	ipn3ke_rpst_link_update(dev, 0);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -198,6 +204,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
 {
 	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
 	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
+	uint16_t i;
 
 	if (hw->retimer.mac_type == IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
 		/* Disable the TX path */
@@ -207,6 +214,11 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
 		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 17/36] net/memif: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/memif/rte_eth_memif.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/drivers/net/memif/rte_eth_memif.c b/drivers/net/memif/rte_eth_memif.c
index 6a8ff5b4eb65..636c8a4af345 100644
--- a/drivers/net/memif/rte_eth_memif.c
+++ b/drivers/net/memif/rte_eth_memif.c
@@ -1358,6 +1358,7 @@ memif_dev_start(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *pmd = dev->data->dev_private;
 	int ret = 0;
+	uint16_t i;
 
 	switch (pmd->role) {
 	case MEMIF_ROLE_CLIENT:
@@ -1372,13 +1373,28 @@ memif_dev_start(struct rte_eth_dev *dev)
 		break;
 	}
 
+	if (ret == 0) {
+		for (i = 0; i < dev->data->nb_rx_queues; i++)
+			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+		for (i = 0; i < dev->data->nb_tx_queues; i++)
+			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	}
+
 	return ret;
 }
 
 static int
 memif_dev_stop(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	memif_disconnect(dev);
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 18/36] net/mana: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/mana/rx.c | 3 +++
 drivers/net/mana/tx.c | 4 ++++
 2 files changed, 7 insertions(+)

diff --git a/drivers/net/mana/rx.c b/drivers/net/mana/rx.c
index 14d908580185..9da7133005a4 100644
--- a/drivers/net/mana/rx.c
+++ b/drivers/net/mana/rx.c
@@ -190,6 +190,8 @@ mana_stop_rx_queues(struct rte_eth_dev *dev)
 
 		memset(&rxq->gdma_rq, 0, sizeof(rxq->gdma_rq));
 		memset(&rxq->gdma_cq, 0, sizeof(rxq->gdma_cq));
+
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 	return 0;
 }
@@ -252,6 +254,7 @@ mana_start_rx_queues(struct rte_eth_dev *dev)
 		}
 
 		ind_tbl[i] = rxq->wq;
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	}
 
 	struct ibv_rwq_ind_table_init_attr ind_table_attr = {
diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c
index 11ba2ee1ac58..fb474e73ec1f 100644
--- a/drivers/net/mana/tx.c
+++ b/drivers/net/mana/tx.c
@@ -51,6 +51,8 @@ mana_stop_tx_queues(struct rte_eth_dev *dev)
 
 		memset(&txq->gdma_sq, 0, sizeof(txq->gdma_sq));
 		memset(&txq->gdma_cq, 0, sizeof(txq->gdma_cq));
+
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	return 0;
@@ -142,6 +144,8 @@ mana_start_tx_queues(struct rte_eth_dev *dev)
 			txq->gdma_cq.id, txq->gdma_cq.buffer,
 			txq->gdma_cq.count, txq->gdma_cq.size,
 			txq->gdma_cq.head);
+
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	}
 
 	return 0;
-- 
2.30.0

[PATCH 19/36] net/mlx4: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/mlx4/mlx4.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index a54016f4a235..1389b606ccb1 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -292,6 +292,7 @@ mlx4_dev_start(struct rte_eth_dev *dev)
 {
 	struct mlx4_priv *priv = dev->data->dev_private;
 	struct rte_flow_error error;
+	uint16_t i;
 	int ret;
 
 	if (priv->started)
@@ -327,6 +328,12 @@ mlx4_dev_start(struct rte_eth_dev *dev)
 	dev->rx_pkt_burst = mlx4_rx_burst;
 	/* Enable datapath on secondary process. */
 	mlx4_mp_req_start_rxtx(dev);
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 err:
 	mlx4_dev_stop(dev);
@@ -345,6 +352,7 @@ static int
 mlx4_dev_stop(struct rte_eth_dev *dev)
 {
 	struct mlx4_priv *priv = dev->data->dev_private;
+	uint16_t i;
 
 	if (!priv->started)
 		return 0;
@@ -359,6 +367,11 @@ mlx4_dev_stop(struct rte_eth_dev *dev)
 	mlx4_rxq_intr_disable(priv);
 	mlx4_rss_deinit(priv);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 20/36] net/mvneta: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/mvneta/mvneta_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/mvneta/mvneta_ethdev.c b/drivers/net/mvneta/mvneta_ethdev.c
index d7953ac7cbac..ad27b3e663ec 100644
--- a/drivers/net/mvneta/mvneta_ethdev.c
+++ b/drivers/net/mvneta/mvneta_ethdev.c
@@ -376,6 +376,10 @@ mvneta_dev_start(struct rte_eth_dev *dev)
 		goto out;
 	}
 
+	/* start rx queues */
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	/* start tx queues */
 	for (i = 0; i < dev->data->nb_tx_queues; i++)
 		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
@@ -412,6 +416,14 @@ mvneta_dev_stop(struct rte_eth_dev *dev)
 
 	priv->ppio = NULL;
 
+	/* stop rx queues */
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
+	/* stop tx queues */
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 21/36] net/mvpp2: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/mvpp2/mrvl_ethdev.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/net/mvpp2/mrvl_ethdev.c b/drivers/net/mvpp2/mrvl_ethdev.c
index 89c83f1c1f82..79d5afe2b3c5 100644
--- a/drivers/net/mvpp2/mrvl_ethdev.c
+++ b/drivers/net/mvpp2/mrvl_ethdev.c
@@ -951,6 +951,9 @@ mrvl_dev_start(struct rte_eth_dev *dev)
 			goto out;
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	mrvl_flow_init(dev);
 	mrvl_mtr_init(dev);
 	mrvl_set_tx_function(dev);
@@ -1076,6 +1079,13 @@ mrvl_flush_bpool(struct rte_eth_dev *dev)
 static int
 mrvl_dev_stop(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return mrvl_dev_set_link_down(dev);
 }
 
-- 
2.30.0

[PATCH 22/36] net/netvsc: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/netvsc/hn_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/netvsc/hn_ethdev.c b/drivers/net/netvsc/hn_ethdev.c
index d0bbc0a4c0c0..f778e3c64649 100644
--- a/drivers/net/netvsc/hn_ethdev.c
+++ b/drivers/net/netvsc/hn_ethdev.c
@@ -990,6 +990,7 @@ static int
 hn_dev_start(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
+	uint16_t i;
 	int error;
 
 	PMD_INIT_FUNC_TRACE();
@@ -1017,6 +1018,11 @@ hn_dev_start(struct rte_eth_dev *dev)
 	if (error == 0)
 		hn_dev_link_update(dev, 0);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return error;
 }
 
@@ -1024,10 +1030,16 @@ static int
 hn_dev_stop(struct rte_eth_dev *dev)
 {
 	struct hn_data *hv = dev->data->dev_private;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
 	dev->data->dev_started = 0;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	rte_dev_event_callback_unregister(NULL, netvsc_hotadd_callback, hv);
 	hn_rndis_set_rxfilter(hv, 0);
 	return hn_vf_stop(dev);
-- 
2.30.0

[PATCH 23/36] net/nfp: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/nfp/flower/nfp_flower.c             |  8 ++++++++
 drivers/net/nfp/flower/nfp_flower_representor.c | 12 ++++++++++++
 drivers/net/nfp/nfp_common.c                    |  2 ++
 drivers/net/nfp/nfp_ethdev.c                    |  6 ++++++
 drivers/net/nfp/nfp_ethdev_vf.c                 |  6 ++++++
 5 files changed, 34 insertions(+)

diff --git a/drivers/net/nfp/flower/nfp_flower.c b/drivers/net/nfp/flower/nfp_flower.c
index 77dab864f319..eb7b40a6eb25 100644
--- a/drivers/net/nfp/flower/nfp_flower.c
+++ b/drivers/net/nfp/flower/nfp_flower.c
@@ -85,6 +85,7 @@ int
 nfp_flower_pf_start(struct rte_eth_dev *dev)
 {
 	int ret;
+	uint16_t i;
 	uint32_t new_ctrl;
 	uint32_t update = 0;
 	struct nfp_net_hw *hw;
@@ -137,6 +138,11 @@ nfp_flower_pf_start(struct rte_eth_dev *dev)
 		return -EIO;
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -159,11 +165,13 @@ nfp_flower_pf_stop(struct rte_eth_dev *dev)
 	for (i = 0; i < dev->data->nb_tx_queues; i++) {
 		this_tx_q = dev->data->tx_queues[i];
 		nfp_net_reset_tx_queue(this_tx_q);
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
 		this_rx_q = dev->data->rx_queues[i];
 		nfp_net_reset_rx_queue(this_rx_q);
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c
index 5f94d20f1b0c..3f97bc9f8a39 100644
--- a/drivers/net/nfp/flower/nfp_flower_representor.c
+++ b/drivers/net/nfp/flower/nfp_flower_representor.c
@@ -303,6 +303,7 @@ nfp_flower_repr_dev_start(struct rte_eth_dev *dev)
 {
 	struct nfp_flower_representor *repr;
 	struct nfp_app_fw_flower *app_fw_flower;
+	uint16_t i;
 
 	repr = dev->data->dev_private;
 	app_fw_flower = repr->app_fw_flower;
@@ -314,6 +315,11 @@ nfp_flower_repr_dev_start(struct rte_eth_dev *dev)
 
 	nfp_flower_cmsg_port_mod(app_fw_flower, repr->port_id, true);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -322,6 +328,7 @@ nfp_flower_repr_dev_stop(struct rte_eth_dev *dev)
 {
 	struct nfp_flower_representor *repr;
 	struct nfp_app_fw_flower *app_fw_flower;
+	uint16_t i;
 
 	repr = dev->data->dev_private;
 	app_fw_flower = repr->app_fw_flower;
@@ -333,6 +340,11 @@ nfp_flower_repr_dev_stop(struct rte_eth_dev *dev)
 				repr->nfp_idx, 0);
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
diff --git a/drivers/net/nfp/nfp_common.c b/drivers/net/nfp/nfp_common.c
index 5092e5869de4..c0d4708f2b3e 100644
--- a/drivers/net/nfp/nfp_common.c
+++ b/drivers/net/nfp/nfp_common.c
@@ -1927,6 +1927,7 @@ nfp_net_stop_rx_queue(struct rte_eth_dev *dev)
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
 		this_rx_q = dev->data->rx_queues[i];
 		nfp_net_reset_rx_queue(this_rx_q);
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 }
 
@@ -1952,6 +1953,7 @@ nfp_net_stop_tx_queue(struct rte_eth_dev *dev)
 	for (i = 0; i < dev->data->nb_tx_queues; i++) {
 		this_tx_q = dev->data->tx_queues[i];
 		nfp_net_reset_tx_queue(this_tx_q);
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 }
 
diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
index e3ff3d80873d..bdfd5530c55f 100644
--- a/drivers/net/nfp/nfp_ethdev.c
+++ b/drivers/net/nfp/nfp_ethdev.c
@@ -65,6 +65,7 @@ nfp_net_start(struct rte_eth_dev *dev)
 	struct rte_eth_conf *dev_conf;
 	struct rte_eth_rxmode *rxmode;
 	uint32_t intr_vector;
+	uint16_t i;
 	int ret;
 
 	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -176,6 +177,11 @@ nfp_net_start(struct rte_eth_dev *dev)
 
 	hw->ctrl = new_ctrl;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 
 error:
diff --git a/drivers/net/nfp/nfp_ethdev_vf.c b/drivers/net/nfp/nfp_ethdev_vf.c
index eaf815d06da5..3e8aeef51ab4 100644
--- a/drivers/net/nfp/nfp_ethdev_vf.c
+++ b/drivers/net/nfp/nfp_ethdev_vf.c
@@ -39,6 +39,7 @@ nfp_netvf_start(struct rte_eth_dev *dev)
 	struct rte_eth_conf *dev_conf;
 	struct rte_eth_rxmode *rxmode;
 	uint32_t intr_vector;
+	uint16_t i;
 	int ret;
 
 	hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
@@ -115,6 +116,11 @@ nfp_netvf_start(struct rte_eth_dev *dev)
 
 	hw->ctrl = new_ctrl;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 
 error:
-- 
2.30.0

[PATCH 24/36] net/ngbe: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/ngbe/ngbe_rxtx.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ngbe/ngbe_rxtx.c b/drivers/net/ngbe/ngbe_rxtx.c
index f31906cc2fe3..ec353a30b1de 100644
--- a/drivers/net/ngbe/ngbe_rxtx.c
+++ b/drivers/net/ngbe/ngbe_rxtx.c
@@ -2415,6 +2415,7 @@ ngbe_dev_clear_queues(struct rte_eth_dev *dev)
 		if (txq != NULL) {
 			txq->ops->release_mbufs(txq);
 			txq->ops->reset(txq);
+			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 		}
 	}
 
@@ -2424,6 +2425,7 @@ ngbe_dev_clear_queues(struct rte_eth_dev *dev)
 		if (rxq != NULL) {
 			ngbe_rx_queue_release_mbufs(rxq);
 			ngbe_reset_rx_queue(adapter, rxq);
+			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 		}
 	}
 }
-- 
2.30.0

[PATCH 25/36] net/null: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/null/rte_eth_null.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 31081af79752..d742bc415c8c 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -192,21 +192,36 @@ eth_dev_configure(struct rte_eth_dev *dev __rte_unused)
 static int
 eth_dev_start(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	if (dev == NULL)
 		return -EINVAL;
 
 	dev->data->dev_link.link_status = RTE_ETH_LINK_UP;
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
 static int
 eth_dev_stop(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	if (dev == NULL)
 		return 0;
 
 	dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 26/36] net/octeon_ep: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/octeon_ep/otx_ep_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/net/octeon_ep/otx_ep_ethdev.c b/drivers/net/octeon_ep/otx_ep_ethdev.c
index 57b965ad0670..970372bbd791 100644
--- a/drivers/net/octeon_ep/otx_ep_ethdev.c
+++ b/drivers/net/octeon_ep/otx_ep_ethdev.c
@@ -156,6 +156,11 @@ otx_ep_dev_start(struct rte_eth_dev *eth_dev)
 	otx_ep_dev_link_update(eth_dev, 0);
 	otx_ep_info("dev started\n");
 
+	for (q = 0; q < eth_dev->data->nb_rx_queues; q++)
+		eth_dev->data->rx_queue_state[q] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (q = 0; q < eth_dev->data->nb_tx_queues; q++)
+		eth_dev->data->tx_queue_state[q] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -164,9 +169,15 @@ static int
 otx_ep_dev_stop(struct rte_eth_dev *eth_dev)
 {
 	struct otx_ep_device *otx_epvf = OTX_EP_DEV(eth_dev);
+	uint16_t i;
 
 	otx_epvf->fn_list.disable_io_queues(otx_epvf);
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 27/36] net/octeontx: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/octeontx/octeontx_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c
index a6ae51a42c4e..2a8378a33ea2 100644
--- a/drivers/net/octeontx/octeontx_ethdev.c
+++ b/drivers/net/octeontx/octeontx_ethdev.c
@@ -732,6 +732,11 @@ octeontx_dev_start(struct rte_eth_dev *dev)
 	}
 
 	/* Success */
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return ret;
 
 pki_port_stop_error:
@@ -746,6 +751,7 @@ static int
 octeontx_dev_stop(struct rte_eth_dev *dev)
 {
 	struct octeontx_nic *nic = octeontx_pmd_priv(dev);
+	uint16_t i;
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
@@ -772,6 +778,11 @@ octeontx_dev_stop(struct rte_eth_dev *dev)
 		return ret;
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 28/36] net/pfe: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/pfe/pfe_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/net/pfe/pfe_ethdev.c b/drivers/net/pfe/pfe_ethdev.c
index 0352a5795096..551f3cf193e3 100644
--- a/drivers/net/pfe/pfe_ethdev.c
+++ b/drivers/net/pfe/pfe_ethdev.c
@@ -241,6 +241,7 @@ pfe_eth_open(struct rte_eth_dev *dev)
 	struct pfe_eth_priv_s *priv = dev->data->dev_private;
 	struct hif_client_s *client;
 	struct hif_shm *hif_shm;
+	uint16_t i;
 	int rc;
 
 	/* Register client driver with HIF */
@@ -318,6 +319,10 @@ pfe_eth_open(struct rte_eth_dev *dev)
 		PFE_PMD_INFO("PFE INTERRUPT Mode enabled");
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 
 err0:
 	return rc;
@@ -361,6 +366,7 @@ static int
 pfe_eth_stop(struct rte_eth_dev *dev/*, int wake*/)
 {
 	struct pfe_eth_priv_s *priv = dev->data->dev_private;
+	uint16_t i;
 
 	dev->data->dev_started = 0;
 
@@ -370,6 +376,11 @@ pfe_eth_stop(struct rte_eth_dev *dev/*, int wake*/)
 	dev->rx_pkt_burst = rte_eth_pkt_burst_dummy;
 	dev->tx_pkt_burst = rte_eth_pkt_burst_dummy;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 29/36] net/ring: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/ring/rte_eth_ring.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index c43dccea7ffe..48953dd7a059 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -113,15 +113,30 @@ eth_dev_start(struct rte_eth_dev *dev)
 static int
 eth_dev_stop(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	dev->data->dev_started = 0;
 	dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	return 0;
 }
 
 static int
 eth_dev_set_link_down(struct rte_eth_dev *dev)
 {
+	uint16_t i;
+
 	dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
+
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 30/36] net/sfc: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/sfc/sfc_repr.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/sfc/sfc_repr.c b/drivers/net/sfc/sfc_repr.c
index 6c7727d56980..278e37477530 100644
--- a/drivers/net/sfc/sfc_repr.c
+++ b/drivers/net/sfc/sfc_repr.c
@@ -263,6 +263,7 @@ static int
 sfc_repr_dev_start(struct rte_eth_dev *dev)
 {
 	struct sfc_repr *sr = sfc_repr_by_eth_dev(dev);
+	uint16_t i;
 	int ret;
 
 	sfcr_info(sr, "entry");
@@ -274,6 +275,11 @@ sfc_repr_dev_start(struct rte_eth_dev *dev)
 	if (ret != 0)
 		goto fail_start;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	sfcr_info(sr, "done");
 
 	return 0;
@@ -338,6 +344,7 @@ static int
 sfc_repr_dev_stop(struct rte_eth_dev *dev)
 {
 	struct sfc_repr *sr = sfc_repr_by_eth_dev(dev);
+	uint16_t i;
 	int ret;
 
 	sfcr_info(sr, "entry");
@@ -352,6 +359,11 @@ sfc_repr_dev_stop(struct rte_eth_dev *dev)
 
 	sfc_repr_unlock(sr);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	sfcr_info(sr, "done");
 
 	return 0;
-- 
2.30.0

[PATCH 31/36] net/softnic: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/softnic/rte_eth_softnic.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/softnic/rte_eth_softnic.c b/drivers/net/softnic/rte_eth_softnic.c
index bcf6664460a1..1b90cf7a21ee 100644
--- a/drivers/net/softnic/rte_eth_softnic.c
+++ b/drivers/net/softnic/rte_eth_softnic.c
@@ -134,6 +134,7 @@ pmd_dev_start(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *p = dev->data->dev_private;
 	int status;
+	uint16_t i;
 
 	/* Firmware */
 	status = softnic_cli_script_process(p,
@@ -146,6 +147,11 @@ pmd_dev_start(struct rte_eth_dev *dev)
 	/* Link UP */
 	dev->data->dev_link.link_status = RTE_ETH_LINK_UP;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -153,6 +159,7 @@ static int
 pmd_dev_stop(struct rte_eth_dev *dev)
 {
 	struct pmd_internals *p = dev->data->dev_private;
+	uint16_t i;
 
 	/* Link DOWN */
 	dev->data->dev_link.link_status = RTE_ETH_LINK_DOWN;
@@ -163,6 +170,11 @@ pmd_dev_stop(struct rte_eth_dev *dev)
 	softnic_softnic_swq_free_keep_rxq_txq(p);
 	softnic_mempool_free(p);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 32/36] net/txgbe: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/txgbe/txgbe_rxtx.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/net/txgbe/txgbe_rxtx.c b/drivers/net/txgbe/txgbe_rxtx.c
index 427f8b82ac80..f4be2bb91c78 100644
--- a/drivers/net/txgbe/txgbe_rxtx.c
+++ b/drivers/net/txgbe/txgbe_rxtx.c
@@ -2805,6 +2805,8 @@ txgbe_dev_clear_queues(struct rte_eth_dev *dev)
 			txq->ops->release_mbufs(txq);
 			txq->ops->reset(txq);
 		}
+
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
@@ -2814,6 +2816,8 @@ txgbe_dev_clear_queues(struct rte_eth_dev *dev)
 			txgbe_rx_queue_release_mbufs(rxq);
 			txgbe_reset_rx_queue(adapter, rxq);
 		}
+
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
 	}
 }
 
@@ -5004,6 +5008,8 @@ txgbevf_dev_rxtx_start(struct rte_eth_dev *dev)
 		} while (--poll_ms && !(txdctl & TXGBE_TXCFG_ENA));
 		if (!poll_ms)
 			PMD_INIT_LOG(ERR, "Could not enable Tx Queue %d", i);
+		else
+			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 	}
 	for (i = 0; i < dev->data->nb_rx_queues; i++) {
 		rxq = dev->data->rx_queues[i];
@@ -5018,6 +5024,8 @@ txgbevf_dev_rxtx_start(struct rte_eth_dev *dev)
 		} while (--poll_ms && !(rxdctl & TXGBE_RXCFG_ENA));
 		if (!poll_ms)
 			PMD_INIT_LOG(ERR, "Could not enable Rx Queue %d", i);
+		else
+			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
 		rte_wmb();
 		wr32(hw, TXGBE_RXWP(i), rxq->nb_rx_desc - 1);
 	}
-- 
2.30.0

[PATCH 33/36] net/vhost: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/vhost/rte_eth_vhost.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c
index 8d37ec977545..d53b8c2cd1fc 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1135,6 +1135,7 @@ eth_dev_start(struct rte_eth_dev *eth_dev)
 {
 	struct pmd_internal *internal = eth_dev->data->dev_private;
 	struct rte_eth_conf *dev_conf = &eth_dev->data->dev_conf;
+	uint16_t i;
 
 	eth_vhost_uninstall_intr(eth_dev);
 	if (dev_conf->intr_conf.rxq && eth_vhost_install_intr(eth_dev) < 0) {
@@ -1150,6 +1151,11 @@ eth_dev_start(struct rte_eth_dev *eth_dev)
 	rte_atomic32_set(&internal->started, 1);
 	update_queuing_status(eth_dev, false);
 
+	for (i = 0; i < eth_dev->data->nb_rx_queues; i++)
+		eth_dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < eth_dev->data->nb_tx_queues; i++)
+		eth_dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return 0;
 }
 
@@ -1157,11 +1163,17 @@ static int
 eth_dev_stop(struct rte_eth_dev *dev)
 {
 	struct pmd_internal *internal = dev->data->dev_private;
+	uint16_t i;
 
 	dev->data->dev_started = 0;
 	rte_atomic32_set(&internal->started, 0);
 	update_queuing_status(dev, true);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 34/36] net/virtio: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/virtio/virtio_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c
index 2c23f1c00e25..3ab56ef769c9 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2417,6 +2417,11 @@ virtio_dev_start(struct rte_eth_dev *dev)
 	set_rxtx_funcs(dev);
 	hw->started = 1;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	/* Initialize Link state */
 	virtio_dev_link_update(dev, 0);
 
@@ -2506,6 +2511,7 @@ virtio_dev_stop(struct rte_eth_dev *dev)
 	struct virtio_hw *hw = dev->data->dev_private;
 	struct rte_eth_link link;
 	struct rte_eth_intr_conf *intr_conf = &dev->data->dev_conf.intr_conf;
+	uint16_t i;
 
 	PMD_INIT_LOG(DEBUG, "stop");
 	dev->data->dev_started = 0;
@@ -2533,6 +2539,11 @@ virtio_dev_stop(struct rte_eth_dev *dev)
 out_unlock:
 	rte_spinlock_unlock(&hw->state_lock);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 35/36] net/vmxnet3: fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue state, which is stored in
dev->data->tx_queue_state and dev->data->rx_queue_state. The
state is maintained by the driver. Users may determine whether
a queue participates in packet forwarding based on the state.
Therefore, the driver needs to modify the queue state in time
according to the actual situation.

Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/vmxnet3/vmxnet3_ethdev.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c
index 76e80e302519..e49191718aea 100644
--- a/drivers/net/vmxnet3/vmxnet3_ethdev.c
+++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c
@@ -1048,6 +1048,7 @@ vmxnet3_dev_start(struct rte_eth_dev *dev)
 {
 	int ret;
 	struct vmxnet3_hw *hw = dev->data->dev_private;
+	uint16_t i;
 
 	PMD_INIT_FUNC_TRACE();
 
@@ -1151,6 +1152,11 @@ vmxnet3_dev_start(struct rte_eth_dev *dev)
 	 */
 	__vmxnet3_dev_link_update(dev, 0);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+
 	return VMXNET3_SUCCESS;
 }
 
@@ -1163,6 +1169,7 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev)
 	struct rte_eth_link link;
 	struct vmxnet3_hw *hw = dev->data->dev_private;
 	struct rte_intr_handle *intr_handle = dev->intr_handle;
+	uint16_t i;
 	int ret;
 
 	PMD_INIT_FUNC_TRACE();
@@ -1218,6 +1225,11 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev)
 	hw->adapter_stopped = 1;
 	dev->data->dev_started = 0;
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++)
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	for (i = 0; i < dev->data->nb_tx_queues; i++)
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+
 	return 0;
 }
 
-- 
2.30.0

[PATCH 36/36] app/testpmd: fix primary process not polling all queues

[Jie Hai]

Here's how the problem arises.
step1: Start the app.
    dpdk-testpmd -a 0000:35:00.0 -l 0-3 -- -i --rxq=10 --txq=10

step2: Perform the following steps and send traffic. As expected,
queue 7 does not send or receive packets, and other queues do.
    port 0 rxq 7 stop
    port 0 txq 7 stop
    set fwd mac
    start

step3: Perform the following steps and send traffic. All queues
are expected to send and receive packets normally, but that's not
the case for queue 7.
    stop
    port stop all
    port start all
    start
    show port xstats all

In fact, only the value of rx_q7_packets for queue 7 is not zero,
which means queue 7 is enabled for the driver but is not involved
in packet receiving and forwarding by software. If we check queue
state by command 'show rxq info 0 7' and 'show txq info 0 7',
we see queue 7 is started as other queues are.
    Rx queue state: started
    Tx queue state: started
The queue 7 is started but cannot forward. That's the problem.

We know that each stream has a read-only "disabled" field that
control if this stream should be used to forward. This field
depends on testpmd local queue state, please see
commit 3c4426db54fc ("app/testpmd: do not poll stopped queues").
DPDK framework maintains ethdev queue state that drivers reported,
which indicates the real state of queues.

There are commands that update these two kind queue state such as
'port X rxq|txq start|stop'. But these operations take effect only
in one stop-start round. In the following stop-start round, the
preceding operations do not take effect anymore. However, only
the ethdev queue state is updated, causing the testpmd and ethdev
state information to diverge and causing unexpected side effects
as above problem.

There was a similar problem for the secondary process, please see
commit 5028f207a4fa ("app/testpmd: fix secondary process packet
forwarding").

This patch applies its workaround with some difference to the
primary process. Not all PMDs implement rte_eth_rx_queue_info_get and
rte_eth_tx_queue_info_get, however they may support deferred_start
with primary process. To not break their behavior, retain the original
testpmd local queue state for those PMDs.

Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
---
 app/test-pmd/testpmd.c | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 938ca035d4f8..079ef3392014 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2424,6 +2424,13 @@ update_rx_queue_state(uint16_t port_id, uint16_t queue_id)
 		ports[port_id].rxq[queue_id].state =
 			rx_qinfo.queue_state;
 	} else if (rc == -ENOTSUP) {
+		/*
+		 * Do not change the rxq state for primary process
+		 * to ensure that the PMDs do not implement
+		 * rte_eth_rx_queue_info_get can forward as before.
+		 */
+		if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+			return;
 		/*
 		 * Set the rxq state to RTE_ETH_QUEUE_STATE_STARTED
 		 * to ensure that the PMDs do not implement
@@ -2449,6 +2456,13 @@ update_tx_queue_state(uint16_t port_id, uint16_t queue_id)
 		ports[port_id].txq[queue_id].state =
 			tx_qinfo.queue_state;
 	} else if (rc == -ENOTSUP) {
+		/*
+		 * Do not change the txq state for primary process
+		 * to ensure that the PMDs do not implement
+		 * rte_eth_tx_queue_info_get can forward as before.
+		 */
+		if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+			return;
 		/*
 		 * Set the txq state to RTE_ETH_QUEUE_STATE_STARTED
 		 * to ensure that the PMDs do not implement
@@ -2463,12 +2477,15 @@ update_tx_queue_state(uint16_t port_id, uint16_t queue_id)
 }
 
 static void
-update_queue_state(void)
+update_queue_state(portid_t pid)
 {
 	portid_t pi;
 	queueid_t qi;
 
 	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
 		for (qi = 0; qi < nb_rxq; qi++)
 			update_rx_queue_state(pi, qi);
 		for (qi = 0; qi < nb_txq; qi++)
@@ -2516,8 +2533,7 @@ start_packet_forwarding(int with_tx_first)
 		return;
 
 	if (stream_init != NULL) {
-		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			update_queue_state();
+		update_queue_state(RTE_PORT_ALL);
 		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
 			stream_init(fwd_streams[i]);
 	}
@@ -3280,8 +3296,7 @@ start_port(portid_t pid)
 		pl[cfg_pi++] = pi;
 	}
 
-	if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-		update_queue_state();
+	update_queue_state(pi);
 
 	if (at_least_one_port_successfully_started && !no_link_check)
 		check_all_ports_link_status(RTE_PORT_ALL);
-- 
2.30.0

Re: [PATCH 00/36] fix Rx and Tx queue state

[David Marchand]

On Fri, Sep 8, 2023 at 1:32 PM Jie Hai <haijie1@huawei.com> wrote:
>
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether
> a queue participates in packet forwarding based on the state,
> for example,

The driver is maintaining this state in dev_start / dev_stop and per
queue start/stop handlers.

>
> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
>
> Therefore, the drivers need to modify the queue state in time
> according to the actual situation, especially when dev_start
> and dev_stop are called. see [3] for more information.
>
> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
>
> This patchset also resubmit the patch [2] and makes some fixes on the patch.

I just had a quick look at some patches and I wonder if a better fix
would be at the ethdev level, rather than fixing all drivers.


-- 
David Marchand

Re: [PATCH 30/36] net/sfc: fix Rx and Tx queue state

[Andrew Rybchenko]

On 9/8/23 14:28, Jie Hai wrote:
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether
> a queue participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time
> according to the actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>   drivers/net/sfc/sfc_repr.c | 12 ++++++++++++
>   1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/net/sfc/sfc_repr.c b/drivers/net/sfc/sfc_repr.c
> index 6c7727d56980..278e37477530 100644
> --- a/drivers/net/sfc/sfc_repr.c
> +++ b/drivers/net/sfc/sfc_repr.c
> @@ -263,6 +263,7 @@ static int
>   sfc_repr_dev_start(struct rte_eth_dev *dev)
>   {
>   	struct sfc_repr *sr = sfc_repr_by_eth_dev(dev);
> +	uint16_t i;
>   	int ret;
>   
>   	sfcr_info(sr, "entry");
> @@ -274,6 +275,11 @@ sfc_repr_dev_start(struct rte_eth_dev *dev)
>   	if (ret != 0)
>   		goto fail_start;
>   
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;

May be I miss something, but the driver does when queue is
really started in sfc_rx_qstart() and sfc_tx_qstart().
Also the patch is wrong in general in the case of deferred
start queues.

Same for stop.

> +
>   	sfcr_info(sr, "done");
>   
>   	return 0;
> @@ -338,6 +344,7 @@ static int
>   sfc_repr_dev_stop(struct rte_eth_dev *dev)
>   {
>   	struct sfc_repr *sr = sfc_repr_by_eth_dev(dev);
> +	uint16_t i;
>   	int ret;
>   
>   	sfcr_info(sr, "entry");
> @@ -352,6 +359,11 @@ sfc_repr_dev_stop(struct rte_eth_dev *dev)
>   
>   	sfc_repr_unlock(sr);
>   
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +
>   	sfcr_info(sr, "done");
>   
>   	return 0;

RE: [PATCH 16/36] net/ipn3ke: fix Rx and Tx queue state

[Xu, Rosen]

Hi,

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Friday, September 8, 2023 7:29 PM
> To: dev@dpdk.org; Xu, Rosen <rosen.xu@intel.com>; Ferruh Yigit
> <ferruh.yigit@intel.com>; Lijun Ou <oulijun@huawei.com>; Chengwen Feng
> <fengchengwen@huawei.com>; Konstantin Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH 16/36] net/ipn3ke: fix Rx and Tx queue state
> 
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether a queue
> participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time according to
> the actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue
> information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  drivers/net/ipn3ke/ipn3ke_representor.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c
> b/drivers/net/ipn3ke/ipn3ke_representor.c
> index c82f8b533370..d904d3f251e3 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -120,6 +120,7 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
>  	uint64_t base_mac;
>  	uint32_t val;
>  	char attr_name[IPN3KE_RAWDEV_ATTR_LEN_MAX];
> +	uint16_t i;
> 
>  	rawdev = hw->rawdev;
> 
> @@ -190,6 +191,11 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
> 
>  	ipn3ke_rpst_link_update(dev, 0);
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +
>  	return 0;
>  }
> 
> @@ -198,6 +204,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)  {
>  	struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
>  	struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
> +	uint16_t i;
> 
>  	if (hw->retimer.mac_type ==
> IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
>  		/* Disable the TX path */
> @@ -207,6 +214,11 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
>  		ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
>  	}
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +
>  	return 0;
>  }
> 
> --
> 2.30.0

Reviewed-by: Rosen Xu <rosen.xu@intel.com>

RE: [PATCH 23/36] net/nfp: fix Rx and Tx queue state

[Chaoyong He]

Thanks, it looks good to me.

Acked-by: Chaoyong He <chaoyong.he@corigine.com>

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Friday, September 8, 2023 7:29 PM
> To: dev@dpdk.org; Chaoyong He <chaoyong.he@corigine.com>; Niklas
> Soderlund <niklas.soderlund@corigine.com>; Chengwen Feng
> <fengchengwen@huawei.com>; Lijun Ou <oulijun@huawei.com>; Konstantin
> Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>;
> Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
> <ferruh.yigit@intel.com>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH 23/36] net/nfp: fix Rx and Tx queue state
> 
> [Some people who received this message don't often get email from
> haijie1@huawei.com. Learn why this is important at
> https://aka.ms/LearnAboutSenderIdentification ]
> 
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether a queue
> participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time according to the
> actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  drivers/net/nfp/flower/nfp_flower.c             |  8 ++++++++
>  drivers/net/nfp/flower/nfp_flower_representor.c | 12 ++++++++++++
>  drivers/net/nfp/nfp_common.c                    |  2 ++
>  drivers/net/nfp/nfp_ethdev.c                    |  6 ++++++
>  drivers/net/nfp/nfp_ethdev_vf.c                 |  6 ++++++
>  5 files changed, 34 insertions(+)
> 
> diff --git a/drivers/net/nfp/flower/nfp_flower.c
> b/drivers/net/nfp/flower/nfp_flower.c
> index 77dab864f319..eb7b40a6eb25 100644
> --- a/drivers/net/nfp/flower/nfp_flower.c
> +++ b/drivers/net/nfp/flower/nfp_flower.c
> @@ -85,6 +85,7 @@ int
>  nfp_flower_pf_start(struct rte_eth_dev *dev)  {
>         int ret;
> +       uint16_t i;
>         uint32_t new_ctrl;
>         uint32_t update = 0;
>         struct nfp_net_hw *hw;
> @@ -137,6 +138,11 @@ nfp_flower_pf_start(struct rte_eth_dev *dev)
>                 return -EIO;
>         }
> 
> +       for (i = 0; i < dev->data->nb_rx_queues; i++)
> +               dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +       for (i = 0; i < dev->data->nb_tx_queues; i++)
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STARTED;
> +
>         return 0;
>  }
> 
> @@ -159,11 +165,13 @@ nfp_flower_pf_stop(struct rte_eth_dev *dev)
>         for (i = 0; i < dev->data->nb_tx_queues; i++) {
>                 this_tx_q = dev->data->tx_queues[i];
>                 nfp_net_reset_tx_queue(this_tx_q);
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STOPPED;
>         }
> 
>         for (i = 0; i < dev->data->nb_rx_queues; i++) {
>                 this_rx_q = dev->data->rx_queues[i];
>                 nfp_net_reset_rx_queue(this_rx_q);
> +               dev->data->rx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STOPPED;
>         }
> 
>         if (rte_eal_process_type() == RTE_PROC_PRIMARY) diff --git
> a/drivers/net/nfp/flower/nfp_flower_representor.c
> b/drivers/net/nfp/flower/nfp_flower_representor.c
> index 5f94d20f1b0c..3f97bc9f8a39 100644
> --- a/drivers/net/nfp/flower/nfp_flower_representor.c
> +++ b/drivers/net/nfp/flower/nfp_flower_representor.c
> @@ -303,6 +303,7 @@ nfp_flower_repr_dev_start(struct rte_eth_dev *dev)
> {
>         struct nfp_flower_representor *repr;
>         struct nfp_app_fw_flower *app_fw_flower;
> +       uint16_t i;
> 
>         repr = dev->data->dev_private;
>         app_fw_flower = repr->app_fw_flower; @@ -314,6 +315,11 @@
> nfp_flower_repr_dev_start(struct rte_eth_dev *dev)
> 
>         nfp_flower_cmsg_port_mod(app_fw_flower, repr->port_id, true);
> 
> +       for (i = 0; i < dev->data->nb_rx_queues; i++)
> +               dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +       for (i = 0; i < dev->data->nb_tx_queues; i++)
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STARTED;
> +
>         return 0;
>  }
> 
> @@ -322,6 +328,7 @@ nfp_flower_repr_dev_stop(struct rte_eth_dev *dev)
> {
>         struct nfp_flower_representor *repr;
>         struct nfp_app_fw_flower *app_fw_flower;
> +       uint16_t i;
> 
>         repr = dev->data->dev_private;
>         app_fw_flower = repr->app_fw_flower; @@ -333,6 +340,11 @@
> nfp_flower_repr_dev_stop(struct rte_eth_dev *dev)
>                                 repr->nfp_idx, 0);
>         }
> 
> +       for (i = 0; i < dev->data->nb_rx_queues; i++)
> +               dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +       for (i = 0; i < dev->data->nb_tx_queues; i++)
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STOPPED;
> +
>         return 0;
>  }
> 
> diff --git a/drivers/net/nfp/nfp_common.c b/drivers/net/nfp/nfp_common.c
> index 5092e5869de4..c0d4708f2b3e 100644
> --- a/drivers/net/nfp/nfp_common.c
> +++ b/drivers/net/nfp/nfp_common.c
> @@ -1927,6 +1927,7 @@ nfp_net_stop_rx_queue(struct rte_eth_dev *dev)
>         for (i = 0; i < dev->data->nb_rx_queues; i++) {
>                 this_rx_q = dev->data->rx_queues[i];
>                 nfp_net_reset_rx_queue(this_rx_q);
> +               dev->data->rx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STOPPED;
>         }
>  }
> 
> @@ -1952,6 +1953,7 @@ nfp_net_stop_tx_queue(struct rte_eth_dev *dev)
>         for (i = 0; i < dev->data->nb_tx_queues; i++) {
>                 this_tx_q = dev->data->tx_queues[i];
>                 nfp_net_reset_tx_queue(this_tx_q);
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STOPPED;
>         }
>  }
> 
> diff --git a/drivers/net/nfp/nfp_ethdev.c b/drivers/net/nfp/nfp_ethdev.c
> index e3ff3d80873d..bdfd5530c55f 100644
> --- a/drivers/net/nfp/nfp_ethdev.c
> +++ b/drivers/net/nfp/nfp_ethdev.c
> @@ -65,6 +65,7 @@ nfp_net_start(struct rte_eth_dev *dev)
>         struct rte_eth_conf *dev_conf;
>         struct rte_eth_rxmode *rxmode;
>         uint32_t intr_vector;
> +       uint16_t i;
>         int ret;
> 
>         hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -176,6 +177,11 @@ nfp_net_start(struct rte_eth_dev *dev)
> 
>         hw->ctrl = new_ctrl;
> 
> +       for (i = 0; i < dev->data->nb_rx_queues; i++)
> +               dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +       for (i = 0; i < dev->data->nb_tx_queues; i++)
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STARTED;
> +
>         return 0;
> 
>  error:
> diff --git a/drivers/net/nfp/nfp_ethdev_vf.c
> b/drivers/net/nfp/nfp_ethdev_vf.c index eaf815d06da5..3e8aeef51ab4
> 100644
> --- a/drivers/net/nfp/nfp_ethdev_vf.c
> +++ b/drivers/net/nfp/nfp_ethdev_vf.c
> @@ -39,6 +39,7 @@ nfp_netvf_start(struct rte_eth_dev *dev)
>         struct rte_eth_conf *dev_conf;
>         struct rte_eth_rxmode *rxmode;
>         uint32_t intr_vector;
> +       uint16_t i;
>         int ret;
> 
>         hw = NFP_NET_DEV_PRIVATE_TO_HW(dev->data->dev_private);
> @@ -115,6 +116,11 @@ nfp_netvf_start(struct rte_eth_dev *dev)
> 
>         hw->ctrl = new_ctrl;
> 
> +       for (i = 0; i < dev->data->nb_rx_queues; i++)
> +               dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +       for (i = 0; i < dev->data->nb_tx_queues; i++)
> +               dev->data->tx_queue_state[i] =
> + RTE_ETH_QUEUE_STATE_STARTED;
> +
>         return 0;
> 
>  error:
> --
> 2.30.0

Re: [PATCH 30/36] net/sfc: fix Rx and Tx queue state

[Jie Hai]

On 2023/9/8 20:01, Andrew Rybchenko wrote:
> On 9/8/23 14:28, Jie Hai wrote:
>> The DPDK framework reports the queue state, which is stored in
>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>> state is maintained by the driver. Users may determine whether
>> a queue participates in packet forwarding based on the state.
>> Therefore, the driver needs to modify the queue state in time
>> according to the actual situation.
>>
>> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue 
>> information")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>>   drivers/net/sfc/sfc_repr.c | 12 ++++++++++++
>>   1 file changed, 12 insertions(+)
>>
>> diff --git a/drivers/net/sfc/sfc_repr.c b/drivers/net/sfc/sfc_repr.c
>> index 6c7727d56980..278e37477530 100644
>> --- a/drivers/net/sfc/sfc_repr.c
>> +++ b/drivers/net/sfc/sfc_repr.c
>> @@ -263,6 +263,7 @@ static int
>>   sfc_repr_dev_start(struct rte_eth_dev *dev)
>>   {
>>       struct sfc_repr *sr = sfc_repr_by_eth_dev(dev);
>> +    uint16_t i;
>>       int ret;
>>       sfcr_info(sr, "entry");
>> @@ -274,6 +275,11 @@ sfc_repr_dev_start(struct rte_eth_dev *dev)
>>       if (ret != 0)
>>           goto fail_start;
>> +    for (i = 0; i < dev->data->nb_rx_queues; i++)
>> +        dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
>> +    for (i = 0; i < dev->data->nb_tx_queues; i++)
>> +        dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> 
> May be I miss something, but the driver does when queue is
> really started in sfc_rx_qstart() and sfc_tx_qstart().
> Also the patch is wrong in general in the case of deferred
> start queues.
> 
> Same for stop.
Hi, Andrew Rybchenko,

Thanks for your review.

There are two ops implementations,
one in file sfc_ethdev.c(eg. sfc_dev_start)
and the other in file sfc_repr.c (eg. sfc_repr_dev_start).
This patch is for the second one, which does not support deferred start, 
please see 'sfc_repr_rx_qcheck_conf' and 'sfc_repr_tx_qcheck_conf'.

The function process of ‘sfc_repr_dev_start’and ‘sfc_repr_dev_stop’ is 
as follows:
sfc_repr_dev_start
   -->sfc_repr_start
     -->sfc_repr_proxy_start_repr
       -->sfc_repr_proxy_start
       	-->sfc_repr_proxy_rxq_start
       	  -->sfc_rx_qstart -- RTE_ETH_QUEUE_STATE_STARTED
       	-->sfc_repr_proxy_txq_start

sfc_repr_dev_stop
   -->sfc_repr_stop
     -->sfc_repr_proxy_stop_repr
       -->sfc_repr_proxy_stop
       	-->sfc_repr_proxy_rxq_stop
       	  -->sfc_rx_qstop -- RTE_ETH_QUEUE_STATE_STOPPED
       	-->sfc_repr_proxy_txq_stop

Since the Rx has been modified, Maybe I should modify the TX queue 
status separately in 'sfc_repr_proxy_txq_start/stop'.

--
Best regards,
Jie Hai
> 
>> +
>>       sfcr_info(sr, "done");
>>       return 0;
>> @@ -338,6 +344,7 @@ static int
>>   sfc_repr_dev_stop(struct rte_eth_dev *dev)
>>   {
>>       struct sfc_repr *sr = sfc_repr_by_eth_dev(dev);
>> +    uint16_t i;
>>       int ret;
>>       sfcr_info(sr, "entry");
>> @@ -352,6 +359,11 @@ sfc_repr_dev_stop(struct rte_eth_dev *dev)
>>       sfc_repr_unlock(sr);
>> +    for (i = 0; i < dev->data->nb_rx_queues; i++)
>> +        dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>> +    for (i = 0; i < dev->data->nb_tx_queues; i++)
>> +        dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>> +
>>       sfcr_info(sr, "done");
>>       return 0;
> 
> .

RE: [PATCH 09/36] net/dpaa: fix Rx and Tx queue state

[Hemant Agrawal]

Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Friday, September 8, 2023 4:59 PM
> To: dev@dpdk.org; Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin
> Saxena <sachin.saxena@nxp.com>; Lijun Ou <oulijun@huawei.com>; Ferruh
> Yigit <ferruh.yigit@intel.com>; Konstantin Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>;
> Thomas Monjalon <thomas@monjalon.net>; Chengwen Feng
> <fengchengwen@huawei.com>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH 09/36] net/dpaa: fix Rx and Tx queue state
> Importance: High
> 
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether a queue
> participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time according to
> the actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue
> information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  drivers/net/dpaa/dpaa_ethdev.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/drivers/net/dpaa/dpaa_ethdev.c
> b/drivers/net/dpaa/dpaa_ethdev.c index a6c86113d125..ef4c06db6a4d
> 100644
> --- a/drivers/net/dpaa/dpaa_ethdev.c
> +++ b/drivers/net/dpaa/dpaa_ethdev.c
> @@ -399,6 +399,7 @@ static void dpaa_interrupt_handler(void *param)
> static int dpaa_eth_dev_start(struct rte_eth_dev *dev)  {
>  	struct dpaa_if *dpaa_intf = dev->data->dev_private;
> +	uint16_t i;
> 
>  	PMD_INIT_FUNC_TRACE();
> 
> @@ -413,12 +414,18 @@ static int dpaa_eth_dev_start(struct rte_eth_dev
> *dev)
> 
>  	fman_if_enable_rx(dev->process_private);
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +
>  	return 0;
>  }
> 
>  static int dpaa_eth_dev_stop(struct rte_eth_dev *dev)  {
>  	struct fman_if *fif = dev->process_private;
> +	uint16_t i;
> 
>  	PMD_INIT_FUNC_TRACE();
>  	dev->data->dev_started = 0;
> @@ -427,6 +434,11 @@ static int dpaa_eth_dev_stop(struct rte_eth_dev
> *dev)
>  		fman_if_disable_rx(fif);
>  	dev->tx_pkt_burst = dpaa_eth_tx_drop_all;
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +
>  	return 0;
>  }
> 
> --
> 2.30.0

RE: [PATCH 10/36] net/dpaa2: fix Rx and Tx queue state

[Hemant Agrawal]

Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Friday, September 8, 2023 4:59 PM
> To: dev@dpdk.org; Hemant Agrawal <hemant.agrawal@nxp.com>; Sachin
> Saxena <sachin.saxena@nxp.com>; Thomas Monjalon
> <thomas@monjalon.net>; Konstantin Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>;
> Chengwen Feng <fengchengwen@huawei.com>; Ferruh Yigit
> <ferruh.yigit@intel.com>; Lijun Ou <oulijun@huawei.com>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH 10/36] net/dpaa2: fix Rx and Tx queue state
> Importance: High
> 
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether a queue
> participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time according to
> the actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue
> information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  drivers/net/dpaa2/dpaa2_ethdev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/drivers/net/dpaa2/dpaa2_ethdev.c
> b/drivers/net/dpaa2/dpaa2_ethdev.c
> index 679f33ae1a08..8e610b6bba30 100644
> --- a/drivers/net/dpaa2/dpaa2_ethdev.c
> +++ b/drivers/net/dpaa2/dpaa2_ethdev.c
> @@ -1278,6 +1278,11 @@ dpaa2_dev_start(struct rte_eth_dev *dev)
>  	if (priv->en_ordered)
>  		dev->tx_pkt_burst = dpaa2_dev_tx_ordered;
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +
>  	return 0;
>  }
> 
> @@ -1295,6 +1300,7 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
>  	struct rte_device *rdev = dev->device;
>  	struct rte_intr_handle *intr_handle;
>  	struct rte_dpaa2_device *dpaa2_dev;
> +	uint16_t i;
> 
>  	dpaa2_dev = container_of(rdev, struct rte_dpaa2_device, device);
>  	intr_handle = dpaa2_dev->intr_handle;
> @@ -1329,6 +1335,11 @@ dpaa2_dev_stop(struct rte_eth_dev *dev)
>  	memset(&link, 0, sizeof(link));
>  	rte_eth_linkstatus_set(dev, &link);
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +
>  	return 0;
>  }
> 
> --
> 2.30.0

RE: [PATCH 31/36] net/softnic: fix Rx and Tx queue state

[Dumitrescu, Cristian]

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Friday, September 8, 2023 12:29 PM
> To: dev@dpdk.org; Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> Lijun Ou <oulijun@huawei.com>; Konstantin Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>;
> Thomas Monjalon <thomas@monjalon.net>; Ferruh Yigit
> <ferruh.yigit@intel.com>; Chengwen Feng <fengchengwen@huawei.com>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH 31/36] net/softnic: fix Rx and Tx queue state
> 
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether
> a queue participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time
> according to the actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---

Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>

Re: [PATCH 00/36] fix Rx and Tx queue state

[Ferruh Yigit]

On 9/8/2023 12:50 PM, David Marchand wrote:
> On Fri, Sep 8, 2023 at 1:32 PM Jie Hai <haijie1@huawei.com> wrote:
>>
>> The DPDK framework reports the queue state, which is stored in
>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>> state is maintained by the driver. Users may determine whether
>> a queue participates in packet forwarding based on the state,
>> for example,
> 
> The driver is maintaining this state in dev_start / dev_stop and per
> queue start/stop handlers.
> 
>>
>> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
>> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
>>
>> Therefore, the drivers need to modify the queue state in time
>> according to the actual situation, especially when dev_start
>> and dev_stop are called. see [3] for more information.
>>
>> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
>>
>> This patchset also resubmit the patch [2] and makes some fixes on the patch.
> 
> I just had a quick look at some patches and I wonder if a better fix
> would be at the ethdev level, rather than fixing all drivers.
> 
> 

I came here to make the same comment,

Jie, I forgot if we discuss this already but,

does it work if we update queue state in 'rte_eth_dev_start()' &
'rte_eth_dev_stop()' when 'dev_start' & 'dev_stop' dev_ops succeeds?

This solves the case driver forgets to update the queue state.

Re: [PATCH 00/36] fix Rx and Tx queue state

[Jie Hai]

On 2023/9/19 0:54, Ferruh Yigit wrote:
> On 9/8/2023 12:50 PM, David Marchand wrote:
>> On Fri, Sep 8, 2023 at 1:32 PM Jie Hai <haijie1@huawei.com> wrote:
>>>
>>> The DPDK framework reports the queue state, which is stored in
>>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>>> state is maintained by the driver. Users may determine whether
>>> a queue participates in packet forwarding based on the state,
>>> for example,
>>
>> The driver is maintaining this state in dev_start / dev_stop and per
>> queue start/stop handlers.
>>
>>>
>>> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
>>> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
>>>
>>> Therefore, the drivers need to modify the queue state in time
>>> according to the actual situation, especially when dev_start
>>> and dev_stop are called. see [3] for more information.
>>>
>>> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
>>>
>>> This patchset also resubmit the patch [2] and makes some fixes on the patch.
>>
>> I just had a quick look at some patches and I wonder if a better fix
>> would be at the ethdev level, rather than fixing all drivers.
>>
>>
> 
> I came here to make the same comment,
> 
> Jie, I forgot if we discuss this already but,
> 
> does it work if we update queue state in 'rte_eth_dev_start()' &
> 'rte_eth_dev_stop()' when 'dev_start' & 'dev_stop' dev_ops succeeds?
> 
> This solves the case driver forgets to update the queue state.
> 
> 
Hi, Furrh and David,

It's OK for dev_stop, but not enough for dev_start.
Some drivers also support deferred_start.
Therefore, not all queues are in the start state after dev_start.

If we want to get that correct queue state at the framework level, I 
offer the following options:

step 1. A capability(e.g. RTE_ETH_DEV_CAPA_DEFERRED_START) is added to 
the framework, indicating whether the driver supports deferred_start.
The capability should be reported by the driver and user can get it by
rte_eth_dev_info_get().
step 2. All drivers that support deferred_start should report configuration
information about deferred_start through
rte_eth_rx_queue_info_get |rte_eth_tx_queue_info_get.
step 3.The framework updates the queue status based on the support and 
configuration of deferred_start.

Is this solution OK?

> .

Re: [PATCH 00/36] fix Rx and Tx queue state

[David Marchand]

Hello,

On Fri, Sep 22, 2023 at 4:41 AM Jie Hai <haijie1@huawei.com> wrote:
> On 2023/9/19 0:54, Ferruh Yigit wrote:
> > On 9/8/2023 12:50 PM, David Marchand wrote:
> >> On Fri, Sep 8, 2023 at 1:32 PM Jie Hai <haijie1@huawei.com> wrote:
> >>>
> >>> The DPDK framework reports the queue state, which is stored in
> >>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> >>> state is maintained by the driver. Users may determine whether
> >>> a queue participates in packet forwarding based on the state,
> >>> for example,
> >>
> >> The driver is maintaining this state in dev_start / dev_stop and per
> >> queue start/stop handlers.
> >>
> >>>
> >>> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
> >>> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
> >>>
> >>> Therefore, the drivers need to modify the queue state in time
> >>> according to the actual situation, especially when dev_start
> >>> and dev_stop are called. see [3] for more information.
> >>>
> >>> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
> >>>
> >>> This patchset also resubmit the patch [2] and makes some fixes on the patch.
> >>
> >> I just had a quick look at some patches and I wonder if a better fix
> >> would be at the ethdev level, rather than fixing all drivers.
> >>
> >>
> >
> > I came here to make the same comment,
> >
> > Jie, I forgot if we discuss this already but,
> >
> > does it work if we update queue state in 'rte_eth_dev_start()' &
> > 'rte_eth_dev_stop()' when 'dev_start' & 'dev_stop' dev_ops succeeds?
> >
> > This solves the case driver forgets to update the queue state.
> >
> >
> Hi, Furrh and David,
>
> It's OK for dev_stop, but not enough for dev_start.
> Some drivers also support deferred_start.
> Therefore, not all queues are in the start state after dev_start.
>
> If we want to get that correct queue state at the framework level, I
> offer the following options:
>
> step 1. A capability(e.g. RTE_ETH_DEV_CAPA_DEFERRED_START) is added to
> the framework, indicating whether the driver supports deferred_start.
> The capability should be reported by the driver and user can get it by
> rte_eth_dev_info_get().
> step 2. All drivers that support deferred_start should report configuration
> information about deferred_start through
> rte_eth_rx_queue_info_get |rte_eth_tx_queue_info_get.
> step 3.The framework updates the queue status based on the support and
> configuration of deferred_start.

rte_eth_dev_start must only update the queue state if
rx_deferred_start is unset (see struct
rte_eth_rxconf::rx_deferred_start).
And the queue state needs to be updated in ethdev
rte_eth_dev_rx_queue_start/stop.

So I don't see where we need a new capability.


-- 
David Marchand

Re: [PATCH 00/36] fix Rx and Tx queue state

[Jie Hai]

On 2023/9/22 14:41, David Marchand wrote:
> Hello,
> 
> On Fri, Sep 22, 2023 at 4:41 AM Jie Hai <haijie1@huawei.com> wrote:
>> On 2023/9/19 0:54, Ferruh Yigit wrote:
>>> On 9/8/2023 12:50 PM, David Marchand wrote:
>>>> On Fri, Sep 8, 2023 at 1:32 PM Jie Hai <haijie1@huawei.com> wrote:
>>>>>
>>>>> The DPDK framework reports the queue state, which is stored in
>>>>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>>>>> state is maintained by the driver. Users may determine whether
>>>>> a queue participates in packet forwarding based on the state,
>>>>> for example,
>>>>
>>>> The driver is maintaining this state in dev_start / dev_stop and per
>>>> queue start/stop handlers.
>>>>
>>>>>
>>>>> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
>>>>> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
>>>>>
>>>>> Therefore, the drivers need to modify the queue state in time
>>>>> according to the actual situation, especially when dev_start
>>>>> and dev_stop are called. see [3] for more information.
>>>>>
>>>>> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
>>>>>
>>>>> This patchset also resubmit the patch [2] and makes some fixes on the patch.
>>>>
>>>> I just had a quick look at some patches and I wonder if a better fix
>>>> would be at the ethdev level, rather than fixing all drivers.
>>>>
>>>>
>>>
>>> I came here to make the same comment,
>>>
>>> Jie, I forgot if we discuss this already but,
>>>
>>> does it work if we update queue state in 'rte_eth_dev_start()' &
>>> 'rte_eth_dev_stop()' when 'dev_start' & 'dev_stop' dev_ops succeeds?
>>>
>>> This solves the case driver forgets to update the queue state.
>>>
>>>
>> Hi, Furrh and David,
>>
>> It's OK for dev_stop, but not enough for dev_start.
>> Some drivers also support deferred_start.
>> Therefore, not all queues are in the start state after dev_start.
>>
>> If we want to get that correct queue state at the framework level, I
>> offer the following options:
>>
>> step 1. A capability(e.g. RTE_ETH_DEV_CAPA_DEFERRED_START) is added to
>> the framework, indicating whether the driver supports deferred_start.
>> The capability should be reported by the driver and user can get it by
>> rte_eth_dev_info_get().
>> step 2. All drivers that support deferred_start should report configuration
>> information about deferred_start through
>> rte_eth_rx_queue_info_get |rte_eth_tx_queue_info_get.
>> step 3.The framework updates the queue status based on the support and
>> configuration of deferred_start.
> 
> rte_eth_dev_start must only update the queue state if
> rx_deferred_start is unset (see struct
> rte_eth_rxconf::rx_deferred_start).
> And the queue state needs to be updated in ethdev
> rte_eth_dev_rx_queue_start/stop.
> 
> So I don't see where we need a new capability.
> 
Hi, David,

Thanks and you are right.
The driver reports whether the queue is configured with deferred_start 
through rxq_info_get & rxq_info_get ops.
The framework can obtain queue configuration through 
rte_eth_rx_queue_info_get & rte_eth_tx_queue_info_get.
According to your suggestion, the prerequisite is that all drivers that 
support deferred_start support
rxq_info_get & rxq_info_get ops reporting it.
Currently, not all drivers support reporting  the deferred_start 
configuration. Therefore, we need to modify some driver.
I'll send V2 after finishing.

Best, regards
Jie Hai
> 

[PATCH v2 0/8] fix Rx and Tx queue state

[Jie Hai]

The DPDK framework reports the queue status, which is stored in
'dev->data->tx_queue_state' and 'dev->data->rx_queue_state' . The state
is currently maintained by the drivers. Users may determine whether
a queue participates in packet forwarding based on the state,
For example,

[1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
[2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")

However, not all drivers correctly report the queue status. This may cause
forwarding problems.

Since it is difficult to modify the queue status of all drivers, we consider
updating the queue status at the framework level. Some drivers support queues
for hairpin, leaving status updating for such queues to the drivers. Some
drivers support deferred_start. Assume that all drivers that support
'deferred_start' can obtain the configuration through 'rte_eth_tx_queue_info_get'
and 'rte_eth_rx_queue_info_get'. So we can directly update the queue status in
'rte_eth_dev_start' and 'rte_eth_dev_stop'.

This patchset does the follow things:
1. update Rx and Tx queue status in dev_start and dev_stop.
2. implement rxq|txq_info_get ops for drivers supporting deferred start.
3. resubmit the patch [2] and makes some fixes on the patch.

Jie Hai (8):
  lib/ethdev: update Rx and Tx queue status
  net/cpfl: support getting queue information
  net/enetc: save deferred start configuratin for queues
  net/enetc: support getting queue information
  net/failsafe: support getting queue information
  net/fm10k: support getting queue information
  net/idpf: support getting queue information
  app/testpmd: fix primary process not polling all queues

 app/test-pmd/testpmd.c              | 25 ++++++++++++++----
 drivers/net/cpfl/cpfl_ethdev.c      |  2 ++
 drivers/net/cpfl/cpfl_rxtx.c        | 26 +++++++++++++++++++
 drivers/net/cpfl/cpfl_rxtx.h        |  4 +++
 drivers/net/enetc/enetc.h           |  2 ++
 drivers/net/enetc/enetc_ethdev.c    | 26 +++++++++++++++++++
 drivers/net/failsafe/failsafe_ops.c | 24 +++++++++++++++++
 drivers/net/fm10k/fm10k_ethdev.c    | 28 ++++++++++++++++++++
 drivers/net/idpf/idpf_ethdev.c      | 22 ++++++++++++++++
 lib/ethdev/rte_ethdev.c             | 40 +++++++++++++++++++++++++++++
 10 files changed, 194 insertions(+), 5 deletions(-)

-- 
2.30.0

[PATCH v2 1/8] lib/ethdev: update Rx and Tx queue status

[Jie Hai]

The DPDK framework reports the queue status, which is stored in
'dev->data->tx_queue_state' and 'dev->data->rx_queue_state'.The
state is currently maintained by the drivers. Users may determine
whether a queue participates in packet forwarding based on the
state. However, not all drivers correctly report the queue status.
This may cause forwarding problems.

Since it is difficult to modify the queue status of all drivers,
we consider updating the queue status at the framework level.
Some drivers support queues for hairpin, leaving status updating
for such queues to the drivers. Some drivers support
'deferred_start'. Assume that all drivers that support
'deferred_start' can obtain the configuration through
'rte_eth_tx_queue_info_get' and 'rte_eth_rx_queue_info_get'. So
we can directly update the queue status in 'rte_eth_dev_start'
and 'rte_eth_dev_stop'.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 lib/ethdev/rte_ethdev.c | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
index 0840d2b5942a..e3aaa71eba9e 100644
--- a/lib/ethdev/rte_ethdev.c
+++ b/lib/ethdev/rte_ethdev.c
@@ -1641,6 +1641,9 @@ rte_eth_dev_start(uint16_t port_id)
 	struct rte_eth_dev_info dev_info;
 	int diag;
 	int ret, ret_stop;
+	uint16_t i;
+	struct rte_eth_rxq_info rxq_info;
+	struct rte_eth_txq_info txq_info;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
 	dev = &rte_eth_devices[port_id];
@@ -1697,6 +1700,30 @@ rte_eth_dev_start(uint16_t port_id)
 		(*dev->dev_ops->link_update)(dev, 0);
 	}
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++) {
+		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
+			continue;
+
+		memset(&rxq_info, 0, sizeof(rxq_info));
+		ret = rte_eth_rx_queue_info_get(port_id, i, &rxq_info);
+		if (ret == 0 && rxq_info.conf.rx_deferred_start != 0)
+			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+		else
+			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	}
+
+	for (i = 0; i < dev->data->nb_tx_queues; i++) {
+		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
+			continue;
+
+		memset(&txq_info, 0, sizeof(txq_info));
+		ret = rte_eth_tx_queue_info_get(port_id, i, &txq_info);
+		if (ret == 0 && txq_info.conf.tx_deferred_start != 0)
+			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+		else
+			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
+	}
+
 	/* expose selection of PMD fast-path functions */
 	eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
 
@@ -1708,6 +1735,7 @@ int
 rte_eth_dev_stop(uint16_t port_id)
 {
 	struct rte_eth_dev *dev;
+	uint16_t i;
 	int ret;
 
 	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
@@ -1731,6 +1759,18 @@ rte_eth_dev_stop(uint16_t port_id)
 		dev->data->dev_started = 0;
 	rte_ethdev_trace_stop(port_id, ret);
 
+	for (i = 0; i < dev->data->nb_rx_queues; i++) {
+		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
+			continue;
+		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	}
+
+	for (i = 0; i < dev->data->nb_tx_queues; i++) {
+		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
+			continue;
+		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
+	}
+
 	return ret;
 }
 
-- 
2.30.0

[PATCH v2 2/8] net/cpfl: support getting queue information

[Jie Hai]

This patch adds support for querying Rx/Tx queue information.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/cpfl/cpfl_ethdev.c |  2 ++
 drivers/net/cpfl/cpfl_rxtx.c   | 26 ++++++++++++++++++++++++++
 drivers/net/cpfl/cpfl_rxtx.h   |  4 ++++
 3 files changed, 32 insertions(+)

diff --git a/drivers/net/cpfl/cpfl_ethdev.c b/drivers/net/cpfl/cpfl_ethdev.c
index c4ca9343c3e0..05604f99ed44 100644
--- a/drivers/net/cpfl/cpfl_ethdev.c
+++ b/drivers/net/cpfl/cpfl_ethdev.c
@@ -1291,6 +1291,8 @@ static const struct eth_dev_ops cpfl_eth_dev_ops = {
 	.tx_queue_stop			= cpfl_tx_queue_stop,
 	.rx_queue_release		= cpfl_dev_rx_queue_release,
 	.tx_queue_release		= cpfl_dev_tx_queue_release,
+	.rxq_info_get			= cpfl_dev_rxq_info_get,
+	.txq_info_get			= cpfl_dev_txq_info_get,
 	.mtu_set			= cpfl_dev_mtu_set,
 	.dev_supported_ptypes_get	= cpfl_dev_supported_ptypes_get,
 	.stats_get			= cpfl_dev_stats_get,
diff --git a/drivers/net/cpfl/cpfl_rxtx.c b/drivers/net/cpfl/cpfl_rxtx.c
index 2ef6871a8509..7636162b332a 100644
--- a/drivers/net/cpfl/cpfl_rxtx.c
+++ b/drivers/net/cpfl/cpfl_rxtx.c
@@ -1606,3 +1606,29 @@ cpfl_set_tx_function(struct rte_eth_dev *dev)
 	}
 #endif /* RTE_ARCH_X86 */
 }
+
+void
+cpfl_dev_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+		      struct rte_eth_rxq_info *qinfo)
+{
+	struct cpfl_rx_queue *cpfl_rxq;
+
+	cpfl_rxq = dev->data->rx_queues[rx_queue_id];
+	qinfo->nb_desc = cpfl_rxq->base.nb_rx_desc;
+	qinfo->conf.rx_free_thresh = cpfl_rxq->base.nb_rx_desc;
+	qinfo->conf.rx_free_thresh = cpfl_rxq->base.rx_free_thresh;
+	qinfo->conf.rx_deferred_start = cpfl_rxq->base.rx_deferred_start;
+}
+
+void
+cpfl_dev_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+		      struct rte_eth_txq_info *qinfo)
+{
+	struct cpfl_tx_queue *cpfl_txq;
+
+	cpfl_txq = dev->data->tx_queues[tx_queue_id];
+	qinfo->nb_desc = cpfl_txq->base.nb_tx_desc;
+	qinfo->conf.tx_rs_thresh = cpfl_txq->base.rs_thresh;
+	qinfo->conf.tx_free_thresh = cpfl_txq->base.free_thresh;
+	qinfo->conf.tx_deferred_start = cpfl_txq->base.tx_deferred_start;
+}
diff --git a/drivers/net/cpfl/cpfl_rxtx.h b/drivers/net/cpfl/cpfl_rxtx.h
index aacd087b56cd..1ec14ed24cd6 100644
--- a/drivers/net/cpfl/cpfl_rxtx.h
+++ b/drivers/net/cpfl/cpfl_rxtx.h
@@ -102,6 +102,10 @@ int cpfl_tx_queue_stop(struct rte_eth_dev *dev, uint16_t tx_queue_id);
 int cpfl_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
 void cpfl_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
 void cpfl_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
+void cpfl_dev_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+			   struct rte_eth_rxq_info *qinfo);
+void cpfl_dev_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+			   struct rte_eth_txq_info *qinfo);
 void cpfl_set_rx_function(struct rte_eth_dev *dev);
 void cpfl_set_tx_function(struct rte_eth_dev *dev);
 int cpfl_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx,
-- 
2.30.0

[PATCH v2 3/8] net/enetc: save deferred start configuratin for queues

[Jie Hai]

This patch adds new fields to 'struct enetc_bdr' to save the
configuratin of Rx and Tx deferred start.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/enetc/enetc.h        | 2 ++
 drivers/net/enetc/enetc_ethdev.c | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/drivers/net/enetc/enetc.h b/drivers/net/enetc/enetc.h
index 7163633bcedf..170854cd1649 100644
--- a/drivers/net/enetc/enetc.h
+++ b/drivers/net/enetc/enetc.h
@@ -74,6 +74,8 @@ struct enetc_bdr {
 	};
 	struct rte_mempool *mb_pool;   /* mbuf pool to populate RX ring. */
 	struct rte_eth_dev *ndev;
+	bool rx_deferred_start;
+	bool tx_deferred_start;
 };
 
 /*
diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index 1b4337bc488c..444792b7f77e 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -326,9 +326,11 @@ enetc_tx_queue_setup(struct rte_eth_dev *dev,
 			       ENETC_TBMR, ENETC_TBMR_EN);
 		dev->data->tx_queue_state[tx_ring->index] =
 			       RTE_ETH_QUEUE_STATE_STARTED;
+		tx_ring->tx_deferred_start = false;
 	} else {
 		dev->data->tx_queue_state[tx_ring->index] =
 			       RTE_ETH_QUEUE_STATE_STOPPED;
+		tx_ring->tx_deferred_start = true;
 	}
 
 	return 0;
@@ -473,9 +475,11 @@ enetc_rx_queue_setup(struct rte_eth_dev *dev,
 			       ENETC_RBMR_EN);
 		dev->data->rx_queue_state[rx_ring->index] =
 			       RTE_ETH_QUEUE_STATE_STARTED;
+		rx_ring->rx_deferred_start = false;
 	} else {
 		dev->data->rx_queue_state[rx_ring->index] =
 			       RTE_ETH_QUEUE_STATE_STOPPED;
+		rx_ring->rx_deferred_start = true;
 	}
 
 	rx_ring->crc_len = (uint8_t)((rx_offloads & RTE_ETH_RX_OFFLOAD_KEEP_CRC) ?
-- 
2.30.0

[PATCH v2 4/8] net/enetc: support getting queue information

[Jie Hai]

This patch adds support for querying Rx/Tx queue information.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/enetc/enetc_ethdev.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/drivers/net/enetc/enetc_ethdev.c b/drivers/net/enetc/enetc_ethdev.c
index 444792b7f77e..07ee07c4d5d8 100644
--- a/drivers/net/enetc/enetc_ethdev.c
+++ b/drivers/net/enetc/enetc_ethdev.c
@@ -828,6 +828,26 @@ enetc_tx_queue_stop(struct rte_eth_dev *dev, uint16_t qidx)
 	return 0;
 }
 
+static void
+enetc_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+		   struct rte_eth_rxq_info *qinfo)
+{
+	struct enetc_bdr *rx_ring;
+
+	rx_ring = dev->data->rx_queues[queue_id];
+	qinfo->conf.rx_deferred_start = rx_ring->rx_deferred_start;
+}
+
+static void
+enetc_txq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
+		   struct rte_eth_txq_info *qinfo)
+{
+	struct enetc_bdr *tx_ring;
+
+	tx_ring = dev->data->tx_queues[queue_id];
+	qinfo->conf.tx_deferred_start = tx_ring->tx_deferred_start;
+}
+
 /*
  * The set of PCI devices this driver supports
  */
@@ -856,10 +876,12 @@ static const struct eth_dev_ops enetc_ops = {
 	.rx_queue_start       = enetc_rx_queue_start,
 	.rx_queue_stop        = enetc_rx_queue_stop,
 	.rx_queue_release     = enetc_rx_queue_release,
+	.rxq_info_get         = enetc_rxq_info_get,
 	.tx_queue_setup       = enetc_tx_queue_setup,
 	.tx_queue_start       = enetc_tx_queue_start,
 	.tx_queue_stop        = enetc_tx_queue_stop,
 	.tx_queue_release     = enetc_tx_queue_release,
+	.txq_info_get         = enetc_txq_info_get,
 	.dev_supported_ptypes_get = enetc_supported_ptypes_get,
 };
 
-- 
2.30.0

[PATCH v2 5/8] net/failsafe: support getting queue information

[Jie Hai]

This patch adds support for querying Rx/Tx queue information.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/failsafe/failsafe_ops.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index 35649b6244a5..e84dbae5563c 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -1565,6 +1565,28 @@ fs_rss_hash_update(struct rte_eth_dev *dev,
 	return 0;
 }
 
+static void
+fs_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+		struct rte_eth_rxq_info *qinfo)
+{
+	struct rxq *rxq;
+
+	rxq = dev->data->rx_queues[rx_queue_id];
+	qinfo->nb_desc = rxq->info.nb_desc;
+	qinfo->conf.rx_deferred_start = rxq->info.conf.rx_deferred_start;
+}
+
+static void
+fs_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+		struct rte_eth_txq_info *qinfo)
+{
+	struct txq *txq;
+
+	txq = dev->data->tx_queues[tx_queue_id];
+	qinfo->nb_desc = txq->info.nb_desc;
+	qinfo->conf.tx_deferred_start = txq->info.conf.tx_deferred_start;
+}
+
 static int
 fs_flow_ops_get(struct rte_eth_dev *dev __rte_unused,
 		const struct rte_flow_ops **ops)
@@ -1604,6 +1626,8 @@ const struct eth_dev_ops failsafe_ops = {
 	.tx_queue_release = fs_tx_queue_release,
 	.rx_queue_intr_enable = fs_rx_intr_enable,
 	.rx_queue_intr_disable = fs_rx_intr_disable,
+	.rxq_info_get = fs_rxq_info_get,
+	.txq_info_get = fs_txq_info_get,
 	.flow_ctrl_get = fs_flow_ctrl_get,
 	.flow_ctrl_set = fs_flow_ctrl_set,
 	.mac_addr_remove = fs_mac_addr_remove,
-- 
2.30.0

[PATCH v2 6/8] net/fm10k: support getting queue information

[Jie Hai]

This patch adds support for querying Rx/Tx queue information.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/fm10k/fm10k_ethdev.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 4d3c4c10cfa4..e12177b7cdba 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2835,6 +2835,32 @@ fm10k_dev_close(struct rte_eth_dev *dev)
 	return ret;
 }
 
+
+static void
+fm10k_dev_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+		  struct rte_eth_rxq_info *qinfo)
+{
+	struct fm10k_rx_queue *rxq;
+
+	rxq = dev->data->rx_queues[rx_queue_id];
+	qinfo->nb_desc = rxq->nb_desc;
+	qinfo->conf.rx_drop_en = rxq->drop_en;
+	qinfo->conf.rx_deferred_start = rxq->rx_deferred_start;
+}
+
+static void
+fm10k_dev_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+		  struct rte_eth_txq_info *qinfo)
+{
+	struct fm10k_tx_queue *txq;
+
+	txq = dev->data->tx_queues[tx_queue_id];
+	qinfo->nb_desc = txq->nb_desc;
+	qinfo->conf.tx_rs_thresh = txq->rs_thresh;
+	qinfo->conf.tx_free_thresh = txq->free_thresh;
+	qinfo->conf.tx_deferred_start = txq->tx_deferred_start;
+}
+
 static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.dev_configure		= fm10k_dev_configure,
 	.dev_start		= fm10k_dev_start,
@@ -2866,6 +2892,8 @@ static const struct eth_dev_ops fm10k_eth_dev_ops = {
 	.tx_queue_release	= fm10k_tx_queue_release,
 	.rx_queue_intr_enable	= fm10k_dev_rx_queue_intr_enable,
 	.rx_queue_intr_disable	= fm10k_dev_rx_queue_intr_disable,
+	.rxq_info_get		= fm10k_dev_rxq_info_get,
+	.txq_info_get		= fm10k_dev_txq_info_get,
 	.reta_update		= fm10k_reta_update,
 	.reta_query		= fm10k_reta_query,
 	.rss_hash_update	= fm10k_rss_hash_update,
-- 
2.30.0

[PATCH v2 7/8] net/idpf: support getting queue information

[Jie Hai]

This patch adds support for querying Rx/Tx queue information.

Signed-off-by: Jie Hai <haijie1@huawei.com>
---
 drivers/net/idpf/idpf_ethdev.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/drivers/net/idpf/idpf_ethdev.c b/drivers/net/idpf/idpf_ethdev.c
index 3af7cf0bb7e0..c3819e4a863c 100644
--- a/drivers/net/idpf/idpf_ethdev.c
+++ b/drivers/net/idpf/idpf_ethdev.c
@@ -835,6 +835,26 @@ idpf_dev_close(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+idpf_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
+		  struct rte_eth_rxq_info *qinfo)
+{
+	struct idpf_rx_queue *rxq;
+
+	rxq = dev->data->rx_queues[rx_queue_id];
+	qinfo->conf.rx_deferred_start = rxq->rx_deferred_start;
+}
+
+static void
+idpf_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
+		  struct rte_eth_txq_info *qinfo)
+{
+	struct idpf_tx_queue *txq;
+
+	txq = dev->data->tx_queues[tx_queue_id];
+	qinfo->conf.tx_deferred_start = txq->tx_deferred_start;
+}
+
 static const struct eth_dev_ops idpf_eth_dev_ops = {
 	.dev_configure			= idpf_dev_configure,
 	.dev_close			= idpf_dev_close,
@@ -850,6 +870,8 @@ static const struct eth_dev_ops idpf_eth_dev_ops = {
 	.tx_queue_stop			= idpf_tx_queue_stop,
 	.rx_queue_release		= idpf_dev_rx_queue_release,
 	.tx_queue_release		= idpf_dev_tx_queue_release,
+	.rxq_info_get			= idpf_rxq_info_get,
+	.txq_info_get			= idpf_txq_info_get,
 	.mtu_set			= idpf_dev_mtu_set,
 	.dev_supported_ptypes_get	= idpf_dev_supported_ptypes_get,
 	.stats_get			= idpf_dev_stats_get,
-- 
2.30.0

[PATCH v2 8/8] app/testpmd: fix primary process not polling all queues

[Jie Hai]

Here's how the problem arises.
step1: Start the app.
    dpdk-testpmd -a 0000:35:00.0 -l 0-3 -- -i --rxq=10 --txq=10

step2: Perform the following steps and send traffic. As expected,
queue 7 does not send or receive packets, and other queues do.
    port 0 rxq 7 stop
    port 0 txq 7 stop
    set fwd mac
    start

step3: Perform the following steps and send traffic. All queues
are expected to send and receive packets normally, but that's not
the case for queue 7.
    stop
    port stop all
    port start all
    start
    show port xstats all

In fact, only the value of rx_q7_packets for queue 7 is not zero,
which means queue 7 is enabled for the driver but is not involved
in packet receiving and forwarding by software. If we check queue
state by command 'show rxq info 0 7' and 'show txq info 0 7',
we see queue 7 is started as other queues are.
    Rx queue state: started
    Tx queue state: started
The queue 7 is started but cannot forward. That's the problem.

We know that each stream has a read-only "disabled" field that
control if this stream should be used to forward. This field
depends on testpmd local queue state, please see
commit 3c4426db54fc ("app/testpmd: do not poll stopped queues").
DPDK framework maintains ethdev queue state that drivers reported,
which indicates the real state of queues.

There are commands that update these two kind queue state such as
'port X rxq|txq start|stop'. But these operations take effect only
in one stop-start round. In the following stop-start round, the
preceding operations do not take effect anymore. However, only
the ethdev queue state is updated, causing the testpmd and ethdev
state information to diverge and causing unexpected side effects
as above problem.

There was a similar problem for the secondary process, please see
commit 5028f207a4fa ("app/testpmd: fix secondary process packet
forwarding").

This patch applies its workaround with some difference to the
primary process. Not all PMDs implement rte_eth_rx_queue_info_get and
rte_eth_tx_queue_info_get, however they may support deferred_start
with primary process. To not break their behavior, retain the original
testpmd local queue state for those PMDs.

Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
Cc: stable@dpdk.org

Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
---
 app/test-pmd/testpmd.c | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 938ca035d4f8..079ef3392014 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2424,6 +2424,13 @@ update_rx_queue_state(uint16_t port_id, uint16_t queue_id)
 		ports[port_id].rxq[queue_id].state =
 			rx_qinfo.queue_state;
 	} else if (rc == -ENOTSUP) {
+		/*
+		 * Do not change the rxq state for primary process
+		 * to ensure that the PMDs do not implement
+		 * rte_eth_rx_queue_info_get can forward as before.
+		 */
+		if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+			return;
 		/*
 		 * Set the rxq state to RTE_ETH_QUEUE_STATE_STARTED
 		 * to ensure that the PMDs do not implement
@@ -2449,6 +2456,13 @@ update_tx_queue_state(uint16_t port_id, uint16_t queue_id)
 		ports[port_id].txq[queue_id].state =
 			tx_qinfo.queue_state;
 	} else if (rc == -ENOTSUP) {
+		/*
+		 * Do not change the txq state for primary process
+		 * to ensure that the PMDs do not implement
+		 * rte_eth_tx_queue_info_get can forward as before.
+		 */
+		if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+			return;
 		/*
 		 * Set the txq state to RTE_ETH_QUEUE_STATE_STARTED
 		 * to ensure that the PMDs do not implement
@@ -2463,12 +2477,15 @@ update_tx_queue_state(uint16_t port_id, uint16_t queue_id)
 }
 
 static void
-update_queue_state(void)
+update_queue_state(portid_t pid)
 {
 	portid_t pi;
 	queueid_t qi;
 
 	RTE_ETH_FOREACH_DEV(pi) {
+		if (pid != pi && pid != (portid_t)RTE_PORT_ALL)
+			continue;
+
 		for (qi = 0; qi < nb_rxq; qi++)
 			update_rx_queue_state(pi, qi);
 		for (qi = 0; qi < nb_txq; qi++)
@@ -2516,8 +2533,7 @@ start_packet_forwarding(int with_tx_first)
 		return;
 
 	if (stream_init != NULL) {
-		if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-			update_queue_state();
+		update_queue_state(RTE_PORT_ALL);
 		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
 			stream_init(fwd_streams[i]);
 	}
@@ -3280,8 +3296,7 @@ start_port(portid_t pid)
 		pl[cfg_pi++] = pi;
 	}
 
-	if (rte_eal_process_type() == RTE_PROC_SECONDARY)
-		update_queue_state();
+	update_queue_state(pi);
 
 	if (at_least_one_port_successfully_started && !no_link_check)
 		check_all_ports_link_status(RTE_PORT_ALL);
-- 
2.30.0

Re: [PATCH v2 1/8] lib/ethdev: update Rx and Tx queue status

[lihuisong (C)]

在 2023/9/28 15:42, Jie Hai 写道:
> The DPDK framework reports the queue status, which is stored in
> 'dev->data->tx_queue_state' and 'dev->data->rx_queue_state'.The
> state is currently maintained by the drivers. Users may determine
> whether a queue participates in packet forwarding based on the
> state. However, not all drivers correctly report the queue status.
> This may cause forwarding problems.
>
> Since it is difficult to modify the queue status of all drivers,
> we consider updating the queue status at the framework level.
> Some drivers support queues for hairpin, leaving status updating
> for such queues to the drivers. Some drivers support
> 'deferred_start'. Assume that all drivers that support
> 'deferred_start' can obtain the configuration through
> 'rte_eth_tx_queue_info_get' and 'rte_eth_rx_queue_info_get'. So
> we can directly update the queue status in 'rte_eth_dev_start'
> and 'rte_eth_dev_stop'.
>
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>   lib/ethdev/rte_ethdev.c | 40 ++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 40 insertions(+)
>
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> index 0840d2b5942a..e3aaa71eba9e 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -1641,6 +1641,9 @@ rte_eth_dev_start(uint16_t port_id)
>   	struct rte_eth_dev_info dev_info;
>   	int diag;
>   	int ret, ret_stop;
> +	uint16_t i;
> +	struct rte_eth_rxq_info rxq_info;
> +	struct rte_eth_txq_info txq_info;
>   
>   	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>   	dev = &rte_eth_devices[port_id];
> @@ -1697,6 +1700,30 @@ rte_eth_dev_start(uint16_t port_id)
>   		(*dev->dev_ops->link_update)(dev, 0);
>   	}
>   
> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> +		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
> +			continue;
> +
> +		memset(&rxq_info, 0, sizeof(rxq_info));
This memset() is unsed because this operation is already in the 
rte_eth_rx/tx_queue_info_get().
> +		ret = rte_eth_rx_queue_info_get(port_id, i, &rxq_info);
> +		if (ret == 0 && rxq_info.conf.rx_deferred_start != 0)
> +			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +		else
> +			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +	}
> +
> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
> +		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
> +			continue;
> +
> +		memset(&txq_info, 0, sizeof(txq_info));
> +		ret = rte_eth_tx_queue_info_get(port_id, i, &txq_info);
> +		if (ret == 0 && txq_info.conf.tx_deferred_start != 0)
> +			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +		else
> +			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +	}
Suggest: extract a functon to start/stop Rx/Tx queue states.
> +
>   	/* expose selection of PMD fast-path functions */
>   	eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
>   
> @@ -1708,6 +1735,7 @@ int
>   rte_eth_dev_stop(uint16_t port_id)
>   {
>   	struct rte_eth_dev *dev;
> +	uint16_t i;
>   	int ret;
>   
>   	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> @@ -1731,6 +1759,18 @@ rte_eth_dev_stop(uint16_t port_id)
>   		dev->data->dev_started = 0;
>   	rte_ethdev_trace_stop(port_id, ret);
>   
> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> +		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
> +			continue;
> +		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +	}
> +
> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
> +		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
> +			continue;
> +		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +	}
> +
>   	return ret;
>   }
>   

Re: [PATCH 00/36] fix Rx and Tx queue state

[Ferruh Yigit]

On 9/22/2023 7:41 AM, David Marchand wrote:
> Hello,
> 
> On Fri, Sep 22, 2023 at 4:41 AM Jie Hai <haijie1@huawei.com> wrote:
>> On 2023/9/19 0:54, Ferruh Yigit wrote:
>>> On 9/8/2023 12:50 PM, David Marchand wrote:
>>>> On Fri, Sep 8, 2023 at 1:32 PM Jie Hai <haijie1@huawei.com> wrote:
>>>>>
>>>>> The DPDK framework reports the queue state, which is stored in
>>>>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>>>>> state is maintained by the driver. Users may determine whether
>>>>> a queue participates in packet forwarding based on the state,
>>>>> for example,
>>>>
>>>> The driver is maintaining this state in dev_start / dev_stop and per
>>>> queue start/stop handlers.
>>>>
>>>>>
>>>>> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
>>>>> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
>>>>>
>>>>> Therefore, the drivers need to modify the queue state in time
>>>>> according to the actual situation, especially when dev_start
>>>>> and dev_stop are called. see [3] for more information.
>>>>>
>>>>> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
>>>>>
>>>>> This patchset also resubmit the patch [2] and makes some fixes on the patch.
>>>>
>>>> I just had a quick look at some patches and I wonder if a better fix
>>>> would be at the ethdev level, rather than fixing all drivers.
>>>>
>>>>
>>>
>>> I came here to make the same comment,
>>>
>>> Jie, I forgot if we discuss this already but,
>>>
>>> does it work if we update queue state in 'rte_eth_dev_start()' &
>>> 'rte_eth_dev_stop()' when 'dev_start' & 'dev_stop' dev_ops succeeds?
>>>
>>> This solves the case driver forgets to update the queue state.
>>>
>>>
>> Hi, Furrh and David,
>>
>> It's OK for dev_stop, but not enough for dev_start.
>> Some drivers also support deferred_start.
>> Therefore, not all queues are in the start state after dev_start.
>>
>> If we want to get that correct queue state at the framework level, I
>> offer the following options:
>>
>> step 1. A capability(e.g. RTE_ETH_DEV_CAPA_DEFERRED_START) is added to
>> the framework, indicating whether the driver supports deferred_start.
>> The capability should be reported by the driver and user can get it by
>> rte_eth_dev_info_get().
>> step 2. All drivers that support deferred_start should report configuration
>> information about deferred_start through
>> rte_eth_rx_queue_info_get |rte_eth_tx_queue_info_get.
>> step 3.The framework updates the queue status based on the support and
>> configuration of deferred_start.
> 
> rte_eth_dev_start must only update the queue state if
> rx_deferred_start is unset (see struct
> rte_eth_rxconf::rx_deferred_start).
> And the queue state needs to be updated in ethdev
> rte_eth_dev_rx_queue_start/stop.
> 
> So I don't see where we need a new capability.
> 
> 

Hi David,

Problem seems 'rte_eth_rxconf::rx_deferred_start' is not stored in
ethdev level, so rte_eth_dev_start() can't use it.


And although application can request it, via rxconf, it is not clear if
driver supports it or if it will take into account, that is where a
capability flag can be useful, but it is a big change for this patch.

Re: [PATCH v2 1/8] lib/ethdev: update Rx and Tx queue status

[Ferruh Yigit]

On 9/28/2023 8:42 AM, Jie Hai wrote:
> The DPDK framework reports the queue status, which is stored in
> 'dev->data->tx_queue_state' and 'dev->data->rx_queue_state'.The
> state is currently maintained by the drivers. Users may determine
> whether a queue participates in packet forwarding based on the
> state. However, not all drivers correctly report the queue status.
> This may cause forwarding problems.
> 
> Since it is difficult to modify the queue status of all drivers,
> we consider updating the queue status at the framework level.
> Some drivers support queues for hairpin, leaving status updating
> for such queues to the drivers. Some drivers support
> 'deferred_start'. Assume that all drivers that support
> 'deferred_start' can obtain the configuration through
> 'rte_eth_tx_queue_info_get' and 'rte_eth_rx_queue_info_get'. So
> we can directly update the queue status in 'rte_eth_dev_start'
> and 'rte_eth_dev_stop'.
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  lib/ethdev/rte_ethdev.c | 40 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
> 
> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
> index 0840d2b5942a..e3aaa71eba9e 100644
> --- a/lib/ethdev/rte_ethdev.c
> +++ b/lib/ethdev/rte_ethdev.c
> @@ -1641,6 +1641,9 @@ rte_eth_dev_start(uint16_t port_id)
>  	struct rte_eth_dev_info dev_info;
>  	int diag;
>  	int ret, ret_stop;
> +	uint16_t i;
> +	struct rte_eth_rxq_info rxq_info;
> +	struct rte_eth_txq_info txq_info;
>  
>  	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>  	dev = &rte_eth_devices[port_id];
> @@ -1697,6 +1700,30 @@ rte_eth_dev_start(uint16_t port_id)
>  		(*dev->dev_ops->link_update)(dev, 0);
>  	}
>  
> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> +		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
> +			continue;
> +
> +		memset(&rxq_info, 0, sizeof(rxq_info));
> +		ret = rte_eth_rx_queue_info_get(port_id, i, &rxq_info);
> +		if (ret == 0 && rxq_info.conf.rx_deferred_start != 0)
> +			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +		else
> +			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +	}
> +
> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
> +		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
> +			continue;
> +
> +		memset(&txq_info, 0, sizeof(txq_info));
> +		ret = rte_eth_tx_queue_info_get(port_id, i, &txq_info);
> +		if (ret == 0 && txq_info.conf.tx_deferred_start != 0)
> +			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +		else
> +			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
> +	}
> +

Hi Jie,

I am not sure about adding queue_info_get() calls a dependency to
start(), since start() is a mandatory API, I am concerned about possible
side affects.
Like if rte_eth_rx_queue_info_get() fails,
Or event though deferred_start is set, can application first call
rx_queue_start(), later start(), like:
 start()
 rx_queue_start()
 stop()
 rx_queue_start()
 start()
Or even applications sets deferred_start, PMD be ignoring without
reflecting that to conf.rx_deferred_start, etc...


Anyway, intention was to move common task, setting queue state, to the
ethdev layer, but because of the deferred_start, in rte_eth_dev_start()
we don't really know the queue state.

We can try to move deferred state information to the ethdev, but that is
similar to setting queue state in the driver, that is why perhaps better
to leave setting state to drivers, as done in the first version of the set.
@Jie, @David, what do you think?


And, @Jie, can you also check the rx_queue_start() & tx_queue_start()
dev_ops of the drivers sets the queue state?
If missing on some drivers, that needs to be added too.


>  	/* expose selection of PMD fast-path functions */
>  	eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
>  
> @@ -1708,6 +1735,7 @@ int
>  rte_eth_dev_stop(uint16_t port_id)
>  {
>  	struct rte_eth_dev *dev;
> +	uint16_t i;
>  	int ret;
>  
>  	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
> @@ -1731,6 +1759,18 @@ rte_eth_dev_stop(uint16_t port_id)
>  		dev->data->dev_started = 0;
>  	rte_ethdev_trace_stop(port_id, ret);
>  
> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
> +		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
> +			continue;
> +		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +	}
> +
> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
> +		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
> +			continue;
> +		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
> +	}
> +
>  	return ret;
>  }
>  

RE: [PATCH v2 2/8] net/cpfl: support getting queue information

[Ali Alnubani]

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Thursday, September 28, 2023 10:43 AM
> To: dev@dpdk.org; Yuying Zhang <yuying.zhang@intel.com>; Beilei Xing
> <beilei.xing@intel.com>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH v2 2/8] net/cpfl: support getting queue information
> 
> This patch adds support for querying Rx/Tx queue information.
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  drivers/net/cpfl/cpfl_ethdev.c |  2 ++
>  drivers/net/cpfl/cpfl_rxtx.c   | 26 ++++++++++++++++++++++++++
>  drivers/net/cpfl/cpfl_rxtx.h   |  4 ++++
>  3 files changed, 32 insertions(+)
> 
> diff --git a/drivers/net/cpfl/cpfl_ethdev.c b/drivers/net/cpfl/cpfl_ethdev.c
> index c4ca9343c3e0..05604f99ed44 100644
> --- a/drivers/net/cpfl/cpfl_ethdev.c
> +++ b/drivers/net/cpfl/cpfl_ethdev.c
> @@ -1291,6 +1291,8 @@ static const struct eth_dev_ops cpfl_eth_dev_ops
> = {
>  	.tx_queue_stop			= cpfl_tx_queue_stop,
>  	.rx_queue_release		= cpfl_dev_rx_queue_release,
>  	.tx_queue_release		= cpfl_dev_tx_queue_release,
> +	.rxq_info_get			= cpfl_dev_rxq_info_get,
> +	.txq_info_get			= cpfl_dev_txq_info_get,
>  	.mtu_set			= cpfl_dev_mtu_set,
>  	.dev_supported_ptypes_get	= cpfl_dev_supported_ptypes_get,
>  	.stats_get			= cpfl_dev_stats_get,
> diff --git a/drivers/net/cpfl/cpfl_rxtx.c b/drivers/net/cpfl/cpfl_rxtx.c
> index 2ef6871a8509..7636162b332a 100644
> --- a/drivers/net/cpfl/cpfl_rxtx.c
> +++ b/drivers/net/cpfl/cpfl_rxtx.c
> @@ -1606,3 +1606,29 @@ cpfl_set_tx_function(struct rte_eth_dev *dev)
>  	}
>  #endif /* RTE_ARCH_X86 */
>  }
> +
> +void
> +cpfl_dev_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
> +		      struct rte_eth_rxq_info *qinfo)
> +{
> +	struct cpfl_rx_queue *cpfl_rxq;
> +
> +	cpfl_rxq = dev->data->rx_queues[rx_queue_id];
> +	qinfo->nb_desc = cpfl_rxq->base.nb_rx_desc;
> +	qinfo->conf.rx_free_thresh = cpfl_rxq->base.nb_rx_desc;
> +	qinfo->conf.rx_free_thresh = cpfl_rxq->base.rx_free_thresh;

qinfo->conf.rx_free_thresh is being set twice.

> +	qinfo->conf.rx_deferred_start = cpfl_rxq->base.rx_deferred_start;
> +}
> +
> +void
> +cpfl_dev_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
> +		      struct rte_eth_txq_info *qinfo)
> +{
> +	struct cpfl_tx_queue *cpfl_txq;
> +
> +	cpfl_txq = dev->data->tx_queues[tx_queue_id];
> +	qinfo->nb_desc = cpfl_txq->base.nb_tx_desc;
> +	qinfo->conf.tx_rs_thresh = cpfl_txq->base.rs_thresh;
> +	qinfo->conf.tx_free_thresh = cpfl_txq->base.free_thresh;
> +	qinfo->conf.tx_deferred_start = cpfl_txq->base.tx_deferred_start;
> +}
> diff --git a/drivers/net/cpfl/cpfl_rxtx.h b/drivers/net/cpfl/cpfl_rxtx.h
> index aacd087b56cd..1ec14ed24cd6 100644
> --- a/drivers/net/cpfl/cpfl_rxtx.h
> +++ b/drivers/net/cpfl/cpfl_rxtx.h
> @@ -102,6 +102,10 @@ int cpfl_tx_queue_stop(struct rte_eth_dev *dev,
> uint16_t tx_queue_id);
>  int cpfl_rx_queue_stop(struct rte_eth_dev *dev, uint16_t rx_queue_id);
>  void cpfl_dev_tx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
>  void cpfl_dev_rx_queue_release(struct rte_eth_dev *dev, uint16_t qid);
> +void cpfl_dev_rxq_info_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
> +			   struct rte_eth_rxq_info *qinfo);
> +void cpfl_dev_txq_info_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
> +			   struct rte_eth_txq_info *qinfo);
>  void cpfl_set_rx_function(struct rte_eth_dev *dev);
>  void cpfl_set_tx_function(struct rte_eth_dev *dev);
>  int cpfl_rx_hairpin_queue_setup(struct rte_eth_dev *dev, uint16_t
> queue_idx,
> --
> 2.30.0

RE: [PATCH v2 0/8] fix Rx and Tx queue state

[Ali Alnubani]

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Thursday, September 28, 2023 10:43 AM
> To: dev@dpdk.org
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [PATCH v2 0/8] fix Rx and Tx queue state
> 
> The DPDK framework reports the queue status, which is stored in
> 'dev->data->tx_queue_state' and 'dev->data->rx_queue_state' . The state
> is currently maintained by the drivers. Users may determine whether
> a queue participates in packet forwarding based on the state,
> For example,
> 
> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
> 
> However, not all drivers correctly report the queue status. This may cause
> forwarding problems.
> 
> Since it is difficult to modify the queue status of all drivers, we consider
> updating the queue status at the framework level. Some drivers support
> queues
> for hairpin, leaving status updating for such queues to the drivers. Some
> drivers support deferred_start. Assume that all drivers that support
> 'deferred_start' can obtain the configuration through
> 'rte_eth_tx_queue_info_get'
> and 'rte_eth_rx_queue_info_get'. So we can directly update the queue status
> in
> 'rte_eth_dev_start' and 'rte_eth_dev_stop'.
> 
> This patchset does the follow things:
> 1. update Rx and Tx queue status in dev_start and dev_stop.
> 2. implement rxq|txq_info_get ops for drivers supporting deferred start.
> 3. resubmit the patch [2] and makes some fixes on the patch.
> 

Hello,

Testpmd stats no longer show any traffic on Nvidia cards with mlx5 driver, specifically because of the changes in app/test-pmd/testpmd.c.

Regards,
Ali

Re: [PATCH v2 1/8] lib/ethdev: update Rx and Tx queue status

[Jie Hai]

On 2023/9/28 21:15, Ferruh Yigit wrote:
> On 9/28/2023 8:42 AM, Jie Hai wrote:
>> The DPDK framework reports the queue status, which is stored in
>> 'dev->data->tx_queue_state' and 'dev->data->rx_queue_state'.The
>> state is currently maintained by the drivers. Users may determine
>> whether a queue participates in packet forwarding based on the
>> state. However, not all drivers correctly report the queue status.
>> This may cause forwarding problems.
>>
>> Since it is difficult to modify the queue status of all drivers,
>> we consider updating the queue status at the framework level.
>> Some drivers support queues for hairpin, leaving status updating
>> for such queues to the drivers. Some drivers support
>> 'deferred_start'. Assume that all drivers that support
>> 'deferred_start' can obtain the configuration through
>> 'rte_eth_tx_queue_info_get' and 'rte_eth_rx_queue_info_get'. So
>> we can directly update the queue status in 'rte_eth_dev_start'
>> and 'rte_eth_dev_stop'.
>>
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>>   lib/ethdev/rte_ethdev.c | 40 ++++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 40 insertions(+)
>>
>> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
>> index 0840d2b5942a..e3aaa71eba9e 100644
>> --- a/lib/ethdev/rte_ethdev.c
>> +++ b/lib/ethdev/rte_ethdev.c
>> @@ -1641,6 +1641,9 @@ rte_eth_dev_start(uint16_t port_id)
>>   	struct rte_eth_dev_info dev_info;
>>   	int diag;
>>   	int ret, ret_stop;
>> +	uint16_t i;
>> +	struct rte_eth_rxq_info rxq_info;
>> +	struct rte_eth_txq_info txq_info;
>>   
>>   	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>>   	dev = &rte_eth_devices[port_id];
>> @@ -1697,6 +1700,30 @@ rte_eth_dev_start(uint16_t port_id)
>>   		(*dev->dev_ops->link_update)(dev, 0);
>>   	}
>>   
>> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
>> +		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
>> +			continue;
>> +
>> +		memset(&rxq_info, 0, sizeof(rxq_info));
>> +		ret = rte_eth_rx_queue_info_get(port_id, i, &rxq_info);
>> +		if (ret == 0 && rxq_info.conf.rx_deferred_start != 0)
>> +			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>> +		else
>> +			dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
>> +	}
>> +
>> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
>> +		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
>> +			continue;
>> +
>> +		memset(&txq_info, 0, sizeof(txq_info));
>> +		ret = rte_eth_tx_queue_info_get(port_id, i, &txq_info);
>> +		if (ret == 0 && txq_info.conf.tx_deferred_start != 0)
>> +			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>> +		else
>> +			dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
>> +	}
>> +
> 
> Hi Jie,
> 
> I am not sure about adding queue_info_get() calls a dependency to
> start(), since start() is a mandatory API, I am concerned about possible
> side affects.
> Like if rte_eth_rx_queue_info_get() fails,Yes, unless we let rte_eth_rx|tx_queue_info_get a mandatory API.
> Or event though deferred_start is set, can application first call
> rx_queue_start(), later start(), like:
>   start()
>   rx_queue_start()
>   stop()
>   rx_queue_start()
>   start()
    start()  --> deferred_start is confugured, the queue is stopped
    rx_queue_start() --> the queue is started
    stop() --> all queues are stopped
    rx_queue_start() --> not supported, port should starts first
    start()

If we change the order as:
    start()
    rx_queue_start()
    stop()
    start()

The last status of the queue for different drivers may be different.
Most drivers starts all queues except the queue setting deferred_start.
For sfc driver, all queues are started and the status of  the queue
setting deferred_start will be reported as stopped, which is not correct.
That's the point, drivers have their own private processing for 
different sequences of deferred_start, start|stop and queue_start|stop.

> Or even applications sets deferred_start, PMD be ignoring without
> reflecting that to conf.rx_deferred_start, etc...

Supppose the app sets the deferred_start,
whether the driver verifies the support the configuration or not,
whether the driver issues the configuration to the hardware or not,
whether the driver saves and reports the configuration or not,
Different driver implementations vary.

For the above three cases,  pay attention to the following:

1) Y-Y-Y That's what we need.
2) Y-Y-N That's what we should add.
3) N-N-Y This will lead to wrong information(e.g.ice_rxtx.c mlx5_txq.c 
mlx5_rxq.c)
4) N-N-N That's what we need, too.
> 
> 
> Anyway, intention was to move common task, setting queue state, to the
> ethdev layer, but because of the deferred_start, in rte_eth_dev_start()
> we don't really know the queue state.
> 
> We can try to move deferred state information to the ethdev, but that is
> similar to setting queue state in the driver, that is why perhaps better
> to leave setting state to drivers, as done in the first version of the set.
> @Jie, @David, what do you think?
> 
I support V1. Whoever changes the queue status is responsible for 
reporting the queue status.
That will maintains the specificity of the drive treatment.

> 
> And, @Jie, can you also check the rx_queue_start() & tx_queue_start()
> dev_ops of the drivers sets the queue state?
> If missing on some drivers, that needs to be added too.
> 
I have checked already, all drivers support rx_queue_start() & 
tx_queue_start() set the queue state.
> 
>>   	/* expose selection of PMD fast-path functions */
>>   	eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
>>   
>> @@ -1708,6 +1735,7 @@ int
>>   rte_eth_dev_stop(uint16_t port_id)
>>   {
>>   	struct rte_eth_dev *dev;
>> +	uint16_t i;
>>   	int ret;
>>   
>>   	RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>> @@ -1731,6 +1759,18 @@ rte_eth_dev_stop(uint16_t port_id)
>>   		dev->data->dev_started = 0;
>>   	rte_ethdev_trace_stop(port_id, ret);
>>   
>> +	for (i = 0; i < dev->data->nb_rx_queues; i++) {
>> +		if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
>> +			continue;
>> +		dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>> +	}
>> +
>> +	for (i = 0; i < dev->data->nb_tx_queues; i++) {
>> +		if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
>> +			continue;
>> +		dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>> +	}
>> +
>>   	return ret;
>>   }
>>   
> 

Re: [PATCH v2 1/8] lib/ethdev: update Rx and Tx queue status

[Ferruh Yigit]

On 10/7/2023 9:36 AM, Jie Hai wrote:
> On 2023/9/28 21:15, Ferruh Yigit wrote:
>> On 9/28/2023 8:42 AM, Jie Hai wrote:
>>> The DPDK framework reports the queue status, which is stored in
>>> 'dev->data->tx_queue_state' and 'dev->data->rx_queue_state'.The
>>> state is currently maintained by the drivers. Users may determine
>>> whether a queue participates in packet forwarding based on the
>>> state. However, not all drivers correctly report the queue status.
>>> This may cause forwarding problems.
>>>
>>> Since it is difficult to modify the queue status of all drivers,
>>> we consider updating the queue status at the framework level.
>>> Some drivers support queues for hairpin, leaving status updating
>>> for such queues to the drivers. Some drivers support
>>> 'deferred_start'. Assume that all drivers that support
>>> 'deferred_start' can obtain the configuration through
>>> 'rte_eth_tx_queue_info_get' and 'rte_eth_rx_queue_info_get'. So
>>> we can directly update the queue status in 'rte_eth_dev_start'
>>> and 'rte_eth_dev_stop'.
>>>
>>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>>> ---
>>>   lib/ethdev/rte_ethdev.c | 40 ++++++++++++++++++++++++++++++++++++++++
>>>   1 file changed, 40 insertions(+)
>>>
>>> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c
>>> index 0840d2b5942a..e3aaa71eba9e 100644
>>> --- a/lib/ethdev/rte_ethdev.c
>>> +++ b/lib/ethdev/rte_ethdev.c
>>> @@ -1641,6 +1641,9 @@ rte_eth_dev_start(uint16_t port_id)
>>>       struct rte_eth_dev_info dev_info;
>>>       int diag;
>>>       int ret, ret_stop;
>>> +    uint16_t i;
>>> +    struct rte_eth_rxq_info rxq_info;
>>> +    struct rte_eth_txq_info txq_info;
>>>         RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>>>       dev = &rte_eth_devices[port_id];
>>> @@ -1697,6 +1700,30 @@ rte_eth_dev_start(uint16_t port_id)
>>>           (*dev->dev_ops->link_update)(dev, 0);
>>>       }
>>>   +    for (i = 0; i < dev->data->nb_rx_queues; i++) {
>>> +        if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
>>> +            continue;
>>> +
>>> +        memset(&rxq_info, 0, sizeof(rxq_info));
>>> +        ret = rte_eth_rx_queue_info_get(port_id, i, &rxq_info);
>>> +        if (ret == 0 && rxq_info.conf.rx_deferred_start != 0)
>>> +            dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>>> +        else
>>> +            dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
>>> +    }
>>> +
>>> +    for (i = 0; i < dev->data->nb_tx_queues; i++) {
>>> +        if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
>>> +            continue;
>>> +
>>> +        memset(&txq_info, 0, sizeof(txq_info));
>>> +        ret = rte_eth_tx_queue_info_get(port_id, i, &txq_info);
>>> +        if (ret == 0 && txq_info.conf.tx_deferred_start != 0)
>>> +            dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>>> +        else
>>> +            dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STARTED;
>>> +    }
>>> +
>>
>> Hi Jie,
>>
>> I am not sure about adding queue_info_get() calls a dependency to
>> start(), since start() is a mandatory API, I am concerned about possible
>> side affects.
>> Like if rte_eth_rx_queue_info_get() fails,Yes, unless we let
>> rte_eth_rx|tx_queue_info_get a mandatory API.
>> Or event though deferred_start is set, can application first call
>> rx_queue_start(), later start(), like:
>>   start()
>>   rx_queue_start()
>>   stop()
>>   rx_queue_start()
>>   start()
>    start()  --> deferred_start is confugured, the queue is stopped
>    rx_queue_start() --> the queue is started
>    stop() --> all queues are stopped
>    rx_queue_start() --> not supported, port should starts first
>    start()
> 
> If we change the order as:
>    start()
>    rx_queue_start()
>    stop()
>    start()
> 
> The last status of the queue for different drivers may be different.
> Most drivers starts all queues except the queue setting deferred_start.
> For sfc driver, all queues are started and the status of  the queue
> setting deferred_start will be reported as stopped, which is not correct.
> That's the point, drivers have their own private processing for
> different sequences of deferred_start, start|stop and queue_start|stop.
> 
>> Or even applications sets deferred_start, PMD be ignoring without
>> reflecting that to conf.rx_deferred_start, etc...
> 
> Supppose the app sets the deferred_start,
> whether the driver verifies the support the configuration or not,
> whether the driver issues the configuration to the hardware or not,
> whether the driver saves and reports the configuration or not,
> Different driver implementations vary.
> 
> For the above three cases,  pay attention to the following:
> 
> 1) Y-Y-Y That's what we need.
> 2) Y-Y-N That's what we should add.
> 3) N-N-Y This will lead to wrong information(e.g.ice_rxtx.c mlx5_txq.c
> mlx5_rxq.c)
> 4) N-N-N That's what we need, too.
>>
>>
>> Anyway, intention was to move common task, setting queue state, to the
>> ethdev layer, but because of the deferred_start, in rte_eth_dev_start()
>> we don't really know the queue state.
>>
>> We can try to move deferred state information to the ethdev, but that is
>> similar to setting queue state in the driver, that is why perhaps better
>> to leave setting state to drivers, as done in the first version of the
>> set.
>> @Jie, @David, what do you think?
>>
> I support V1. Whoever changes the queue status is responsible for
> reporting the queue status.
> That will maintains the specificity of the drive treatment.
> 

Agreed, thanks Jie, let me switch back to v1.

>>
>> And, @Jie, can you also check the rx_queue_start() & tx_queue_start()
>> dev_ops of the drivers sets the queue state?
>> If missing on some drivers, that needs to be added too.
>>
> I have checked already, all drivers support rx_queue_start() &
> tx_queue_start() set the queue state.
>>
>>>       /* expose selection of PMD fast-path functions */
>>>       eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev);
>>>   @@ -1708,6 +1735,7 @@ int
>>>   rte_eth_dev_stop(uint16_t port_id)
>>>   {
>>>       struct rte_eth_dev *dev;
>>> +    uint16_t i;
>>>       int ret;
>>>         RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV);
>>> @@ -1731,6 +1759,18 @@ rte_eth_dev_stop(uint16_t port_id)
>>>           dev->data->dev_started = 0;
>>>       rte_ethdev_trace_stop(port_id, ret);
>>>   +    for (i = 0; i < dev->data->nb_rx_queues; i++) {
>>> +        if (rte_eth_dev_is_rx_hairpin_queue(dev, i))
>>> +            continue;
>>> +        dev->data->rx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>>> +    }
>>> +
>>> +    for (i = 0; i < dev->data->nb_tx_queues; i++) {
>>> +        if (rte_eth_dev_is_tx_hairpin_queue(dev, i))
>>> +            continue;
>>> +        dev->data->tx_queue_state[i] = RTE_ETH_QUEUE_STATE_STOPPED;
>>> +    }
>>> +
>>>       return ret;
>>>   }
>>>   
>>

Re: [PATCH 00/36] fix Rx and Tx queue state

[Ferruh Yigit]

On 9/8/2023 12:28 PM, Jie Hai wrote:
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether
> a queue participates in packet forwarding based on the state,
> for example,
> 
> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
> 
> Therefore, the drivers need to modify the queue state in time
> according to the actual situation, especially when dev_start
> and dev_stop are called. see [3] for more information.
> 
> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
> 
> This patchset also resubmit the patch [2] and makes some fixes on the patch.
> 
> Jie Hai (36):
>   net/axgbe: fix Rx and Tx queue state
>   net/af_packet: fix Rx and Tx queue state
>   net/af_xdp: fix Rx and Tx queue state
>   net/avp: fix Rx and Tx queue state
>   net/bnx2x: fix Rx and Tx queue state
>   net/bnxt: fix Rx and Tx queue state
>   net/bonding: fix Rx and Tx queue state
>   net/cxgbe: fix Rx and Tx queue state
>   net/dpaa: fix Rx and Tx queue state
>   net/dpaa2: fix Rx and Tx queue state
>   net/e1000: fix Rx and Tx queue state
>   net/ena: fix Rx and Tx queue state
>   net/enetc: fix Rx and Tx queue state
>   net/enic: fix Rx and Tx queue state
>   net/hinic: fix Rx and Tx queue state
>   net/ipn3ke: fix Rx and Tx queue state
>   net/memif: fix Rx and Tx queue state
>   net/mana: fix Rx and Tx queue state
>   net/mlx4: fix Rx and Tx queue state
>   net/mvneta: fix Rx and Tx queue state
>   net/mvpp2: fix Rx and Tx queue state
>   net/netvsc: fix Rx and Tx queue state
>   net/nfp: fix Rx and Tx queue state
>   net/ngbe: fix Rx and Tx queue state
>   net/null: fix Rx and Tx queue state
>   net/octeon_ep: fix Rx and Tx queue state
>   net/octeontx: fix Rx and Tx queue state
>   net/pfe: fix Rx and Tx queue state
>   net/ring: fix Rx and Tx queue state
>   net/sfc: fix Rx and Tx queue state
>   net/softnic: fix Rx and Tx queue state
>   net/txgbe: fix Rx and Tx queue state
>   net/vhost: fix Rx and Tx queue state
>   net/virtio: fix Rx and Tx queue state
>   net/vmxnet3: fix Rx and Tx queue state
>   app/testpmd: fix primary process not polling all queues
> 

For series,
Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>


Except net/mana (it has separate patch by its maintainer),
Series applied to dpdk-next-net/main, thanks.

Re: [PATCH 00/36] fix Rx and Tx queue state

[Ferruh Yigit]

On 10/16/2023 12:51 PM, Ferruh Yigit wrote:
> On 9/8/2023 12:28 PM, Jie Hai wrote:
>> The DPDK framework reports the queue state, which is stored in
>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>> state is maintained by the driver. Users may determine whether
>> a queue participates in packet forwarding based on the state,
>> for example,
>>
>> [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
>> [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
>>
>> Therefore, the drivers need to modify the queue state in time
>> according to the actual situation, especially when dev_start
>> and dev_stop are called. see [3] for more information.
>>
>> [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
>>
>> This patchset also resubmit the patch [2] and makes some fixes on the patch.
>>
>> Jie Hai (36):
>>   net/axgbe: fix Rx and Tx queue state
>>   net/af_packet: fix Rx and Tx queue state
>>   net/af_xdp: fix Rx and Tx queue state
>>   net/avp: fix Rx and Tx queue state
>>   net/bnx2x: fix Rx and Tx queue state
>>   net/bnxt: fix Rx and Tx queue state
>>   net/bonding: fix Rx and Tx queue state
>>   net/cxgbe: fix Rx and Tx queue state
>>   net/dpaa: fix Rx and Tx queue state
>>   net/dpaa2: fix Rx and Tx queue state
>>   net/e1000: fix Rx and Tx queue state
>>   net/ena: fix Rx and Tx queue state
>>   net/enetc: fix Rx and Tx queue state
>>   net/enic: fix Rx and Tx queue state
>>   net/hinic: fix Rx and Tx queue state
>>   net/ipn3ke: fix Rx and Tx queue state
>>   net/memif: fix Rx and Tx queue state
>>   net/mana: fix Rx and Tx queue state
>>   net/mlx4: fix Rx and Tx queue state
>>   net/mvneta: fix Rx and Tx queue state
>>   net/mvpp2: fix Rx and Tx queue state
>>   net/netvsc: fix Rx and Tx queue state
>>   net/nfp: fix Rx and Tx queue state
>>   net/ngbe: fix Rx and Tx queue state
>>   net/null: fix Rx and Tx queue state
>>   net/octeon_ep: fix Rx and Tx queue state
>>   net/octeontx: fix Rx and Tx queue state
>>   net/pfe: fix Rx and Tx queue state
>>   net/ring: fix Rx and Tx queue state
>>   net/sfc: fix Rx and Tx queue state
>>   net/softnic: fix Rx and Tx queue state
>>   net/txgbe: fix Rx and Tx queue state
>>   net/vhost: fix Rx and Tx queue state
>>   net/virtio: fix Rx and Tx queue state
>>   net/vmxnet3: fix Rx and Tx queue state
>>   app/testpmd: fix primary process not polling all queues
>>
> 
> For series,
> Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>
> 
> 
> Except net/mana (it has separate patch by its maintainer),
> Series applied to dpdk-next-net/main, thanks.
> 

net/netvsc also dropped from this set and maintainer version [1] merged.

[1]
https://patchwork.dpdk.org/project/dpdk/patch/1693333769-10296-1-git-send-email-longli@linuxonhyperv.com/

Re: [PATCH 00/36] fix Rx and Tx queue state

[Thomas Monjalon]

16/10/2023 13:51, Ferruh Yigit:
> On 9/8/2023 12:28 PM, Jie Hai wrote:
> > The DPDK framework reports the queue state, which is stored in
> > dev->data->tx_queue_state and dev->data->rx_queue_state. The
> > state is maintained by the driver. Users may determine whether
> > a queue participates in packet forwarding based on the state,
> > for example,
> > 
> > [1] 5028f207a4fa ("app/testpmd: fix secondary process packet forwarding"
> > [2] 141a520b35f7 ("app/testpmd: fix primary process not polling all queues")
> > 
> > Therefore, the drivers need to modify the queue state in time
> > according to the actual situation, especially when dev_start
> > and dev_stop are called. see [3] for more information.
> > 
> > [3] https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/
> > 
> > This patchset also resubmit the patch [2] and makes some fixes on the patch.
> > 
> > Jie Hai (36):
> >   net/axgbe: fix Rx and Tx queue state
> >   net/af_packet: fix Rx and Tx queue state
> >   net/af_xdp: fix Rx and Tx queue state
> >   net/avp: fix Rx and Tx queue state
> >   net/bnx2x: fix Rx and Tx queue state
> >   net/bnxt: fix Rx and Tx queue state
> >   net/bonding: fix Rx and Tx queue state
> >   net/cxgbe: fix Rx and Tx queue state
> >   net/dpaa: fix Rx and Tx queue state
> >   net/dpaa2: fix Rx and Tx queue state
> >   net/e1000: fix Rx and Tx queue state
> >   net/ena: fix Rx and Tx queue state
> >   net/enetc: fix Rx and Tx queue state
> >   net/enic: fix Rx and Tx queue state
> >   net/hinic: fix Rx and Tx queue state
> >   net/ipn3ke: fix Rx and Tx queue state
> >   net/memif: fix Rx and Tx queue state
> >   net/mana: fix Rx and Tx queue state
> >   net/mlx4: fix Rx and Tx queue state
> >   net/mvneta: fix Rx and Tx queue state
> >   net/mvpp2: fix Rx and Tx queue state
> >   net/netvsc: fix Rx and Tx queue state
> >   net/nfp: fix Rx and Tx queue state
> >   net/ngbe: fix Rx and Tx queue state
> >   net/null: fix Rx and Tx queue state
> >   net/octeon_ep: fix Rx and Tx queue state
> >   net/octeontx: fix Rx and Tx queue state
> >   net/pfe: fix Rx and Tx queue state
> >   net/ring: fix Rx and Tx queue state
> >   net/sfc: fix Rx and Tx queue state
> >   net/softnic: fix Rx and Tx queue state
> >   net/txgbe: fix Rx and Tx queue state
> >   net/vhost: fix Rx and Tx queue state
> >   net/virtio: fix Rx and Tx queue state
> >   net/vmxnet3: fix Rx and Tx queue state
> >   app/testpmd: fix primary process not polling all queues
> > 
> 
> For series,
> Acked-by: Ferruh Yigit <ferruh.yigit@amd.com>

I see this series as a failure in having common code in ethdev.
I hope we can revert it later to have a better ethdev code
which update queue states taking deferred start into account.
It may require to rework all drivers to have a common behaviour,
but it would be a better improvement.

RE: [EXT] [PATCH 27/36] net/octeontx: fix Rx and Tx queue state

[Harman Kalra]

Hi Jie,

What issue did you observe which the following patch resolves? Can you please provide steps.
Because we are setting the respective queue state in octeontx_dev_tx_queue_start/stop() ethdev_ops

Thanks
Harman

> -----Original Message-----
> From: Jie Hai <haijie1@huawei.com>
> Sent: Friday, September 8, 2023 4:59 PM
> To: dev@dpdk.org; Harman Kalra <hkalra@marvell.com>; Chengwen Feng
> <fengchengwen@huawei.com>; Lijun Ou <oulijun@huawei.com>; Thomas
> Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
> Konstantin Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>
> Cc: haijie1@huawei.com; lihuisong@huawei.com
> Subject: [EXT] [PATCH 27/36] net/octeontx: fix Rx and Tx queue state
> 
> External Email
> 
> ----------------------------------------------------------------------
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether a queue
> participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time according to
> the actual situation.
> 
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue
> information")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
>  drivers/net/octeontx/octeontx_ethdev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/drivers/net/octeontx/octeontx_ethdev.c
> b/drivers/net/octeontx/octeontx_ethdev.c
> index a6ae51a42c4e..2a8378a33ea2 100644
> --- a/drivers/net/octeontx/octeontx_ethdev.c
> +++ b/drivers/net/octeontx/octeontx_ethdev.c
> @@ -732,6 +732,11 @@ octeontx_dev_start(struct rte_eth_dev *dev)
>  	}
> 
>  	/* Success */
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +
>  	return ret;
> 
>  pki_port_stop_error:
> @@ -746,6 +751,7 @@ static int
>  octeontx_dev_stop(struct rte_eth_dev *dev)  {
>  	struct octeontx_nic *nic = octeontx_pmd_priv(dev);
> +	uint16_t i;
>  	int ret;
> 
>  	PMD_INIT_FUNC_TRACE();
> @@ -772,6 +778,11 @@ octeontx_dev_stop(struct rte_eth_dev *dev)
>  		return ret;
>  	}
> 
> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
> +		dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
> +		dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +
>  	return 0;
>  }
> 
> --
> 2.30.0

Re: [EXT] [PATCH 27/36] net/octeontx: fix Rx and Tx queue state

[Jie Hai]

On 2023/11/2 17:59, Harman Kalra wrote:
> Hi Jie,
> 
> What issue did you observe which the following patch resolves? Can you please provide steps.
> Because we are setting the respective queue state in octeontx_dev_tx_queue_start/stop() ethdev_ops
> 
> Thanks
> Harman
> 
Hi, Harman,

If dev_start are called, all queues except for the deferred one should 
forward packets.

I have not tested for this driver, please try the following steps to see 
if there is any problem.

For the original question, please see commit:
37b68fa1445c app/testpmd: fix primary process not polling all queues applied

If applied this patch without drivers reporting their queue state in
dev_start and dev_stop ops, perform the following opertation:

step1: Start the testpmd app.
     dpdk-testpmd -a 0000:35:00.0 -l 0-3 -- -i --rxq=10 --txq=10

step2: Perform the following steps and send traffic. As expected,
queue 7 does not send or receive packets, and other queues do.
     port 0 rxq 7 stop
     port 0 txq 7 stop
     set fwd mac
     start

step3: Perform the following steps and send traffic. All queues
are expected to send and receive packets normally, but that's not
the case for queue 7. The queue may be enabled on hardware, but
not allowed to participate in forwarding in app level for the wrong state.
     stop
     port stop all
     port start all
     start
     show port xstats all

If drivers support checking queue state by command 'show rxq info 0 7'
  and 'show txq info 0 7', we see queue 7 is stopped as other queues are
started. That makes queue 7 not forward.

Please refer to
https://inbox.dpdk.org/dev/20230721160422.3848154-1-ferruh.yigit@amd.com/

and

https://inbox.dpdk.org/dev/20230928074305.2991100-9-haijie1@huawei.com/.

Thanks,
Jie Hai

>> -----Original Message-----
>> From: Jie Hai <haijie1@huawei.com>
>> Sent: Friday, September 8, 2023 4:59 PM
>> To: dev@dpdk.org; Harman Kalra <hkalra@marvell.com>; Chengwen Feng
>> <fengchengwen@huawei.com>; Lijun Ou <oulijun@huawei.com>; Thomas
>> Monjalon <thomas@monjalon.net>; Ferruh Yigit <ferruh.yigit@intel.com>;
>> Konstantin Ananyev
>> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>
>> Cc: haijie1@huawei.com; lihuisong@huawei.com
>> Subject: [EXT] [PATCH 27/36] net/octeontx: fix Rx and Tx queue state
>>
>> External Email
>>
>> ----------------------------------------------------------------------
>> The DPDK framework reports the queue state, which is stored in
>> dev->data->tx_queue_state and dev->data->rx_queue_state. The
>> state is maintained by the driver. Users may determine whether a queue
>> participates in packet forwarding based on the state.
>> Therefore, the driver needs to modify the queue state in time according to
>> the actual situation.
>>
>> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue
>> information")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>>   drivers/net/octeontx/octeontx_ethdev.c | 11 +++++++++++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/drivers/net/octeontx/octeontx_ethdev.c
>> b/drivers/net/octeontx/octeontx_ethdev.c
>> index a6ae51a42c4e..2a8378a33ea2 100644
>> --- a/drivers/net/octeontx/octeontx_ethdev.c
>> +++ b/drivers/net/octeontx/octeontx_ethdev.c
>> @@ -732,6 +732,11 @@ octeontx_dev_start(struct rte_eth_dev *dev)
>>   	}
>>
>>   	/* Success */
>> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
>> +		dev->data->rx_queue_state[i] =
>> RTE_ETH_QUEUE_STATE_STARTED;
>> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
>> +		dev->data->tx_queue_state[i] =
>> RTE_ETH_QUEUE_STATE_STARTED;
>> +
>>   	return ret;
>>
>>   pki_port_stop_error:
>> @@ -746,6 +751,7 @@ static int
>>   octeontx_dev_stop(struct rte_eth_dev *dev)  {
>>   	struct octeontx_nic *nic = octeontx_pmd_priv(dev);
>> +	uint16_t i;
>>   	int ret;
>>
>>   	PMD_INIT_FUNC_TRACE();
>> @@ -772,6 +778,11 @@ octeontx_dev_stop(struct rte_eth_dev *dev)
>>   		return ret;
>>   	}
>>
>> +	for (i = 0; i < dev->data->nb_rx_queues; i++)
>> +		dev->data->rx_queue_state[i] =
>> RTE_ETH_QUEUE_STATE_STOPPED;
>> +	for (i = 0; i < dev->data->nb_tx_queues; i++)
>> +		dev->data->tx_queue_state[i] =
>> RTE_ETH_QUEUE_STATE_STOPPED;
>> +
>>   	return 0;
>>   }
>>
>> --
>> 2.30.0
> 
> .