[PATCH v2] app/testpmd: fix incorrect queues state of secondary process

[PATCH v2] app/testpmd: fix incorrect queues state of secondary process

[peng1x.zhang]

From: Peng Zhang <peng1x.zhang@intel.com>

Primary process could set up queues state correctly when starting port,
but under multi-process scenario, "stream_init" function would get wrong
queues state for secondary process.

This commit is to get queues state from ethdev which is located in
shared memory.

Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng1x.zhang@intel.com>
---
 app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index addcbcac85..70f907d96b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -75,6 +75,8 @@
 
 #include "testpmd.h"
 
+#include <ethdev_driver.h>
+
 #ifndef MAP_HUGETLB
 /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
 #define HUGE_FLAG (0x40000)
@@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
 	if (!pkt_fwd_shared_rxq_check())
 		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_port];
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_port];
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}
 			stream_init(fwd_streams[i]);
+		}
+	}
 
 	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
 	if (port_fwd_begin != NULL) {
-- 
2.25.1

[PATCH] app/testpmd: fix secondary process cannot dump packet

[peng1x.zhang]

From: Peng Zhang <peng1x.zhang@intel.com>

The origin design is whether testpmd is primary or not, if state of
receive queue is stop, then packets will not be dumped for show.
While to secondary process, receive queue will not be set up, and state
will still be stop even if testpmd is started. So packets of stated
secondary process cannot be dumped for show.

The current design is to secondary process state of queue will be set
to start after testpmd is started. Then packets of started secondary
process can be dumped for show.

Fixes: a550baf24af9 ("app/testpmd: support multi-process")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng1x.zhang@intel.com>
---
 app/test-pmd/testpmd.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 205d98ee3d..93ba7e7c9b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -3007,6 +3007,18 @@ start_port(portid_t pid)
 			if (setup_hairpin_queues(pi, p_pi, cnt_pi) != 0)
 				return -1;
 		}
+
+		if (port->need_reconfig_queues > 0 && !is_proc_primary()) {
+			struct rte_eth_rxconf *rx_conf;
+			for (qi = 0; qi < nb_rxq; qi++) {
+				rx_conf = &(port->rxq[qi].conf);
+				ports[pi].rxq[qi].state =
+					rx_conf->rx_deferred_start ?
+					RTE_ETH_QUEUE_STATE_STOPPED :
+					RTE_ETH_QUEUE_STATE_STARTED;
+			}
+		}
+
 		configure_rxtx_dump_callbacks(verbose_level);
 		if (clear_ptypes) {
 			diag = rte_eth_dev_set_ptypes(pi, RTE_PTYPE_UNKNOWN,
-- 
2.25.1

[PATCH v2] app/testpmd: fix incorrect queues state of secondary process

[peng1x.zhang]

From: Peng Zhang <peng1x.zhang@intel.com>

Primary process could set up queues state correctly when starting port,
but under multi-process scenario, "stream_init" function would get wrong
queues state for secondary process.

This commit is to get queues state from ethdev which is located in
shared memory.

Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
Cc: stable@dpdk.org

Signed-off-by: Peng Zhang <peng1x.zhang@intel.com>
---
 app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index addcbcac85..70f907d96b 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -75,6 +75,8 @@
 
 #include "testpmd.h"
 
+#include <ethdev_driver.h>
+
 #ifndef MAP_HUGETLB
 /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
 #define HUGE_FLAG (0x40000)
@@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
 	if (!pkt_fwd_shared_rxq_check())
 		return;
 
-	if (stream_init != NULL)
-		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
+	if (stream_init != NULL) {
+		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
+			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+				struct fwd_stream *fs = fwd_streams[i];
+				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
+
+				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
+				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
+
+				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_port];
+				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
+				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_port];
+				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
+			}
 			stream_init(fwd_streams[i]);
+		}
+	}
 
 	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
 	if (port_fwd_begin != NULL) {
-- 
2.25.1

Re: [PATCH v2] app/testpmd: fix incorrect queues state of secondary process

[Singh, Aman Deep]

Hi Peng,

On 8/19/2022 3:39 PM, peng1x.zhang@intel.com wrote:
> From: Peng Zhang <peng1x.zhang@intel.com>
>
> Primary process could set up queues state correctly when starting port,
> but under multi-process scenario, "stream_init" function would get wrong
> queues state for secondary process.
>
> This commit is to get queues state from ethdev which is located in
> shared memory.
>
> Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
> Cc: stable@dpdk.org
>
> Signed-off-by: Peng Zhang <peng1x.zhang@intel.com>
> ---
>   app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
>   1 file changed, 18 insertions(+), 2 deletions(-)
>
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index addcbcac85..70f907d96b 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -75,6 +75,8 @@
>   
>   #include "testpmd.h"
>   
> +#include <ethdev_driver.h>
> +
>   #ifndef MAP_HUGETLB
>   /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
>   #define HUGE_FLAG (0x40000)
> @@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
>   	if (!pkt_fwd_shared_rxq_check())
>   		return;
>   
> -	if (stream_init != NULL)
> -		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
> +	if (stream_init != NULL) {
> +		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
> +			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> +				struct fwd_stream *fs = fwd_streams[i];
> +				struct rte_eth_dev_data *dev_rx_data, *dev_tx_data;
> +
> +				dev_rx_data = (&rte_eth_devices[fs->rx_port])->data;
> +				dev_tx_data = (&rte_eth_devices[fs->tx_port])->data;
> +
> +				uint8_t rx_state = dev_rx_data->rx_queue_state[fs->rx_port];

To get the queue state, the array rx_queue_state[] should be indexed by the queue number.
Using fs->rx_port may not give desired queue's state.

> +				ports[fs->rx_port].rxq[fs->rx_queue].state = rx_state;
> +				uint8_t tx_state = dev_tx_data->tx_queue_state[fs->tx_port];

Same as rx queue above. We might need to root cause the issue further.

> +				ports[fs->tx_port].txq[fs->tx_queue].state = tx_state;
> +			}
>   			stream_init(fwd_streams[i]);
> +		}
> +	}
>   
>   	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
>   	if (port_fwd_begin != NULL) {

RE: [PATCH v2] app/testpmd: fix incorrect queues state of secondary process

[Zhang, Peng1X]

Hi Aman Deep,

Thanks for your comment, I will follow it. And root cause is to secondary process, it cannot get queue state directly, but from share memory. So when starting port, secondary process cannot get correct queue state even if queue has been started.

> -----Original Message-----
> From: Singh, Aman Deep <aman.deep.singh@intel.com>
> Sent: Thursday, August 25, 2022 2:22 AM
> To: Zhang, Peng1X <peng1x.zhang@intel.com>; dev@dpdk.org
> Cc: Zhang, Yuying <yuying.zhang@intel.com>; stable@dpdk.org
> Subject: Re: [PATCH v2] app/testpmd: fix incorrect queues state of secondary
> process
> 
> Hi Peng,
> 
> On 8/19/2022 3:39 PM, peng1x.zhang@intel.com wrote:
> > From: Peng Zhang <peng1x.zhang@intel.com>
> >
> > Primary process could set up queues state correctly when starting
> > port, but under multi-process scenario, "stream_init" function would
> > get wrong queues state for secondary process.
> >
> > This commit is to get queues state from ethdev which is located in
> > shared memory.
> >
> > Fixes: 3c4426db54fc ("app/testpmd: do not poll stopped queues")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Peng Zhang <peng1x.zhang@intel.com>
> > ---
> >   app/test-pmd/testpmd.c | 20 ++++++++++++++++++--
> >   1 file changed, 18 insertions(+), 2 deletions(-)
> >
> > diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index
> > addcbcac85..70f907d96b 100644
> > --- a/app/test-pmd/testpmd.c
> > +++ b/app/test-pmd/testpmd.c
> > @@ -75,6 +75,8 @@
> >
> >   #include "testpmd.h"
> >
> > +#include <ethdev_driver.h>
> > +
> >   #ifndef MAP_HUGETLB
> >   /* FreeBSD may not have MAP_HUGETLB (in fact, it probably doesn't) */
> >   #define HUGE_FLAG (0x40000)
> > @@ -2402,9 +2404,23 @@ start_packet_forwarding(int with_tx_first)
> >   	if (!pkt_fwd_shared_rxq_check())
> >   		return;
> >
> > -	if (stream_init != NULL)
> > -		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++)
> > +	if (stream_init != NULL) {
> > +		for (i = 0; i < cur_fwd_config.nb_fwd_streams; i++) {
> > +			if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> > +				struct fwd_stream *fs = fwd_streams[i];
> > +				struct rte_eth_dev_data *dev_rx_data,
> *dev_tx_data;
> > +
> > +				dev_rx_data = (&rte_eth_devices[fs-
> >rx_port])->data;
> > +				dev_tx_data = (&rte_eth_devices[fs->tx_port])-
> >data;
> > +
> > +				uint8_t rx_state = dev_rx_data-
> >rx_queue_state[fs->rx_port];
> 
> To get the queue state, the array rx_queue_state[] should be indexed by the
> queue number.
> Using fs->rx_port may not give desired queue's state.
> 
> > +				ports[fs->rx_port].rxq[fs->rx_queue].state =
> rx_state;
> > +				uint8_t tx_state = dev_tx_data-
> >tx_queue_state[fs->tx_port];
> 
> Same as rx queue above. We might need to root cause the issue further.
> 
> > +				ports[fs->tx_port].txq[fs->tx_queue].state =
> tx_state;
> > +			}
> >   			stream_init(fwd_streams[i]);
> > +		}
> > +	}
> >
> >   	port_fwd_begin = cur_fwd_config.fwd_eng->port_fwd_begin;
> >   	if (port_fwd_begin != NULL) {