[dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Hemant Agrawal]

From: Jun Yang <jun.yang@nxp.com>

The existing forwarding mode usages the total number of
queues as the minimum of rxq and txq.
It finds the txq as the same index as rxq.
However in some scenarios, specially for flow control
the number of rxq and txq can be different.
This patch maxes the txq and function of rxq for all such
scenario instead of keeping 1:1 relationship between the two.

Now packets from all RXQs can be forwarded to TXQs

Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
 app/test-pmd/config.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index d59968278..efa409453 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -2130,8 +2130,6 @@ rss_fwd_config_setup(void)
 	streamid_t  sm_id;
 
 	nb_q = nb_rxq;
-	if (nb_q > nb_txq)
-		nb_q = nb_txq;
 	cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
 	cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
 	cur_fwd_config.nb_fwd_streams =
@@ -2154,7 +2152,7 @@ rss_fwd_config_setup(void)
 		fs->rx_port = fwd_ports_ids[rxp];
 		fs->rx_queue = rxq;
 		fs->tx_port = fwd_ports_ids[txp];
-		fs->tx_queue = rxq;
+		fs->tx_queue = (rxq % nb_txq);
 		fs->peer_addr = fs->tx_port;
 		fs->retry_enabled = retry_enabled;
 		rxp++;
-- 
2.17.1

Re: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Ananyev, Konstantin]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Hemant Agrawal
> Sent: Wednesday, December 11, 2019 5:30 AM
> To: dev@dpdk.org
> Cc: Jun Yang <jun.yang@nxp.com>
> Subject: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ
> 
> From: Jun Yang <jun.yang@nxp.com>
> 
> The existing forwarding mode usages the total number of
> queues as the minimum of rxq and txq.
> It finds the txq as the same index as rxq.
> However in some scenarios, specially for flow control
> the number of rxq and txq can be different.
> This patch maxes the txq and function of rxq for all such
> scenario instead of keeping 1:1 relationship between the two.
> 
> Now packets from all RXQs can be forwarded to TXQs
> 
> Signed-off-by: Jun Yang <jun.yang@nxp.com>
> ---
>  app/test-pmd/config.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> index d59968278..efa409453 100644
> --- a/app/test-pmd/config.c
> +++ b/app/test-pmd/config.c
> @@ -2130,8 +2130,6 @@ rss_fwd_config_setup(void)
>  	streamid_t  sm_id;
> 
>  	nb_q = nb_rxq;
> -	if (nb_q > nb_txq)
> -		nb_q = nb_txq;
>  	cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
>  	cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
>  	cur_fwd_config.nb_fwd_streams =
> @@ -2154,7 +2152,7 @@ rss_fwd_config_setup(void)
>  		fs->rx_port = fwd_ports_ids[rxp];
>  		fs->rx_queue = rxq;
>  		fs->tx_port = fwd_ports_ids[txp];
> -		fs->tx_queue = rxq;
> +		fs->tx_queue = (rxq % nb_txq);

But does it mean that now 2 lcores cah use the same TX queue?
If so, then how it supposed to work?

>  		fs->peer_addr = fs->tx_port;
>  		fs->retry_enabled = retry_enabled;
>  		rxp++;
> --
> 2.17.1

Re: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Jerin Jacob]

On Wed, Dec 11, 2019 at 3:29 PM Ananyev, Konstantin
<konstantin.ananyev@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Hemant Agrawal
> > Sent: Wednesday, December 11, 2019 5:30 AM
> > To: dev@dpdk.org
> > Cc: Jun Yang <jun.yang@nxp.com>
> > Subject: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ
> >
> > From: Jun Yang <jun.yang@nxp.com>
> >
> > The existing forwarding mode usages the total number of
> > queues as the minimum of rxq and txq.
> > It finds the txq as the same index as rxq.
> > However in some scenarios, specially for flow control
> > the number of rxq and txq can be different.
> > This patch maxes the txq and function of rxq for all such
> > scenario instead of keeping 1:1 relationship between the two.
> >
> > Now packets from all RXQs can be forwarded to TXQs

Allow this feature only for DEV_TX_OFFLOAD_MT_LOCKFREE devices.
Please probe DEV_TX_OFFLOAD_MT_LOCKFREE() capability first to
avoid breaking contract on the other devices.

> >
> > Signed-off-by: Jun Yang <jun.yang@nxp.com>
> > ---
> >  app/test-pmd/config.c | 4 +---
> >  1 file changed, 1 insertion(+), 3 deletions(-)
> >
> > diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> > index d59968278..efa409453 100644
> > --- a/app/test-pmd/config.c
> > +++ b/app/test-pmd/config.c
> > @@ -2130,8 +2130,6 @@ rss_fwd_config_setup(void)
> >       streamid_t  sm_id;
> >
> >       nb_q = nb_rxq;
> > -     if (nb_q > nb_txq)
> > -             nb_q = nb_txq;
> >       cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
> >       cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
> >       cur_fwd_config.nb_fwd_streams =
> > @@ -2154,7 +2152,7 @@ rss_fwd_config_setup(void)
> >               fs->rx_port = fwd_ports_ids[rxp];
> >               fs->rx_queue = rxq;
> >               fs->tx_port = fwd_ports_ids[txp];
> > -             fs->tx_queue = rxq;
> > +             fs->tx_queue = (rxq % nb_txq);
>
> But does it mean that now 2 lcores cah use the same TX queue?
> If so, then how it supposed to work?

See above.


>
> >               fs->peer_addr = fs->tx_port;
> >               fs->retry_enabled = retry_enabled;
> >               rxp++;
> > --
> > 2.17.1
>

Re: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Hemant Agrawal]

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>

> On Wed, Dec 11, 2019 at 3:29 PM Ananyev, Konstantin
> <konstantin.ananyev@intel.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: dev <dev-bounces@dpdk.org> On Behalf Of Hemant Agrawal
> > > Sent: Wednesday, December 11, 2019 5:30 AM
> > > To: dev@dpdk.org
> > > Cc: Jun Yang <jun.yang@nxp.com>
> > > Subject: [dpdk-dev] [PATCH] app/testpmd: support unequal number of
> > > RXQ and TXQ
> > >
> > > From: Jun Yang <jun.yang@nxp.com>
> > >
> > > The existing forwarding mode usages the total number of queues as
> > > the minimum of rxq and txq.
> > > It finds the txq as the same index as rxq.
> > > However in some scenarios, specially for flow control the number of
> > > rxq and txq can be different.
> > > This patch maxes the txq and function of rxq for all such scenario
> > > instead of keeping 1:1 relationship between the two.
> > >
> > > Now packets from all RXQs can be forwarded to TXQs
> 
> Allow this feature only for DEV_TX_OFFLOAD_MT_LOCKFREE devices.
> Please probe DEV_TX_OFFLOAD_MT_LOCKFREE() capability first to avoid
> breaking contract on the other devices.

[Hemant] Agree with the suggestion.

> 
> > >
> > > Signed-off-by: Jun Yang <jun.yang@nxp.com>
> > > ---
> > >  app/test-pmd/config.c | 4 +---
> > >  1 file changed, 1 insertion(+), 3 deletions(-)
> > >
> > > diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c index
> > > d59968278..efa409453 100644
> > > --- a/app/test-pmd/config.c
> > > +++ b/app/test-pmd/config.c
> > > @@ -2130,8 +2130,6 @@ rss_fwd_config_setup(void)
> > >       streamid_t  sm_id;
> > >
> > >       nb_q = nb_rxq;
> > > -     if (nb_q > nb_txq)
> > > -             nb_q = nb_txq;
> > >       cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
> > >       cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
> > >       cur_fwd_config.nb_fwd_streams = @@ -2154,7 +2152,7 @@
> > > rss_fwd_config_setup(void)
> > >               fs->rx_port = fwd_ports_ids[rxp];
> > >               fs->rx_queue = rxq;
> > >               fs->tx_port = fwd_ports_ids[txp];
> > > -             fs->tx_queue = rxq;
> > > +             fs->tx_queue = (rxq % nb_txq);
> >
> > But does it mean that now 2 lcores cah use the same TX queue?
> > If so, then how it supposed to work?
> 
> See above.
> 
> 
> >
> > >               fs->peer_addr = fs->tx_port;
> > >               fs->retry_enabled = retry_enabled;
> > >               rxp++;
> > > --
> > > 2.17.1
> >

Re: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Stephen Hemminger]

On Wed, 11 Dec 2019 15:56:17 +0530
Jerin Jacob <jerinjacobk@gmail.com> wrote:

> >  
> > > -----Original Message-----
> > > From: dev <dev-bounces@dpdk.org> On Behalf Of Hemant Agrawal
> > > Sent: Wednesday, December 11, 2019 5:30 AM
> > > To: dev@dpdk.org
> > > Cc: Jun Yang <jun.yang@nxp.com>
> > > Subject: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ
> > >
> > > From: Jun Yang <jun.yang@nxp.com>
> > >
> > > The existing forwarding mode usages the total number of
> > > queues as the minimum of rxq and txq.
> > > It finds the txq as the same index as rxq.
> > > However in some scenarios, specially for flow control
> > > the number of rxq and txq can be different.
> > > This patch maxes the txq and function of rxq for all such
> > > scenario instead of keeping 1:1 relationship between the two.
> > >
> > > Now packets from all RXQs can be forwarded to TXQs  
> 
> Allow this feature only for DEV_TX_OFFLOAD_MT_LOCKFREE devices.
> Please probe DEV_TX_OFFLOAD_MT_LOCKFREE() capability first to
> avoid breaking contract on the other devices.
> 
> > >

Dropping this version of the patch. Please resubmit a new version
with the lock free check.

Re: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Nipun Gupta]

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Thursday, September 30, 2021 7:16 PM
> To: Nipun Gupta <nipun.gupta@nxp.com>; dev@dpdk.org
> Cc: xiaoyun.li@intel.com; orika@nvidia.com; thomas@monjalon.net; Hemant
> Agrawal <hemant.agrawal@nxp.com>; Jun Yang <jun.yang@nxp.com>
> Subject: Re: [PATCH] app/testpmd: support unequal number of RXQ and TXQ
> 
> On 9/28/2021 12:08 PM, nipun.gupta@nxp.com wrote:
> > From: Jun Yang <jun.yang@nxp.com>
> >
> > The existing forwarding mode configures the total number of
> > queues as the minimum of rxq and txq, so eventually the number
> > of txq are same as rxq.
> > However in some scenarios, specially for flow control the
> > number of rxq and txq can be different.
> > This patch makes the txq and function of rxq for all such
> > scenario instead of keeping 1:1 relationship between the two.
> >
> 
> Hi Nipun,
> 
> I expect more code exists that assumes Rx queue number and Tx queue number
> is
> equal, did you able to test unequal queue numbers with below change?

Hi Ferruh,

Yes, this works with flow, but I will double check on latest code without using
flow too. Also, we will check for any other code which shall need change w.r.t.
this functionality.

> 
> And can you please describe your usecase more, is the device has uneven Rx/Tx
> queues?

Yes, you are correct. In our case, Rx queues which we can use can be much higher
than Tx queues, when rte flow is being used.

> 
> > Signed-off-by: Jun Yang <jun.yang@nxp.com>
> > ---
> >  app/test-pmd/config.c | 6 ++----
> >  1 file changed, 2 insertions(+), 4 deletions(-)
> >
> > diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> > index f5765b34f7..7e17f233ba 100644
> > --- a/app/test-pmd/config.c
> > +++ b/app/test-pmd/config.c
> > @@ -3000,8 +3000,6 @@ rss_fwd_config_setup(void)
> >  	int end;
> >
> >  	nb_q = nb_rxq;
> > -	if (nb_q > nb_txq)
> > -		nb_q = nb_txq;
> >  	cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
> >  	cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
> >  	cur_fwd_config.nb_fwd_streams =
> > @@ -3038,7 +3036,7 @@ rss_fwd_config_setup(void)
> >  		fs->rx_port = fwd_ports_ids[rxp];
> >  		fs->rx_queue = rxq;
> >  		fs->tx_port = fwd_ports_ids[txp];
> > -		fs->tx_queue = rxq;
> > +		fs->tx_queue = (rxq % nb_txq);
> 
> Is this assumes number of Rx queue is always more than number of Tx queue?

Yes, but on re-thought as per your comment, we shall need to revisit this logic.
I will have a relook on this part too.

Thanks,
Nipun

> 
> >  		fs->peer_addr = fs->tx_port;
> >  		fs->retry_enabled = retry_enabled;
> >  		rxp++;
> > @@ -3253,7 +3251,7 @@ fwd_config_setup(void)
> >  		return;
> >  	}
> >
> > -	if ((nb_rxq > 1) && (nb_txq > 1)){
> > +	if ((nb_rxq > 1) && (nb_txq > 1)) {
> >  		if (dcb_config) {
> >  			for (i = 0; i < nb_fwd_ports; i++) {
> >  				pt_id = fwd_ports_ids[i];
> >

Re: [dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[Ferruh Yigit]

On 9/28/2021 12:08 PM, nipun.gupta@nxp.com wrote:
> From: Jun Yang <jun.yang@nxp.com>
> 
> The existing forwarding mode configures the total number of
> queues as the minimum of rxq and txq, so eventually the number
> of txq are same as rxq.
> However in some scenarios, specially for flow control the
> number of rxq and txq can be different.
> This patch makes the txq and function of rxq for all such
> scenario instead of keeping 1:1 relationship between the two.
> 

Hi Nipun,

I expect more code exists that assumes Rx queue number and Tx queue number is
equal, did you able to test unequal queue numbers with below change?

And can you please describe your usecase more, is the device has uneven Rx/Tx
queues?

> Signed-off-by: Jun Yang <jun.yang@nxp.com>
> ---
>  app/test-pmd/config.c | 6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
> index f5765b34f7..7e17f233ba 100644
> --- a/app/test-pmd/config.c
> +++ b/app/test-pmd/config.c
> @@ -3000,8 +3000,6 @@ rss_fwd_config_setup(void)
>  	int end;
>  
>  	nb_q = nb_rxq;
> -	if (nb_q > nb_txq)
> -		nb_q = nb_txq;
>  	cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
>  	cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
>  	cur_fwd_config.nb_fwd_streams =
> @@ -3038,7 +3036,7 @@ rss_fwd_config_setup(void)
>  		fs->rx_port = fwd_ports_ids[rxp];
>  		fs->rx_queue = rxq;
>  		fs->tx_port = fwd_ports_ids[txp];
> -		fs->tx_queue = rxq;
> +		fs->tx_queue = (rxq % nb_txq);

Is this assumes number of Rx queue is always more than number of Tx queue?

>  		fs->peer_addr = fs->tx_port;
>  		fs->retry_enabled = retry_enabled;
>  		rxp++;
> @@ -3253,7 +3251,7 @@ fwd_config_setup(void)
>  		return;
>  	}
>  
> -	if ((nb_rxq > 1) && (nb_txq > 1)){
> +	if ((nb_rxq > 1) && (nb_txq > 1)) {
>  		if (dcb_config) {
>  			for (i = 0; i < nb_fwd_ports; i++) {
>  				pt_id = fwd_ports_ids[i];
> 

[dpdk-dev] [PATCH] app/testpmd: support unequal number of RXQ and TXQ

[nipun.gupta]

From: Jun Yang <jun.yang@nxp.com>

The existing forwarding mode configures the total number of
queues as the minimum of rxq and txq, so eventually the number
of txq are same as rxq.
However in some scenarios, specially for flow control the
number of rxq and txq can be different.
This patch makes the txq and function of rxq for all such
scenario instead of keeping 1:1 relationship between the two.

Signed-off-by: Jun Yang <jun.yang@nxp.com>
---
 app/test-pmd/config.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/app/test-pmd/config.c b/app/test-pmd/config.c
index f5765b34f7..7e17f233ba 100644
--- a/app/test-pmd/config.c
+++ b/app/test-pmd/config.c
@@ -3000,8 +3000,6 @@ rss_fwd_config_setup(void)
 	int end;
 
 	nb_q = nb_rxq;
-	if (nb_q > nb_txq)
-		nb_q = nb_txq;
 	cur_fwd_config.nb_fwd_lcores = (lcoreid_t) nb_fwd_lcores;
 	cur_fwd_config.nb_fwd_ports = nb_fwd_ports;
 	cur_fwd_config.nb_fwd_streams =
@@ -3038,7 +3036,7 @@ rss_fwd_config_setup(void)
 		fs->rx_port = fwd_ports_ids[rxp];
 		fs->rx_queue = rxq;
 		fs->tx_port = fwd_ports_ids[txp];
-		fs->tx_queue = rxq;
+		fs->tx_queue = (rxq % nb_txq);
 		fs->peer_addr = fs->tx_port;
 		fs->retry_enabled = retry_enabled;
 		rxp++;
@@ -3253,7 +3251,7 @@ fwd_config_setup(void)
 		return;
 	}
 
-	if ((nb_rxq > 1) && (nb_txq > 1)){
+	if ((nb_rxq > 1) && (nb_txq > 1)) {
 		if (dcb_config) {
 			for (i = 0; i < nb_fwd_ports; i++) {
 				pt_id = fwd_ports_ids[i];
-- 
2.17.1