[dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs being used

[dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs being used

[Moti Haimovsky]

This patch verifies that the number of Rx queues configured for RSS
is supported by the device hardware.
RSS support in mlx4 requires contiguous chunk of QPs to be reserved,
there is a hardware limitation on the amount of contiguous QPs which
is reported by the hardware. Ignoring this value will cause Rx queues
creation to fail.

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
---
 drivers/net/mlx4/mlx4.c     | 3 +++
 drivers/net/mlx4/mlx4.h     | 1 +
 drivers/net/mlx4/mlx4_rxq.c | 6 ++++++
 3 files changed, 10 insertions(+)

diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c
index c67d2c0..defc0d4 100644
--- a/drivers/net/mlx4/mlx4.c
+++ b/drivers/net/mlx4/mlx4.c
@@ -673,6 +673,9 @@ struct mlx4_conf {
 						   &device_attr_ex);
 		DEBUG("supported RSS hash fields mask: %016" PRIx64,
 		      priv->hw_rss_sup);
+		priv->hw_rss_max_qps =
+			device_attr_ex.rss_caps.max_rwq_indirection_table_size;
+		DEBUG("MAX RSS queues %d", priv->hw_rss_max_qps);
 		priv->hw_fcs_strip = !!(device_attr_ex.raw_packet_caps &
 					IBV_RAW_PACKET_CAP_SCATTER_FCS);
 		DEBUG("FCS stripping toggling is %ssupported",
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index 89d8c38..e6fb934 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -95,6 +95,7 @@ struct priv {
 	uint32_t hw_fcs_strip:1; /**< FCS stripping toggling is supported. */
 	uint32_t tso:1; /**< Transmit segmentation offload is supported. */
 	uint32_t tso_max_payload_sz; /**< Max supported TSO payload size. */
+	uint32_t hw_rss_max_qps; /**< Max Rx Queues supported by RSS. */
 	uint64_t hw_rss_sup; /**< Supported RSS hash fields (Verbs format). */
 	struct rte_intr_handle intr_handle; /**< Port interrupt handle. */
 	struct mlx4_drop *drop; /**< Shared resources for drop flow rules. */
diff --git a/drivers/net/mlx4/mlx4_rxq.c b/drivers/net/mlx4/mlx4_rxq.c
index 0cd9560..e8bef1d 100644
--- a/drivers/net/mlx4/mlx4_rxq.c
+++ b/drivers/net/mlx4/mlx4_rxq.c
@@ -338,6 +338,12 @@ struct mlx4_rss *
 
 	if (priv->rss_init)
 		return 0;
+	if (priv->dev->data->nb_rx_queues > priv->hw_rss_max_qps) {
+		ERROR("RSS does not support more than %d queues",
+		      priv->hw_rss_max_qps);
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
 	/* Prepare range for RSS contexts before creating the first WQ. */
 	ret = mlx4_glue->dv_set_context_attr
 		(priv->ctx,
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs being used

[Matan Azrad]

Hi Moti

Small comment below.

From: Mordechay Haimovsky
> This patch verifies that the number of Rx queues configured for RSS is
> supported by the device hardware.
> RSS support in mlx4 requires contiguous chunk of QPs to be reserved, there is a
> hardware limitation on the amount of contiguous QPs which is reported by the
> hardware. Ignoring this value will cause Rx queues creation to fail.
> 
> Signed-off-by: Moti Haimovsky <motih@mellanox.com>
> ---
>  drivers/net/mlx4/mlx4.c     | 3 +++
>  drivers/net/mlx4/mlx4.h     | 1 +
>  drivers/net/mlx4/mlx4_rxq.c | 6 ++++++
>  3 files changed, 10 insertions(+)
> 
> diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index
> c67d2c0..defc0d4 100644
> --- a/drivers/net/mlx4/mlx4.c
> +++ b/drivers/net/mlx4/mlx4.c
> @@ -673,6 +673,9 @@ struct mlx4_conf {
>  						   &device_attr_ex);
>  		DEBUG("supported RSS hash fields mask: %016" PRIx64,
>  		      priv->hw_rss_sup);
> +		priv->hw_rss_max_qps =
> +
> 	device_attr_ex.rss_caps.max_rwq_indirection_table_size;
> +		DEBUG("MAX RSS queues %d", priv->hw_rss_max_qps);
>  		priv->hw_fcs_strip = !!(device_attr_ex.raw_packet_caps &
> 
> 	IBV_RAW_PACKET_CAP_SCATTER_FCS);
>  		DEBUG("FCS stripping toggling is %ssupported", diff --git
> a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 89d8c38..e6fb934
> 100644
> --- a/drivers/net/mlx4/mlx4.h
> +++ b/drivers/net/mlx4/mlx4.h
> @@ -95,6 +95,7 @@ struct priv {
>  	uint32_t hw_fcs_strip:1; /**< FCS stripping toggling is supported. */
>  	uint32_t tso:1; /**< Transmit segmentation offload is supported. */
>  	uint32_t tso_max_payload_sz; /**< Max supported TSO payload size.
> */
> +	uint32_t hw_rss_max_qps; /**< Max Rx Queues supported by RSS. */
>  	uint64_t hw_rss_sup; /**< Supported RSS hash fields (Verbs format). */
>  	struct rte_intr_handle intr_handle; /**< Port interrupt handle. */
>  	struct mlx4_drop *drop; /**< Shared resources for drop flow rules. */
> diff --git a/drivers/net/mlx4/mlx4_rxq.c b/drivers/net/mlx4/mlx4_rxq.c index
> 0cd9560..e8bef1d 100644
> --- a/drivers/net/mlx4/mlx4_rxq.c
> +++ b/drivers/net/mlx4/mlx4_rxq.c
> @@ -338,6 +338,12 @@ struct mlx4_rss *
> 
>  	if (priv->rss_init)
>  		return 0;
> +	if (priv->dev->data->nb_rx_queues > priv->hw_rss_max_qps) {
> +		ERROR("RSS does not support more than %d queues",
> +		      priv->hw_rss_max_qps);
> +		rte_errno = EINVAL;

What's about E2BIG ?

> +		return -rte_errno;
> +	}
>  	/* Prepare range for RSS contexts before creating the first WQ. */
>  	ret = mlx4_glue->dv_set_context_attr
>  		(priv->ctx,
> --
> 1.8.3.1

Besides that;
Acked-by: Matan Azrad <matan@mellanox.com>

Re: [dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs being used

[Matan Azrad]

Hi

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Matan Azrad
> Sent: Thursday, July 26, 2018 5:36 PM
> To: Mordechay Haimovsky <motih@mellanox.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs being
> used
> 
> Hi Moti
> 
> Small comment below.
> 
> From: Mordechay Haimovsky
> > This patch verifies that the number of Rx queues configured for RSS is
> > supported by the device hardware.
> > RSS support in mlx4 requires contiguous chunk of QPs to be reserved,
> > there is a hardware limitation on the amount of contiguous QPs which
> > is reported by the hardware. Ignoring this value will cause Rx queues creation
> to fail.
> >
> > Signed-off-by: Moti Haimovsky <motih@mellanox.com>
> > ---
> >  drivers/net/mlx4/mlx4.c     | 3 +++
> >  drivers/net/mlx4/mlx4.h     | 1 +
> >  drivers/net/mlx4/mlx4_rxq.c | 6 ++++++
> >  3 files changed, 10 insertions(+)
> >
> > diff --git a/drivers/net/mlx4/mlx4.c b/drivers/net/mlx4/mlx4.c index
> > c67d2c0..defc0d4 100644
> > --- a/drivers/net/mlx4/mlx4.c
> > +++ b/drivers/net/mlx4/mlx4.c
> > @@ -673,6 +673,9 @@ struct mlx4_conf {
> >  						   &device_attr_ex);
> >  		DEBUG("supported RSS hash fields mask: %016" PRIx64,
> >  		      priv->hw_rss_sup);
> > +		priv->hw_rss_max_qps =
> > +
> > 	device_attr_ex.rss_caps.max_rwq_indirection_table_size;
> > +		DEBUG("MAX RSS queues %d", priv->hw_rss_max_qps);
> >  		priv->hw_fcs_strip = !!(device_attr_ex.raw_packet_caps &
> >
> > 	IBV_RAW_PACKET_CAP_SCATTER_FCS);
> >  		DEBUG("FCS stripping toggling is %ssupported", diff --git
> > a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index
> > 89d8c38..e6fb934
> > 100644
> > --- a/drivers/net/mlx4/mlx4.h
> > +++ b/drivers/net/mlx4/mlx4.h
> > @@ -95,6 +95,7 @@ struct priv {
> >  	uint32_t hw_fcs_strip:1; /**< FCS stripping toggling is supported. */
> >  	uint32_t tso:1; /**< Transmit segmentation offload is supported. */
> >  	uint32_t tso_max_payload_sz; /**< Max supported TSO payload size.
> > */
> > +	uint32_t hw_rss_max_qps; /**< Max Rx Queues supported by RSS. */
> >  	uint64_t hw_rss_sup; /**< Supported RSS hash fields (Verbs format). */
> >  	struct rte_intr_handle intr_handle; /**< Port interrupt handle. */
> >  	struct mlx4_drop *drop; /**< Shared resources for drop flow rules.
> > */ diff --git a/drivers/net/mlx4/mlx4_rxq.c
> > b/drivers/net/mlx4/mlx4_rxq.c index 0cd9560..e8bef1d 100644
> > --- a/drivers/net/mlx4/mlx4_rxq.c
> > +++ b/drivers/net/mlx4/mlx4_rxq.c
> > @@ -338,6 +338,12 @@ struct mlx4_rss *
> >
> >  	if (priv->rss_init)
> >  		return 0;
> > +	if (priv->dev->data->nb_rx_queues > priv->hw_rss_max_qps) {
> > +		ERROR("RSS does not support more than %d queues",
> > +		      priv->hw_rss_max_qps);
> > +		rte_errno = EINVAL;
> 
> What's about E2BIG ?


My mistake,
The description of E2BIG related to the argument list and not generally for too big parameters,
so no need to change the EINVAL.

> > +		return -rte_errno;
> > +	}
> >  	/* Prepare range for RSS contexts before creating the first WQ. */
> >  	ret = mlx4_glue->dv_set_context_attr
> >  		(priv->ctx,
> > --
> > 1.8.3.1
> 
> Besides that;
> Acked-by: Matan Azrad <matan@mellanox.com>

Re: [dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs being used

[Shahaf Shuler]

Sunday, July 29, 2018 11:12 AM, Matan Azrad:
> Subject: Re: [dpdk-dev] [PATCH] net/mlx4: verify RSS supports num Rx Qs
> being used
> > Hi Moti
> >
> > Small comment below.
> >
> > From: Mordechay Haimovsky
> > > This patch verifies that the number of Rx queues configured for RSS
> > > is supported by the device hardware.
> > > RSS support in mlx4 requires contiguous chunk of QPs to be reserved,
> > > there is a hardware limitation on the amount of contiguous QPs which
> > > is reported by the hardware. Ignoring this value will cause Rx
> > > queues creation
> > to fail.
> > >
> > > Signed-off-by: Moti Haimovsky <motih@mellanox.com>

Applied to next-net-mlx (with commit title change), thanks.