[PATCH 0/3] net/mlx5: E-Switch and validation fixes

[PATCH 0/3] net/mlx5: E-Switch and validation fixes

[Dariusz Sosnowski]

Patch 1 - Fixes a bug with fdb_def_rule_en device argument,
used to control default E-Switch flow rules created by mlx5 PMD.

Patches 2-3 - Fixes for flow rule validation in async flow API,
which were found during testing.

Dariusz Sosnowski (3):
  net/mlx5: fix disabling E-Switch default flow rules
  net/mlx5: fix action configuration validation
  net/mlx5: fix RSS and queue action validation

 drivers/net/mlx5/mlx5_flow_hw.c | 182 +++++++++++++++++---------------
 drivers/net/mlx5/mlx5_trigger.c |   4 +-
 drivers/net/mlx5/mlx5_txq.c     |  13 ++-
 3 files changed, 107 insertions(+), 92 deletions(-)

--
2.39.2

[PATCH 1/3] net/mlx5: fix disabling E-Switch default flow rules

[Dariusz Sosnowski]

`fdb_def_rule_en` devarg controls whether mlx5 PMD creates default
E-Switch flow rules for:

- Transferring traffic from wire, VFs and SFs to group 1 (default jump).
- Providing default behavior for application traffic (default SQ miss
  flow rules).

With these flow rules, applications effectively create transfer flow
rules in group 1 and higher (application group is translated to one
higher) allowing for faster insertion on all groups and providing
ability to forward to VF, SF and wire on any group.

By default, these rules are created (`fdb_def_rule_en` == 1).

When these default flow rules are disabled (`fdb_def_rule_en` == 0)
with HW Steering flow engine (`dv_flow_en` == 2) only creation of
default jump rules was disabled. Also, necessary template table and
pattern/actions templates were created as well,
but they were never used.
SQ miss flow rules were still created.
This is a bug, because with `fdb_def_rule_en` == 0, application should
not expect any default E-Switch flow rules.

This patch fixes that by disabling all default E-Switch flow rules
creation and disabling creating templates for these flow rules,
when `fdb_def_rule_en` == 0.
If an application needs to run with these flow rules disabled,
and requires flow rules providing SQ miss flow rules functionality,
then application must explicitly create similar flow rules.

Fixes: 1939eb6f660c ("net/mlx5: support flow port action with HWS")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_hw.c | 142 ++++++++++++++++++--------------
 drivers/net/mlx5/mlx5_trigger.c |   4 +-
 drivers/net/mlx5/mlx5_txq.c     |  13 ++-
 3 files changed, 91 insertions(+), 68 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index e7d8c251a0..fe7df7305f 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -10580,6 +10580,7 @@ flow_hw_create_ctrl_tables(struct rte_eth_dev *dev, struct rte_flow_error *error
 	struct mlx5_flow_hw_ctrl_fdb *hw_ctrl_fdb;
 	uint32_t xmeta = priv->sh->config.dv_xmeta_en;
 	uint32_t repr_matching = priv->sh->config.repr_matching;
+	uint32_t fdb_def_rule = priv->sh->config.fdb_def_rule;
 
 	MLX5_ASSERT(priv->hw_ctrl_fdb == NULL);
 	hw_ctrl_fdb = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*hw_ctrl_fdb), 0, SOCKET_ID_ANY);
@@ -10590,70 +10591,79 @@ flow_hw_create_ctrl_tables(struct rte_eth_dev *dev, struct rte_flow_error *error
 		goto err;
 	}
 	priv->hw_ctrl_fdb = hw_ctrl_fdb;
-	/* Create templates and table for default SQ miss flow rules - root table. */
-	hw_ctrl_fdb->esw_mgr_items_tmpl = flow_hw_create_ctrl_esw_mgr_pattern_template(dev, error);
-	if (!hw_ctrl_fdb->esw_mgr_items_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create E-Switch Manager item"
-			" template for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->regc_jump_actions_tmpl = flow_hw_create_ctrl_regc_jump_actions_template
-			(dev, error);
-	if (!hw_ctrl_fdb->regc_jump_actions_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create REG_C set and jump action template"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->hw_esw_sq_miss_root_tbl = flow_hw_create_ctrl_sq_miss_root_table
-			(dev, hw_ctrl_fdb->esw_mgr_items_tmpl, hw_ctrl_fdb->regc_jump_actions_tmpl,
-			 error);
-	if (!hw_ctrl_fdb->hw_esw_sq_miss_root_tbl) {
-		DRV_LOG(ERR, "port %u failed to create table for default sq miss (root table)"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	/* Create templates and table for default SQ miss flow rules - non-root table. */
-	hw_ctrl_fdb->regc_sq_items_tmpl = flow_hw_create_ctrl_regc_sq_pattern_template(dev, error);
-	if (!hw_ctrl_fdb->regc_sq_items_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create SQ item template for"
-			" control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->port_actions_tmpl = flow_hw_create_ctrl_port_actions_template(dev, error);
-	if (!hw_ctrl_fdb->port_actions_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create port action template"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->hw_esw_sq_miss_tbl = flow_hw_create_ctrl_sq_miss_table
-			(dev, hw_ctrl_fdb->regc_sq_items_tmpl, hw_ctrl_fdb->port_actions_tmpl,
-			 error);
-	if (!hw_ctrl_fdb->hw_esw_sq_miss_tbl) {
-		DRV_LOG(ERR, "port %u failed to create table for default sq miss (non-root table)"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	/* Create templates and table for default FDB jump flow rules. */
-	hw_ctrl_fdb->port_items_tmpl = flow_hw_create_ctrl_port_pattern_template(dev, error);
-	if (!hw_ctrl_fdb->port_items_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create SQ item template for"
-			" control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->jump_one_actions_tmpl = flow_hw_create_ctrl_jump_actions_template
-			(dev, MLX5_HW_LOWEST_USABLE_GROUP, error);
-	if (!hw_ctrl_fdb->jump_one_actions_tmpl) {
-		DRV_LOG(ERR, "port %u failed to create jump action template"
-			" for control flows", dev->data->port_id);
-		goto err;
-	}
-	hw_ctrl_fdb->hw_esw_zero_tbl = flow_hw_create_ctrl_jump_table
-			(dev, hw_ctrl_fdb->port_items_tmpl, hw_ctrl_fdb->jump_one_actions_tmpl,
-			 error);
-	if (!hw_ctrl_fdb->hw_esw_zero_tbl) {
-		DRV_LOG(ERR, "port %u failed to create table for default jump to group 1"
-			" for control flows", dev->data->port_id);
-		goto err;
+	if (fdb_def_rule) {
+		/* Create templates and table for default SQ miss flow rules - root table. */
+		hw_ctrl_fdb->esw_mgr_items_tmpl =
+				flow_hw_create_ctrl_esw_mgr_pattern_template(dev, error);
+		if (!hw_ctrl_fdb->esw_mgr_items_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create E-Switch Manager item"
+				" template for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->regc_jump_actions_tmpl =
+				flow_hw_create_ctrl_regc_jump_actions_template(dev, error);
+		if (!hw_ctrl_fdb->regc_jump_actions_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create REG_C set and jump action template"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->hw_esw_sq_miss_root_tbl =
+				flow_hw_create_ctrl_sq_miss_root_table
+					(dev, hw_ctrl_fdb->esw_mgr_items_tmpl,
+					 hw_ctrl_fdb->regc_jump_actions_tmpl, error);
+		if (!hw_ctrl_fdb->hw_esw_sq_miss_root_tbl) {
+			DRV_LOG(ERR, "port %u failed to create table for default sq miss (root table)"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		/* Create templates and table for default SQ miss flow rules - non-root table. */
+		hw_ctrl_fdb->regc_sq_items_tmpl =
+				flow_hw_create_ctrl_regc_sq_pattern_template(dev, error);
+		if (!hw_ctrl_fdb->regc_sq_items_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create SQ item template for"
+				" control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->port_actions_tmpl =
+				flow_hw_create_ctrl_port_actions_template(dev, error);
+		if (!hw_ctrl_fdb->port_actions_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create port action template"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->hw_esw_sq_miss_tbl =
+				flow_hw_create_ctrl_sq_miss_table
+					(dev, hw_ctrl_fdb->regc_sq_items_tmpl,
+					 hw_ctrl_fdb->port_actions_tmpl, error);
+		if (!hw_ctrl_fdb->hw_esw_sq_miss_tbl) {
+			DRV_LOG(ERR, "port %u failed to create table for default sq miss (non-root table)"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		/* Create templates and table for default FDB jump flow rules. */
+		hw_ctrl_fdb->port_items_tmpl =
+				flow_hw_create_ctrl_port_pattern_template(dev, error);
+		if (!hw_ctrl_fdb->port_items_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create SQ item template for"
+				" control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->jump_one_actions_tmpl =
+				flow_hw_create_ctrl_jump_actions_template
+					(dev, MLX5_HW_LOWEST_USABLE_GROUP, error);
+		if (!hw_ctrl_fdb->jump_one_actions_tmpl) {
+			DRV_LOG(ERR, "port %u failed to create jump action template"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
+		hw_ctrl_fdb->hw_esw_zero_tbl = flow_hw_create_ctrl_jump_table
+				(dev, hw_ctrl_fdb->port_items_tmpl,
+				 hw_ctrl_fdb->jump_one_actions_tmpl, error);
+		if (!hw_ctrl_fdb->hw_esw_zero_tbl) {
+			DRV_LOG(ERR, "port %u failed to create table for default jump to group 1"
+				" for control flows", dev->data->port_id);
+			goto err;
+		}
 	}
 	/* Create templates and table for default Tx metadata copy flow rule. */
 	if (!repr_matching && xmeta == MLX5_XMETA_MODE_META32_HWS) {
@@ -15383,6 +15393,8 @@ mlx5_flow_hw_esw_destroy_sq_miss_flow(struct rte_eth_dev *dev, uint32_t sqn)
 	}
 	proxy_dev = &rte_eth_devices[proxy_port_id];
 	proxy_priv = proxy_dev->data->dev_private;
+	/* FDB default flow rules must be enabled. */
+	MLX5_ASSERT(proxy_priv->sh->config.fdb_def_rule);
 	if (!proxy_priv->dr_ctx)
 		return 0;
 	if (!proxy_priv->hw_ctrl_fdb ||
@@ -15447,6 +15459,8 @@ mlx5_flow_hw_esw_create_default_jump_flow(struct rte_eth_dev *dev)
 	}
 	proxy_dev = &rte_eth_devices[proxy_port_id];
 	proxy_priv = proxy_dev->data->dev_private;
+	/* FDB default flow rules must be enabled. */
+	MLX5_ASSERT(proxy_priv->sh->config.fdb_def_rule);
 	if (!proxy_priv->dr_ctx) {
 		DRV_LOG(DEBUG, "Transfer proxy port (port %u) of port %u must be configured "
 			       "for HWS to create default FDB jump rule. Default rule will "
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index 6fa7c01cd0..a65a460731 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -1504,7 +1504,9 @@ mlx5_traffic_enable_hws(struct rte_eth_dev *dev)
 		if (!txq)
 			continue;
 		queue = mlx5_txq_get_sqn(txq);
-		if ((priv->representor || priv->master) && config->dv_esw_en) {
+		if ((priv->representor || priv->master) &&
+		    config->dv_esw_en &&
+		    config->fdb_def_rule) {
 			if (mlx5_flow_hw_esw_create_sq_miss_flow(dev, queue, false)) {
 				mlx5_txq_release(dev, i);
 				goto error;
diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c
index 8eb1ae1f03..f05534e168 100644
--- a/drivers/net/mlx5/mlx5_txq.c
+++ b/drivers/net/mlx5/mlx5_txq.c
@@ -1363,11 +1363,18 @@ rte_pmd_mlx5_external_sq_enable(uint16_t port_id, uint32_t sq_num)
 	}
 #ifdef HAVE_MLX5_HWS_SUPPORT
 	if (priv->sh->config.dv_flow_en == 2) {
-		if (mlx5_flow_hw_esw_create_sq_miss_flow(dev, sq_num, true))
-			return -rte_errno;
+		bool sq_miss_created = false;
+
+		if (priv->sh->config.fdb_def_rule) {
+			if (mlx5_flow_hw_esw_create_sq_miss_flow(dev, sq_num, true))
+				return -rte_errno;
+			sq_miss_created = true;
+		}
+
 		if (priv->sh->config.repr_matching &&
 		    mlx5_flow_hw_tx_repr_matching_flow(dev, sq_num, true)) {
-			mlx5_flow_hw_esw_destroy_sq_miss_flow(dev, sq_num);
+			if (sq_miss_created)
+				mlx5_flow_hw_esw_destroy_sq_miss_flow(dev, sq_num);
 			return -rte_errno;
 		}
 		return 0;
-- 
2.39.2

[PATCH 2/3] net/mlx5: fix action configuration validation

[Dariusz Sosnowski]

Checking if action configuration is required should be checked based on
action type recorded in the actions template, not on user action.

Also, adds a missing internal RSS action type to configuration check
skip list.

Fixes: 57c7b94301ee ("net/mlx5: add async flow operation validation")

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_hw.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index fe7df7305f..39d1cd96d4 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -16388,10 +16388,11 @@ flow_hw_validate_rule_actions(struct rte_eth_dev *dev,
 		user_action = &actions[act_data->action_src];
 
 		/* Skip actions which do not require conf. */
-		switch ((int)user_action->type) {
+		switch ((int)act_data->type) {
 		case RTE_FLOW_ACTION_TYPE_COUNT:
 		case MLX5_RTE_FLOW_ACTION_TYPE_COUNT:
 		case MLX5_RTE_FLOW_ACTION_TYPE_METER_MARK:
+		case MLX5_RTE_FLOW_ACTION_TYPE_RSS:
 			continue;
 		default:
 			break;
-- 
2.39.2

[PATCH 3/3] net/mlx5: fix RSS and queue action validation

[Dariusz Sosnowski]

mlx5 PMD supports configuration where
Rx queues managed by DPDK are not set up.
Externally allocated RQs can be used by mapping them to some
DPDK Rx queue indexes using rte_pmd_mlx5_external_rx_queue_id_map()
API. In this case, mlx5 PMD will allow creating flow rules which
reference such external RQ.

HWS validation of RSS and QUEUE unmasked flow actions in actions
templates worked by constructing a "mock" action which was then checked.
This procedure incorrectly assumed that queue index 0 can be used as
"always valid queue", which is not the case in scenario mentioned above,
because queue 0 was not set up

This patch fixes that by removing "mock" actions, since there's no real
data available for validation. RSS and QUEUE validation in unmasked
action case only checks flow attributes.

Fixes: d6dc072aeb12 ("net/mlx5: validate flow actions in table creation")

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_hw.c | 37 +++++++++++++--------------------
 1 file changed, 14 insertions(+), 23 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 39d1cd96d4..d243b59b71 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -6806,8 +6806,6 @@ mlx5_hw_validate_action_mark(struct rte_eth_dev *dev,
 					      &attr, error);
 }
 
-#define MLX5_FLOW_DEFAULT_INGRESS_QUEUE 0
-
 static int
 mlx5_hw_validate_action_queue(struct rte_eth_dev *dev,
 			      const struct rte_flow_action *template_action,
@@ -6817,22 +6815,22 @@ mlx5_hw_validate_action_queue(struct rte_eth_dev *dev,
 			      struct rte_flow_error *error)
 {
 	const struct rte_flow_action_queue *queue_mask = template_mask->conf;
-	const struct rte_flow_action *action =
-		queue_mask && queue_mask->index ? template_action :
-		&(const struct rte_flow_action) {
-		.type = RTE_FLOW_ACTION_TYPE_QUEUE,
-		.conf = &(const struct rte_flow_action_queue) {
-			.index = MLX5_FLOW_DEFAULT_INGRESS_QUEUE
-		}
-	};
 	const struct rte_flow_attr attr = {
 		.ingress = template_attr->ingress,
 		.egress = template_attr->egress,
 		.transfer = template_attr->transfer
 	};
+	bool masked = queue_mask != NULL && queue_mask->index;
 
-	return mlx5_flow_validate_action_queue(action, action_flags,
-					       dev, &attr, error);
+	if (template_attr->egress || template_attr->transfer)
+		return rte_flow_error_set(error, EINVAL,
+					  RTE_FLOW_ERROR_TYPE_ATTR, NULL,
+					  "QUEUE action supported for ingress only");
+	if (masked)
+		return mlx5_flow_validate_action_queue(template_action, action_flags, dev,
+						       &attr, error);
+	else
+		return 0;
 }
 
 static int
@@ -6844,22 +6842,15 @@ mlx5_hw_validate_action_rss(struct rte_eth_dev *dev,
 			      struct rte_flow_error *error)
 {
 	const struct rte_flow_action_rss *mask = template_mask->conf;
-	const struct rte_flow_action *action = mask ? template_action :
-		&(const struct rte_flow_action) {
-		.type = RTE_FLOW_ACTION_TYPE_RSS,
-		.conf = &(const struct rte_flow_action_rss) {
-			.queue_num = 1,
-			.queue = (uint16_t [1]) {
-				MLX5_FLOW_DEFAULT_INGRESS_QUEUE
-			}
-		}
-	};
 
 	if (template_attr->egress || template_attr->transfer)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ATTR, NULL,
 					  "RSS action supported for ingress only");
-	return mlx5_validate_action_rss(dev, action, error);
+	if (mask != NULL)
+		return mlx5_validate_action_rss(dev, template_action, error);
+	else
+		return 0;
 }
 
 static int
-- 
2.39.2

Re: [PATCH 0/3] net/mlx5: E-Switch and validation fixes

[Raslan Darawsheh]

Hi,

From: Dariusz Sosnowski <dsosnowski@nvidia.com>
Sent: Thursday, July 18, 2024 12:57 PM
To: Slava Ovsiienko; Bing Zhao; Ori Kam; Suanming Mou; Matan Azrad
Cc: dev@dpdk.org; Raslan Darawsheh
Subject: [PATCH 0/3] net/mlx5: E-Switch and validation fixes

Patch 1 - Fixes a bug with fdb_def_rule_en device argument,
used to control default E-Switch flow rules created by mlx5 PMD.

Patches 2-3 - Fixes for flow rule validation in async flow API,
which were found during testing.

Dariusz Sosnowski (3):
  net/mlx5: fix disabling E-Switch default flow rules
  net/mlx5: fix action configuration validation
  net/mlx5: fix RSS and queue action validation

 drivers/net/mlx5/mlx5_flow_hw.c | 182 +++++++++++++++++---------------
 drivers/net/mlx5/mlx5_trigger.c |   4 +-
 drivers/net/mlx5/mlx5_txq.c     |  13 ++-
 3 files changed, 107 insertions(+), 92 deletions(-)


series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh