[PATCH v1 0/2] Fix src port match in meter hierarchy

[PATCH v1 0/2] Fix src port match in meter hierarchy

[Shun Hao]

In meter hierarchy, if policy flow contains modify_field or port_id
action, a src port matching will be added to determine the position of
meter in action list.

Currently the src port is static, using the port id policy belongs to.
This patch set enables creation of multiple policy flows using src port
from user flow dynamically to match different src port, so the meter
hierarchy can always be used by different src ports.

Shun Hao (2):
  net/mlx5: fix meter hierarchy with modify header
  net/mlx5: fix meter policy with port ID destination

 doc/guides/nics/mlx5.rst        |   3 +
 drivers/net/mlx5/mlx5.h         |   8 +-
 drivers/net/mlx5/mlx5_flow.c    |   2 +-
 drivers/net/mlx5/mlx5_flow_dv.c | 126 +++++++++++++++++---------------
 4 files changed, 77 insertions(+), 62 deletions(-)

-- 
2.20.0

[PATCH v1 1/2] net/mlx5: fix meter hierarchy with modify header

[Shun Hao]

If any meter in the hierarchy has a policy flow containing set_tag or
modify_field action, the policy flow must match the src port to which
the policy belongs, to determine the order of modify_hdr and
meter action. But the meter hierarchy will not be able to use by
user flow that matches another src port.

To use this type of meter hierarchy for other src ports, we need to add
a new policy flow matching the new src port from the user flow
dynamically. But then it cannot be used by flow matching all ports.

Fixes: ca7e6051 ("net/mlx5: limit meter flow when matching all ports")
Cc: stable@dpdk.org

Signed-off-by: Shun Hao <shunh@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 doc/guides/nics/mlx5.rst        |  3 +++
 drivers/net/mlx5/mlx5.h         |  6 ++++--
 drivers/net/mlx5/mlx5_flow.c    |  2 +-
 drivers/net/mlx5/mlx5_flow_dv.c | 35 ++++++++++++++++++++++-----------
 4 files changed, 31 insertions(+), 15 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index d5f9375a4e..4f0db21dde 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -511,6 +511,9 @@ Limitations
     if meter has drop count
     or meter hierarchy contains any meter that uses drop count,
     it cannot be used by flow rule matching all ports.
+  - When using DV flow engine (``dv_flow_en`` = 1),
+    if meter hierarchy contains any meter that has MODIFY_FIELD/SET_TAG,
+    it cannot be used by flow matching all ports.
   - When using HWS flow engine (``dv_flow_en`` = 2),
     only meter mark action is supported.
 
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index cbe2d88b9e..73d0329500 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -846,8 +846,10 @@ struct mlx5_flow_meter_policy {
 	/* Is queue action in policy table. */
 	uint32_t is_hierarchy:1;
 	/* Is meter action in policy table. */
-	uint32_t hierarchy_drop_cnt:1;
-	/* Is any meter in hierarchy contains drop_cnt. */
+	uint32_t match_port:1;
+	/* If policy flows match src port. */
+	uint32_t hierarchy_match_port:1;
+	/* Is any meter in hierarchy contains policy flow that matches src port. */
 	uint32_t skip_r:1;
 	/* If red color policy is skipped. */
 	uint32_t skip_y:1;
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index ea88882b88..de75c05602 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -5471,7 +5471,7 @@ flow_meter_split_prep(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_REPRESENTED_PORT:
 			if (mlx5_flow_get_item_vport_id(dev, items, &flow_src_port, NULL, error))
 				return -rte_errno;
-			if (!fm->def_policy && wks->policy->is_hierarchy &&
+			if (!fm->def_policy && wks->policy->hierarchy_match_port &&
 			    flow_src_port != priv->representor_id) {
 				if (flow_drv_mtr_hierarchy_rule_create(dev,
 								flow, fm,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index acd7ea8b79..bb898d7569 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -5445,11 +5445,12 @@ mlx5_flow_validate_action_meter(struct rte_eth_dev *dev,
 							"have different src port.");
 			}
 			/* When flow matching all src ports, meter should not have drop count. */
-			if (all_ports && (fm->drop_cnt || mtr_policy->hierarchy_drop_cnt))
+			if (all_ports && (fm->drop_cnt || mtr_policy->hierarchy_match_port))
 				return rte_flow_error_set(error, EINVAL,
 							  RTE_FLOW_ERROR_TYPE_ITEM_SPEC, NULL,
-							  "Meter drop count not supported "
-							  "when matching all ports.");
+							  "Meter drop count or "
+							  "modify_field/set_tag in meter hierarchy "
+							  "not supported when matching all ports.");
 		} else if (mtr_policy->is_rss) {
 			struct mlx5_flow_meter_policy *fp;
 			struct mlx5_meter_policy_action_container *acg;
@@ -16661,8 +16662,8 @@ __flow_dv_create_domain_policy_acts(struct rte_eth_dev *dev,
 				mtr_policy->dev = next_policy->dev;
 				if (next_policy->mark)
 					mtr_policy->mark = 1;
-				if (next_fm->drop_cnt || next_policy->hierarchy_drop_cnt)
-					mtr_policy->hierarchy_drop_cnt = 1;
+				mtr_policy->hierarchy_match_port =
+							next_policy->hierarchy_match_port;
 				action_flags |=
 				MLX5_FLOW_ACTION_METER_WITH_TERMINATED_POLICY;
 				break;
@@ -17408,6 +17409,10 @@ __flow_dv_create_policy_acts_rules(struct rte_eth_dev *dev,
 			"Failed to create policy rules per domain.");
 		goto err_exit;
 	}
+	if (match_src_port) {
+		mtr_policy->match_port = match_src_port;
+		mtr_policy->hierarchy_match_port = match_src_port;
+	}
 	return 0;
 err_exit:
 	for (i = 0; i < RTE_COLORS; i++)
@@ -18030,7 +18035,7 @@ mlx5_meter_hierarchy_skip_tag_rule(struct mlx5_priv *priv,
 					 NULL, "Failed to find next meter in hierarchy.");
 		goto exit;
 	}
-	if (!(*next_fm)->drop_cnt) {
+	if (!mtr_policy->match_port) {
 		*skip = true;
 		goto exit;
 	}
@@ -18130,6 +18135,9 @@ flow_dv_meter_hierarchy_rule_create(struct rte_eth_dev *dev,
 		rte_spinlock_lock(&mtr_policy->sl);
 		sub_policy = mtr_policy->sub_policys[domain][0];
 		for (j = 0; j < MLX5_MTR_RTE_COLORS; j++) {
+			uint8_t act_n = 0;
+			struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
+
 			if (mtr_policy->act_cnt[j].fate_action != MLX5_FLOW_FATE_MTR)
 				continue;
 			color_rule = mlx5_malloc(MLX5_MEM_ZERO,
@@ -18160,15 +18168,18 @@ flow_dv_meter_hierarchy_rule_create(struct rte_eth_dev *dev,
 			next_sub_policy = next_policy->sub_policys[domain][0];
 			tbl_data = container_of(next_sub_policy->tbl_rsc,
 						struct mlx5_flow_tbl_data_entry, tbl);
+			modify_hdr = mtr_policy->act_cnt[j].modify_hdr;
 			if (mtr_first) {
-				acts.dv_actions[0] = mtr_action;
-				acts.dv_actions[1] = mtr_policy->act_cnt[j].modify_hdr->action;
+				acts.dv_actions[act_n++] = mtr_action;
+				if (modify_hdr)
+					acts.dv_actions[act_n++] = modify_hdr->action;
 			} else {
-				acts.dv_actions[0] = mtr_policy->act_cnt[j].modify_hdr->action;
-				acts.dv_actions[1] = mtr_action;
+				if (modify_hdr)
+					acts.dv_actions[act_n++] = modify_hdr->action;
+				acts.dv_actions[act_n++] = mtr_action;
 			}
-			acts.dv_actions[2] = tbl_data->jump.action;
-			acts.actions_n = 3;
+			acts.dv_actions[act_n++] = tbl_data->jump.action;
+			acts.actions_n = act_n;
 			if (__flow_dv_create_policy_matcher(dev, color_reg_c_idx,
 						MLX5_MTR_POLICY_MATCHER_PRIO, sub_policy,
 						&attr, true, item, &color_rule->matcher, error)) {
-- 
2.20.0

[PATCH v1 2/2] net/mlx5: fix meter policy with port ID destination

[Shun Hao]

When a meter policy contains a fate action of port_id, the policy flow
must match the src port to which the policy belongs. However, this meter
cannot be used by a flow that matches another src port.

This patch fixes this by adding a new policy flow matching the new src
port from the user flow dynamically, but then the meter cannot be used
by a flow that matches all the ports.

Fixes: 48fbc1be ("net/mlx5: fix meter policy flow match item")
Cc: stable@dpdk.org

Signed-off-by: Shun Hao <shunh@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.h         |   2 -
 drivers/net/mlx5/mlx5_flow_dv.c | 105 ++++++++++++++++----------------
 2 files changed, 53 insertions(+), 54 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 73d0329500..7d82cb7c8f 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -832,8 +832,6 @@ struct mlx5_meter_policy_action_container {
 
 /* Flow meter policy parameter structure. */
 struct mlx5_flow_meter_policy {
-	struct rte_eth_dev *dev;
-	/* The port dev on which policy is created. */
 	uint32_t is_rss:1;
 	/* Is RSS policy table. */
 	uint32_t ingress:1;
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index bb898d7569..362d236c89 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -5428,22 +5428,6 @@ mlx5_flow_validate_action_meter(struct rte_eth_dev *dev,
 				return -rte_errno;
 		}
 		if (attr->transfer) {
-			if (mtr_policy->dev) {
-				/**
-				 * When policy has fate action of port_id,
-				 * the flow should have the same src port as policy.
-				 */
-				struct mlx5_priv *policy_port_priv =
-						mtr_policy->dev->data->dev_private;
-
-				if (all_ports || flow_src_port != policy_port_priv->representor_id)
-					return rte_flow_error_set(error,
-							rte_errno,
-							RTE_FLOW_ERROR_TYPE_ITEM_SPEC,
-							NULL,
-							"Flow and meter policy "
-							"have different src port.");
-			}
 			/* When flow matching all src ports, meter should not have drop count. */
 			if (all_ports && (fm->drop_cnt || mtr_policy->hierarchy_match_port))
 				return rte_flow_error_set(error, EINVAL,
@@ -16659,7 +16643,6 @@ __flow_dv_create_domain_policy_acts(struct rte_eth_dev *dev,
 				act_cnt->next_mtr_id = next_fm->meter_id;
 				act_cnt->next_sub_policy = NULL;
 				mtr_policy->is_hierarchy = 1;
-				mtr_policy->dev = next_policy->dev;
 				if (next_policy->mark)
 					mtr_policy->mark = 1;
 				mtr_policy->hierarchy_match_port =
@@ -17342,7 +17325,6 @@ __flow_dv_create_policy_acts_rules(struct rte_eth_dev *dev,
 				acts[i].dv_actions[acts[i].actions_n] =
 					port_action->action;
 				acts[i].actions_n++;
-				mtr_policy->dev = dev;
 				match_src_port = true;
 				break;
 			case MLX5_FLOW_FATE_DROP:
@@ -18027,13 +18009,13 @@ mlx5_meter_hierarchy_skip_tag_rule(struct mlx5_priv *priv,
 	*next_fm = NULL;
 	*skip = false;
 	rte_spinlock_lock(&mtr_policy->sl);
-	if (!mtr_policy->is_hierarchy)
-		goto exit;
-	*next_fm = mlx5_flow_meter_hierarchy_next_meter(priv, mtr_policy, NULL);
-	if (!*next_fm) {
-		ret = rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
-					 NULL, "Failed to find next meter in hierarchy.");
-		goto exit;
+	if (mtr_policy->is_hierarchy) {
+		*next_fm = mlx5_flow_meter_hierarchy_next_meter(priv, mtr_policy, NULL);
+		if (!*next_fm) {
+			ret = rte_flow_error_set(error, EINVAL, RTE_FLOW_ERROR_TYPE_ACTION,
+						NULL, "Failed to find next meter in hierarchy.");
+			goto exit;
+		}
 	}
 	if (!mtr_policy->match_port) {
 		*skip = true;
@@ -18041,7 +18023,8 @@ mlx5_meter_hierarchy_skip_tag_rule(struct mlx5_priv *priv,
 	}
 	sub_policy = mtr_policy->sub_policys[domain][0];
 	for (i = 0; i < MLX5_MTR_RTE_COLORS; i++) {
-		if (mtr_policy->act_cnt[i].fate_action != MLX5_FLOW_FATE_MTR)
+		if (mtr_policy->act_cnt[i].fate_action != MLX5_FLOW_FATE_MTR &&
+		    mtr_policy->act_cnt[i].fate_action != MLX5_FLOW_FATE_PORT_ID)
 			continue;
 		TAILQ_FOREACH(color_rule, &sub_policy->color_rules[i], next_port)
 			if (color_rule->src_port == src_port) {
@@ -18115,7 +18098,7 @@ flow_dv_meter_hierarchy_rule_create(struct rte_eth_dev *dev,
 		if (mlx5_meter_hierarchy_skip_tag_rule(priv, mtr_policy, src_port,
 						       &next_fm, &skip, error))
 			goto err_exit;
-		if (next_fm && !skip) {
+		if (!skip) {
 			fm_info[fm_cnt].fm_policy = mtr_policy;
 			fm_info[fm_cnt].next_fm = next_fm;
 			if (++fm_cnt >= MLX5_MTR_CHAIN_MAX_NUM) {
@@ -18137,8 +18120,10 @@ flow_dv_meter_hierarchy_rule_create(struct rte_eth_dev *dev,
 		for (j = 0; j < MLX5_MTR_RTE_COLORS; j++) {
 			uint8_t act_n = 0;
 			struct mlx5_flow_dv_modify_hdr_resource *modify_hdr;
+			struct mlx5_flow_dv_port_id_action_resource *port_action;
 
-			if (mtr_policy->act_cnt[j].fate_action != MLX5_FLOW_FATE_MTR)
+			if (mtr_policy->act_cnt[j].fate_action != MLX5_FLOW_FATE_MTR &&
+			    mtr_policy->act_cnt[j].fate_action != MLX5_FLOW_FATE_PORT_ID)
 				continue;
 			color_rule = mlx5_malloc(MLX5_MEM_ZERO,
 						 sizeof(struct mlx5_sub_policy_color_rule),
@@ -18151,35 +18136,51 @@ flow_dv_meter_hierarchy_rule_create(struct rte_eth_dev *dev,
 				goto err_exit;
 			}
 			color_rule->src_port = src_port;
-			next_fm = fm_info[i].next_fm;
-			if (mlx5_flow_meter_attach(priv, next_fm, &attr, error)) {
-				mlx5_free(color_rule);
-				rte_spinlock_unlock(&mtr_policy->sl);
-				goto err_exit;
-			}
-			fm_info[i].tag_rule[j] = color_rule;
-			TAILQ_INSERT_TAIL(&sub_policy->color_rules[j], color_rule, next_port);
-			/* Prepare to create color rule. */
-			mtr_action = (next_fm->color_aware && j == RTE_COLOR_YELLOW) ?
-								next_fm->meter_action_y :
-								next_fm->meter_action_g;
-			next_policy = mlx5_flow_meter_policy_find(dev, next_fm->policy_id, NULL);
-			MLX5_ASSERT(next_policy);
-			next_sub_policy = next_policy->sub_policys[domain][0];
-			tbl_data = container_of(next_sub_policy->tbl_rsc,
-						struct mlx5_flow_tbl_data_entry, tbl);
 			modify_hdr = mtr_policy->act_cnt[j].modify_hdr;
-			if (mtr_first) {
-				acts.dv_actions[act_n++] = mtr_action;
-				if (modify_hdr)
-					acts.dv_actions[act_n++] = modify_hdr->action;
+			/* Prepare to create color rule. */
+			if (mtr_policy->act_cnt[j].fate_action == MLX5_FLOW_FATE_MTR) {
+				next_fm = fm_info[i].next_fm;
+				if (mlx5_flow_meter_attach(priv, next_fm, &attr, error)) {
+					mlx5_free(color_rule);
+					rte_spinlock_unlock(&mtr_policy->sl);
+					goto err_exit;
+				}
+				mtr_action = (next_fm->color_aware && j == RTE_COLOR_YELLOW) ?
+									next_fm->meter_action_y :
+									next_fm->meter_action_g;
+				next_policy = mlx5_flow_meter_policy_find(dev, next_fm->policy_id,
+									  NULL);
+				MLX5_ASSERT(next_policy);
+				next_sub_policy = next_policy->sub_policys[domain][0];
+				tbl_data = container_of(next_sub_policy->tbl_rsc,
+							struct mlx5_flow_tbl_data_entry, tbl);
+				if (mtr_first) {
+					acts.dv_actions[act_n++] = mtr_action;
+					if (modify_hdr)
+						acts.dv_actions[act_n++] = modify_hdr->action;
+				} else {
+					if (modify_hdr)
+						acts.dv_actions[act_n++] = modify_hdr->action;
+					acts.dv_actions[act_n++] = mtr_action;
+				}
+				acts.dv_actions[act_n++] = tbl_data->jump.action;
+				acts.actions_n = act_n;
 			} else {
+				port_action =
+					mlx5_ipool_get(priv->sh->ipool[MLX5_IPOOL_PORT_ID],
+						       mtr_policy->act_cnt[j].rix_port_id_action);
+				if (!port_action) {
+					mlx5_free(color_rule);
+					rte_spinlock_unlock(&mtr_policy->sl);
+					goto err_exit;
+				}
 				if (modify_hdr)
 					acts.dv_actions[act_n++] = modify_hdr->action;
-				acts.dv_actions[act_n++] = mtr_action;
+				acts.dv_actions[act_n++] = port_action->action;
+				acts.actions_n = act_n;
 			}
-			acts.dv_actions[act_n++] = tbl_data->jump.action;
-			acts.actions_n = act_n;
+			fm_info[i].tag_rule[j] = color_rule;
+			TAILQ_INSERT_TAIL(&sub_policy->color_rules[j], color_rule, next_port);
 			if (__flow_dv_create_policy_matcher(dev, color_reg_c_idx,
 						MLX5_MTR_POLICY_MATCHER_PRIO, sub_policy,
 						&attr, true, item, &color_rule->matcher, error)) {
-- 
2.20.0

RE: [PATCH v1 0/2] Fix src port match in meter hierarchy

[Raslan Darawsheh]

Hi,
> -----Original Message-----
> From: Shun Hao <shunh@nvidia.com>
> Sent: Tuesday, November 8, 2022 12:04 PM
> To: Slava Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Ori Kam <orika@nvidia.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>
> Subject: [PATCH v1 0/2] Fix src port match in meter hierarchy
> 
> In meter hierarchy, if policy flow contains modify_field or port_id action, a src
> port matching will be added to determine the position of meter in action list.
> 
> Currently the src port is static, using the port id policy belongs to.
> This patch set enables creation of multiple policy flows using src port from
> user flow dynamically to match different src port, so the meter hierarchy can
> always be used by different src ports.
> 
> Shun Hao (2):
>   net/mlx5: fix meter hierarchy with modify header
>   net/mlx5: fix meter policy with port ID destination
> 
>  doc/guides/nics/mlx5.rst        |   3 +
>  drivers/net/mlx5/mlx5.h         |   8 +-
>  drivers/net/mlx5/mlx5_flow.c    |   2 +-
>  drivers/net/mlx5/mlx5_flow_dv.c | 126 +++++++++++++++++---------------
>  4 files changed, 77 insertions(+), 62 deletions(-)
> 
> --
> 2.20.0

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh