[PATCH 0/2] net/mlx5: fix representor matching

[PATCH 0/2] net/mlx5: fix representor matching

[Dariusz Sosnowski]

This patch series addresses a few issues with representor
matching mode available in mlx5 PMD:

- First patch fixes an issue with internal flow group
  index translation occurring when both extended metadata mode
  and representor matching was enabled.
- Second patch fixes an issue with isolated mode occurring
  when representor matching was disabled.

Dariusz Sosnowski (2):
  net/mlx5: fix egress group translation in HWS
  net/mlx5: fix isolated mode when repr matching is disabled

 doc/guides/nics/mlx5.rst         |  3 +++
 drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.c     |  4 ++++
 drivers/net/mlx5/mlx5_flow_hw.c  | 14 +++++++++-----
 4 files changed, 32 insertions(+), 5 deletions(-)

-- 
2.25.1

[PATCH 1/2] net/mlx5: fix egress group translation in HWS

[Dariusz Sosnowski]

With HW Steering enabled creating egress template tables and
egress flow rules on E-Switch setups is allowed.
To enable it, PMD creates a set of default egress flow rules
responsible for:

- Storing representor ID (vport tag is used) in HW register.
  This is used for traffic source identification.
- Copying software metadata to proper HW register to allow
  preserving metadata across domains.

Structure of these flow rules and whether they are inserted
depend on the device configuration.
There are the following cases:

1. repr_matching=1 and dv_xmeta_en=4
   - An egress flow rule in group 0 is created for each Tx queue;
   - Flow rule matching SQ number - fills unused REG_C_0 bits
     with vport tag, copies REG_A to REG_C_1 and jumps to group 1.
2. repr_matching=1 and dv_xmeta_en=0
   - An egress flow rule in group 0 is created for each Tx queue;
   - Flow rule matching SQ number - fills unused REG_C_0 bits
     with vport tag and jumps to group 1.
3. repr_matching=0 and dv_xmeta_en=4
   - A single egress flow rule in group 0 is created;
   - Flow rule matches all E-Switch manager TX traffic,
     copies REG_A to REG_C and jumps to group 1.
4. repr_matching=0 and dv_xmeta_en=0 - no default flow rules are added.

When default egress flow rules are required, they are inserted in
group 0 and this group is reserved for PMD purposes.
User created template tables must be created in higher groups.
As a result, on template table creation PMD is translating
the provided group (incrementing it in that case).

Before this patch, a condition used to check if translation of egress
flow group is needed was incorrect. It did not allow translation
if both representor matching AND extended metadata mode were enabled.

This patch fixes this condition - translation is allowed if and only if
representor matching OR extended metadata mode is enabled.

Fixes: 483181f7b6dd ("net/mlx5: support device control of representor matching")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_hw.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index a9c7045a3e..d3d86fe24d 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -3260,14 +3260,18 @@ flow_hw_translate_group(struct rte_eth_dev *dev,
 						  "group index not supported");
 		*table_group = group + 1;
 	} else if (config->dv_esw_en &&
-		   !(config->repr_matching && config->dv_xmeta_en == MLX5_XMETA_MODE_META32_HWS) &&
+		   (config->repr_matching || config->dv_xmeta_en == MLX5_XMETA_MODE_META32_HWS) &&
 		   cfg->external &&
 		   flow_attr->egress) {
 		/*
-		 * On E-Switch setups, egress group translation is not done if and only if
-		 * representor matching is disabled and legacy metadata mode is selected.
-		 * In all other cases, egree group 0 is reserved for representor tagging flows
-		 * and metadata copy flows.
+		 * On E-Switch setups, default egress flow rules are inserted to allow
+		 * representor matching and/or preserving metadata across steering domains.
+		 * These flow rules are inserted in group 0 and this group is reserved by PMD
+		 * for these purposes.
+		 *
+		 * As a result, if representor matching or extended metadata mode is enabled,
+		 * group provided by the user must be incremented to avoid inserting flow rules
+		 * in group 0.
 		 */
 		if (group > MLX5_HW_MAX_EGRESS_GROUP)
 			return rte_flow_error_set(error, EINVAL,
-- 
2.25.1

RE: [PATCH 1/2] net/mlx5: fix egress group translation in HWS

[Suanming Mou]

> -----Original Message-----
> From: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Sent: Sunday, February 26, 2023 4:18 AM
> To: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>
> Cc: dev@dpdk.org; stable@dpdk.org; Ori Kam <orika@nvidia.com>
> Subject: [PATCH 1/2] net/mlx5: fix egress group translation in HWS
> 
> With HW Steering enabled creating egress template tables and egress flow rules
> on E-Switch setups is allowed.
> To enable it, PMD creates a set of default egress flow rules responsible for:
> 
> - Storing representor ID (vport tag is used) in HW register.
>   This is used for traffic source identification.
> - Copying software metadata to proper HW register to allow
>   preserving metadata across domains.
> 
> Structure of these flow rules and whether they are inserted depend on the
> device configuration.
> There are the following cases:
> 
> 1. repr_matching=1 and dv_xmeta_en=4
>    - An egress flow rule in group 0 is created for each Tx queue;
>    - Flow rule matching SQ number - fills unused REG_C_0 bits
>      with vport tag, copies REG_A to REG_C_1 and jumps to group 1.
> 2. repr_matching=1 and dv_xmeta_en=0
>    - An egress flow rule in group 0 is created for each Tx queue;
>    - Flow rule matching SQ number - fills unused REG_C_0 bits
>      with vport tag and jumps to group 1.
> 3. repr_matching=0 and dv_xmeta_en=4
>    - A single egress flow rule in group 0 is created;
>    - Flow rule matches all E-Switch manager TX traffic,
>      copies REG_A to REG_C and jumps to group 1.
> 4. repr_matching=0 and dv_xmeta_en=0 - no default flow rules are added.
> 
> When default egress flow rules are required, they are inserted in group 0 and
> this group is reserved for PMD purposes.
> User created template tables must be created in higher groups.
> As a result, on template table creation PMD is translating the provided group
> (incrementing it in that case).
> 
> Before this patch, a condition used to check if translation of egress flow group is
> needed was incorrect. It did not allow translation if both representor matching
> AND extended metadata mode were enabled.
> 
> This patch fixes this condition - translation is allowed if and only if representor
> matching OR extended metadata mode is enabled.
> 
> Fixes: 483181f7b6dd ("net/mlx5: support device control of representor
> matching")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>

[PATCH 2/2] net/mlx5: fix isolated mode when repr matching is disabled

[Dariusz Sosnowski]

In HW steering mode, when running on an E-Switch setup,
mlx5 PMD provides an ability to enable or disable
representor matching (through `repr_matching_en` device argument).
If representor matching is enabled, any ingress or egress flow rule,
created on any port representor will match traffic related
to that specific port.
If it is disabled, flow rule created on one of the ports,
will match traffic related to all ports.

As a result, when representor matching is disabled,
PMD cannot correctly create control flow rules for receiving
default traffic according to port configuration.
Since each port representor in the same switch domain,
can have different port configuration and flow rules
do not differentiate between ports, these flow rules cannot be
correctly applied.
In that case, each port works in de facto isolated mode.

This patch makes sure that if representor matching is disabled,
port is forced into isolated mode. Disabling flow isolated is forbidden.

Fixes: 483181f7b6dd ("net/mlx5: support device control of representor matching")
Cc: stable@dpdk.org

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 doc/guides/nics/mlx5.rst         |  3 +++
 drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.c     |  4 ++++
 3 files changed, 23 insertions(+)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 6510e74fb9..350fb0287e 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -1137,6 +1137,9 @@ for an additional list of options shared with other mlx5 drivers.
   - 0. If representor matching is disabled, then there will be no implicit
     item added. As a result, ingress flow rules will match traffic
     coming to any port, not only the port on which flow rule is created.
+    Because of that, default flow rules for ingress traffic cannot be created
+    and port starts in isolated mode by default. Port cannot be switched back
+    to non-isolated mode.
 
   - 1. If representor matching is enabled (default setting),
     then each ingress pattern template has an implicit REPRESENTED_PORT
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a71474c90a..2cce0a7f0f 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1613,6 +1613,22 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 			err = EINVAL;
 			goto error;
 		}
+		/*
+		 * If representor matching is disabled, PMD cannot create default flow rules
+		 * to receive traffic for all ports, since implicit source port match is not added.
+		 * Isolated mode is forced.
+		 */
+		if (priv->sh->config.dv_esw_en && !priv->sh->config.repr_matching) {
+			err = mlx5_flow_isolate(eth_dev, 1, NULL);
+			if (err < 0) {
+				err = -err;
+				goto error;
+			}
+			DRV_LOG(WARNING, "port %u ingress traffic is restricted to defined "
+					 "flow rules (isolated mode) since representor "
+					 "matching is disabled",
+				eth_dev->data->port_id);
+		}
 		return eth_dev;
 #else
 		DRV_LOG(ERR, "DV support is missing for HWS.");
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index a6a426caf7..7ee30bdd1f 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -8077,6 +8077,10 @@ mlx5_flow_isolate(struct rte_eth_dev *dev,
 				   "port must be stopped first");
 		return -rte_errno;
 	}
+	if (!enable && !priv->sh->config.repr_matching)
+		return rte_flow_error_set(error, ENOTSUP, RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+					  "isolated mode cannot be disabled when "
+					  "representor matching is disabled");
 	priv->isolated = !!enable;
 	if (enable)
 		dev->dev_ops = &mlx5_dev_ops_isolate;
-- 
2.25.1

RE: [PATCH 2/2] net/mlx5: fix isolated mode when repr matching is disabled

[Suanming Mou]

> -----Original Message-----
> From: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Sent: Sunday, February 26, 2023 4:18 AM
> To: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>
> Cc: dev@dpdk.org; stable@dpdk.org; Ori Kam <orika@nvidia.com>
> Subject: [PATCH 2/2] net/mlx5: fix isolated mode when repr matching is disabled
> 
> In HW steering mode, when running on an E-Switch setup,
> mlx5 PMD provides an ability to enable or disable representor matching (through
> `repr_matching_en` device argument).
> If representor matching is enabled, any ingress or egress flow rule, created on
> any port representor will match traffic related to that specific port.
> If it is disabled, flow rule created on one of the ports, will match traffic related
> to all ports.
> 
> As a result, when representor matching is disabled, PMD cannot correctly create
> control flow rules for receiving default traffic according to port configuration.
> Since each port representor in the same switch domain, can have different port
> configuration and flow rules do not differentiate between ports, these flow
> rules cannot be correctly applied.
> In that case, each port works in de facto isolated mode.
> 
> This patch makes sure that if representor matching is disabled, port is forced
> into isolated mode. Disabling flow isolated is forbidden.
> 
> Fixes: 483181f7b6dd ("net/mlx5: support device control of representor
> matching")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>

Thanks.

RE: [PATCH 0/2] net/mlx5: fix representor matching

[Matan Azrad]

From: Dariusz Sosnowski
> This patch series addresses a few issues with representor matching mode
> available in mlx5 PMD:
> 
> - First patch fixes an issue with internal flow group
>   index translation occurring when both extended metadata mode
>   and representor matching was enabled.
> - Second patch fixes an issue with isolated mode occurring
>   when representor matching was disabled.
> 
> Dariusz Sosnowski (2):
>   net/mlx5: fix egress group translation in HWS
>   net/mlx5: fix isolated mode when repr matching is disabled
> 
>  doc/guides/nics/mlx5.rst         |  3 +++
>  drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
>  drivers/net/mlx5/mlx5_flow.c     |  4 ++++
>  drivers/net/mlx5/mlx5_flow_hw.c  | 14 +++++++++-----
>  4 files changed, 32 insertions(+), 5 deletions(-)
> 
Series-acked-by: Matan Azrad <matan@nvidia.com>

RE: [PATCH 0/2] net/mlx5: fix representor matching

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Sent: Saturday, February 25, 2023 10:18 PM
> To: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>
> Cc: dev@dpdk.org; stable@dpdk.org; Ori Kam <orika@nvidia.com>
> Subject: [PATCH 0/2] net/mlx5: fix representor matching
> 
> This patch series addresses a few issues with representor matching mode
> available in mlx5 PMD:
> 
> - First patch fixes an issue with internal flow group
>   index translation occurring when both extended metadata mode
>   and representor matching was enabled.
> - Second patch fixes an issue with isolated mode occurring
>   when representor matching was disabled.
> 
> Dariusz Sosnowski (2):
>   net/mlx5: fix egress group translation in HWS
>   net/mlx5: fix isolated mode when repr matching is disabled
> 
>  doc/guides/nics/mlx5.rst         |  3 +++
>  drivers/net/mlx5/linux/mlx5_os.c | 16 ++++++++++++++++
>  drivers/net/mlx5/mlx5_flow.c     |  4 ++++
>  drivers/net/mlx5/mlx5_flow_hw.c  | 14 +++++++++-----
>  4 files changed, 32 insertions(+), 5 deletions(-)
> 
> --
> 2.25.1

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh