[PATCH 0/2] net/mlx5/hws: fix matcher clean up for FDB tables

[PATCH 0/2] net/mlx5/hws: fix matcher clean up for FDB tables

[Dariusz Sosnowski]

Before these patches, if an application was configured to run with
HW Steering and E-Switch enabled, on EAL cleanup the assertion in
mlx5_dev_hw_global_release() was triggered - PD release was unsuccessful.

Root cause of this issue was linked to an inability to destroy RTC objects
used internally in mlx5, in HW Steering implementation.
PMD was unable to destroy RTC objects, because of dangling
references to those objects. More specifically, if all matchers
connected to a single flow table were created, this flow table
was still referencing RTC objects when theye were being destroyed.

This patch series fixes that behavior.
Matcher uninitilization is updated to remove the references to RTC objects
from flow table object if the last matcher related to the flow
table was destroyed.

Erez Shitrit (2):
  net/mlx5/hws: fix order of destroying default tables
  net/mlx5/hws: fix disconnecting matcher

 drivers/net/mlx5/hws/mlx5dr_matcher.c | 35 +++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_table.c   |  2 +-
 2 files changed, 36 insertions(+), 1 deletion(-)

-- 
2.25.1

[PATCH 1/2] net/mlx5/hws: fix order of destroying default tables

[Dariusz Sosnowski]

From: Erez Shitrit <erezsh@nvidia.com>

This patch fixes the order dereferencing default FDB miss table and
destroying the flow table object. Flow table should be destroyed
before the dereference.

Fixes: 394cc7ba4033 ("net/mlx5/hws: add table object")
Cc: valex@nvidia.com

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_table.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_table.c b/drivers/net/mlx5/hws/mlx5dr_table.c
index d3f77e4780..33052abce8 100644
--- a/drivers/net/mlx5/hws/mlx5dr_table.c
+++ b/drivers/net/mlx5/hws/mlx5dr_table.c
@@ -131,8 +131,8 @@ mlx5dr_table_create_default_ft(struct mlx5dr_table *tbl)
 void mlx5dr_table_destroy_default_ft(struct mlx5dr_table *tbl,
 				     struct mlx5dr_devx_obj *ft_obj)
 {
-	mlx5dr_table_down_default_fdb_miss_tbl(tbl);
 	mlx5dr_cmd_destroy_obj(ft_obj);
+	mlx5dr_table_down_default_fdb_miss_tbl(tbl);
 }
 
 static int mlx5dr_table_init(struct mlx5dr_table *tbl)
-- 
2.25.1

[PATCH 2/2] net/mlx5/hws: fix disconnecting matcher

[Dariusz Sosnowski]

From: Erez Shitrit <erezsh@nvidia.com>

This patch fixes the matcher disconnection handling, by removing the RTC
references from flow table if the currently removed matcher was the last
one for the given table. As a result RTC in this matcher can be
correctly freed, since there are no dangling references to the RTC.

Fixes: c467608215b2 ("net/mlx5/hws: add matcher object")
Cc: valex@nvidia.com

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_matcher.c | 35 +++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/net/mlx5/hws/mlx5dr_matcher.c b/drivers/net/mlx5/hws/mlx5dr_matcher.c
index 67adfeec6c..2e444c1179 100644
--- a/drivers/net/mlx5/hws/mlx5dr_matcher.c
+++ b/drivers/net/mlx5/hws/mlx5dr_matcher.c
@@ -36,6 +36,30 @@ static void mlx5dr_matcher_destroy_end_ft(struct mlx5dr_matcher *matcher)
 	mlx5dr_table_destroy_default_ft(matcher->tbl, matcher->end_ft);
 }
 
+static int mlx5dr_matcher_free_rtc_pointing(uint32_t fw_ft_type,
+					    enum mlx5dr_table_type type,
+					    struct mlx5dr_devx_obj *devx_obj)
+{
+	struct mlx5dr_cmd_ft_modify_attr ft_attr = {0};
+	int ret;
+
+	if (type != MLX5DR_TABLE_TYPE_FDB)
+		return 0;
+
+	ft_attr.modify_fs = MLX5_IFC_MODIFY_FLOW_TABLE_RTC_ID;
+	ft_attr.type = fw_ft_type;
+	ft_attr.rtc_id_0 = 0;
+	ft_attr.rtc_id_1 = 0;
+
+	ret = mlx5dr_cmd_flow_table_modify(devx_obj, &ft_attr);
+	if (ret) {
+		DR_LOG(ERR, "Failed to disconnect previous RTC");
+		return ret;
+	}
+
+	return 0;
+}
+
 static int mlx5dr_matcher_connect(struct mlx5dr_matcher *matcher)
 {
 	struct mlx5dr_cmd_ft_modify_attr ft_attr = {0};
@@ -148,6 +172,17 @@ static int mlx5dr_matcher_disconnect(struct mlx5dr_matcher *matcher)
 
 	LIST_REMOVE(matcher, next);
 
+	if (!next) {
+		/* ft no longer points to any RTC, drop refcount */
+		ret = mlx5dr_matcher_free_rtc_pointing(tbl->fw_ft_type,
+						       tbl->type,
+						       prev_ft);
+		if (ret) {
+			DR_LOG(ERR, "Failed to reset last RTC refcount");
+			return ret;
+		}
+	}
+
 	return 0;
 }
 
-- 
2.25.1

RE: [PATCH 0/2] net/mlx5/hws: fix matcher clean up for FDB tables

[Matan Azrad]

From: Dariusz Sosnowski
 
> Before these patches, if an application was configured to run with HW
> Steering and E-Switch enabled, on EAL cleanup the assertion in
> mlx5_dev_hw_global_release() was triggered - PD release was unsuccessful.
> 
> Root cause of this issue was linked to an inability to destroy RTC objects used
> internally in mlx5, in HW Steering implementation.
> PMD was unable to destroy RTC objects, because of dangling references to
> those objects. More specifically, if all matchers connected to a single flow
> table were created, this flow table was still referencing RTC objects when
> theye were being destroyed.
> 
> This patch series fixes that behavior.
> Matcher uninitilization is updated to remove the references to RTC objects
> from flow table object if the last matcher related to the flow table was
> destroyed.
> 
> Erez Shitrit (2):
>   net/mlx5/hws: fix order of destroying default tables
>   net/mlx5/hws: fix disconnecting matcher
> 
>  drivers/net/mlx5/hws/mlx5dr_matcher.c | 35
> +++++++++++++++++++++++++++
>  drivers/net/mlx5/hws/mlx5dr_table.c   |  2 +-
>  2 files changed, 36 insertions(+), 1 deletion(-)

Series-acked-by: Matan Azrad <matan@nvidia.com>

> --
> 2.25.1

RE: [PATCH 0/2] net/mlx5/hws: fix matcher clean up for FDB tables

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Dariusz Sosnowski <dsosnowski@nvidia.com>
> Sent: Wednesday, November 9, 2022 11:54 AM
> To: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Alex Vesker <valex@nvidia.com>; Erez Shitrit
> <erezsh@nvidia.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>
> Subject: [PATCH 0/2] net/mlx5/hws: fix matcher clean up for FDB tables
> 
> Before these patches, if an application was configured to run with HW
> Steering and E-Switch enabled, on EAL cleanup the assertion in
> mlx5_dev_hw_global_release() was triggered - PD release was unsuccessful.
> 
> Root cause of this issue was linked to an inability to destroy RTC objects used
> internally in mlx5, in HW Steering implementation.
> PMD was unable to destroy RTC objects, because of dangling references to
> those objects. More specifically, if all matchers connected to a single flow
> table were created, this flow table was still referencing RTC objects when
> theye were being destroyed.
> 
> This patch series fixes that behavior.
> Matcher uninitilization is updated to remove the references to RTC objects
> from flow table object if the last matcher related to the flow table was
> destroyed.
> 
> Erez Shitrit (2):
>   net/mlx5/hws: fix order of destroying default tables
>   net/mlx5/hws: fix disconnecting matcher
> 
>  drivers/net/mlx5/hws/mlx5dr_matcher.c | 35
> +++++++++++++++++++++++++++
>  drivers/net/mlx5/hws/mlx5dr_table.c   |  2 +-
>  2 files changed, 36 insertions(+), 1 deletion(-)
> 
> --
> 2.25.1

series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh