[PATCH v2 0/4] net/mlx5: implement send to kernel action

[PATCH v2 0/4] net/mlx5: implement send to kernel action

[Michael Savisko]

This patchset provides mlx5 implementation of new rte_flow
action RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL.

Depends-on: patch-6838dd4bf976 ("ethdev: add send to kernel action")

Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
v2:
- updated after code review

v1:
https://patches.dpdk.org/project/dpdk/list/?series=24734

---
Michael Savisko (4):
  common/mlx5: new glue callback for send to kernel action
  net/mlx5: introduce new mlx5 action flag
  net/mlx5: add send to kernel action resource holder
  net/mlx5: translation of rte flow send to kernel action

 doc/guides/nics/features/default.ini    |  1 +
 doc/guides/nics/features/mlx5.ini       |  1 +
 drivers/common/mlx5/linux/meson.build   |  2 +
 drivers/common/mlx5/linux/mlx5_glue.c   | 17 ++++++
 drivers/common/mlx5/linux/mlx5_glue.h   |  2 +
 drivers/net/mlx5/linux/mlx5_flow_os.h   | 22 ++++++++
 drivers/net/mlx5/linux/mlx5_os.c        | 13 +++++
 drivers/net/mlx5/mlx5.h                 |  6 ++
 drivers/net/mlx5/mlx5_flow.c            | 32 +++++++++++
 drivers/net/mlx5/mlx5_flow.h            | 10 +++-
 drivers/net/mlx5/mlx5_flow_dv.c         | 74 +++++++++++++++++++++++--
 drivers/net/mlx5/mlx5_flow_hw.c         |  3 +
 drivers/net/mlx5/windows/mlx5_flow_os.h | 24 ++++++++
 13 files changed, 200 insertions(+), 7 deletions(-)

-- 
2.27.0

[PATCH v2 1/4] common/mlx5: new glue callback for send to kernel action

[Michael Savisko]

Add new glue callback dr_create_flow_action_send_to_kernel.
Default callback invokes mlx5dv_dr_action_create_dest_root_table().

Add static inline mlx5_flow_os_create_flow_action_send_to_kernel(),
which calls dr_create_flow_action_send_to_kernel glue callback.

Define HAVE_MLX5DV_DR_ACTION_CREATE_DEST_ROOT_TABLE macro if function
mlx5dv_dr_action_create_dest_root_table exists in infiniband/mlx5dv.h

Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/meson.build   |  2 ++
 drivers/common/mlx5/linux/mlx5_glue.c   | 17 +++++++++++++++++
 drivers/common/mlx5/linux/mlx5_glue.h   |  2 ++
 drivers/net/mlx5/linux/mlx5_flow_os.h   | 22 ++++++++++++++++++++++
 drivers/net/mlx5/windows/mlx5_flow_os.h | 24 ++++++++++++++++++++++++
 5 files changed, 67 insertions(+)

diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build
index e77b46d157..b044f95700 100644
--- a/drivers/common/mlx5/linux/meson.build
+++ b/drivers/common/mlx5/linux/meson.build
@@ -213,6 +213,8 @@ has_sym_args = [
             'ibv_reg_mr_iova' ],
         [ 'HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR', 'infiniband/verbs.h',
             'ibv_import_device' ],
+        [ 'HAVE_MLX5DV_DR_ACTION_CREATE_DEST_ROOT_TABLE', 'infiniband/mlx5dv.h',
+            'mlx5dv_dr_action_create_dest_root_table' ],
 ]
 if  libmtcr_ul_found
     has_sym_args += [
diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c
index 450dd6a06a..b954df0784 100644
--- a/drivers/common/mlx5/linux/mlx5_glue.c
+++ b/drivers/common/mlx5/linux/mlx5_glue.c
@@ -1434,6 +1434,21 @@ mlx5_glue_dv_free_pp(struct mlx5dv_pp *pp)
 #endif
 }
 
+static void *
+mlx5_glue_dr_create_flow_action_send_to_kernel(void *tbl, uint16_t priority)
+{
+#ifdef HAVE_MLX5DV_DR_ACTION_CREATE_DEST_ROOT_TABLE
+	struct mlx5dv_dr_table *table = (struct mlx5dv_dr_table *)tbl;
+
+	return mlx5dv_dr_action_create_dest_root_table(table, priority);
+#else
+	RTE_SET_USED(tbl);
+	RTE_SET_USED(priority);
+	errno = ENOTSUP;
+	return NULL;
+#endif
+}
+
 __rte_cache_aligned
 const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) {
 	.version = MLX5_GLUE_VERSION,
@@ -1561,4 +1576,6 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) {
 	.dv_free_var = mlx5_glue_dv_free_var,
 	.dv_alloc_pp = mlx5_glue_dv_alloc_pp,
 	.dv_free_pp = mlx5_glue_dv_free_pp,
+	.dr_create_flow_action_send_to_kernel =
+		mlx5_glue_dr_create_flow_action_send_to_kernel,
 };
diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h
index c4903a6dce..9616dfdd06 100644
--- a/drivers/common/mlx5/linux/mlx5_glue.h
+++ b/drivers/common/mlx5/linux/mlx5_glue.h
@@ -373,6 +373,8 @@ struct mlx5_glue {
 	void *(*dv_create_flow_action_aso)
 			(struct mlx5dv_dr_domain *domain, void *aso_obj,
 			 uint32_t offset, uint32_t flags, uint8_t return_reg_c);
+	void *(*dr_create_flow_action_send_to_kernel)(void *tbl,
+						      uint16_t priority);
 };
 
 extern const struct mlx5_glue *mlx5_glue;
diff --git a/drivers/net/mlx5/linux/mlx5_flow_os.h b/drivers/net/mlx5/linux/mlx5_flow_os.h
index bcb48b3e56..ed71289322 100644
--- a/drivers/net/mlx5/linux/mlx5_flow_os.h
+++ b/drivers/net/mlx5/linux/mlx5_flow_os.h
@@ -368,6 +368,28 @@ mlx5_flow_os_create_flow_action_default_miss(void **action)
 	return (*action) ? 0 : -1;
 }
 
+/**
+ * Create flow action: send_to_kernel.
+ *
+ * @param[in] tbl
+ *   Pointer to destination root table.
+ * @param[in] priority
+ *   Priority to which traffic will arrive.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or -1 on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_send_to_kernel(void *tbl, uint16_t priority,
+					  void **action)
+{
+	*action = mlx5_glue->dr_create_flow_action_send_to_kernel(tbl,
+								  priority);
+	return (*action) ? 0 : -1;
+}
+
 /**
  * Create flow action: dest_devx_tir
  *
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.h b/drivers/net/mlx5/windows/mlx5_flow_os.h
index 347ec64580..1c1c17fc41 100644
--- a/drivers/net/mlx5/windows/mlx5_flow_os.h
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.h
@@ -326,6 +326,30 @@ mlx5_flow_os_create_flow_action_default_miss(void **action)
 	return 0;
 }
 
+/**
+ * Create flow action: send_to_kernel.
+ *
+ * @param[in] tbl
+ *   Pointer to destination root table.
+ * @param[in] priority
+ *   Priority to which traffic will arrive.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or -1 on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_send_to_kernel(void *tbl, uint16_t priority,
+					  void **action)
+{
+	RTE_SET_USED(tbl);
+	RTE_SET_USED(priority);
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
 /**
  * Create flow action: sampler
  *
-- 
2.27.0

[PATCH v2 2/4] net/mlx5: introduce new mlx5 action flag

[Michael Savisko]

Add new mlx5 action flag MLX5_FLOW_ACTION_SEND_TO_KERNEL.

Add element MLX5_FLOW_FATE_SEND_TO_KERNEL in enum mlx5_flow_fate_type.
For that purpose field 'fate_action' in structure mlx5_flow_handle must be
expanded from 3 bits to 4 bits.

Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.h    |  7 +++++--
 drivers/net/mlx5/mlx5_flow_dv.c | 17 +++++++++++++++++
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 0fa1735b1a..18a77ec619 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -277,12 +277,14 @@ enum mlx5_feature_name {
 #define MLX5_FLOW_ACTION_MODIFY_FIELD (1ull << 39)
 #define MLX5_FLOW_ACTION_METER_WITH_TERMINATED_POLICY (1ull << 40)
 #define MLX5_FLOW_ACTION_CT (1ull << 41)
+#define MLX5_FLOW_ACTION_SEND_TO_KERNEL (1ull << 42)
 
 #define MLX5_FLOW_FATE_ACTIONS \
 	(MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_QUEUE | \
 	 MLX5_FLOW_ACTION_RSS | MLX5_FLOW_ACTION_JUMP | \
 	 MLX5_FLOW_ACTION_DEFAULT_MISS | \
-	 MLX5_FLOW_ACTION_METER_WITH_TERMINATED_POLICY)
+	 MLX5_FLOW_ACTION_METER_WITH_TERMINATED_POLICY | \
+	 MLX5_FLOW_ACTION_SEND_TO_KERNEL)
 
 #define MLX5_FLOW_FATE_ESWITCH_ACTIONS \
 	(MLX5_FLOW_ACTION_DROP | MLX5_FLOW_ACTION_PORT_ID | \
@@ -477,6 +479,7 @@ enum mlx5_flow_fate_type {
 	MLX5_FLOW_FATE_DEFAULT_MISS,
 	MLX5_FLOW_FATE_SHARED_RSS,
 	MLX5_FLOW_FATE_MTR,
+	MLX5_FLOW_FATE_SEND_TO_KERNEL,
 	MLX5_FLOW_FATE_MAX,
 };
 
@@ -709,7 +712,7 @@ struct mlx5_flow_handle {
 	void *drv_flow; /**< pointer to driver flow object. */
 	uint32_t split_flow_id:27; /**< Sub flow unique match flow id. */
 	uint32_t is_meter_flow_id:1; /**< Indicate if flow_id is for meter. */
-	uint32_t fate_action:3; /**< Fate action type. */
+	uint32_t fate_action:4; /**< Fate action type. */
 	union {
 		uint32_t rix_hrxq; /**< Hash Rx queue object index. */
 		uint32_t rix_jump; /**< Index to the jump action resource. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 91f287af5c..6c355efed7 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -7970,6 +7970,12 @@ flow_dv_validate(struct rte_eth_dev *dev, const struct rte_flow_attr *attr,
 			 * list it here as a supported type
 			 */
 			break;
+#ifdef HAVE_MLX5DV_DR_ACTION_CREATE_DEST_ROOT_TABLE
+		case RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL:
+			action_flags |= MLX5_FLOW_ACTION_SEND_TO_KERNEL;
+			++actions_n;
+			break;
+#endif
 		default:
 			return rte_flow_error_set(error, ENOTSUP,
 						  RTE_FLOW_ERROR_TYPE_ACTION,
@@ -13694,6 +13700,11 @@ flow_dv_translate(struct rte_eth_dev *dev,
 			actions_n++;
 			action_flags |= MLX5_FLOW_ACTION_CT;
 			break;
+		case RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL:
+			return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION,
+				NULL, "send to kernel action is not supported.");
+			break;
 		case RTE_FLOW_ACTION_TYPE_END:
 			actions_end = true;
 			if (mhdr_res->actions_num) {
@@ -14742,6 +14753,12 @@ flow_dv_fate_resource_release(struct rte_eth_dev *dev,
 		flow_dv_port_id_action_resource_release(dev,
 				handle->rix_port_id_action);
 		break;
+	case MLX5_FLOW_FATE_SEND_TO_KERNEL:
+		/* In case of send_to_kernel action the actual release of
+		 * resource is done when all shared DR resources are released
+		 * since this resource is created once and always reused.
+		 */
+		break;
 	default:
 		DRV_LOG(DEBUG, "Incorrect fate action:%d", handle->fate_action);
 		break;
-- 
2.27.0

RE: [PATCH v2 2/4] net/mlx5: introduce new mlx5 action flag

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Michael Savisko <michaelsav@nvidia.com>
> Sent: Wednesday, October 19, 2022 9:40 PM
> To: dev@dpdk.org
> Cc: Michael Savisko <michaelsav@nvidia.com>; Ori Kam <orika@nvidia.com>;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> <asafp@nvidia.com>; Matan Azrad <matan@nvidia.com>
> Subject: [PATCH v2 2/4] net/mlx5: introduce new mlx5 action flag
> 
> Add new mlx5 action flag MLX5_FLOW_ACTION_SEND_TO_KERNEL.
> 
> Add element MLX5_FLOW_FATE_SEND_TO_KERNEL in enum
> mlx5_flow_fate_type.
> For that purpose field 'fate_action' in structure mlx5_flow_handle must be
> expanded from 3 bits to 4 bits.
> 
> Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
> Acked-by: Matan Azrad <matan@nvidia.com>
> ---
>  drivers/net/mlx5/mlx5_flow.h    |  7 +++++--
>  drivers/net/mlx5/mlx5_flow_dv.c | 17 +++++++++++++++++

Missing doc update for features list, I'll handle during integration

Kindest regards,
Raslan Darawsheh

[PATCH v2 3/4] net/mlx5: add send to kernel action resource holder

[Michael Savisko]

Add new structure mlx5_send_to_kernel_action which will hold
together allocated action resource and a reference to used table.
A new structure member of this type added to struct mlx5_dev_ctx_shared.
The member will be initialized upon first created send_to_kernel
action and will be reused for all future actions of this type.
Release of these resources will be done when all shared DR
resources are being released in mlx5_os_free_shared_dr().

Change function flow_dv_tbl_resource_release() from
static to external.

Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 13 +++++++++++++
 drivers/net/mlx5/mlx5.h          |  6 ++++++
 drivers/net/mlx5/mlx5_flow.h     |  2 ++
 drivers/net/mlx5/mlx5_flow_dv.c  |  6 +-----
 4 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 60677eb8d7..3e505d8f4c 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -700,6 +700,19 @@ mlx5_os_free_shared_dr(struct mlx5_priv *priv)
 		mlx5_glue->destroy_flow_action(sh->pop_vlan_action);
 		sh->pop_vlan_action = NULL;
 	}
+	if (sh->send_to_kernel_action.action) {
+		void *action = sh->send_to_kernel_action.action;
+
+		mlx5_glue->destroy_flow_action(action);
+		sh->send_to_kernel_action.action = NULL;
+	}
+	if (sh->send_to_kernel_action.tbl) {
+		struct mlx5_flow_tbl_resource *tbl =
+				sh->send_to_kernel_action.tbl;
+
+		flow_dv_tbl_resource_release(sh, tbl);
+		sh->send_to_kernel_action.tbl = NULL;
+	}
 #endif /* HAVE_MLX5DV_DR */
 	if (sh->default_miss_action)
 		mlx5_glue->destroy_flow_action
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 3c9e6bad53..c2c3ed81fa 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1180,6 +1180,11 @@ struct mlx5_flex_item {
 	struct mlx5_flex_pattern_field map[MLX5_FLEX_ITEM_MAPPING_NUM];
 };
 
+struct mlx5_send_to_kernel_action {
+	void *action;
+	void *tbl;
+};
+
 /*
  * Shared Infiniband device context for Master/Representors
  * which belong to same IB device with multiple IB ports.
@@ -1231,6 +1236,7 @@ struct mlx5_dev_ctx_shared {
 	/* Direct Rules tables for FDB, NIC TX+RX */
 	void *dr_drop_action; /* Pointer to DR drop action, any domain. */
 	void *pop_vlan_action; /* Pointer to DR pop VLAN action. */
+	struct mlx5_send_to_kernel_action send_to_kernel_action;
 	struct mlx5_hlist *encaps_decaps; /* Encap/decap action hash list. */
 	struct mlx5_hlist *modify_cmds;
 	struct mlx5_hlist *tag_table;
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 18a77ec619..525db05969 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1918,6 +1918,8 @@ struct mlx5_flow_tbl_resource *flow_dv_tbl_resource_get(struct rte_eth_dev *dev,
 		bool external, const struct mlx5_flow_tunnel *tunnel,
 		uint32_t group_id, uint8_t dummy,
 		uint32_t table_id, struct rte_flow_error *error);
+int flow_dv_tbl_resource_release(struct mlx5_dev_ctx_shared *sh,
+				 struct mlx5_flow_tbl_resource *tbl);
 
 struct mlx5_list_entry *flow_dv_tag_create_cb(void *tool_ctx, void *cb_ctx);
 int flow_dv_tag_match_cb(void *tool_ctx, struct mlx5_list_entry *entry,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 6c355efed7..bcd9926b81 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -75,10 +75,6 @@ union flow_dv_attr {
 	uint32_t attr;
 };
 
-static int
-flow_dv_tbl_resource_release(struct mlx5_dev_ctx_shared *sh,
-			     struct mlx5_flow_tbl_resource *tbl);
-
 static int
 flow_dv_encap_decap_resource_release(struct rte_eth_dev *dev,
 				     uint32_t encap_decap_idx);
@@ -10928,7 +10924,7 @@ flow_dv_tbl_remove_cb(void *tool_ctx, struct mlx5_list_entry *entry)
  * @return
  *   Returns 0 if table was released, else return 1;
  */
-static int
+int
 flow_dv_tbl_resource_release(struct mlx5_dev_ctx_shared *sh,
 			     struct mlx5_flow_tbl_resource *tbl)
 {
-- 
2.27.0

[PATCH v2 4/4] net/mlx5: translation of rte flow send to kernel action

[Michael Savisko]

Introduce mlx5_get_send_to_kernel_priority() function which returns
value of priority which must be used to jump back to table 0 in order
to send traffic to kernel. This function returns lowest priority.

Add flow_dv_translate_action_send_to_kernel() function which
will allocate rdma-core send_to_kernel action object.
Called from flow_dv_translate().

Fail translation of RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL action in
HW steering.

Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 doc/guides/nics/features/default.ini |  1 +
 doc/guides/nics/features/mlx5.ini    |  1 +
 drivers/net/mlx5/mlx5_flow.c         | 32 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow.h         |  1 +
 drivers/net/mlx5/mlx5_flow_dv.c      | 57 ++++++++++++++++++++++++++--
 drivers/net/mlx5/mlx5_flow_hw.c      |  3 ++
 6 files changed, 92 insertions(+), 3 deletions(-)

diff --git a/doc/guides/nics/features/default.ini b/doc/guides/nics/features/default.ini
index b450ef7e1a..5ed2b90843 100644
--- a/doc/guides/nics/features/default.ini
+++ b/doc/guides/nics/features/default.ini
@@ -198,3 +198,4 @@ set_ttl              =
 vf                   =
 vxlan_decap          =
 vxlan_encap          =
+send_to_kernel       =
diff --git a/doc/guides/nics/features/mlx5.ini b/doc/guides/nics/features/mlx5.ini
index e5974063c8..9f158ef171 100644
--- a/doc/guides/nics/features/mlx5.ini
+++ b/doc/guides/nics/features/mlx5.ini
@@ -128,3 +128,4 @@ set_tp_src           = Y
 set_ttl              = Y
 vxlan_decap          = Y
 vxlan_encap          = Y
+send_to_kernel       = Y
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index e4744b0a67..de998a9720 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -10967,6 +10967,38 @@ mlx5_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
 	return  res;
 }
 
+/**
+ * Get the priority for sending traffic to kernel table.
+ *
+ * @param[in] dev
+ *   Pointer to the Ethernet device structure.
+ *
+ * @return
+ *   On success: the value of priority for sending traffic to kernel table
+ *   On failure: -1
+ */
+uint32_t
+mlx5_get_send_to_kernel_priority(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint32_t res;
+
+	switch (priv->sh->flow_max_priority) {
+	case RTE_DIM(priority_map_5):
+		res = 15;
+		break;
+	case RTE_DIM(priority_map_3):
+		res = 7;
+		break;
+	default:
+		DRV_LOG(ERR,
+			"port %u maximum priority: %d expected 8/16",
+			dev->data->port_id, priv->sh->flow_max_priority);
+		res = (uint32_t)-1;
+	}
+	return res;
+}
+
 /**
  * Get the E-Switch Manager vport id.
  *
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 525db05969..8e97fa188a 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1753,6 +1753,7 @@ uint32_t mlx5_get_lowest_priority(struct rte_eth_dev *dev,
 uint16_t mlx5_get_matcher_priority(struct rte_eth_dev *dev,
 				   const struct rte_flow_attr *attr,
 				   uint32_t subpriority, bool external);
+uint32_t mlx5_get_send_to_kernel_priority(struct rte_eth_dev *dev);
 int mlx5_flow_get_reg_id(struct rte_eth_dev *dev,
 				     enum mlx5_feature_name feature,
 				     uint32_t id,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index bcd9926b81..4bdcb1815b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -12192,6 +12192,51 @@ flow_dv_translate_action_sample(struct rte_eth_dev *dev,
 	return 0;
 }
 
+static void *
+flow_dv_translate_action_send_to_kernel(struct rte_eth_dev *dev,
+					struct rte_flow_error *error)
+{
+	struct mlx5_flow_tbl_resource *tbl;
+	struct mlx5_dev_ctx_shared *sh;
+	uint32_t priority;
+	void *action;
+	int ret;
+
+	sh = MLX5_SH(dev);
+	if (sh->send_to_kernel_action.action)
+		return sh->send_to_kernel_action.action;
+	priority = mlx5_get_send_to_kernel_priority(dev);
+	if (priority == (uint32_t)-1) {
+		rte_flow_error_set(error, ENOTSUP,
+				   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+				   "required priority is not available");
+		return NULL;
+	}
+	tbl = flow_dv_tbl_resource_get(dev, 0, 0, 0, false, NULL, 0, 0, 0,
+				       error);
+	if (!tbl) {
+		rte_flow_error_set(error, ENODATA,
+				   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+				   "cannot find destination root table");
+		return NULL;
+	}
+	ret = mlx5_flow_os_create_flow_action_send_to_kernel(tbl->obj,
+				priority, &action);
+	if (ret) {
+		rte_flow_error_set(error, ENOMEM,
+				   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+				   "cannot create action");
+		goto err;
+	}
+	MLX5_ASSERT(action);
+	sh->send_to_kernel_action.action = action;
+	sh->send_to_kernel_action.tbl = tbl;
+	return action;
+err:
+	flow_dv_tbl_resource_release(sh, tbl);
+	return NULL;
+}
+
 /**
  * Convert Sample action to DV specification.
  *
@@ -13697,9 +13742,15 @@ flow_dv_translate(struct rte_eth_dev *dev,
 			action_flags |= MLX5_FLOW_ACTION_CT;
 			break;
 		case RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL:
-			return rte_flow_error_set(error, ENOTSUP,
-				RTE_FLOW_ERROR_TYPE_ACTION,
-				NULL, "send to kernel action is not supported.");
+			dev_flow->dv.actions[actions_n] =
+				flow_dv_translate_action_send_to_kernel(dev,
+							error);
+			if (!dev_flow->dv.actions[actions_n])
+				return -rte_errno;
+			actions_n++;
+			action_flags |= MLX5_FLOW_ACTION_SEND_TO_KERNEL;
+			dev_flow->handle->fate_action =
+					MLX5_FLOW_FATE_SEND_TO_KERNEL;
 			break;
 		case RTE_FLOW_ACTION_TYPE_END:
 			actions_end = true;
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 12498794a5..b168ff9e7e 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -707,6 +707,9 @@ flow_hw_actions_translate(struct rte_eth_dev *dev,
 			reformat_pos = i++;
 			refmt_type = MLX5DR_ACTION_REFORMAT_TYPE_TNL_L2_TO_L2;
 			break;
+		case RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL:
+			DRV_LOG(ERR, "send to kernel action is not supported in HW steering.");
+			goto err;
 		case RTE_FLOW_ACTION_TYPE_END:
 			actions_end = true;
 			break;
-- 
2.27.0

RE: [PATCH v2 0/4] net/mlx5: implement send to kernel action

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Michael Savisko <michaelsav@nvidia.com>
> Sent: Wednesday, October 19, 2022 9:40 PM
> To: dev@dpdk.org
> Cc: Michael Savisko <michaelsav@nvidia.com>; Ori Kam <orika@nvidia.com>;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> <asafp@nvidia.com>; Matan Azrad <matan@nvidia.com>
> Subject: [PATCH v2 0/4] net/mlx5: implement send to kernel action
> 
> This patchset provides mlx5 implementation of new rte_flow action
> RTE_FLOW_ACTION_TYPE_SEND_TO_KERNEL.
> 
> Depends-on: patch-6838dd4bf976 ("ethdev: add send to kernel action")
> 
> Signed-off-by: Michael Savisko <michaelsav@nvidia.com>
> Acked-by: Matan Azrad <matan@nvidia.com>
> ---
> v2:
> - updated after code review
> 
> v1:
> https://patches.dpdk.org/project/dpdk/list/?series=24734
> 
> ---
> Michael Savisko (4):
>   common/mlx5: new glue callback for send to kernel action
>   net/mlx5: introduce new mlx5 action flag
>   net/mlx5: add send to kernel action resource holder
>   net/mlx5: translation of rte flow send to kernel action
> 
>  doc/guides/nics/features/default.ini    |  1 +
>  doc/guides/nics/features/mlx5.ini       |  1 +
>  drivers/common/mlx5/linux/meson.build   |  2 +
>  drivers/common/mlx5/linux/mlx5_glue.c   | 17 ++++++
>  drivers/common/mlx5/linux/mlx5_glue.h   |  2 +
>  drivers/net/mlx5/linux/mlx5_flow_os.h   | 22 ++++++++
>  drivers/net/mlx5/linux/mlx5_os.c        | 13 +++++
>  drivers/net/mlx5/mlx5.h                 |  6 ++
>  drivers/net/mlx5/mlx5_flow.c            | 32 +++++++++++
>  drivers/net/mlx5/mlx5_flow.h            | 10 +++-
>  drivers/net/mlx5/mlx5_flow_dv.c         | 74 +++++++++++++++++++++++--
>  drivers/net/mlx5/mlx5_flow_hw.c         |  3 +
>  drivers/net/mlx5/windows/mlx5_flow_os.h | 24 ++++++++
>  13 files changed, 200 insertions(+), 7 deletions(-)
> 
> --
> 2.27.0

Series applied to next-net-mlx,
With a small fix for the missing doc on the features list, 

Kindest regards,
Raslan Darawsheh