[RFC 0/9] support flex item matching and modify field

[RFC 0/9] support flex item matching and modify field

[Rongwei Liu]

Add flex item matching and modify field support when dv_flow_en=2.

Rongwei Liu (9):
  ethdev: add flex item modify field support
  app/testpmd: add flex item modify field cmdline support
  app/testpmd: pass flex handle into matching mask
  net/mlx5: enable hws flex item create
  net/mlx5: add IPv6 protocol as flex item input
  net/mlx5/hws: add hws flex item matching support
  net/mlx5/hws: add flex item modify field implementation
  net/mlx5: return error for sws modify field
  app/testpmd: raw encap with flex item support

 app/test-pmd/cmdline_flow.c            | 123 ++++++++++++++---
 doc/guides/nics/features/mlx5.ini      |   1 +
 doc/guides/nics/mlx5.rst               |   2 +
 doc/guides/prog_guide/rte_flow.rst     |  36 ++---
 doc/guides/rel_notes/release_22_03.rst |   4 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  14 +-
 drivers/common/mlx5/mlx5_devx_cmds.h   |   7 +-
 drivers/common/mlx5/mlx5_prm.h         |  23 +++-
 drivers/net/mlx5/hws/mlx5dr_definer.c  | 141 +++++++++++++++++++
 drivers/net/mlx5/linux/mlx5_os.c       |  27 ++--
 drivers/net/mlx5/mlx5.c                |  17 ++-
 drivers/net/mlx5/mlx5.h                |   8 +-
 drivers/net/mlx5/mlx5_flow.h           |   4 +
 drivers/net/mlx5/mlx5_flow_dv.c        | 184 ++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_flex.c      | 102 +++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c        |  64 ++++++++-
 lib/ethdev/rte_flow.h                  |   8 +-
 17 files changed, 663 insertions(+), 102 deletions(-)

-- 
2.27.0

[RFC 1/9] ethdev: add flex item modify field support

[Rongwei Liu]

Add flex item as modify field destination.
Add "struct rte_flow_item_flex_handle *flex_handle" into
"struct rte_flow_action_modify_data" as union with existed
"level" member. This new member is dedicated for modifying
flex item.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 doc/guides/prog_guide/rte_flow.rst     | 36 ++++++++++++++------------
 doc/guides/rel_notes/release_22_03.rst |  4 +++
 lib/ethdev/rte_flow.h                  |  8 ++++--
 3 files changed, 29 insertions(+), 19 deletions(-)

diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 3e6242803d..ea20145224 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -2952,23 +2952,25 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx, xxx, xxx}.
 
 .. table:: destination/source field definition
 
-   +---------------+----------------------------------------------------------+
-   | Field         | Value                                                    |
-   +===============+==========================================================+
-   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
-   +---------------+----------------------------------------------------------+
-   | ``level``     | encapsulation level of a packet field or tag array index |
-   +---------------+----------------------------------------------------------+
-   | ``offset``    | number of bits to skip at the beginning                  |
-   +---------------+----------------------------------------------------------+
-   | ``value``     | immediate value buffer (source field only, not           |
-   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
-   | ``pvalue``    | pointer to immediate value data (source field only, not  |
-   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
+   +-----------------+----------------------------------------------------------+
+   | Field           | Value                                                    |
+   +=================+==========================================================+
+   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
+   +-----------------+----------------------------------------------------------+
+   | ``level``       | encapsulation level of a packet field or tag array index |
+   +-----------------+----------------------------------------------------------+
+   | ``flex_handle`` | flex item handle of a packet field                       |
+   +-----------------+----------------------------------------------------------+
+   | ``offset``      | number of bits to skip at the beginning                  |
+   +-----------------+----------------------------------------------------------+
+   | ``value``       | immediate value buffer (source field only, not           |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
+   |                 | field type                                               |
+   +-----------------+----------------------------------------------------------+
+   | ``pvalue``      | pointer to immediate value data (source field only, not  |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
+   |                 | field type                                               |
+   +-----------------+----------------------------------------------------------+
 
 Action: ``CONNTRACK``
 ^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst
index 0923707cb8..5fc5aff8a4 100644
--- a/doc/guides/rel_notes/release_22_03.rst
+++ b/doc/guides/rel_notes/release_22_03.rst
@@ -207,6 +207,10 @@ API Changes
 * ethdev: Old public macros and enumeration constants without ``RTE_ETH_`` prefix,
   which are kept for backward compatibility, are marked as deprecated.
 
+* ethdev: added a new field:
+
+  - modify flex item: ``rte_flow_action_modify_data.flex_handle``
+
 * cryptodev: The asymmetric session handling was modified to use a single
   mempool object. An API ``rte_cryptodev_asym_session_pool_create`` was added
   to create a mempool with element size big enough to hold the generic asymmetric
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index b60987db4b..d535722773 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
 	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
 	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
+	RTE_FLOW_FIELD_FLEX_ITEM,       /**< Flex item. */
 };
 
 /**
@@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
 	RTE_STD_C11
 	union {
 		struct {
-			/** Encapsulation level or tag index. */
-			uint32_t level;
+			/**< Encapsulation level or tag index or flex item handle. */
+			union {
+				uint32_t level;
+				struct rte_flow_item_flex_handle *flex_handle;
+			};
 			/** Number of bits to skip from a field. */
 			uint32_t offset;
 		};
-- 
2.27.0

RE: [RFC 1/9] ethdev: add flex item modify field support

[Ori Kam]

Hi Rongwei,


> -----Original Message-----
> From: Rongwei Liu <rongweil@nvidia.com>
> Sent: Wednesday, 21 December 2022 10:40
> 
> Add flex item as modify field destination.
> Add "struct rte_flow_item_flex_handle *flex_handle" into
> "struct rte_flow_action_modify_data" as union with existed
> "level" member. This new member is dedicated for modifying
> flex item.
> 
> Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
> ---
>  doc/guides/prog_guide/rte_flow.rst     | 36 ++++++++++++++------------
>  doc/guides/rel_notes/release_22_03.rst |  4 +++
>  lib/ethdev/rte_flow.h                  |  8 ++++--
>  3 files changed, 29 insertions(+), 19 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst
> b/doc/guides/prog_guide/rte_flow.rst
> index 3e6242803d..ea20145224 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -2952,23 +2952,25 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx,
> xxx, xxx}.
> 
>  .. table:: destination/source field definition
> 
> -   +---------------+----------------------------------------------------------+
> -   | Field         | Value                                                    |
> -
> +===============+=========================================
> =================+
> -   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
> -   +---------------+----------------------------------------------------------+
> -   | ``level``     | encapsulation level of a packet field or tag array index |
> -   +---------------+----------------------------------------------------------+
> -   | ``offset``    | number of bits to skip at the beginning                  |
> -   +---------------+----------------------------------------------------------+
> -   | ``value``     | immediate value buffer (source field only, not           |
> -   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
> -   |               | field type                                               |
> -   +---------------+----------------------------------------------------------+
> -   | ``pvalue``    | pointer to immediate value data (source field only, not  |
> -   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
> -   |               | field type                                               |
> -   +---------------+----------------------------------------------------------+
> +   +-----------------+----------------------------------------------------------+
> +   | Field           | Value                                                    |
> +
> +=================+=======================================
> ===================+
> +   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
> +   +-----------------+----------------------------------------------------------+
> +   | ``level``       | encapsulation level of a packet field or tag array index |
> +   +-----------------+----------------------------------------------------------+
> +   | ``flex_handle`` | flex item handle of a packet field                       |
> +   +-----------------+----------------------------------------------------------+
> +   | ``offset``      | number of bits to skip at the beginning                  |
> +   +-----------------+----------------------------------------------------------+
> +   | ``value``       | immediate value buffer (source field only, not           |
> +   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
> +   |                 | field type                                               |
> +   +-----------------+----------------------------------------------------------+
> +   | ``pvalue``      | pointer to immediate value data (source field only, not  |
> +   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
> +   |                 | field type                                               |
> +   +-----------------+----------------------------------------------------------+
> 
>  Action: ``CONNTRACK``
>  ^^^^^^^^^^^^^^^^^^^^^
> diff --git a/doc/guides/rel_notes/release_22_03.rst
> b/doc/guides/rel_notes/release_22_03.rst
> index 0923707cb8..5fc5aff8a4 100644
> --- a/doc/guides/rel_notes/release_22_03.rst
> +++ b/doc/guides/rel_notes/release_22_03.rst
> @@ -207,6 +207,10 @@ API Changes
>  * ethdev: Old public macros and enumeration constants without
> ``RTE_ETH_`` prefix,
>    which are kept for backward compatibility, are marked as deprecated.
> 
> +* ethdev: added a new field:
> +
> +  - modify flex item: ``rte_flow_action_modify_data.flex_handle``
> +
>  * cryptodev: The asymmetric session handling was modified to use a single
>    mempool object. An API ``rte_cryptodev_asym_session_pool_create`` was
> added
>    to create a mempool with element size big enough to hold the generic
> asymmetric
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index b60987db4b..d535722773 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
>  	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
>  	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
>  	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
> +	RTE_FLOW_FIELD_FLEX_ITEM,       /**< Flex item. */
>  };
> 
>  /**
> @@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
>  	RTE_STD_C11
>  	union {
>  		struct {
> -			/** Encapsulation level or tag index. */
> -			uint32_t level;
> +			/**< Encapsulation level or tag index or flex item
> handle. */
> +			union {
> +				uint32_t level;
> +				struct rte_flow_item_flex_handle
> *flex_handle;
> +			};
>  			/** Number of bits to skip from a field. */
>  			uint32_t offset;
>  		};
> --
> 2.27.0

Acked-by: Ori Kam <orika@nvidia.com>
Thanks,
Ori

[PATCH v2 00/11] add flex item support

[Rongwei Liu]

Support flex item matching and modify field in async flow.
Syntax follows sync flow exactly.

Rongwei Liu (11):
  ethdev: add flex item modify field support
  app/testpmd: pass flex handle into matching mask
  net/mlx5: enable hws flex item create
  net/mlx5: add IPv6 protocol as flex item input
  net/mlx5: adopt new flex item prm definition
  net/mlx5/hws: add hws flex item matching support
  net/mlx5: add flex item modify field implementation
  net/mlx5: return error for sws modify field
  app/testpmd: raw encap with flex item support
  doc/mlx5: update mlx5 doc
  app/testpmd: adjust cleanup sequence when quitting

 app/test-pmd/cmdline_flow.c            | 123 +++++++++++++---
 app/test-pmd/testpmd.c                 |   2 +-
 doc/guides/nics/mlx5.rst               |   2 +-
 doc/guides/prog_guide/rte_flow.rst     |  38 ++---
 doc/guides/rel_notes/release_23_03.rst |   4 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  14 +-
 drivers/common/mlx5/mlx5_devx_cmds.h   |   7 +-
 drivers/common/mlx5/mlx5_prm.h         |  29 +++-
 drivers/net/mlx5/hws/mlx5dr_definer.c  |  83 +++++++++++
 drivers/net/mlx5/linux/mlx5_os.c       |  27 ++--
 drivers/net/mlx5/mlx5.c                |  17 ++-
 drivers/net/mlx5/mlx5.h                |   9 +-
 drivers/net/mlx5/mlx5_flow.h           |   4 +
 drivers/net/mlx5/mlx5_flow_dv.c        | 186 ++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_flex.c      | 149 +++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c        |  64 ++++++++-
 lib/ethdev/rte_flow.h                  |   8 +-
 17 files changed, 661 insertions(+), 105 deletions(-)

-- 
2.27.0

[PATCH v2 01/11] ethdev: add flex item modify field support

[Rongwei Liu]

Add flex item as modify field destination.
Add "struct rte_flow_item_flex_handle *flex_handle" into
"struct rte_flow_action_modify_data" as union with existed
"level" member. This new member is dedicated for modifying
flex item.

Add flex item modify field cmdline support. Now user can use
testpmd cli to specify which flex item to be modified, either
source or destination.

Syntax is as below:
modify_field op set dst_type flex_item dst_level 0
dst_offset 16 src_type value src_value 0x123456781020 width 8

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c            | 89 ++++++++++++++++++++++++--
 doc/guides/prog_guide/rte_flow.rst     | 38 ++++++-----
 doc/guides/rel_notes/release_23_03.rst |  4 ++
 lib/ethdev/rte_flow.h                  |  8 ++-
 4 files changed, 113 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 88108498e0..323c07253d 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -601,10 +601,12 @@ enum index {
 	ACTION_MODIFY_FIELD_DST_TYPE,
 	ACTION_MODIFY_FIELD_DST_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_DST_LEVEL,
+	ACTION_MODIFY_FIELD_DST_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_DST_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_TYPE,
 	ACTION_MODIFY_FIELD_SRC_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_SRC_LEVEL,
+	ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_SRC_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_VALUE,
 	ACTION_MODIFY_FIELD_SRC_POINTER,
@@ -807,7 +809,8 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color", NULL
+	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color",
+	"hash_result", "flex_item", NULL
 };
 
 static const char *const meter_colors[] = {
@@ -2282,6 +2285,10 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 				const char *str, unsigned int len, void *buf,
 				unsigned int size);
 static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+				const char *str, unsigned int len, void *buf,
+				unsigned int size);
+static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
 			 const char *str, unsigned int len, void *buf,
 			 unsigned int size);
@@ -5976,11 +5983,15 @@ static const struct token token_list[] = {
 		.name = "dst_level",
 		.help = "destination field level",
 		.next = NEXT(action_modify_field_dst,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					dst.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_DST_LEVEL_VALUE] = {
+		.name = "{dst_level}",
+		.help = "destination field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_DST_OFFSET] = {
 		.name = "dst_offset",
 		.help = "destination field bit offset",
@@ -6007,11 +6018,15 @@ static const struct token token_list[] = {
 		.name = "src_level",
 		.help = "source field level",
 		.next = NEXT(action_modify_field_src,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					src.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE] = {
+		.name = "{src_level}",
+		.help = "source field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_SRC_OFFSET] = {
 		.name = "src_offset",
 		.help = "source field bit offset",
@@ -8477,6 +8492,66 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 	return len;
 }
 
+/** Parse level for modify_field command. */
+static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+			 const char *str, unsigned int len, void *buf,
+			 unsigned int size)
+{
+	struct rte_flow_action_modify_field *action;
+	struct flex_item *fp;
+	uint32_t val;
+	struct buffer *out = buf;
+	char *end;
+
+	(void)token;
+	(void)size;
+	if (ctx->curr != ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		ctx->curr != ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)
+		return -1;
+	if (!ctx->object)
+		return len;
+	action = ctx->object;
+	errno = 0;
+	val = strtoumax(str, &end, 0);
+	if (errno || (size_t)(end - str) != len)
+		return -1;
+	/* No need to validate action template mask value */
+	if (out->args.vc.masks) {
+		if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)
+			action->dst.level = val;
+		else
+			action->src.level = val;
+		return len;
+	}
+	if ((ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		action->dst.field == RTE_FLOW_FIELD_FLEX_ITEM) ||
+		(ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE &&
+		action->src.field == RTE_FLOW_FIELD_FLEX_ITEM)) {
+		if (val >= FLEX_MAX_PARSERS_NUM) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+		fp = flex_items[ctx->port][val];
+		if (!fp) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+	}
+	if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE) {
+		if (action->dst.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->dst.level = val;
+		else
+			action->dst.flex_handle = fp->flex_handle;
+	} else if (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE) {
+		if (action->src.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->src.level = val;
+		else
+			action->src.flex_handle = fp->flex_handle;
+	}
+	return len;
+}
+
 /** Parse the conntrack update, not a rte_flow_action. */
 static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 3e6242803d..e9baf31001 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -2952,23 +2952,27 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx, xxx, xxx}.
 
 .. table:: destination/source field definition
 
-   +---------------+----------------------------------------------------------+
-   | Field         | Value                                                    |
-   +===============+==========================================================+
-   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
-   +---------------+----------------------------------------------------------+
-   | ``level``     | encapsulation level of a packet field or tag array index |
-   +---------------+----------------------------------------------------------+
-   | ``offset``    | number of bits to skip at the beginning                  |
-   +---------------+----------------------------------------------------------+
-   | ``value``     | immediate value buffer (source field only, not           |
-   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
-   | ``pvalue``    | pointer to immediate value data (source field only, not  |
-   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
+   +-----------------+----------------------------------------------------------+
+   | Field           | Value                                                    |
+   +=================+==========================================================+
+   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
+   +-----------------+----------------------------------------------------------+
+   | ``level``       | encapsulation level of a packet field or tag array index |
+   +-----------------+----------------------------------------------------------+
+   | ``flex_handle`` | flex item handle of a packet field                       |
+   +-----------------+----------------------------------------------------------+
+   | ``offset``      | number of bits to skip at the beginning                  |
+   +-----------------+----------------------------------------------------------+
+   | ``value``       | immediate value buffer (source field only, not           |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
+   |                 | field type                                               |
+   |                 | This field is only 16 bytes, maybe not big enough for    |
+   |                 | all NICs' flex item                                      |
+   +-----------------+----------------------------------------------------------+
+   | ``pvalue``      | pointer to immediate value data (source field only, not  |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
+   |                 | field type                                               |
+   +-----------------+----------------------------------------------------------+
 
 Action: ``CONNTRACK``
 ^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index b8c5b68d6c..c673205e5e 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -56,6 +56,10 @@ New Features
      =======================================================
 
 
+* ethdev: added a new field:
+
+  - modify flex item: ``rte_flow_action_modify_data.flex_handle``
+
 Removed Items
 -------------
 
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index b60987db4b..c66a65351d 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
 	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
 	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
+	RTE_FLOW_FIELD_FLEX_ITEM,	/**< Flex item. */
 };
 
 /**
@@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
 	RTE_STD_C11
 	union {
 		struct {
-			/** Encapsulation level or tag index. */
-			uint32_t level;
+			/**< Encapsulation level or tag index or flex item handle. */
+			union {
+				uint32_t level;
+				struct rte_flow_item_flex_handle *flex_handle;
+			};
 			/** Number of bits to skip from a field. */
 			uint32_t offset;
 		};
-- 
2.27.0

Re: [PATCH v2 01/11] ethdev: add flex item modify field support

[Andrew Rybchenko]

On 1/19/23 07:58, Rongwei Liu wrote:
> Add flex item as modify field destination.
> Add "struct rte_flow_item_flex_handle *flex_handle" into
> "struct rte_flow_action_modify_data" as union with existed
> "level" member. This new member is dedicated for modifying
> flex item.
> 
> Add flex item modify field cmdline support. Now user can use
> testpmd cli to specify which flex item to be modified, either
> source or destination.
> 
> Syntax is as below:
> modify_field op set dst_type flex_item dst_level 0
> dst_offset 16 src_type value src_value 0x123456781020 width 8
> 
> Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>

[snip]

> diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
> index b8c5b68d6c..c673205e5e 100644
> --- a/doc/guides/rel_notes/release_23_03.rst
> +++ b/doc/guides/rel_notes/release_23_03.rst
> @@ -56,6 +56,10 @@ New Features
>        =======================================================
>   
>   

It should be just one empty line here

> +* ethdev: added a new field:

"added a new field' is too generic.

> +
> +  - modify flex item: ``rte_flow_action_modify_data.flex_handle``
> +

And two empty lines here.

>   Removed Items
>   -------------
>   
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index b60987db4b..c66a65351d 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
>   	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
>   	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
>   	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
> +	RTE_FLOW_FIELD_FLEX_ITEM,	/**< Flex item. */
>   };
>   
>   /**
> @@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
>   	RTE_STD_C11
>   	union {
>   		struct {
> -			/** Encapsulation level or tag index. */
> -			uint32_t level;
> +			/**< Encapsulation level or tag index or flex item handle. */

Have you tried to generate documentation? If it is a union documentation 
it should be /**, not /**<.
In general, it is better to document union from overall
point of view. What is it logically? Do not define union
as just a union of its fields.

> +			union {
> +				uint32_t level;
> +				struct rte_flow_item_flex_handle *flex_handle;

Union items documentation missing.

> +			};
>   			/** Number of bits to skip from a field. */
>   			uint32_t offset;
>   		};

RE: [PATCH v2 01/11] ethdev: add flex item modify field support

[Rongwei Liu]

Hi Andrew

BR
Rongwei

> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, January 20, 2023 17:08
> To: Rongwei Liu <rongweil@nvidia.com>; Matan Azrad <matan@nvidia.com>;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>;
> NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Aman
> Singh <aman.deep.singh@intel.com>; Yuying Zhang
> <yuying.zhang@intel.com>; Ferruh Yigit <ferruh.yigit@amd.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>
> Subject: Re: [PATCH v2 01/11] ethdev: add flex item modify field support
> 
> External email: Use caution opening links or attachments
> 
> 
> On 1/19/23 07:58, Rongwei Liu wrote:
> > Add flex item as modify field destination.
> > Add "struct rte_flow_item_flex_handle *flex_handle" into "struct
> > rte_flow_action_modify_data" as union with existed "level" member.
> > This new member is dedicated for modifying flex item.
> >
> > Add flex item modify field cmdline support. Now user can use testpmd
> > cli to specify which flex item to be modified, either source or
> > destination.
> >
> > Syntax is as below:
> > modify_field op set dst_type flex_item dst_level 0 dst_offset 16
> > src_type value src_value 0x123456781020 width 8
> >
> > Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
> > Acked-by: Ori Kam <orika@nvidia.com>
> 
> [snip]
> 
> > diff --git a/doc/guides/rel_notes/release_23_03.rst
> > b/doc/guides/rel_notes/release_23_03.rst
> > index b8c5b68d6c..c673205e5e 100644
> > --- a/doc/guides/rel_notes/release_23_03.rst
> > +++ b/doc/guides/rel_notes/release_23_03.rst
> > @@ -56,6 +56,10 @@ New Features
> >        =======================================================
> >
> >
> 
> It should be just one empty line here
> 
Sure.
> > +* ethdev: added a new field:
> 
> "added a new field' is too generic.
> 
> > +
> > +  - modify flex item: ``rte_flow_action_modify_data.flex_handle``
> > +
> 
> And two empty lines here.
> 
Sure.
> >   Removed Items
> >   -------------
> >
> > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h index
> > b60987db4b..c66a65351d 100644
> > --- a/lib/ethdev/rte_flow.h
> > +++ b/lib/ethdev/rte_flow.h
> > @@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
> >       RTE_FLOW_FIELD_IPV6_ECN,        /**< IPv6 ECN. */
> >       RTE_FLOW_FIELD_GTP_PSC_QFI,     /**< GTP QFI. */
> >       RTE_FLOW_FIELD_METER_COLOR,     /**< Meter color marker. */
> > +     RTE_FLOW_FIELD_FLEX_ITEM,       /**< Flex item. */
> >   };
> >
> >   /**
> > @@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
> >       RTE_STD_C11
> >       union {
> >               struct {
> > -                     /** Encapsulation level or tag index. */
> > -                     uint32_t level;
> > +                     /**< Encapsulation level or tag index or flex
> > + item handle. */
> 
> Have you tried to generate documentation? If it is a union documentation it
> should be /**, not /**<.
Sure. Sorry, I followed wrong existed examples.
> In general, it is better to document union from overall point of view. What is it
> logically? Do not define union as just a union of its fields.
Currently, 'flex_handle' is documents in rte_flow.rst file " table:: destination/source field definition" as a new row.  
From API aspect, when modifying flex item, user should specify the pointer of the flex item instead of ID.
That' why it was added as a union. 
> 
> > +                     union {
> > +                             uint32_t level;
> > +                             struct rte_flow_item_flex_handle
> > + *flex_handle;
> 
> Union items documentation missing.
See above. Do we need another place to document the union again?
> 
> > +                     };
> >                       /** Number of bits to skip from a field. */
> >                       uint32_t offset;
> >               };

RE: [PATCH v2 01/11] ethdev: add flex item modify field support

[Rongwei Liu]

Hi Andrew

BR
Rongwei

> -----Original Message-----
> From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
> Sent: Friday, January 20, 2023 17:08
> To: Rongwei Liu <rongweil@nvidia.com>; Matan Azrad <matan@nvidia.com>;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>;
> NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Aman
> Singh <aman.deep.singh@intel.com>; Yuying Zhang
> <yuying.zhang@intel.com>; Ferruh Yigit <ferruh.yigit@amd.com>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>
> Subject: Re: [PATCH v2 01/11] ethdev: add flex item modify field support
> 
> External email: Use caution opening links or attachments
> 
> 
> On 1/19/23 07:58, Rongwei Liu wrote:
> > Add flex item as modify field destination.
> > Add "struct rte_flow_item_flex_handle *flex_handle" into "struct
> > rte_flow_action_modify_data" as union with existed "level" member.
> > This new member is dedicated for modifying flex item.
> >
> > Add flex item modify field cmdline support. Now user can use testpmd
> > cli to specify which flex item to be modified, either source or
> > destination.
> >
> > Syntax is as below:
> > modify_field op set dst_type flex_item dst_level 0 dst_offset 16
> > src_type value src_value 0x123456781020 width 8
> >
> > Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
> > Acked-by: Ori Kam <orika@nvidia.com>
> 
> [snip]
> 
> > diff --git a/doc/guides/rel_notes/release_23_03.rst
> > b/doc/guides/rel_notes/release_23_03.rst
> > index b8c5b68d6c..c673205e5e 100644
> > --- a/doc/guides/rel_notes/release_23_03.rst
> > +++ b/doc/guides/rel_notes/release_23_03.rst
> > @@ -56,6 +56,10 @@ New Features
> >        =======================================================
> >
> >
> 
> It should be just one empty line here
> 
> > +* ethdev: added a new field:
> 
> "added a new field' is too generic.
> 
> > +
> > +  - modify flex item: ``rte_flow_action_modify_data.flex_handle``
> > +
> 
> And two empty lines here.
> 
> >   Removed Items
> >   -------------
> >
> > diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h index
> > b60987db4b..c66a65351d 100644
> > --- a/lib/ethdev/rte_flow.h
> > +++ b/lib/ethdev/rte_flow.h
> > @@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
> >       RTE_FLOW_FIELD_IPV6_ECN,        /**< IPv6 ECN. */
> >       RTE_FLOW_FIELD_GTP_PSC_QFI,     /**< GTP QFI. */
> >       RTE_FLOW_FIELD_METER_COLOR,     /**< Meter color marker. */
> > +     RTE_FLOW_FIELD_FLEX_ITEM,       /**< Flex item. */
> >   };
> >
> >   /**
> > @@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
> >       RTE_STD_C11
> >       union {
> >               struct {
> > -                     /** Encapsulation level or tag index. */
> > -                     uint32_t level;
> > +                     /**< Encapsulation level or tag index or flex
> > + item handle. */
> 
> Have you tried to generate documentation? If it is a union documentation it
> should be /**, not /**<.
> In general, it is better to document union from overall point of view. What is it
> logically? Do not define union as just a union of its fields.
> 
> > +                     union {
> > +                             uint32_t level;
> > +                             struct rte_flow_item_flex_handle
> > + *flex_handle;
> 
> Union items documentation missing.
Added it in the "rte_flow.rst" following ``level`` segment.
> 
> > +                     };
> >                       /** Number of bits to skip from a field. */
> >                       uint32_t offset;
> >               };

[PATCH v3 00/11] add flex item support

[Rongwei Liu]

Support flex item matching and modify field in async flow.
Syntax follows sync flow exactly.

v3: enhance format, add flex_handle document.

Rongwei Liu (11):
  ethdev: add flex item modify field support
  app/testpmd: pass flex handle into matching mask
  net/mlx5: enable hws flex item create
  net/mlx5: add IPv6 protocol as flex item input
  net/mlx5: adopt new flex item prm definition
  net/mlx5/hws: add hws flex item matching support
  net/mlx5: add flex item modify field implementation
  net/mlx5: return error for sws modify field
  app/testpmd: raw encap with flex item support
  doc/mlx5: update mlx5 doc
  app/testpmd: adjust cleanup sequence when quitting

 app/test-pmd/cmdline_flow.c            | 123 +++++++++++++---
 app/test-pmd/testpmd.c                 |   2 +-
 doc/guides/nics/mlx5.rst               |   2 +-
 doc/guides/prog_guide/rte_flow.rst     |  41 +++---
 doc/guides/rel_notes/release_23_03.rst |   4 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  14 +-
 drivers/common/mlx5/mlx5_devx_cmds.h   |   7 +-
 drivers/common/mlx5/mlx5_prm.h         |  29 +++-
 drivers/net/mlx5/hws/mlx5dr_definer.c  |  83 +++++++++++
 drivers/net/mlx5/linux/mlx5_os.c       |  27 ++--
 drivers/net/mlx5/mlx5.c                |  17 ++-
 drivers/net/mlx5/mlx5.h                |   9 +-
 drivers/net/mlx5/mlx5_flow.h           |   4 +
 drivers/net/mlx5/mlx5_flow_dv.c        | 186 ++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_flex.c      | 149 +++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c        |  64 ++++++++-
 lib/ethdev/rte_flow.h                  |   8 +-
 17 files changed, 664 insertions(+), 105 deletions(-)

-- 
2.27.0

[PATCH v3 01/11] ethdev: add flex item modify field support

[Rongwei Liu]

Add flex item as modify field destination.
Add "struct rte_flow_item_flex_handle *flex_handle" into
"struct rte_flow_action_modify_data" as union with existed
"level" member. This new member is dedicated for modifying
flex item.

Add flex item modify field cmdline support. Now user can use
testpmd cli to specify which flex item to be modified, either
source or destination.

Syntax is as below:
modify_field op set dst_type flex_item dst_level 0
dst_offset 16 src_type value src_value 0x123456781020 width 8

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c            | 89 ++++++++++++++++++++++++--
 doc/guides/prog_guide/rte_flow.rst     | 41 +++++++-----
 doc/guides/rel_notes/release_23_03.rst |  4 ++
 lib/ethdev/rte_flow.h                  |  8 ++-
 4 files changed, 116 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 88108498e0..323c07253d 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -601,10 +601,12 @@ enum index {
 	ACTION_MODIFY_FIELD_DST_TYPE,
 	ACTION_MODIFY_FIELD_DST_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_DST_LEVEL,
+	ACTION_MODIFY_FIELD_DST_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_DST_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_TYPE,
 	ACTION_MODIFY_FIELD_SRC_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_SRC_LEVEL,
+	ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_SRC_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_VALUE,
 	ACTION_MODIFY_FIELD_SRC_POINTER,
@@ -807,7 +809,8 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color", NULL
+	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color",
+	"hash_result", "flex_item", NULL
 };
 
 static const char *const meter_colors[] = {
@@ -2282,6 +2285,10 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 				const char *str, unsigned int len, void *buf,
 				unsigned int size);
 static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+				const char *str, unsigned int len, void *buf,
+				unsigned int size);
+static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
 			 const char *str, unsigned int len, void *buf,
 			 unsigned int size);
@@ -5976,11 +5983,15 @@ static const struct token token_list[] = {
 		.name = "dst_level",
 		.help = "destination field level",
 		.next = NEXT(action_modify_field_dst,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					dst.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_DST_LEVEL_VALUE] = {
+		.name = "{dst_level}",
+		.help = "destination field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_DST_OFFSET] = {
 		.name = "dst_offset",
 		.help = "destination field bit offset",
@@ -6007,11 +6018,15 @@ static const struct token token_list[] = {
 		.name = "src_level",
 		.help = "source field level",
 		.next = NEXT(action_modify_field_src,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					src.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE] = {
+		.name = "{src_level}",
+		.help = "source field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_SRC_OFFSET] = {
 		.name = "src_offset",
 		.help = "source field bit offset",
@@ -8477,6 +8492,66 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 	return len;
 }
 
+/** Parse level for modify_field command. */
+static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+			 const char *str, unsigned int len, void *buf,
+			 unsigned int size)
+{
+	struct rte_flow_action_modify_field *action;
+	struct flex_item *fp;
+	uint32_t val;
+	struct buffer *out = buf;
+	char *end;
+
+	(void)token;
+	(void)size;
+	if (ctx->curr != ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		ctx->curr != ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)
+		return -1;
+	if (!ctx->object)
+		return len;
+	action = ctx->object;
+	errno = 0;
+	val = strtoumax(str, &end, 0);
+	if (errno || (size_t)(end - str) != len)
+		return -1;
+	/* No need to validate action template mask value */
+	if (out->args.vc.masks) {
+		if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)
+			action->dst.level = val;
+		else
+			action->src.level = val;
+		return len;
+	}
+	if ((ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		action->dst.field == RTE_FLOW_FIELD_FLEX_ITEM) ||
+		(ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE &&
+		action->src.field == RTE_FLOW_FIELD_FLEX_ITEM)) {
+		if (val >= FLEX_MAX_PARSERS_NUM) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+		fp = flex_items[ctx->port][val];
+		if (!fp) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+	}
+	if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE) {
+		if (action->dst.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->dst.level = val;
+		else
+			action->dst.flex_handle = fp->flex_handle;
+	} else if (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE) {
+		if (action->src.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->src.level = val;
+		else
+			action->src.flex_handle = fp->flex_handle;
+	}
+	return len;
+}
+
 /** Parse the conntrack update, not a rte_flow_action. */
 static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 3e6242803d..9ffd7baa7a 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -2905,6 +2905,9 @@ encapsulation level, from outermost to innermost (lower to higher values).
 For the tag array (in case of multiple tags are supported and present)
 ``level`` translates directly into the array index.
 
+``flex_handle`` is used to specify the flex item pointer which is being
+modified. ``flex_handle`` and ``level`` are mutually exclusive.
+
 ``offset`` specifies the number of bits to skip from a field's start.
 That allows performing a partial copy of the needed part or to divide a big
 packet field into multiple smaller fields. Alternatively, ``offset`` allows
@@ -2952,23 +2955,27 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx, xxx, xxx}.
 
 .. table:: destination/source field definition
 
-   +---------------+----------------------------------------------------------+
-   | Field         | Value                                                    |
-   +===============+==========================================================+
-   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
-   +---------------+----------------------------------------------------------+
-   | ``level``     | encapsulation level of a packet field or tag array index |
-   +---------------+----------------------------------------------------------+
-   | ``offset``    | number of bits to skip at the beginning                  |
-   +---------------+----------------------------------------------------------+
-   | ``value``     | immediate value buffer (source field only, not           |
-   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
-   | ``pvalue``    | pointer to immediate value data (source field only, not  |
-   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
+   +-----------------+----------------------------------------------------------+
+   | Field           | Value                                                    |
+   +=================+==========================================================+
+   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
+   +-----------------+----------------------------------------------------------+
+   | ``level``       | encapsulation level of a packet field or tag array index |
+   +-----------------+----------------------------------------------------------+
+   | ``flex_handle`` | flex item handle of a packet field                       |
+   +-----------------+----------------------------------------------------------+
+   | ``offset``      | number of bits to skip at the beginning                  |
+   +-----------------+----------------------------------------------------------+
+   | ``value``       | immediate value buffer (source field only, not           |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
+   |                 | field type                                               |
+   |                 | This field is only 16 bytes, maybe not big enough for    |
+   |                 | all NICs' flex item                                      |
+   +-----------------+----------------------------------------------------------+
+   | ``pvalue``      | pointer to immediate value data (source field only, not  |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
+   |                 | field type                                               |
+   +-----------------+----------------------------------------------------------+
 
 Action: ``CONNTRACK``
 ^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index b8c5b68d6c..27d3a0e0da 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -55,6 +55,10 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* ethdev: added a new field:
+
+  - modify flex item: ``rte_flow_action_modify_data.flex_handle``.
+
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index b60987db4b..ba9f3d50a3 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
 	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
 	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
+	RTE_FLOW_FIELD_FLEX_ITEM,	/**< Flex item. */
 };
 
 /**
@@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
 	RTE_STD_C11
 	union {
 		struct {
-			/** Encapsulation level or tag index. */
-			uint32_t level;
+			/** Encapsulation level or tag index or flex item handle. */
+			union {
+				uint32_t level;
+				struct rte_flow_item_flex_handle *flex_handle;
+			};
 			/** Number of bits to skip from a field. */
 			uint32_t offset;
 		};
-- 
2.27.0

[PATCH v3 02/11] app/testpmd: pass flex handle into matching mask

[Rongwei Liu]

In async flow create API, there is only mask information when
creating flow table but flex item handle is required to parse
the HW sample information.

Pass the flex item handle instead of UINT64/32_MAX to mask.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 323c07253d..f5d7a67def 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -10068,8 +10068,8 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	}
 	if (offset == offsetof(struct rte_flow_item_flex, handle)) {
 		const struct flex_item *fp;
-		struct rte_flow_item_flex *item_flex = ctx->object;
-		handle = (uint16_t)(uintptr_t)item_flex->handle;
+		spec = ctx->object;
+		handle = (uint16_t)(uintptr_t)spec->handle;
 		if (handle >= FLEX_MAX_PARSERS_NUM) {
 			printf("Bad flex item handle\n");
 			return -1;
@@ -10079,7 +10079,9 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 			printf("Bad flex item handle\n");
 			return -1;
 		}
-		item_flex->handle = fp->flex_handle;
+		spec->handle = fp->flex_handle;
+		mask = spec + 2; /* spec, last, mask */
+		mask->handle = fp->flex_handle;
 	} else if (offset == offsetof(struct rte_flow_item_flex, pattern)) {
 		handle = (uint16_t)(uintptr_t)
 			((struct rte_flow_item_flex *)ctx->object)->pattern;
-- 
2.27.0

[PATCH v3 03/11] net/mlx5: enable hws flex item create

[Rongwei Liu]

Enable flex item create and destroy with dv_flow_en=2

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 27 +++++++++++++++------------
 drivers/net/mlx5/mlx5_flow_hw.c  |  2 ++
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a71474c90a..f5b3edea99 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -474,10 +474,20 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 	err = mlx5_alloc_table_hash_list(priv);
 	if (err)
 		goto error;
-	if (priv->sh->config.dv_flow_en == 2)
-		return 0;
 	/* The resources below are only valid with DV support. */
 #ifdef HAVE_IBV_FLOW_DV_SUPPORT
+	/* Init shared flex parsers list, no need lcore_share */
+	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
+	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
+					       mlx5_flex_parser_create_cb,
+					       mlx5_flex_parser_match_cb,
+					       mlx5_flex_parser_remove_cb,
+					       mlx5_flex_parser_clone_cb,
+					       mlx5_flex_parser_clone_free_cb);
+	if (!sh->flex_parsers_dv)
+		goto error;
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 	/* Init port id action list. */
 	snprintf(s, sizeof(s), "%s_port_id_action_list", sh->ibdev_name);
 	sh->port_id_action_list = mlx5_list_create(s, sh, true,
@@ -518,16 +528,9 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 					      flow_dv_dest_array_clone_free_cb);
 	if (!sh->dest_array_list)
 		goto error;
-	/* Init shared flex parsers list, no need lcore_share */
-	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
-	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
-					       mlx5_flex_parser_create_cb,
-					       mlx5_flex_parser_match_cb,
-					       mlx5_flex_parser_remove_cb,
-					       mlx5_flex_parser_clone_cb,
-					       mlx5_flex_parser_clone_free_cb);
-	if (!sh->flex_parsers_dv)
-		goto error;
+#else
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 #endif
 #ifdef HAVE_MLX5DV_DR
 	void *domain;
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 20c71ff7f0..44953451d5 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -8336,6 +8336,8 @@ const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops = {
 	.query = flow_hw_query,
 	.get_aged_flows = flow_hw_get_aged_flows,
 	.get_q_aged_flows = flow_hw_get_q_aged_flows,
+	.item_create = flow_dv_item_create,
+	.item_release = flow_dv_item_release,
 };
 
 /**
-- 
2.27.0

[PATCH v3 04/11] net/mlx5: add IPv6 protocol as flex item input

[Rongwei Liu]

Support IPv6 protocol as new flex item input link.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_flex.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index fb08910ddb..bec07b13c1 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -1043,6 +1043,22 @@ mlx5_flex_arc_in_udp(const struct rte_flow_item *item,
 	return rte_be_to_cpu_16(spec->hdr.dst_port);
 }
 
+static int
+mlx5_flex_arc_in_ipv6(const struct rte_flow_item *item,
+		      struct rte_flow_error *error)
+{
+	const struct rte_flow_item_ipv6 *spec = item->spec;
+	const struct rte_flow_item_ipv6 *mask = item->mask;
+	struct rte_flow_item_ipv6 ip = { .hdr.proto = 0xff };
+
+	if (memcmp(mask, &ip, sizeof(struct rte_flow_item_ipv6))) {
+		return rte_flow_error_set
+			(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item,
+			 "invalid ipv6 item mask, full mask is desired");
+	}
+	return spec->hdr.proto;
+}
+
 static int
 mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 			   const struct rte_flow_item_flex_conf *conf,
@@ -1089,6 +1105,9 @@ mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 		case RTE_FLOW_ITEM_TYPE_UDP:
 			ret = mlx5_flex_arc_in_udp(rte_item, error);
 			break;
+		case RTE_FLOW_ITEM_TYPE_IPV6:
+			ret = mlx5_flex_arc_in_ipv6(rte_item, error);
+			break;
 		default:
 			MLX5_ASSERT(false);
 			return rte_flow_error_set
-- 
2.27.0

[PATCH v3 05/11] net/mlx5: adopt new flex item prm definition

[Rongwei Liu]

Per newest PRM definition, sample_id stands for 3 parts
of information instead of single uint32_t id: sample_id +
modify_filed_id + format_select_dw.

Also new FW capability bits have been introduces to identify
the new capability.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 14 +++++++++++---
 drivers/common/mlx5/mlx5_devx_cmds.h |  7 ++++++-
 drivers/common/mlx5/mlx5_prm.h       | 28 ++++++++++++++++++++++++++--
 drivers/net/mlx5/mlx5.c              | 15 +++++++++++----
 drivers/net/mlx5/mlx5.h              |  3 ++-
 drivers/net/mlx5/mlx5_flow_flex.c    | 14 +++++++++++---
 6 files changed, 67 insertions(+), 14 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index e3a4927d0f..1f65ea7dcb 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -607,7 +607,8 @@ mlx5_devx_cmd_query_hca_vdpa_attr(void *ctx,
 
 int
 mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				  uint32_t ids[], uint32_t num)
+				  struct mlx5_ext_sample_id ids[],
+				  uint32_t num, uint8_t *anchor)
 {
 	uint32_t in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {0};
 	uint32_t out[MLX5_ST_SZ_DW(create_flex_parser_out)] = {0};
@@ -636,6 +637,7 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			(void *)flex_obj);
 		return -rte_errno;
 	}
+	*anchor = MLX5_GET(parse_graph_flex, flex, head_anchor_id);
 	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
 		void *s_off = (void *)((char *)sample + i *
 			      MLX5_ST_SZ_BYTES(parse_graph_flow_match_sample));
@@ -645,8 +647,8 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			      flow_match_sample_en);
 		if (!en)
 			continue;
-		ids[idx++] = MLX5_GET(parse_graph_flow_match_sample, s_off,
-				  flow_match_sample_field_id);
+		ids[idx++].id = MLX5_GET(parse_graph_flow_match_sample, s_off,
+					 flow_match_sample_field_id);
 	}
 	if (num != idx) {
 		rte_errno = EINVAL;
@@ -794,6 +796,12 @@ mlx5_devx_cmd_query_hca_parse_graph_node_cap
 					 max_num_arc_out);
 	attr->max_num_sample = MLX5_GET(parse_graph_node_cap, hcattr,
 					max_num_sample);
+	attr->anchor_en = MLX5_GET(parse_graph_node_cap, hcattr, anchor_en);
+	attr->ext_sample_id = MLX5_GET(parse_graph_node_cap, hcattr, ext_sample_id);
+	attr->sample_tunnel_inner2 = MLX5_GET(parse_graph_node_cap, hcattr,
+					      sample_tunnel_inner2);
+	attr->zero_size_supported = MLX5_GET(parse_graph_node_cap, hcattr,
+					     zero_size_supported);
 	attr->sample_id_in_out = MLX5_GET(parse_graph_node_cap, hcattr,
 					  sample_id_in_out);
 	attr->max_base_header_length = MLX5_GET(parse_graph_node_cap, hcattr,
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index c94b9eac06..5b33010155 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -114,6 +114,10 @@ struct mlx5_hca_flex_attr {
 	uint8_t  max_num_arc_out;
 	uint8_t  max_num_sample;
 	uint8_t  max_num_prog_sample:5;	/* From HCA CAP 2 */
+	uint8_t  anchor_en:1;
+	uint8_t  ext_sample_id:1;
+	uint8_t  sample_tunnel_inner2:1;
+	uint8_t  zero_size_supported:1;
 	uint8_t  sample_id_in_out:1;
 	uint16_t max_base_header_length;
 	uint8_t  max_sample_base_offset;
@@ -706,7 +710,8 @@ int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir,
 			     struct mlx5_devx_modify_tir_attr *tir_attr);
 __rte_internal
 int mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				      uint32_t ids[], uint32_t num);
+				      struct mlx5_ext_sample_id ids[],
+				      uint32_t num, uint8_t *anchor);
 
 __rte_internal
 struct mlx5_devx_obj *
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 3790dc84b8..ce6cd98fd7 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1893,7 +1893,11 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 max_num_arc_in[0x08];
 	u8 max_num_arc_out[0x08];
 	u8 max_num_sample[0x08];
-	u8 reserved_at_78[0x07];
+	u8 reserved_at_78[0x03];
+	u8 anchor_en[0x1];
+	u8 ext_sample_id[0x1];
+	u8 sample_tunnel_inner2[0x1];
+	u8 zero_size_supported[0x1];
 	u8 sample_id_in_out[0x1];
 	u8 max_base_header_length[0x10];
 	u8 reserved_at_90[0x08];
@@ -1903,6 +1907,24 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 header_length_mask_width[0x08];
 };
 
+/* ext_sample_id structure, see PRM Table: Flow Match Sample ID Format. */
+struct mlx5_ext_sample_id {
+	union {
+		struct {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+			uint32_t format_select_dw:8;
+			uint32_t modify_field_id:12;
+			uint32_t sample_id:12;
+#else
+			uint32_t sample_id:12;
+			uint32_t modify_field_id:12;
+			uint32_t format_select_dw:8;
+#endif
+		};
+		uint32_t id;
+	};
+};
+
 struct mlx5_ifc_flow_table_prop_layout_bits {
 	u8 ft_support[0x1];
 	u8 flow_tag[0x1];
@@ -4522,7 +4544,9 @@ struct mlx5_ifc_parse_graph_flex_bits {
 	u8 header_length_mode[0x4];
 	u8 header_length_field_offset[0x10];
 	u8 next_header_field_offset[0x10];
-	u8 reserved_at_160[0x1b];
+	u8 reserved_at_160[0x12];
+	u8 head_anchor_id[0x6];
+	u8 reserved_at_178[0x3];
 	u8 next_header_field_size[0x5];
 	u8 header_length_field_mask[0x20];
 	u8 reserved_at_224[0x20];
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index b8643cebdd..0b97c4e78d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -964,11 +964,13 @@ int
 mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
 	struct mlx5_devx_graph_node_attr node = {
 		.modify_field_select = 0,
 	};
-	uint32_t ids[8];
+	struct mlx5_ext_sample_id ids[8];
+	uint8_t anchor_id;
 	int ret;
 
 	if (!priv->sh->cdev->config.hca_attr.parse_graph_flex_node) {
@@ -1004,15 +1006,20 @@ mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->num = 2;
-	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num);
+	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num, &anchor_id);
 	if (ret) {
 		DRV_LOG(ERR, "Failed to query sample IDs.");
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->offset[0] = 0x0;
 	prf->offset[1] = sizeof(uint32_t);
-	prf->ids[0] = ids[0];
-	prf->ids[1] = ids[1];
+	if (attr->ext_sample_id) {
+		prf->ids[0] = ids[0].sample_id;
+		prf->ids[1] = ids[1].sample_id;
+	} else {
+		prf->ids[0] = ids[0].id;
+		prf->ids[1] = ids[1].id;
+	}
 	return 0;
 }
 
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 16b33e1548..83fb316ad8 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1307,9 +1307,10 @@ struct mlx5_lag {
 struct mlx5_flex_parser_devx {
 	struct mlx5_list_entry entry;  /* List element at the beginning. */
 	uint32_t num_samples;
+	uint8_t anchor_id;
 	void *devx_obj;
 	struct mlx5_devx_graph_node_attr devx_conf;
-	uint32_t sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
+	struct mlx5_ext_sample_id sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
 };
 
 /* Pattern field descriptor - how to translate flex pattern into samples. */
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index bec07b13c1..24b7226ee6 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -226,15 +226,18 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	void *misc4_m = MLX5_ADDR_OF(fte_match_param, matcher,
 				     misc_parameters_4);
 	void *misc4_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_4);
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_flex_item *tp;
 	uint32_t i, pos = 0;
+	uint32_t sample_id;
 
 	RTE_SET_USED(dev);
 	MLX5_ASSERT(item->spec && item->mask);
 	spec = item->spec;
 	mask = item->mask;
 	tp = (struct mlx5_flex_item *)spec->handle;
-	MLX5_ASSERT(mlx5_flex_index(dev->data->dev_private, tp) >= 0);
+	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
 		uint32_t id = map->reg_id;
@@ -257,9 +260,13 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 			MLX5_ASSERT(id < num_samples);
 			id += num_samples;
 		}
+		if (attr->ext_sample_id)
+			sample_id = tp->devx_fp->sample_ids[id].sample_id;
+		else
+			sample_id = tp->devx_fp->sample_ids[id].id;
 		mlx5_flex_set_match_sample(misc4_m, misc4_v,
 					   def, msk & def, val & msk & def,
-					   tp->devx_fp->sample_ids[id], id);
+					   sample_id, id);
 		pos += map->width;
 	}
 }
@@ -1317,7 +1324,8 @@ mlx5_flex_parser_create_cb(void *list_ctx, void *ctx)
 	/* Query the firmware assigned sample ids. */
 	ret = mlx5_devx_cmd_query_parse_samples(fp->devx_obj,
 						fp->sample_ids,
-						fp->num_samples);
+						fp->num_samples,
+						&fp->anchor_id);
 	if (ret)
 		goto error;
 	DRV_LOG(DEBUG, "DEVx flex parser %p created, samples num: %u",
-- 
2.27.0

[PATCH v3 06/11] net/mlx5/hws: add hws flex item matching support

[Rongwei Liu]

Support flex item matching in hws and syntax follows
sws exactly.

Flex item should be created in advance and follow current
json mapping logic.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c |  83 ++++++++++++++++++
 drivers/net/mlx5/mlx5.c               |   2 +-
 drivers/net/mlx5/mlx5.h               |   6 ++
 drivers/net/mlx5/mlx5_flow.h          |   1 +
 drivers/net/mlx5/mlx5_flow_dv.c       |   2 +-
 drivers/net/mlx5/mlx5_flow_flex.c     | 116 ++++++++++++++++++++++----
 drivers/net/mlx5/mlx5_flow_hw.c       |  48 ++++++++++-
 7 files changed, 239 insertions(+), 19 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 6b98eb8c96..a6378afb10 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -293,6 +293,43 @@ mlx5dr_definer_integrity_set(struct mlx5dr_definer_fc *fc,
 	DR_SET(tag, ok1_bits, fc->byte_off, fc->bit_off, fc->bit_mask);
 }
 
+static void
+mlx5dr_definer_flex_parser_set(struct mlx5dr_definer_fc *fc,
+			       const void *item,
+			       uint8_t *tag, bool is_inner)
+{
+	const struct rte_flow_item_flex *flex = item;
+	uint32_t byte_off, val, idx;
+	int ret;
+
+	val = 0;
+	byte_off = MLX5_BYTE_OFF(definer_hl, flex_parser.flex_parser_0);
+	idx = fc->fname - MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+	byte_off -= idx * sizeof(uint32_t);
+	ret = mlx5_flex_get_parser_value_per_byte_off(flex, flex->handle, byte_off,
+						      false, is_inner, &val);
+	if (ret == -1 || !val)
+		return;
+
+	DR_SET(tag, val, fc->byte_off, 0, fc->bit_mask);
+}
+
+static void
+mlx5dr_definer_flex_parser_inner_set(struct mlx5dr_definer_fc *fc,
+				     const void *item,
+				     uint8_t *tag)
+{
+	mlx5dr_definer_flex_parser_set(fc, item, tag, true);
+}
+
+static void
+mlx5dr_definer_flex_parser_outer_set(struct mlx5dr_definer_fc *fc,
+				     const void *item,
+				     uint8_t *tag)
+{
+	mlx5dr_definer_flex_parser_set(fc, item, tag, false);
+}
+
 static void
 mlx5dr_definer_gre_key_set(struct mlx5dr_definer_fc *fc,
 			   const void *item_spec,
@@ -1465,6 +1502,47 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_flex_parser(struct mlx5dr_definer_conv_data *cd,
+				     struct rte_flow_item *item,
+				     int item_idx)
+{
+	uint32_t base_off = MLX5_BYTE_OFF(definer_hl, flex_parser.flex_parser_0);
+	const struct rte_flow_item_flex *v, *m;
+	enum mlx5dr_definer_fname fname;
+	struct mlx5dr_definer_fc *fc;
+	uint32_t i, mask, byte_off;
+	bool is_inner = cd->tunnel;
+	int ret;
+
+	m = item->mask;
+	v = item->spec;
+	mask = 0;
+	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
+		byte_off = base_off - i * sizeof(uint32_t);
+		ret = mlx5_flex_get_parser_value_per_byte_off(m, v->handle, byte_off,
+							      true, is_inner, &mask);
+		if (ret == -1) {
+			rte_errno = EINVAL;
+			return rte_errno;
+		}
+
+		if (!mask)
+			continue;
+
+		fname = MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+		fname += (enum mlx5dr_definer_fname)i;
+		fc = &cd->fc[fname];
+		fc->byte_off = byte_off;
+		fc->item_idx = item_idx;
+		fc->tag_set = cd->tunnel ? &mlx5dr_definer_flex_parser_inner_set :
+					   &mlx5dr_definer_flex_parser_outer_set;
+		fc->tag_mask_set = &mlx5dr_definer_ones_set;
+		fc->bit_mask = mask;
+	}
+	return 0;
+}
+
 static int
 mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 				struct mlx5dr_match_template *mt,
@@ -1581,6 +1659,11 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			ret = mlx5dr_definer_conv_item_meter_color(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_METER_COLOR;
 			break;
+		case RTE_FLOW_ITEM_TYPE_FLEX:
+			ret = mlx5dr_definer_conv_item_flex_parser(&cd, items, i);
+			item_flags |= cd.tunnel ? MLX5_FLOW_ITEM_INNER_FLEX :
+						  MLX5_FLOW_ITEM_OUTER_FLEX;
+			break;
 		default:
 			DR_LOG(ERR, "Unsupported item type %d", items->type);
 			rte_errno = ENOTSUP;
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 0b97c4e78d..8297129bd1 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1034,7 +1034,7 @@ static void
 mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
+	struct mlx5_ecpri_parser_profile *prf = &priv->sh->ecpri_parser;
 
 	if (prf->obj)
 		mlx5_devx_cmd_destroy(prf->obj);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 83fb316ad8..2cc3959d01 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -2243,6 +2243,12 @@ void mlx5_flex_item_port_cleanup(struct rte_eth_dev *dev);
 void mlx5_flex_flow_translate_item(struct rte_eth_dev *dev, void *matcher,
 				   void *key, const struct rte_flow_item *item,
 				   bool is_inner);
+int mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			    uint32_t idx, uint32_t *pos,
+			    bool is_inner, uint32_t *def);
+int mlx5_flex_get_parser_value_per_byte_off(const struct rte_flow_item_flex *item,
+					    void *flex, uint32_t byte_off,
+					    bool is_mask, bool tunnel, uint32_t *value);
 int mlx5_flex_acquire_index(struct rte_eth_dev *dev,
 			    struct rte_flow_item_flex_handle *handle,
 			    bool acquire);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e376dcae93..c8761c4e5a 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1225,6 +1225,7 @@ struct rte_flow_pattern_template {
 	 * tag pattern item for representor matching.
 	 */
 	bool implicit_tag;
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Flow action template struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 7ca909999b..284f18da11 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10588,7 +10588,7 @@ flow_dv_translate_item_flex(struct rte_eth_dev *dev, void *matcher, void *key,
 		(const struct rte_flow_item_flex *)item->spec;
 	int index = mlx5_flex_acquire_index(dev, spec->handle, false);
 
-	MLX5_ASSERT(index >= 0 && index <= (int)(sizeof(uint32_t) * CHAR_BIT));
+	MLX5_ASSERT(index >= 0 && index < (int)(sizeof(uint32_t) * CHAR_BIT));
 	if (index < 0)
 		return;
 	if (!(dev_flow->handle->flex_item & RTE_BIT32(index))) {
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index 24b7226ee6..aa317fc958 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -198,6 +198,99 @@ mlx5_flex_set_match_sample(void *misc4_m, void *misc4_v,
 	}
 #undef SET_FP_MATCH_SAMPLE_ID
 }
+
+/**
+ * Get the flex parser sample id and corresponding mask
+ * per shift and width information.
+ *
+ * @param[in] tp
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] idx
+ *   Mapping index.
+ * @param[in, out] pos
+ *   Where to search the value and mask.
+ * @param[in] is_inner
+ *   For inner matching or not.
+ * @param[in, def] def
+ *   Mask generated by mapping shift and width.
+ *
+ * @return
+ *   0 on success, -1 to ignore.
+ */
+int
+mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			uint32_t idx, uint32_t *pos,
+			bool is_inner, uint32_t *def)
+{
+	const struct mlx5_flex_pattern_field *map = tp->map + idx;
+	uint32_t id = map->reg_id;
+
+	*def = (RTE_BIT64(map->width) - 1) << map->shift;
+	/* Skip placeholders for DUMMY fields. */
+	if (id == MLX5_INVALID_SAMPLE_REG_ID) {
+		*pos += map->width;
+		return -1;
+	}
+	MLX5_ASSERT(map->width);
+	MLX5_ASSERT(id < tp->devx_fp->num_samples);
+	if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
+		uint32_t num_samples = tp->devx_fp->num_samples / 2;
+
+		MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
+		MLX5_ASSERT(id < num_samples);
+		id += num_samples;
+	}
+	return id;
+}
+
+/**
+ * Get the flex parser mapping value per definer format_select_dw.
+ *
+ * @param[in] item
+ *   Rte flex item pointer.
+ * @param[in] flex
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] byte_off
+ *   Mlx5 flex item format_select_dw.
+ * @param[in] is_mask
+ *   Spec or mask.
+ * @param[in] tunnel
+ *   Tunnel mode or not.
+ * @param[in, def] value
+ *   Value calculated for this flex parser, either spec or mask.
+ *
+ * @return
+ *   0 on success, -1 for error.
+ */
+int
+mlx5_flex_get_parser_value_per_byte_off(const struct rte_flow_item_flex *item,
+					void *flex, uint32_t byte_off,
+					bool is_mask, bool tunnel, uint32_t *value)
+{
+	struct mlx5_flex_pattern_field *map;
+	struct mlx5_flex_item *tp = flex;
+	uint32_t def, i, pos, val;
+	int id;
+
+	*value = 0;
+	for (i = 0, pos = 0; i < tp->mapnum && pos < item->length * CHAR_BIT; i++) {
+		map = tp->map + i;
+		id = mlx5_flex_get_sample_id(tp, i, &pos, tunnel, &def);
+		if (id == -1)
+			continue;
+		if (id >= (int)tp->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return -1;
+		if (byte_off == tp->devx_fp->sample_ids[id].format_select_dw * sizeof(uint32_t)) {
+			val = mlx5_flex_get_bitfield(item, pos, map->width, map->shift);
+			if (is_mask)
+				val &= RTE_BE32(def);
+			*value |= val;
+		}
+		pos += map->width;
+	}
+	return 0;
+}
+
 /**
  * Translate item pattern into matcher fields according to translation
  * array.
@@ -240,26 +333,17 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
-		uint32_t id = map->reg_id;
-		uint32_t def = (RTE_BIT64(map->width) - 1) << map->shift;
-		uint32_t val, msk;
+		uint32_t val, msk, def;
+		int id = mlx5_flex_get_sample_id(tp, i, &pos, is_inner, &def);
 
-		/* Skip placeholders for DUMMY fields. */
-		if (id == MLX5_INVALID_SAMPLE_REG_ID) {
-			pos += map->width;
+		if (id == -1)
 			continue;
-		}
+		MLX5_ASSERT(id < (int)tp->devx_fp->num_samples);
+		if (id >= (int)tp->devx_fp->num_samples ||
+		    id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
 		val = mlx5_flex_get_bitfield(spec, pos, map->width, map->shift);
 		msk = mlx5_flex_get_bitfield(mask, pos, map->width, map->shift);
-		MLX5_ASSERT(map->width);
-		MLX5_ASSERT(id < tp->devx_fp->num_samples);
-		if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
-			uint32_t num_samples = tp->devx_fp->num_samples / 2;
-
-			MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
-			MLX5_ASSERT(id < num_samples);
-			id += num_samples;
-		}
 		if (attr->ext_sample_id)
 			sample_id = tp->devx_fp->sample_ids[id].sample_id;
 		else
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 44953451d5..742b0b0d43 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4331,6 +4331,36 @@ flow_hw_set_vlan_vid_construct(struct rte_eth_dev *dev,
 					      &modify_action);
 }
 
+static int
+flow_hw_flex_item_acquire(struct rte_eth_dev *dev,
+			  struct rte_flow_item_flex_handle *handle,
+			  uint8_t *flex_item)
+{
+	int index = mlx5_flex_acquire_index(dev, handle, false);
+
+	MLX5_ASSERT(index >= 0 && index < (int)(sizeof(uint32_t) * CHAR_BIT));
+	if (index < 0)
+		return -1;
+	if (!(*flex_item & RTE_BIT32(index))) {
+		/* Don't count same flex item again. */
+		if (mlx5_flex_acquire_index(dev, handle, true) != index)
+			MLX5_ASSERT(false);
+		*flex_item |= (uint8_t)RTE_BIT32(index);
+	}
+	return 0;
+}
+
+static void
+flow_hw_flex_item_release(struct rte_eth_dev *dev, uint8_t *flex_item)
+{
+	while (*flex_item) {
+		int index = rte_bsf32(*flex_item);
+
+		mlx5_flex_release_index(dev, index);
+		*flex_item &= ~(uint8_t)RTE_BIT32(index);
+	}
+}
+
 /**
  * Create flow action template.
  *
@@ -4732,6 +4762,7 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_ICMP:
 		case RTE_FLOW_ITEM_TYPE_ICMP6:
 		case RTE_FLOW_ITEM_TYPE_CONNTRACK:
+		case RTE_FLOW_ITEM_TYPE_FLEX:
 			break;
 		case RTE_FLOW_ITEM_TYPE_INTEGRITY:
 			/*
@@ -4809,6 +4840,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 		.mask = &tag_m,
 		.last = NULL
 	};
+	unsigned int i = 0;
 
 	if (flow_hw_pattern_validate(dev, attr, items, error))
 		return NULL;
@@ -4868,6 +4900,19 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 			it->implicit_tag = true;
 		mlx5_free(copied_items);
 	}
+	for (i = 0; items[i].type != RTE_FLOW_ITEM_TYPE_END; ++i) {
+		if (items[i].type == RTE_FLOW_ITEM_TYPE_FLEX) {
+			const struct rte_flow_item_flex *spec =
+				(const struct rte_flow_item_flex *)items[i].spec;
+			struct rte_flow_item_flex_handle *handle = spec->handle;
+
+			if (flow_hw_flex_item_acquire(dev, handle, &it->flex_item)) {
+				claim_zero(mlx5dr_match_template_destroy(it->mt));
+				mlx5_free(it);
+				return NULL;
+			}
+		}
+	}
 	__atomic_fetch_add(&it->refcnt, 1, __ATOMIC_RELAXED);
 	LIST_INSERT_HEAD(&priv->flow_hw_itt, it, next);
 	return it;
@@ -4887,7 +4932,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 			      struct rte_flow_pattern_template *template,
 			      struct rte_flow_error *error __rte_unused)
 {
@@ -4900,6 +4945,7 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "item template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	claim_zero(mlx5dr_match_template_destroy(template->mt));
 	mlx5_free(template);
 	return 0;
-- 
2.27.0

[PATCH v3 07/11] net/mlx5: add flex item modify field implementation

[Rongwei Liu]

Add flex item modify field HWS implementation.
The minimum modify boundary is one byte.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h  |   1 +
 drivers/net/mlx5/mlx5_flow.h    |   3 +
 drivers/net/mlx5/mlx5_flow_dv.c | 165 +++++++++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c |  14 ++-
 4 files changed, 170 insertions(+), 13 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index ce6cd98fd7..0c2a516e9d 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -759,6 +759,7 @@ enum mlx5_modification_field {
 	MLX5_MODI_OUT_IP_ECN = 0x73,
 	MLX5_MODI_TUNNEL_HDR_DW_1 = 0x75,
 	MLX5_MODI_GTPU_FIRST_EXT_DW_0 = 0x76,
+	MLX5_MODI_INVALID = INT_MAX,
 };
 
 /* Total number of metadata reg_c's. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index c8761c4e5a..c71fa1c0ad 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1080,6 +1080,8 @@ struct field_modify_info {
 	uint32_t size; /* Size of field in protocol header, in bytes. */
 	uint32_t offset; /* Offset of field in protocol header, in bytes. */
 	enum mlx5_modification_field id;
+	uint32_t shift;
+	uint8_t is_flex; /* Temporary indicator for flex item modify filed WA. */
 };
 
 /* HW steering flow attributes. */
@@ -1244,6 +1246,7 @@ struct rte_flow_actions_template {
 	uint16_t mhdr_off; /* Offset of DR modify header action. */
 	uint32_t refcnt; /* Reference counter. */
 	uint16_t rx_cpy_pos; /* Action position of Rx metadata to be copied. */
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Jump action struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 284f18da11..92a5914d4b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -414,10 +414,15 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			++field;
 			continue;
 		}
-		/* Deduce actual data width in bits from mask value. */
-		off_b = rte_bsf32(mask) + carry_b;
-		size_b = sizeof(uint32_t) * CHAR_BIT -
-			 off_b - __builtin_clz(mask);
+		if (type == MLX5_MODIFICATION_TYPE_COPY && field->is_flex) {
+			off_b = 32 - field->shift + carry_b - field->size * CHAR_BIT;
+			size_b = field->size * CHAR_BIT - carry_b;
+		} else {
+			/* Deduce actual data width in bits from mask value. */
+			off_b = rte_bsf32(mask) + carry_b;
+			size_b = sizeof(uint32_t) * CHAR_BIT -
+				 off_b - __builtin_clz(mask);
+		}
 		MLX5_ASSERT(size_b);
 		actions[i] = (struct mlx5_modification_cmd) {
 			.action_type = type,
@@ -437,40 +442,46 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			 * Destination field overflow. Copy leftovers of
 			 * a source field to the next destination field.
 			 */
-			carry_b = 0;
 			if ((size_b > dcopy->size * CHAR_BIT - dcopy->offset) &&
 			    dcopy->size != 0) {
 				actions[i].length =
 					dcopy->size * CHAR_BIT - dcopy->offset;
-				carry_b = actions[i].length;
+				carry_b += actions[i].length;
 				next_field = false;
+			} else {
+				carry_b = 0;
 			}
 			/*
 			 * Not enough bits in a source filed to fill a
 			 * destination field. Switch to the next source.
 			 */
 			if ((size_b < dcopy->size * CHAR_BIT - dcopy->offset) &&
-			    (size_b == field->size * CHAR_BIT - off_b)) {
-				actions[i].length =
-					field->size * CHAR_BIT - off_b;
+			    ((size_b == field->size * CHAR_BIT - off_b) ||
+			     field->is_flex)) {
+				actions[i].length = size_b;
 				dcopy->offset += actions[i].length;
 				next_dcopy = false;
 			}
-			if (next_dcopy)
-				++dcopy;
 		} else {
 			MLX5_ASSERT(item->spec);
 			data = flow_dv_fetch_field((const uint8_t *)item->spec +
 						   field->offset, field->size);
 			/* Shift out the trailing masked bits from data. */
 			data = (data & mask) >> off_b;
+			if (field->is_flex)
+				actions[i].offset = 32 - field->shift - field->size * CHAR_BIT;
 			actions[i].data1 = rte_cpu_to_be_32(data);
 		}
 		/* Convert entire record to expected big-endian format. */
 		actions[i].data0 = rte_cpu_to_be_32(actions[i].data0);
+		if ((type != MLX5_MODIFICATION_TYPE_COPY ||
+		     dcopy->id != (enum mlx5_modification_field)UINT32_MAX) &&
+		    field->id != (enum mlx5_modification_field)UINT32_MAX)
+			++i;
+		if (next_dcopy && type == MLX5_MODIFICATION_TYPE_COPY)
+			++dcopy;
 		if (next_field)
 			++field;
-		++i;
 	} while (field->size);
 	if (resource->actions_num == i)
 		return rte_flow_error_set(error, EINVAL,
@@ -1421,6 +1432,131 @@ flow_modify_info_mask_32_masked(uint32_t length, uint32_t off, uint32_t post_mas
 	return rte_cpu_to_be_32(mask & post_mask);
 }
 
+static void
+mlx5_modify_flex_item(const struct rte_eth_dev *dev,
+		      const struct mlx5_flex_item *flex,
+		      const struct rte_flow_action_modify_data *data,
+		      struct field_modify_info *info,
+		      uint32_t *mask, uint32_t width)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
+	uint32_t i, j;
+	int id = 0;
+	uint32_t pos = 0;
+	const struct mlx5_flex_pattern_field *map;
+	uint32_t offset = data->offset;
+	uint32_t width_left = width;
+	uint32_t def;
+	uint32_t cur_width = 0;
+	uint32_t tmp_ofs;
+	uint32_t idx = 0;
+	struct field_modify_info tmp;
+	int tmp_id;
+
+	if (!attr->ext_sample_id) {
+		DRV_LOG(ERR, "FW doesn't support modify field with flex item.");
+		return;
+	}
+	/*
+	 * search for the mapping instance until Accumulated width is no
+	 * less than data->offset.
+	 */
+	for (i = 0; i < flex->mapnum; i++) {
+		if (flex->map[i].width + pos > data->offset)
+			break;
+		pos += flex->map[i].width;
+	}
+	if (i >= flex->mapnum)
+		return;
+	tmp_ofs = pos < data->offset ? data->offset - pos : 0;
+	for (j = i; i < flex->mapnum && width_left > 0; ) {
+		map = flex->map + i;
+		id = mlx5_flex_get_sample_id(flex, i, &pos, false, &def);
+		if (id == -1) {
+			i++;
+			/* All left length is dummy */
+			if (pos >= data->offset + width)
+				return;
+			cur_width = map->width;
+		/* One mapping instance covers the whole width. */
+		} else if (pos + map->width >= (data->offset + width)) {
+			cur_width = width_left;
+		} else {
+			cur_width = cur_width + map->width - tmp_ofs;
+			pos += map->width;
+			/*
+			 * Continue to search next until:
+			 * 1. Another flex parser ID.
+			 * 2. Width has been covered.
+			 */
+			for (j = i + 1; j < flex->mapnum; j++) {
+				tmp_id = mlx5_flex_get_sample_id(flex, j, &pos, false, &def);
+				if (tmp_id == -1) {
+					i = j;
+					pos -= flex->map[j].width;
+					break;
+				}
+				if (id >= (int)flex->devx_fp->num_samples ||
+				    id >= MLX5_GRAPH_NODE_SAMPLE_NUM ||
+				    tmp_id >= (int)flex->devx_fp->num_samples ||
+				    tmp_id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+					return;
+				if (flex->devx_fp->sample_ids[id].id !=
+						flex->devx_fp->sample_ids[tmp_id].id ||
+				    flex->map[j].shift != flex->map[j - 1].width +
+							  flex->map[j - 1].shift) {
+					i = j;
+					break;
+				}
+				if ((pos + flex->map[j].width) >= (data->offset + width)) {
+					cur_width = width_left;
+					break;
+				}
+				pos += flex->map[j].width;
+				cur_width += flex->map[j].width;
+			}
+		}
+		if (cur_width > width_left)
+			cur_width = width_left;
+		else if (cur_width < width_left && (j == flex->mapnum || i == flex->mapnum))
+			return;
+
+		MLX5_ASSERT(id < (int)flex->devx_fp->num_samples);
+		if (id >= (int)flex->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
+		/* Use invalid entry as placeholder for DUMMY mapping. */
+		info[idx] = (struct field_modify_info){cur_width / CHAR_BIT, offset / CHAR_BIT,
+			     id == -1 ? MLX5_MODI_INVALID :
+			     (enum mlx5_modification_field)
+			     flex->devx_fp->sample_ids[id].modify_field_id,
+			     map->shift + tmp_ofs, 1};
+		offset += cur_width;
+		width_left -= cur_width;
+		if (!mask) {
+			info[idx].offset = (32 - cur_width - map->shift - tmp_ofs);
+			info[idx].size = cur_width / CHAR_BIT + info[idx].offset / CHAR_BIT;
+		}
+		cur_width = 0;
+		tmp_ofs = 0;
+		idx++;
+	}
+	if (unlikely(width_left > 0)) {
+		MLX5_ASSERT(false);
+		return;
+	}
+	if (mask)
+		memset(mask, 0xff, data->offset / CHAR_BIT + width / CHAR_BIT);
+	/* Re-order the info to follow IPv6 address. */
+	for (i = 0; i < idx / 2; i++) {
+		tmp = info[i];
+		MLX5_ASSERT(info[i].id);
+		MLX5_ASSERT(info[idx - 1 - i].id);
+		info[i] = info[idx - 1 - i];
+		info[idx - 1 - i] = tmp;
+	}
+}
+
 void
 mlx5_flow_field_id_to_modify_info
 		(const struct rte_flow_action_modify_data *data,
@@ -1883,6 +2019,11 @@ mlx5_flow_field_id_to_modify_info
 				info[idx].offset = data->offset;
 		}
 		break;
+	case RTE_FLOW_FIELD_FLEX_ITEM:
+		MLX5_ASSERT(data->flex_handle != NULL && !(data->offset & 0x7));
+		mlx5_modify_flex_item(dev, (const struct mlx5_flex_item *)data->flex_handle,
+				      data, info, mask, width);
+		break;
 	case RTE_FLOW_FIELD_POINTER:
 	case RTE_FLOW_FIELD_VALUE:
 	default:
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 742b0b0d43..2e263880d7 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4550,6 +4550,17 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
 			at->actions[i].conf = actions->conf;
 			at->masks[i].conf = masks->conf;
 		}
+		if (actions->type == RTE_FLOW_ACTION_TYPE_MODIFY_FIELD) {
+			const struct rte_flow_action_modify_field *info = actions->conf;
+
+			if ((info->dst.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			     flow_hw_flex_item_acquire(dev, info->dst.flex_handle,
+						       &at->flex_item)) ||
+			     (info->src.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			      flow_hw_flex_item_acquire(dev, info->src.flex_handle,
+							&at->flex_item)))
+				goto error;
+		}
 	}
 	at->tmpl = flow_hw_dr_actions_template_create(at);
 	if (!at->tmpl)
@@ -4581,7 +4592,7 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_actions_template_destroy(struct rte_eth_dev *dev,
 				 struct rte_flow_actions_template *template,
 				 struct rte_flow_error *error __rte_unused)
 {
@@ -4594,6 +4605,7 @@ flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "action template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	if (template->tmpl)
 		mlx5dr_action_template_destroy(template->tmpl);
 	mlx5_free(template);
-- 
2.27.0

[PATCH v3 08/11] net/mlx5: return error for sws modify field

[Rongwei Liu]

Return unsupported error message when application tries to
modify flex item field.

Validation of packet modifications actions for SW Steering checked
if either source or destination field of MODIFY_FIELD action
was a flex item.
When DEC_TTL action is used, DEC_TTL action does not have any
action configuration and dereferencing source or destination field
is invalid, so validation of source and destination field types
should be moved to MODIFY_FIELD specific validation function, then
field types are validated if and only if action type is MODIFY_FIELD.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 92a5914d4b..a7c0d5bf17 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4828,6 +4828,7 @@ flow_dv_validate_action_modify_hdr(const uint64_t action_flags,
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION_CONF,
 					  NULL, "action configuration not set");
+
 	if (action_flags & MLX5_FLOW_ACTION_ENCAP)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION, NULL,
@@ -5153,17 +5154,21 @@ flow_dv_validate_action_modify_field(struct rte_eth_dev *dev,
 	struct mlx5_hca_attr *hca_attr = &priv->sh->cdev->config.hca_attr;
 	const struct rte_flow_action_modify_field *action_modify_field =
 		action->conf;
-	uint32_t dst_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->dst.field,
-				-1, attr, error);
-	uint32_t src_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->src.field,
-				dst_width, attr, error);
+	uint32_t dst_width, src_width;
 
 	ret = flow_dv_validate_action_modify_hdr(action_flags, action, error);
 	if (ret)
 		return ret;
-
+	if (action_modify_field->src.field == RTE_FLOW_FIELD_FLEX_ITEM ||
+	    action_modify_field->dst.field == RTE_FLOW_FIELD_FLEX_ITEM)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"flex item fields modification"
+				" is not supported");
+	dst_width = mlx5_flow_item_field_width(dev, action_modify_field->dst.field,
+					       -1, attr, error);
+	src_width = mlx5_flow_item_field_width(dev, action_modify_field->src.field,
+					       dst_width, attr, error);
 	if (action_modify_field->width == 0)
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
-- 
2.27.0

[PATCH v3 09/11] app/testpmd: raw encap with flex item support

[Rongwei Liu]

Application should retrieve raw_encap buffer from
spec->pattern if it is flex item.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index f5d7a67def..50c8ec5594 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -11193,6 +11193,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 	uint16_t proto = 0;
 	uint16_t idx = in->port; /* We borrow port field as index */
 	int gtp_psc = -1; /* GTP PSC option index. */
+	const void *src_spec;
 
 	if (in->command == SET_SAMPLE_ACTIONS)
 		return cmd_set_raw_parsed_sample(in);
@@ -11216,6 +11217,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 		item = in->args.vc.pattern + i;
 		if (item->spec == NULL)
 			item->spec = flow_item_default_mask(item);
+		src_spec = item->spec;
 		switch (item->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			size = sizeof(struct rte_ether_hdr);
@@ -11343,9 +11345,13 @@ cmd_set_raw_parsed(const struct buffer *in)
 			size = sizeof(struct rte_flow_item_pfcp);
 			break;
 		case RTE_FLOW_ITEM_TYPE_FLEX:
-			size = item->spec ?
-				((const struct rte_flow_item_flex *)
-				item->spec)->length : 0;
+			if (item->spec != NULL) {
+				size = ((const struct rte_flow_item_flex *)item->spec)->length;
+				src_spec = ((const struct rte_flow_item_flex *)item->spec)->pattern;
+			} else {
+				size = 0;
+				src_spec = NULL;
+			}
 			break;
 		case RTE_FLOW_ITEM_TYPE_GRE_OPTION:
 			size = 0;
@@ -11378,12 +11384,14 @@ cmd_set_raw_parsed(const struct buffer *in)
 			fprintf(stderr, "Error - Not supported item\n");
 			goto error;
 		}
-		*total_size += size;
-		rte_memcpy(data_tail - (*total_size), item->spec, size);
-		/* update some fields which cannot be set by cmdline */
-		update_fields((data_tail - (*total_size)), item,
-			      upper_layer);
-		upper_layer = proto;
+		if (size) {
+			*total_size += size;
+			rte_memcpy(data_tail - (*total_size), src_spec, size);
+			/* update some fields which cannot be set by cmdline */
+			update_fields((data_tail - (*total_size)), item,
+				      upper_layer);
+			upper_layer = proto;
+		}
 	}
 	if (verbose_level & 0x1)
 		printf("total data size is %zu\n", (*total_size));
-- 
2.27.0

[PATCH v3 10/11] doc/mlx5: update mlx5 doc

[Rongwei Liu]

Add flex item matching and modify field feature into
mlx5 documents.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 doc/guides/nics/mlx5.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index b23ca35b8f..a2634c378f 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -106,7 +106,7 @@ Features
 - Sub-Function representors.
 - Sub-Function.
 - Matching on represented port.
-
+- Modify flex item field.
 
 Limitations
 -----------
-- 
2.27.0

[PATCH v3 11/11] app/testpmd: adjust cleanup sequence when quitting

[Rongwei Liu]

If flex item is referenced in async flow either by
pattern template or action template, currently testpmd
complains "flex item has flow references". Flex items should
be flushed after async flow resources cleanup.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/testpmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 134d79a555..e35f7a0e7a 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -3298,10 +3298,10 @@ flush_port_owned_resources(portid_t pi)
 {
 	mcast_addr_pool_destroy(pi);
 	port_flow_flush(pi);
-	port_flex_item_flush(pi);
 	port_flow_template_table_flush(pi);
 	port_flow_pattern_template_flush(pi);
 	port_flow_actions_template_flush(pi);
+	port_flex_item_flush(pi);
 	port_action_handle_flush(pi);
 }
 
-- 
2.27.0

[PATCH v3 00/11] add flex item support

[Rongwei Liu]

Support flex item matching and modify field in async flow.
Syntax follows sync flow exactly.

v3: enhance format, add flex_handle document.

Rongwei Liu (11):
  ethdev: add flex item modify field support
  app/testpmd: pass flex handle into matching mask
  net/mlx5: enable hws flex item create
  net/mlx5: add IPv6 protocol as flex item input
  net/mlx5: adopt new flex item prm definition
  net/mlx5/hws: add hws flex item matching support
  net/mlx5: add flex item modify field implementation
  net/mlx5: return error for sws modify field
  app/testpmd: raw encap with flex item support
  doc/mlx5: update mlx5 doc
  app/testpmd: adjust cleanup sequence when quitting

 app/test-pmd/cmdline_flow.c            | 123 +++++++++++++---
 app/test-pmd/testpmd.c                 |   2 +-
 doc/guides/nics/mlx5.rst               |   1 +
 doc/guides/prog_guide/rte_flow.rst     |  41 +++---
 doc/guides/rel_notes/release_23_03.rst |   4 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  14 +-
 drivers/common/mlx5/mlx5_devx_cmds.h   |   7 +-
 drivers/common/mlx5/mlx5_prm.h         |  29 +++-
 drivers/net/mlx5/hws/mlx5dr_definer.c  |  83 +++++++++++
 drivers/net/mlx5/linux/mlx5_os.c       |  27 ++--
 drivers/net/mlx5/mlx5.c                |  17 ++-
 drivers/net/mlx5/mlx5.h                |   9 +-
 drivers/net/mlx5/mlx5_flow.h           |   4 +
 drivers/net/mlx5/mlx5_flow_dv.c        | 186 ++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_flex.c      | 149 +++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c        |  64 ++++++++-
 lib/ethdev/rte_flow.h                  |   8 +-
 17 files changed, 664 insertions(+), 104 deletions(-)

-- 
2.27.0

[PATCH v3 01/11] ethdev: add flex item modify field support

[Rongwei Liu]

Add flex item as modify field destination.
Add "struct rte_flow_item_flex_handle *flex_handle" into
"struct rte_flow_action_modify_data" as union with existed
"level" member. This new member is dedicated for modifying
flex item.

Add flex item modify field cmdline support. Now user can use
testpmd cli to specify which flex item to be modified, either
source or destination.

Syntax is as below:
modify_field op set dst_type flex_item dst_level 0
dst_offset 16 src_type value src_value 0x123456781020 width 8

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c            | 89 ++++++++++++++++++++++++--
 doc/guides/prog_guide/rte_flow.rst     | 41 +++++++-----
 doc/guides/rel_notes/release_23_03.rst |  4 ++
 lib/ethdev/rte_flow.h                  |  8 ++-
 4 files changed, 116 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 88108498e0..7c12d63cbc 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -601,10 +601,12 @@ enum index {
 	ACTION_MODIFY_FIELD_DST_TYPE,
 	ACTION_MODIFY_FIELD_DST_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_DST_LEVEL,
+	ACTION_MODIFY_FIELD_DST_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_DST_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_TYPE,
 	ACTION_MODIFY_FIELD_SRC_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_SRC_LEVEL,
+	ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_SRC_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_VALUE,
 	ACTION_MODIFY_FIELD_SRC_POINTER,
@@ -807,7 +809,8 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color", NULL
+	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color",
+	"flex_item", NULL
 };
 
 static const char *const meter_colors[] = {
@@ -2282,6 +2285,10 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 				const char *str, unsigned int len, void *buf,
 				unsigned int size);
 static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+				const char *str, unsigned int len, void *buf,
+				unsigned int size);
+static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
 			 const char *str, unsigned int len, void *buf,
 			 unsigned int size);
@@ -5976,11 +5983,15 @@ static const struct token token_list[] = {
 		.name = "dst_level",
 		.help = "destination field level",
 		.next = NEXT(action_modify_field_dst,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					dst.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_DST_LEVEL_VALUE] = {
+		.name = "{dst_level}",
+		.help = "destination field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_DST_OFFSET] = {
 		.name = "dst_offset",
 		.help = "destination field bit offset",
@@ -6007,11 +6018,15 @@ static const struct token token_list[] = {
 		.name = "src_level",
 		.help = "source field level",
 		.next = NEXT(action_modify_field_src,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					src.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE] = {
+		.name = "{src_level}",
+		.help = "source field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_SRC_OFFSET] = {
 		.name = "src_offset",
 		.help = "source field bit offset",
@@ -8477,6 +8492,66 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 	return len;
 }
 
+/** Parse level for modify_field command. */
+static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+			 const char *str, unsigned int len, void *buf,
+			 unsigned int size)
+{
+	struct rte_flow_action_modify_field *action;
+	struct flex_item *fp;
+	uint32_t val;
+	struct buffer *out = buf;
+	char *end;
+
+	(void)token;
+	(void)size;
+	if (ctx->curr != ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		ctx->curr != ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)
+		return -1;
+	if (!ctx->object)
+		return len;
+	action = ctx->object;
+	errno = 0;
+	val = strtoumax(str, &end, 0);
+	if (errno || (size_t)(end - str) != len)
+		return -1;
+	/* No need to validate action template mask value */
+	if (out->args.vc.masks) {
+		if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)
+			action->dst.level = val;
+		else
+			action->src.level = val;
+		return len;
+	}
+	if ((ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		action->dst.field == RTE_FLOW_FIELD_FLEX_ITEM) ||
+		(ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE &&
+		action->src.field == RTE_FLOW_FIELD_FLEX_ITEM)) {
+		if (val >= FLEX_MAX_PARSERS_NUM) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+		fp = flex_items[ctx->port][val];
+		if (!fp) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+	}
+	if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE) {
+		if (action->dst.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->dst.level = val;
+		else
+			action->dst.flex_handle = fp->flex_handle;
+	} else if (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE) {
+		if (action->src.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->src.level = val;
+		else
+			action->src.flex_handle = fp->flex_handle;
+	}
+	return len;
+}
+
 /** Parse the conntrack update, not a rte_flow_action. */
 static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 3e6242803d..9ffd7baa7a 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -2905,6 +2905,9 @@ encapsulation level, from outermost to innermost (lower to higher values).
 For the tag array (in case of multiple tags are supported and present)
 ``level`` translates directly into the array index.
 
+``flex_handle`` is used to specify the flex item pointer which is being
+modified. ``flex_handle`` and ``level`` are mutually exclusive.
+
 ``offset`` specifies the number of bits to skip from a field's start.
 That allows performing a partial copy of the needed part or to divide a big
 packet field into multiple smaller fields. Alternatively, ``offset`` allows
@@ -2952,23 +2955,27 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx, xxx, xxx}.
 
 .. table:: destination/source field definition
 
-   +---------------+----------------------------------------------------------+
-   | Field         | Value                                                    |
-   +===============+==========================================================+
-   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
-   +---------------+----------------------------------------------------------+
-   | ``level``     | encapsulation level of a packet field or tag array index |
-   +---------------+----------------------------------------------------------+
-   | ``offset``    | number of bits to skip at the beginning                  |
-   +---------------+----------------------------------------------------------+
-   | ``value``     | immediate value buffer (source field only, not           |
-   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
-   | ``pvalue``    | pointer to immediate value data (source field only, not  |
-   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
+   +-----------------+----------------------------------------------------------+
+   | Field           | Value                                                    |
+   +=================+==========================================================+
+   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
+   +-----------------+----------------------------------------------------------+
+   | ``level``       | encapsulation level of a packet field or tag array index |
+   +-----------------+----------------------------------------------------------+
+   | ``flex_handle`` | flex item handle of a packet field                       |
+   +-----------------+----------------------------------------------------------+
+   | ``offset``      | number of bits to skip at the beginning                  |
+   +-----------------+----------------------------------------------------------+
+   | ``value``       | immediate value buffer (source field only, not           |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
+   |                 | field type                                               |
+   |                 | This field is only 16 bytes, maybe not big enough for    |
+   |                 | all NICs' flex item                                      |
+   +-----------------+----------------------------------------------------------+
+   | ``pvalue``      | pointer to immediate value data (source field only, not  |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
+   |                 | field type                                               |
+   +-----------------+----------------------------------------------------------+
 
 Action: ``CONNTRACK``
 ^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index c15f6fbb9f..3fb6e738e2 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -69,6 +69,10 @@ New Features
     ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
     required for eth_rx, eth_tx, crypto and timer eventdev adapters.
 
+* ethdev: added a new field:
+
+  - modify flex item: ``rte_flow_action_modify_data.flex_handle``.
+
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index b60987db4b..ba9f3d50a3 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
 	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
 	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
+	RTE_FLOW_FIELD_FLEX_ITEM,	/**< Flex item. */
 };
 
 /**
@@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
 	RTE_STD_C11
 	union {
 		struct {
-			/** Encapsulation level or tag index. */
-			uint32_t level;
+			/** Encapsulation level or tag index or flex item handle. */
+			union {
+				uint32_t level;
+				struct rte_flow_item_flex_handle *flex_handle;
+			};
 			/** Number of bits to skip from a field. */
 			uint32_t offset;
 		};
-- 
2.27.0

RE: [PATCH v3 01/11] ethdev: add flex item modify field support

[Rongwei Liu]

Hi Andrew,
	Can you share some comments on the modified version v3?

BR
Rongwei

> -----Original Message-----
> From: Rongwei Liu <rongweil@nvidia.com>
> Sent: Monday, January 30, 2023 21:20
> To: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; NBU-Contact-
> Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Aman Singh
> <aman.deep.singh@intel.com>; Yuying Zhang <yuying.zhang@intel.com>;
> Ferruh Yigit <ferruh.yigit@amd.com>; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>
> Cc: dev@dpdk.org; Raslan Darawsheh <rasland@nvidia.com>
> Subject: [PATCH v3 01/11] ethdev: add flex item modify field support
> 
> External email: Use caution opening links or attachments
> 
> 
> Add flex item as modify field destination.
> Add "struct rte_flow_item_flex_handle *flex_handle" into "struct
> rte_flow_action_modify_data" as union with existed "level" member. This new
> member is dedicated for modifying flex item.
> 
> Add flex item modify field cmdline support. Now user can use testpmd cli to
> specify which flex item to be modified, either source or destination.
> 
> Syntax is as below:
> modify_field op set dst_type flex_item dst_level 0 dst_offset 16 src_type value
> src_value 0x123456781020 width 8
> 
> Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>
> ---
>  app/test-pmd/cmdline_flow.c            | 89 ++++++++++++++++++++++++--
>  doc/guides/prog_guide/rte_flow.rst     | 41 +++++++-----
>  doc/guides/rel_notes/release_23_03.rst |  4 ++
>  lib/ethdev/rte_flow.h                  |  8 ++-
>  4 files changed, 116 insertions(+), 26 deletions(-)
> 
> diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
> index 88108498e0..7c12d63cbc 100644
> --- a/app/test-pmd/cmdline_flow.c
> +++ b/app/test-pmd/cmdline_flow.c
> @@ -601,10 +601,12 @@ enum index {
>         ACTION_MODIFY_FIELD_DST_TYPE,
>         ACTION_MODIFY_FIELD_DST_TYPE_VALUE,
>         ACTION_MODIFY_FIELD_DST_LEVEL,
> +       ACTION_MODIFY_FIELD_DST_LEVEL_VALUE,
>         ACTION_MODIFY_FIELD_DST_OFFSET,
>         ACTION_MODIFY_FIELD_SRC_TYPE,
>         ACTION_MODIFY_FIELD_SRC_TYPE_VALUE,
>         ACTION_MODIFY_FIELD_SRC_LEVEL,
> +       ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE,
>         ACTION_MODIFY_FIELD_SRC_OFFSET,
>         ACTION_MODIFY_FIELD_SRC_VALUE,
>         ACTION_MODIFY_FIELD_SRC_POINTER, @@ -807,7 +809,8 @@ static
> const char *const modify_field_ids[] = {
>         "udp_port_src", "udp_port_dst",
>         "vxlan_vni", "geneve_vni", "gtp_teid",
>         "tag", "mark", "meta", "pointer", "value",
> -       "ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color", NULL
> +       "ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color",
> +       "flex_item", NULL
>  };
> 
>  static const char *const meter_colors[] = { @@ -2282,6 +2285,10 @@
> parse_vc_modify_field_id(struct context *ctx, const struct token *token,
>                                 const char *str, unsigned int len, void *buf,
>                                 unsigned int size);  static int
> +parse_vc_modify_field_level(struct context *ctx, const struct token *token,
> +                               const char *str, unsigned int len, void *buf,
> +                               unsigned int size); static int
>  parse_vc_action_conntrack_update(struct context *ctx, const struct token
> *token,
>                          const char *str, unsigned int len, void *buf,
>                          unsigned int size); @@ -5976,11 +5983,15 @@ static const
> struct token token_list[] = {
>                 .name = "dst_level",
>                 .help = "destination field level",
>                 .next = NEXT(action_modify_field_dst,
> -                            NEXT_ENTRY(COMMON_UNSIGNED)),
> -               .args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
> -                                       dst.level)),
> +
> + NEXT_ENTRY(ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)),
>                 .call = parse_vc_conf,
>         },
> +       [ACTION_MODIFY_FIELD_DST_LEVEL_VALUE] = {
> +               .name = "{dst_level}",
> +               .help = "destination field level value",
> +               .call = parse_vc_modify_field_level,
> +               .comp = comp_none,
> +       },
>         [ACTION_MODIFY_FIELD_DST_OFFSET] = {
>                 .name = "dst_offset",
>                 .help = "destination field bit offset", @@ -6007,11 +6018,15 @@
> static const struct token token_list[] = {
>                 .name = "src_level",
>                 .help = "source field level",
>                 .next = NEXT(action_modify_field_src,
> -                            NEXT_ENTRY(COMMON_UNSIGNED)),
> -               .args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
> -                                       src.level)),
> +
> + NEXT_ENTRY(ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)),
>                 .call = parse_vc_conf,
>         },
> +       [ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE] = {
> +               .name = "{src_level}",
> +               .help = "source field level value",
> +               .call = parse_vc_modify_field_level,
> +               .comp = comp_none,
> +       },
>         [ACTION_MODIFY_FIELD_SRC_OFFSET] = {
>                 .name = "src_offset",
>                 .help = "source field bit offset", @@ -8477,6 +8492,66 @@
> parse_vc_modify_field_id(struct context *ctx, const struct token *token,
>         return len;
>  }
> 
> +/** Parse level for modify_field command. */ static int
> +parse_vc_modify_field_level(struct context *ctx, const struct token *token,
> +                        const char *str, unsigned int len, void *buf,
> +                        unsigned int size) {
> +       struct rte_flow_action_modify_field *action;
> +       struct flex_item *fp;
> +       uint32_t val;
> +       struct buffer *out = buf;
> +       char *end;
> +
> +       (void)token;
> +       (void)size;
> +       if (ctx->curr != ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
> +               ctx->curr != ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)
> +               return -1;
> +       if (!ctx->object)
> +               return len;
> +       action = ctx->object;
> +       errno = 0;
> +       val = strtoumax(str, &end, 0);
> +       if (errno || (size_t)(end - str) != len)
> +               return -1;
> +       /* No need to validate action template mask value */
> +       if (out->args.vc.masks) {
> +               if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)
> +                       action->dst.level = val;
> +               else
> +                       action->src.level = val;
> +               return len;
> +       }
> +       if ((ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
> +               action->dst.field == RTE_FLOW_FIELD_FLEX_ITEM) ||
> +               (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE &&
> +               action->src.field == RTE_FLOW_FIELD_FLEX_ITEM)) {
> +               if (val >= FLEX_MAX_PARSERS_NUM) {
> +                       printf("Bad flex item handle\n");
> +                       return -1;
> +               }
> +               fp = flex_items[ctx->port][val];
> +               if (!fp) {
> +                       printf("Bad flex item handle\n");
> +                       return -1;
> +               }
> +       }
> +       if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE) {
> +               if (action->dst.field != RTE_FLOW_FIELD_FLEX_ITEM)
> +                       action->dst.level = val;
> +               else
> +                       action->dst.flex_handle = fp->flex_handle;
> +       } else if (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE) {
> +               if (action->src.field != RTE_FLOW_FIELD_FLEX_ITEM)
> +                       action->src.level = val;
> +               else
> +                       action->src.flex_handle = fp->flex_handle;
> +       }
> +       return len;
> +}
> +
>  /** Parse the conntrack update, not a rte_flow_action. */  static int
> parse_vc_action_conntrack_update(struct context *ctx, const struct token
> *token, diff --git a/doc/guides/prog_guide/rte_flow.rst
> b/doc/guides/prog_guide/rte_flow.rst
> index 3e6242803d..9ffd7baa7a 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -2905,6 +2905,9 @@ encapsulation level, from outermost to innermost
> (lower to higher values).
>  For the tag array (in case of multiple tags are supported and present)
> ``level`` translates directly into the array index.
> 
> +``flex_handle`` is used to specify the flex item pointer which is being
> +modified. ``flex_handle`` and ``level`` are mutually exclusive.
> +
>  ``offset`` specifies the number of bits to skip from a field's start.
>  That allows performing a partial copy of the needed part or to divide a big
> packet field into multiple smaller fields. Alternatively, ``offset`` allows @@ -
> 2952,23 +2955,27 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx, xxx,
> xxx}.
> 
>  .. table:: destination/source field definition
> 
> -   +---------------+----------------------------------------------------------+
> -   | Field         | Value                                                    |
> -
> +===============+===============================================
> ===========+
> -   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
> -   +---------------+----------------------------------------------------------+
> -   | ``level``     | encapsulation level of a packet field or tag array index |
> -   +---------------+----------------------------------------------------------+
> -   | ``offset``    | number of bits to skip at the beginning                  |
> -   +---------------+----------------------------------------------------------+
> -   | ``value``     | immediate value buffer (source field only, not           |
> -   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
> -   |               | field type                                               |
> -   +---------------+----------------------------------------------------------+
> -   | ``pvalue``    | pointer to immediate value data (source field only, not  |
> -   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
> -   |               | field type                                               |
> -   +---------------+----------------------------------------------------------+
> +   +-----------------+----------------------------------------------------------+
> +   | Field           | Value                                                    |
> +
> +=================+=============================================
> =============+
> +   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
> +   +-----------------+----------------------------------------------------------+
> +   | ``level``       | encapsulation level of a packet field or tag array index |
> +   +-----------------+----------------------------------------------------------+
> +   | ``flex_handle`` | flex item handle of a packet field                       |
> +   +-----------------+----------------------------------------------------------+
> +   | ``offset``      | number of bits to skip at the beginning                  |
> +   +-----------------+----------------------------------------------------------+
> +   | ``value``       | immediate value buffer (source field only, not           |
> +   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
> +   |                 | field type                                               |
> +   |                 | This field is only 16 bytes, maybe not big enough for    |
> +   |                 | all NICs' flex item                                      |
> +   +-----------------+----------------------------------------------------------+
> +   | ``pvalue``      | pointer to immediate value data (source field only, not  |
> +   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
> +   |                 | field type                                               |
> +
> + +-----------------+---------------------------------------------------
> + -------+
> 
>  Action: ``CONNTRACK``
>  ^^^^^^^^^^^^^^^^^^^^^
> diff --git a/doc/guides/rel_notes/release_23_03.rst
> b/doc/guides/rel_notes/release_23_03.rst
> index c15f6fbb9f..3fb6e738e2 100644
> --- a/doc/guides/rel_notes/release_23_03.rst
> +++ b/doc/guides/rel_notes/release_23_03.rst
> @@ -69,6 +69,10 @@ New Features
>      ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
>      required for eth_rx, eth_tx, crypto and timer eventdev adapters.
> 
> +* ethdev: added a new field:
> +
> +  - modify flex item: ``rte_flow_action_modify_data.flex_handle``.
> +
> 
>  Removed Items
>  -------------
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h index
> b60987db4b..ba9f3d50a3 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
>         RTE_FLOW_FIELD_IPV6_ECN,        /**< IPv6 ECN. */
>         RTE_FLOW_FIELD_GTP_PSC_QFI,     /**< GTP QFI. */
>         RTE_FLOW_FIELD_METER_COLOR,     /**< Meter color marker. */
> +       RTE_FLOW_FIELD_FLEX_ITEM,       /**< Flex item. */
>  };
> 
>  /**
> @@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
>         RTE_STD_C11
>         union {
>                 struct {
> -                       /** Encapsulation level or tag index. */
> -                       uint32_t level;
> +                       /** Encapsulation level or tag index or flex item handle. */
> +                       union {
> +                               uint32_t level;
> +                               struct rte_flow_item_flex_handle *flex_handle;
> +                       };
>                         /** Number of bits to skip from a field. */
>                         uint32_t offset;
>                 };
> --
> 2.27.0

[PATCH v4 0/4] add flex item support

[Rongwei Liu]

Support flex item matching and modify field in async flow.
Syntax follows sync flow exactly.

v4: split ethdev part.
v3: enhance format, add flex_handle document.

Rongwei Liu (4):
  ethdev: add flex item modify field support
  app/testpmd: pass flex handle into matching mask
  app/testpmd: raw encap with flex item support
  app/testpmd: adjust cleanup sequence when quitting

 app/test-pmd/cmdline_flow.c            | 123 +++++++++++++++++++++----
 app/test-pmd/testpmd.c                 |   2 +-
 doc/guides/prog_guide/rte_flow.rst     |  41 +++++----
 doc/guides/rel_notes/release_23_03.rst |   4 +
 lib/ethdev/rte_flow.h                  |   8 +-
 5 files changed, 139 insertions(+), 39 deletions(-)

-- 
2.27.0

[PATCH v4 1/4] ethdev: add flex item modify field support

[Rongwei Liu]

Add flex item as modify field destination.
Add "struct rte_flow_item_flex_handle *flex_handle" into
"struct rte_flow_action_modify_data" as union with existed
"level" member. This new member is dedicated for modifying
flex item.

Add flex item modify field cmdline support. Now user can use
testpmd cli to specify which flex item to be modified, either
source or destination.

Syntax is as below:
modify_field op set dst_type flex_item dst_level 0
dst_offset 16 src_type value src_value 0x123456781020 width 8

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c            | 89 ++++++++++++++++++++++++--
 doc/guides/prog_guide/rte_flow.rst     | 41 +++++++-----
 doc/guides/rel_notes/release_23_03.rst |  4 ++
 lib/ethdev/rte_flow.h                  |  8 ++-
 4 files changed, 116 insertions(+), 26 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 88108498e0..7c12d63cbc 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -601,10 +601,12 @@ enum index {
 	ACTION_MODIFY_FIELD_DST_TYPE,
 	ACTION_MODIFY_FIELD_DST_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_DST_LEVEL,
+	ACTION_MODIFY_FIELD_DST_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_DST_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_TYPE,
 	ACTION_MODIFY_FIELD_SRC_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_SRC_LEVEL,
+	ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_SRC_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_VALUE,
 	ACTION_MODIFY_FIELD_SRC_POINTER,
@@ -807,7 +809,8 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color", NULL
+	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color",
+	"flex_item", NULL
 };
 
 static const char *const meter_colors[] = {
@@ -2282,6 +2285,10 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 				const char *str, unsigned int len, void *buf,
 				unsigned int size);
 static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+				const char *str, unsigned int len, void *buf,
+				unsigned int size);
+static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
 			 const char *str, unsigned int len, void *buf,
 			 unsigned int size);
@@ -5976,11 +5983,15 @@ static const struct token token_list[] = {
 		.name = "dst_level",
 		.help = "destination field level",
 		.next = NEXT(action_modify_field_dst,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					dst.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_DST_LEVEL_VALUE] = {
+		.name = "{dst_level}",
+		.help = "destination field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_DST_OFFSET] = {
 		.name = "dst_offset",
 		.help = "destination field bit offset",
@@ -6007,11 +6018,15 @@ static const struct token token_list[] = {
 		.name = "src_level",
 		.help = "source field level",
 		.next = NEXT(action_modify_field_src,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					src.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE] = {
+		.name = "{src_level}",
+		.help = "source field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_SRC_OFFSET] = {
 		.name = "src_offset",
 		.help = "source field bit offset",
@@ -8477,6 +8492,66 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 	return len;
 }
 
+/** Parse level for modify_field command. */
+static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+			 const char *str, unsigned int len, void *buf,
+			 unsigned int size)
+{
+	struct rte_flow_action_modify_field *action;
+	struct flex_item *fp;
+	uint32_t val;
+	struct buffer *out = buf;
+	char *end;
+
+	(void)token;
+	(void)size;
+	if (ctx->curr != ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		ctx->curr != ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)
+		return -1;
+	if (!ctx->object)
+		return len;
+	action = ctx->object;
+	errno = 0;
+	val = strtoumax(str, &end, 0);
+	if (errno || (size_t)(end - str) != len)
+		return -1;
+	/* No need to validate action template mask value */
+	if (out->args.vc.masks) {
+		if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)
+			action->dst.level = val;
+		else
+			action->src.level = val;
+		return len;
+	}
+	if ((ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		action->dst.field == RTE_FLOW_FIELD_FLEX_ITEM) ||
+		(ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE &&
+		action->src.field == RTE_FLOW_FIELD_FLEX_ITEM)) {
+		if (val >= FLEX_MAX_PARSERS_NUM) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+		fp = flex_items[ctx->port][val];
+		if (!fp) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+	}
+	if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE) {
+		if (action->dst.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->dst.level = val;
+		else
+			action->dst.flex_handle = fp->flex_handle;
+	} else if (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE) {
+		if (action->src.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->src.level = val;
+		else
+			action->src.flex_handle = fp->flex_handle;
+	}
+	return len;
+}
+
 /** Parse the conntrack update, not a rte_flow_action. */
 static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
diff --git a/doc/guides/prog_guide/rte_flow.rst b/doc/guides/prog_guide/rte_flow.rst
index 3e6242803d..9ffd7baa7a 100644
--- a/doc/guides/prog_guide/rte_flow.rst
+++ b/doc/guides/prog_guide/rte_flow.rst
@@ -2905,6 +2905,9 @@ encapsulation level, from outermost to innermost (lower to higher values).
 For the tag array (in case of multiple tags are supported and present)
 ``level`` translates directly into the array index.
 
+``flex_handle`` is used to specify the flex item pointer which is being
+modified. ``flex_handle`` and ``level`` are mutually exclusive.
+
 ``offset`` specifies the number of bits to skip from a field's start.
 That allows performing a partial copy of the needed part or to divide a big
 packet field into multiple smaller fields. Alternatively, ``offset`` allows
@@ -2952,23 +2955,27 @@ value as sequence of bytes {xxx, xxx, 0x85, xxx, xxx, xxx}.
 
 .. table:: destination/source field definition
 
-   +---------------+----------------------------------------------------------+
-   | Field         | Value                                                    |
-   +===============+==========================================================+
-   | ``field``     | ID: packet field, mark, meta, tag, immediate, pointer    |
-   +---------------+----------------------------------------------------------+
-   | ``level``     | encapsulation level of a packet field or tag array index |
-   +---------------+----------------------------------------------------------+
-   | ``offset``    | number of bits to skip at the beginning                  |
-   +---------------+----------------------------------------------------------+
-   | ``value``     | immediate value buffer (source field only, not           |
-   |               | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
-   | ``pvalue``    | pointer to immediate value data (source field only, not  |
-   |               | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
-   |               | field type                                               |
-   +---------------+----------------------------------------------------------+
+   +-----------------+----------------------------------------------------------+
+   | Field           | Value                                                    |
+   +=================+==========================================================+
+   | ``field``       | ID: packet field, mark, meta, tag, immediate, pointer    |
+   +-----------------+----------------------------------------------------------+
+   | ``level``       | encapsulation level of a packet field or tag array index |
+   +-----------------+----------------------------------------------------------+
+   | ``flex_handle`` | flex item handle of a packet field                       |
+   +-----------------+----------------------------------------------------------+
+   | ``offset``      | number of bits to skip at the beginning                  |
+   +-----------------+----------------------------------------------------------+
+   | ``value``       | immediate value buffer (source field only, not           |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_VALUE      |
+   |                 | field type                                               |
+   |                 | This field is only 16 bytes, maybe not big enough for    |
+   |                 | all NICs' flex item                                      |
+   +-----------------+----------------------------------------------------------+
+   | ``pvalue``      | pointer to immediate value data (source field only, not  |
+   |                 | applicable to destination) for RTE_FLOW_FIELD_POINTER    |
+   |                 | field type                                               |
+   +-----------------+----------------------------------------------------------+
 
 Action: ``CONNTRACK``
 ^^^^^^^^^^^^^^^^^^^^^
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index c15f6fbb9f..3fb6e738e2 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -69,6 +69,10 @@ New Features
     ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
     required for eth_rx, eth_tx, crypto and timer eventdev adapters.
 
+* ethdev: added a new field:
+
+  - modify flex item: ``rte_flow_action_modify_data.flex_handle``.
+
 
 Removed Items
 -------------
diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
index b60987db4b..ba9f3d50a3 100644
--- a/lib/ethdev/rte_flow.h
+++ b/lib/ethdev/rte_flow.h
@@ -3528,6 +3528,7 @@ enum rte_flow_field_id {
 	RTE_FLOW_FIELD_IPV6_ECN,	/**< IPv6 ECN. */
 	RTE_FLOW_FIELD_GTP_PSC_QFI,	/**< GTP QFI. */
 	RTE_FLOW_FIELD_METER_COLOR,	/**< Meter color marker. */
+	RTE_FLOW_FIELD_FLEX_ITEM,	/**< Flex item. */
 };
 
 /**
@@ -3541,8 +3542,11 @@ struct rte_flow_action_modify_data {
 	RTE_STD_C11
 	union {
 		struct {
-			/** Encapsulation level or tag index. */
-			uint32_t level;
+			/** Encapsulation level or tag index or flex item handle. */
+			union {
+				uint32_t level;
+				struct rte_flow_item_flex_handle *flex_handle;
+			};
 			/** Number of bits to skip from a field. */
 			uint32_t offset;
 		};
-- 
2.27.0

Re: [PATCH v4 1/4] ethdev: add flex item modify field support

[Ferruh Yigit]

On 2/6/2023 3:39 AM, Rongwei Liu wrote:
> diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
> index c15f6fbb9f..3fb6e738e2 100644
> --- a/doc/guides/rel_notes/release_23_03.rst
> +++ b/doc/guides/rel_notes/release_23_03.rst
> @@ -69,6 +69,10 @@ New Features
>      ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter
>      required for eth_rx, eth_tx, crypto and timer eventdev adapters.
>  
> +* ethdev: added a new field:
> +
> +  - modify flex item: ``rte_flow_action_modify_data.flex_handle``.
> +
>  

Release notes update syntax and location is wrong, fixed in next-net.

[PATCH v4 2/4] app/testpmd: pass flex handle into matching mask

[Rongwei Liu]

In async flow create API, there is only mask information when
creating flow table but flex item handle is required to parse
the HW sample information.

Pass the flex item handle instead of UINT64/32_MAX to mask.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 7c12d63cbc..9037432cc8 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -10068,8 +10068,8 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	}
 	if (offset == offsetof(struct rte_flow_item_flex, handle)) {
 		const struct flex_item *fp;
-		struct rte_flow_item_flex *item_flex = ctx->object;
-		handle = (uint16_t)(uintptr_t)item_flex->handle;
+		spec = ctx->object;
+		handle = (uint16_t)(uintptr_t)spec->handle;
 		if (handle >= FLEX_MAX_PARSERS_NUM) {
 			printf("Bad flex item handle\n");
 			return -1;
@@ -10079,7 +10079,9 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 			printf("Bad flex item handle\n");
 			return -1;
 		}
-		item_flex->handle = fp->flex_handle;
+		spec->handle = fp->flex_handle;
+		mask = spec + 2; /* spec, last, mask */
+		mask->handle = fp->flex_handle;
 	} else if (offset == offsetof(struct rte_flow_item_flex, pattern)) {
 		handle = (uint16_t)(uintptr_t)
 			((struct rte_flow_item_flex *)ctx->object)->pattern;
-- 
2.27.0

[PATCH v4 3/4] app/testpmd: raw encap with flex item support

[Rongwei Liu]

Application should retrieve raw_encap buffer from
spec->pattern if it is flex item.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 9037432cc8..3b2975ee83 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -11193,6 +11193,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 	uint16_t proto = 0;
 	uint16_t idx = in->port; /* We borrow port field as index */
 	int gtp_psc = -1; /* GTP PSC option index. */
+	const void *src_spec;
 
 	if (in->command == SET_SAMPLE_ACTIONS)
 		return cmd_set_raw_parsed_sample(in);
@@ -11216,6 +11217,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 		item = in->args.vc.pattern + i;
 		if (item->spec == NULL)
 			item->spec = flow_item_default_mask(item);
+		src_spec = item->spec;
 		switch (item->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			size = sizeof(struct rte_ether_hdr);
@@ -11343,9 +11345,13 @@ cmd_set_raw_parsed(const struct buffer *in)
 			size = sizeof(struct rte_flow_item_pfcp);
 			break;
 		case RTE_FLOW_ITEM_TYPE_FLEX:
-			size = item->spec ?
-				((const struct rte_flow_item_flex *)
-				item->spec)->length : 0;
+			if (item->spec != NULL) {
+				size = ((const struct rte_flow_item_flex *)item->spec)->length;
+				src_spec = ((const struct rte_flow_item_flex *)item->spec)->pattern;
+			} else {
+				size = 0;
+				src_spec = NULL;
+			}
 			break;
 		case RTE_FLOW_ITEM_TYPE_GRE_OPTION:
 			size = 0;
@@ -11378,12 +11384,14 @@ cmd_set_raw_parsed(const struct buffer *in)
 			fprintf(stderr, "Error - Not supported item\n");
 			goto error;
 		}
-		*total_size += size;
-		rte_memcpy(data_tail - (*total_size), item->spec, size);
-		/* update some fields which cannot be set by cmdline */
-		update_fields((data_tail - (*total_size)), item,
-			      upper_layer);
-		upper_layer = proto;
+		if (size) {
+			*total_size += size;
+			rte_memcpy(data_tail - (*total_size), src_spec, size);
+			/* update some fields which cannot be set by cmdline */
+			update_fields((data_tail - (*total_size)), item,
+				      upper_layer);
+			upper_layer = proto;
+		}
 	}
 	if (verbose_level & 0x1)
 		printf("total data size is %zu\n", (*total_size));
-- 
2.27.0

[PATCH v4 4/4] app/testpmd: adjust cleanup sequence when quitting

[Rongwei Liu]

If flex item is referenced in async flow either by
pattern template or action template, currently testpmd
complains "flex item has flow references". Flex items should
be flushed after async flow resources cleanup.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Aman Singh <aman.deep.singh@intel.com>
---
 app/test-pmd/testpmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 134d79a555..e35f7a0e7a 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -3298,10 +3298,10 @@ flush_port_owned_resources(portid_t pi)
 {
 	mcast_addr_pool_destroy(pi);
 	port_flow_flush(pi);
-	port_flex_item_flush(pi);
 	port_flow_template_table_flush(pi);
 	port_flow_pattern_template_flush(pi);
 	port_flow_actions_template_flush(pi);
+	port_flex_item_flush(pi);
 	port_action_handle_flush(pi);
 }
 
-- 
2.27.0

Re: [PATCH v4 0/4] add flex item support

[Ferruh Yigit]

On 2/6/2023 3:39 AM, Rongwei Liu wrote:
> Support flex item matching and modify field in async flow.
> Syntax follows sync flow exactly.
> 
> v4: split ethdev part.
> v3: enhance format, add flex_handle document.
> 
> Rongwei Liu (4):
>   ethdev: add flex item modify field support
>   app/testpmd: pass flex handle into matching mask
>   app/testpmd: raw encap with flex item support
>   app/testpmd: adjust cleanup sequence when quitting

Series applied to dpdk-next-net/main, thanks.

[PATCH v3 02/11] app/testpmd: pass flex handle into matching mask

[Rongwei Liu]

In async flow create API, there is only mask information when
creating flow table but flex item handle is required to parse
the HW sample information.

Pass the flex item handle instead of UINT64/32_MAX to mask.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 7c12d63cbc..9037432cc8 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -10068,8 +10068,8 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	}
 	if (offset == offsetof(struct rte_flow_item_flex, handle)) {
 		const struct flex_item *fp;
-		struct rte_flow_item_flex *item_flex = ctx->object;
-		handle = (uint16_t)(uintptr_t)item_flex->handle;
+		spec = ctx->object;
+		handle = (uint16_t)(uintptr_t)spec->handle;
 		if (handle >= FLEX_MAX_PARSERS_NUM) {
 			printf("Bad flex item handle\n");
 			return -1;
@@ -10079,7 +10079,9 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 			printf("Bad flex item handle\n");
 			return -1;
 		}
-		item_flex->handle = fp->flex_handle;
+		spec->handle = fp->flex_handle;
+		mask = spec + 2; /* spec, last, mask */
+		mask->handle = fp->flex_handle;
 	} else if (offset == offsetof(struct rte_flow_item_flex, pattern)) {
 		handle = (uint16_t)(uintptr_t)
 			((struct rte_flow_item_flex *)ctx->object)->pattern;
-- 
2.27.0

[PATCH v3 03/11] net/mlx5: enable hws flex item create

[Rongwei Liu]

Enable flex item create and destroy with dv_flow_en=2

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 27 +++++++++++++++------------
 drivers/net/mlx5/mlx5_flow_hw.c  |  2 ++
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a71474c90a..f5b3edea99 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -474,10 +474,20 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 	err = mlx5_alloc_table_hash_list(priv);
 	if (err)
 		goto error;
-	if (priv->sh->config.dv_flow_en == 2)
-		return 0;
 	/* The resources below are only valid with DV support. */
 #ifdef HAVE_IBV_FLOW_DV_SUPPORT
+	/* Init shared flex parsers list, no need lcore_share */
+	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
+	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
+					       mlx5_flex_parser_create_cb,
+					       mlx5_flex_parser_match_cb,
+					       mlx5_flex_parser_remove_cb,
+					       mlx5_flex_parser_clone_cb,
+					       mlx5_flex_parser_clone_free_cb);
+	if (!sh->flex_parsers_dv)
+		goto error;
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 	/* Init port id action list. */
 	snprintf(s, sizeof(s), "%s_port_id_action_list", sh->ibdev_name);
 	sh->port_id_action_list = mlx5_list_create(s, sh, true,
@@ -518,16 +528,9 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 					      flow_dv_dest_array_clone_free_cb);
 	if (!sh->dest_array_list)
 		goto error;
-	/* Init shared flex parsers list, no need lcore_share */
-	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
-	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
-					       mlx5_flex_parser_create_cb,
-					       mlx5_flex_parser_match_cb,
-					       mlx5_flex_parser_remove_cb,
-					       mlx5_flex_parser_clone_cb,
-					       mlx5_flex_parser_clone_free_cb);
-	if (!sh->flex_parsers_dv)
-		goto error;
+#else
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 #endif
 #ifdef HAVE_MLX5DV_DR
 	void *domain;
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 952247d3cf..9406360a10 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -8340,6 +8340,8 @@ const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops = {
 	.query = flow_hw_query,
 	.get_aged_flows = flow_hw_get_aged_flows,
 	.get_q_aged_flows = flow_hw_get_q_aged_flows,
+	.item_create = flow_dv_item_create,
+	.item_release = flow_dv_item_release,
 };
 
 /**
-- 
2.27.0

[PATCH v3 04/11] net/mlx5: add IPv6 protocol as flex item input

[Rongwei Liu]

Support IPv6 protocol as new flex item input link.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_flex.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index fb08910ddb..bec07b13c1 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -1043,6 +1043,22 @@ mlx5_flex_arc_in_udp(const struct rte_flow_item *item,
 	return rte_be_to_cpu_16(spec->hdr.dst_port);
 }
 
+static int
+mlx5_flex_arc_in_ipv6(const struct rte_flow_item *item,
+		      struct rte_flow_error *error)
+{
+	const struct rte_flow_item_ipv6 *spec = item->spec;
+	const struct rte_flow_item_ipv6 *mask = item->mask;
+	struct rte_flow_item_ipv6 ip = { .hdr.proto = 0xff };
+
+	if (memcmp(mask, &ip, sizeof(struct rte_flow_item_ipv6))) {
+		return rte_flow_error_set
+			(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item,
+			 "invalid ipv6 item mask, full mask is desired");
+	}
+	return spec->hdr.proto;
+}
+
 static int
 mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 			   const struct rte_flow_item_flex_conf *conf,
@@ -1089,6 +1105,9 @@ mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 		case RTE_FLOW_ITEM_TYPE_UDP:
 			ret = mlx5_flex_arc_in_udp(rte_item, error);
 			break;
+		case RTE_FLOW_ITEM_TYPE_IPV6:
+			ret = mlx5_flex_arc_in_ipv6(rte_item, error);
+			break;
 		default:
 			MLX5_ASSERT(false);
 			return rte_flow_error_set
-- 
2.27.0

[PATCH v3 05/11] net/mlx5: adopt new flex item prm definition

[Rongwei Liu]

Per newest PRM definition, sample_id stands for 3 parts
of information instead of single uint32_t id: sample_id +
modify_filed_id + format_select_dw.

Also new FW capability bits have been introduces to identify
the new capability.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 14 +++++++++++---
 drivers/common/mlx5/mlx5_devx_cmds.h |  7 ++++++-
 drivers/common/mlx5/mlx5_prm.h       | 28 ++++++++++++++++++++++++++--
 drivers/net/mlx5/mlx5.c              | 15 +++++++++++----
 drivers/net/mlx5/mlx5.h              |  3 ++-
 drivers/net/mlx5/mlx5_flow_flex.c    | 14 +++++++++++---
 6 files changed, 67 insertions(+), 14 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index e3a4927d0f..1f65ea7dcb 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -607,7 +607,8 @@ mlx5_devx_cmd_query_hca_vdpa_attr(void *ctx,
 
 int
 mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				  uint32_t ids[], uint32_t num)
+				  struct mlx5_ext_sample_id ids[],
+				  uint32_t num, uint8_t *anchor)
 {
 	uint32_t in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {0};
 	uint32_t out[MLX5_ST_SZ_DW(create_flex_parser_out)] = {0};
@@ -636,6 +637,7 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			(void *)flex_obj);
 		return -rte_errno;
 	}
+	*anchor = MLX5_GET(parse_graph_flex, flex, head_anchor_id);
 	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
 		void *s_off = (void *)((char *)sample + i *
 			      MLX5_ST_SZ_BYTES(parse_graph_flow_match_sample));
@@ -645,8 +647,8 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			      flow_match_sample_en);
 		if (!en)
 			continue;
-		ids[idx++] = MLX5_GET(parse_graph_flow_match_sample, s_off,
-				  flow_match_sample_field_id);
+		ids[idx++].id = MLX5_GET(parse_graph_flow_match_sample, s_off,
+					 flow_match_sample_field_id);
 	}
 	if (num != idx) {
 		rte_errno = EINVAL;
@@ -794,6 +796,12 @@ mlx5_devx_cmd_query_hca_parse_graph_node_cap
 					 max_num_arc_out);
 	attr->max_num_sample = MLX5_GET(parse_graph_node_cap, hcattr,
 					max_num_sample);
+	attr->anchor_en = MLX5_GET(parse_graph_node_cap, hcattr, anchor_en);
+	attr->ext_sample_id = MLX5_GET(parse_graph_node_cap, hcattr, ext_sample_id);
+	attr->sample_tunnel_inner2 = MLX5_GET(parse_graph_node_cap, hcattr,
+					      sample_tunnel_inner2);
+	attr->zero_size_supported = MLX5_GET(parse_graph_node_cap, hcattr,
+					     zero_size_supported);
 	attr->sample_id_in_out = MLX5_GET(parse_graph_node_cap, hcattr,
 					  sample_id_in_out);
 	attr->max_base_header_length = MLX5_GET(parse_graph_node_cap, hcattr,
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index c94b9eac06..5b33010155 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -114,6 +114,10 @@ struct mlx5_hca_flex_attr {
 	uint8_t  max_num_arc_out;
 	uint8_t  max_num_sample;
 	uint8_t  max_num_prog_sample:5;	/* From HCA CAP 2 */
+	uint8_t  anchor_en:1;
+	uint8_t  ext_sample_id:1;
+	uint8_t  sample_tunnel_inner2:1;
+	uint8_t  zero_size_supported:1;
 	uint8_t  sample_id_in_out:1;
 	uint16_t max_base_header_length;
 	uint8_t  max_sample_base_offset;
@@ -706,7 +710,8 @@ int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir,
 			     struct mlx5_devx_modify_tir_attr *tir_attr);
 __rte_internal
 int mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				      uint32_t ids[], uint32_t num);
+				      struct mlx5_ext_sample_id ids[],
+				      uint32_t num, uint8_t *anchor);
 
 __rte_internal
 struct mlx5_devx_obj *
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 9294f65e24..b32dc735a1 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1894,7 +1894,11 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 max_num_arc_in[0x08];
 	u8 max_num_arc_out[0x08];
 	u8 max_num_sample[0x08];
-	u8 reserved_at_78[0x07];
+	u8 reserved_at_78[0x03];
+	u8 anchor_en[0x1];
+	u8 ext_sample_id[0x1];
+	u8 sample_tunnel_inner2[0x1];
+	u8 zero_size_supported[0x1];
 	u8 sample_id_in_out[0x1];
 	u8 max_base_header_length[0x10];
 	u8 reserved_at_90[0x08];
@@ -1904,6 +1908,24 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 header_length_mask_width[0x08];
 };
 
+/* ext_sample_id structure, see PRM Table: Flow Match Sample ID Format. */
+struct mlx5_ext_sample_id {
+	union {
+		struct {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+			uint32_t format_select_dw:8;
+			uint32_t modify_field_id:12;
+			uint32_t sample_id:12;
+#else
+			uint32_t sample_id:12;
+			uint32_t modify_field_id:12;
+			uint32_t format_select_dw:8;
+#endif
+		};
+		uint32_t id;
+	};
+};
+
 struct mlx5_ifc_flow_table_prop_layout_bits {
 	u8 ft_support[0x1];
 	u8 flow_tag[0x1];
@@ -4542,7 +4564,9 @@ struct mlx5_ifc_parse_graph_flex_bits {
 	u8 header_length_mode[0x4];
 	u8 header_length_field_offset[0x10];
 	u8 next_header_field_offset[0x10];
-	u8 reserved_at_160[0x1b];
+	u8 reserved_at_160[0x12];
+	u8 head_anchor_id[0x6];
+	u8 reserved_at_178[0x3];
 	u8 next_header_field_size[0x5];
 	u8 header_length_field_mask[0x20];
 	u8 reserved_at_224[0x20];
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index b8643cebdd..0b97c4e78d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -964,11 +964,13 @@ int
 mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
 	struct mlx5_devx_graph_node_attr node = {
 		.modify_field_select = 0,
 	};
-	uint32_t ids[8];
+	struct mlx5_ext_sample_id ids[8];
+	uint8_t anchor_id;
 	int ret;
 
 	if (!priv->sh->cdev->config.hca_attr.parse_graph_flex_node) {
@@ -1004,15 +1006,20 @@ mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->num = 2;
-	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num);
+	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num, &anchor_id);
 	if (ret) {
 		DRV_LOG(ERR, "Failed to query sample IDs.");
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->offset[0] = 0x0;
 	prf->offset[1] = sizeof(uint32_t);
-	prf->ids[0] = ids[0];
-	prf->ids[1] = ids[1];
+	if (attr->ext_sample_id) {
+		prf->ids[0] = ids[0].sample_id;
+		prf->ids[1] = ids[1].sample_id;
+	} else {
+		prf->ids[0] = ids[0].id;
+		prf->ids[1] = ids[1].id;
+	}
 	return 0;
 }
 
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 16b33e1548..83fb316ad8 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1307,9 +1307,10 @@ struct mlx5_lag {
 struct mlx5_flex_parser_devx {
 	struct mlx5_list_entry entry;  /* List element at the beginning. */
 	uint32_t num_samples;
+	uint8_t anchor_id;
 	void *devx_obj;
 	struct mlx5_devx_graph_node_attr devx_conf;
-	uint32_t sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
+	struct mlx5_ext_sample_id sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
 };
 
 /* Pattern field descriptor - how to translate flex pattern into samples. */
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index bec07b13c1..24b7226ee6 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -226,15 +226,18 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	void *misc4_m = MLX5_ADDR_OF(fte_match_param, matcher,
 				     misc_parameters_4);
 	void *misc4_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_4);
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_flex_item *tp;
 	uint32_t i, pos = 0;
+	uint32_t sample_id;
 
 	RTE_SET_USED(dev);
 	MLX5_ASSERT(item->spec && item->mask);
 	spec = item->spec;
 	mask = item->mask;
 	tp = (struct mlx5_flex_item *)spec->handle;
-	MLX5_ASSERT(mlx5_flex_index(dev->data->dev_private, tp) >= 0);
+	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
 		uint32_t id = map->reg_id;
@@ -257,9 +260,13 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 			MLX5_ASSERT(id < num_samples);
 			id += num_samples;
 		}
+		if (attr->ext_sample_id)
+			sample_id = tp->devx_fp->sample_ids[id].sample_id;
+		else
+			sample_id = tp->devx_fp->sample_ids[id].id;
 		mlx5_flex_set_match_sample(misc4_m, misc4_v,
 					   def, msk & def, val & msk & def,
-					   tp->devx_fp->sample_ids[id], id);
+					   sample_id, id);
 		pos += map->width;
 	}
 }
@@ -1317,7 +1324,8 @@ mlx5_flex_parser_create_cb(void *list_ctx, void *ctx)
 	/* Query the firmware assigned sample ids. */
 	ret = mlx5_devx_cmd_query_parse_samples(fp->devx_obj,
 						fp->sample_ids,
-						fp->num_samples);
+						fp->num_samples,
+						&fp->anchor_id);
 	if (ret)
 		goto error;
 	DRV_LOG(DEBUG, "DEVx flex parser %p created, samples num: %u",
-- 
2.27.0

[PATCH v3 06/11] net/mlx5/hws: add hws flex item matching support

[Rongwei Liu]

Support flex item matching in hws and syntax follows
sws exactly.

Flex item should be created in advance and follow current
json mapping logic.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c |  83 ++++++++++++++++++
 drivers/net/mlx5/mlx5.c               |   2 +-
 drivers/net/mlx5/mlx5.h               |   6 ++
 drivers/net/mlx5/mlx5_flow.h          |   1 +
 drivers/net/mlx5/mlx5_flow_dv.c       |   2 +-
 drivers/net/mlx5/mlx5_flow_flex.c     | 116 ++++++++++++++++++++++----
 drivers/net/mlx5/mlx5_flow_hw.c       |  48 ++++++++++-
 7 files changed, 239 insertions(+), 19 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 6b98eb8c96..a6378afb10 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -293,6 +293,43 @@ mlx5dr_definer_integrity_set(struct mlx5dr_definer_fc *fc,
 	DR_SET(tag, ok1_bits, fc->byte_off, fc->bit_off, fc->bit_mask);
 }
 
+static void
+mlx5dr_definer_flex_parser_set(struct mlx5dr_definer_fc *fc,
+			       const void *item,
+			       uint8_t *tag, bool is_inner)
+{
+	const struct rte_flow_item_flex *flex = item;
+	uint32_t byte_off, val, idx;
+	int ret;
+
+	val = 0;
+	byte_off = MLX5_BYTE_OFF(definer_hl, flex_parser.flex_parser_0);
+	idx = fc->fname - MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+	byte_off -= idx * sizeof(uint32_t);
+	ret = mlx5_flex_get_parser_value_per_byte_off(flex, flex->handle, byte_off,
+						      false, is_inner, &val);
+	if (ret == -1 || !val)
+		return;
+
+	DR_SET(tag, val, fc->byte_off, 0, fc->bit_mask);
+}
+
+static void
+mlx5dr_definer_flex_parser_inner_set(struct mlx5dr_definer_fc *fc,
+				     const void *item,
+				     uint8_t *tag)
+{
+	mlx5dr_definer_flex_parser_set(fc, item, tag, true);
+}
+
+static void
+mlx5dr_definer_flex_parser_outer_set(struct mlx5dr_definer_fc *fc,
+				     const void *item,
+				     uint8_t *tag)
+{
+	mlx5dr_definer_flex_parser_set(fc, item, tag, false);
+}
+
 static void
 mlx5dr_definer_gre_key_set(struct mlx5dr_definer_fc *fc,
 			   const void *item_spec,
@@ -1465,6 +1502,47 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_flex_parser(struct mlx5dr_definer_conv_data *cd,
+				     struct rte_flow_item *item,
+				     int item_idx)
+{
+	uint32_t base_off = MLX5_BYTE_OFF(definer_hl, flex_parser.flex_parser_0);
+	const struct rte_flow_item_flex *v, *m;
+	enum mlx5dr_definer_fname fname;
+	struct mlx5dr_definer_fc *fc;
+	uint32_t i, mask, byte_off;
+	bool is_inner = cd->tunnel;
+	int ret;
+
+	m = item->mask;
+	v = item->spec;
+	mask = 0;
+	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
+		byte_off = base_off - i * sizeof(uint32_t);
+		ret = mlx5_flex_get_parser_value_per_byte_off(m, v->handle, byte_off,
+							      true, is_inner, &mask);
+		if (ret == -1) {
+			rte_errno = EINVAL;
+			return rte_errno;
+		}
+
+		if (!mask)
+			continue;
+
+		fname = MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+		fname += (enum mlx5dr_definer_fname)i;
+		fc = &cd->fc[fname];
+		fc->byte_off = byte_off;
+		fc->item_idx = item_idx;
+		fc->tag_set = cd->tunnel ? &mlx5dr_definer_flex_parser_inner_set :
+					   &mlx5dr_definer_flex_parser_outer_set;
+		fc->tag_mask_set = &mlx5dr_definer_ones_set;
+		fc->bit_mask = mask;
+	}
+	return 0;
+}
+
 static int
 mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 				struct mlx5dr_match_template *mt,
@@ -1581,6 +1659,11 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			ret = mlx5dr_definer_conv_item_meter_color(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_METER_COLOR;
 			break;
+		case RTE_FLOW_ITEM_TYPE_FLEX:
+			ret = mlx5dr_definer_conv_item_flex_parser(&cd, items, i);
+			item_flags |= cd.tunnel ? MLX5_FLOW_ITEM_INNER_FLEX :
+						  MLX5_FLOW_ITEM_OUTER_FLEX;
+			break;
 		default:
 			DR_LOG(ERR, "Unsupported item type %d", items->type);
 			rte_errno = ENOTSUP;
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 0b97c4e78d..8297129bd1 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1034,7 +1034,7 @@ static void
 mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
+	struct mlx5_ecpri_parser_profile *prf = &priv->sh->ecpri_parser;
 
 	if (prf->obj)
 		mlx5_devx_cmd_destroy(prf->obj);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 83fb316ad8..2cc3959d01 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -2243,6 +2243,12 @@ void mlx5_flex_item_port_cleanup(struct rte_eth_dev *dev);
 void mlx5_flex_flow_translate_item(struct rte_eth_dev *dev, void *matcher,
 				   void *key, const struct rte_flow_item *item,
 				   bool is_inner);
+int mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			    uint32_t idx, uint32_t *pos,
+			    bool is_inner, uint32_t *def);
+int mlx5_flex_get_parser_value_per_byte_off(const struct rte_flow_item_flex *item,
+					    void *flex, uint32_t byte_off,
+					    bool is_mask, bool tunnel, uint32_t *value);
 int mlx5_flex_acquire_index(struct rte_eth_dev *dev,
 			    struct rte_flow_item_flex_handle *handle,
 			    bool acquire);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e376dcae93..c8761c4e5a 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1225,6 +1225,7 @@ struct rte_flow_pattern_template {
 	 * tag pattern item for representor matching.
 	 */
 	bool implicit_tag;
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Flow action template struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 7ca909999b..284f18da11 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10588,7 +10588,7 @@ flow_dv_translate_item_flex(struct rte_eth_dev *dev, void *matcher, void *key,
 		(const struct rte_flow_item_flex *)item->spec;
 	int index = mlx5_flex_acquire_index(dev, spec->handle, false);
 
-	MLX5_ASSERT(index >= 0 && index <= (int)(sizeof(uint32_t) * CHAR_BIT));
+	MLX5_ASSERT(index >= 0 && index < (int)(sizeof(uint32_t) * CHAR_BIT));
 	if (index < 0)
 		return;
 	if (!(dev_flow->handle->flex_item & RTE_BIT32(index))) {
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index 24b7226ee6..aa317fc958 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -198,6 +198,99 @@ mlx5_flex_set_match_sample(void *misc4_m, void *misc4_v,
 	}
 #undef SET_FP_MATCH_SAMPLE_ID
 }
+
+/**
+ * Get the flex parser sample id and corresponding mask
+ * per shift and width information.
+ *
+ * @param[in] tp
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] idx
+ *   Mapping index.
+ * @param[in, out] pos
+ *   Where to search the value and mask.
+ * @param[in] is_inner
+ *   For inner matching or not.
+ * @param[in, def] def
+ *   Mask generated by mapping shift and width.
+ *
+ * @return
+ *   0 on success, -1 to ignore.
+ */
+int
+mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			uint32_t idx, uint32_t *pos,
+			bool is_inner, uint32_t *def)
+{
+	const struct mlx5_flex_pattern_field *map = tp->map + idx;
+	uint32_t id = map->reg_id;
+
+	*def = (RTE_BIT64(map->width) - 1) << map->shift;
+	/* Skip placeholders for DUMMY fields. */
+	if (id == MLX5_INVALID_SAMPLE_REG_ID) {
+		*pos += map->width;
+		return -1;
+	}
+	MLX5_ASSERT(map->width);
+	MLX5_ASSERT(id < tp->devx_fp->num_samples);
+	if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
+		uint32_t num_samples = tp->devx_fp->num_samples / 2;
+
+		MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
+		MLX5_ASSERT(id < num_samples);
+		id += num_samples;
+	}
+	return id;
+}
+
+/**
+ * Get the flex parser mapping value per definer format_select_dw.
+ *
+ * @param[in] item
+ *   Rte flex item pointer.
+ * @param[in] flex
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] byte_off
+ *   Mlx5 flex item format_select_dw.
+ * @param[in] is_mask
+ *   Spec or mask.
+ * @param[in] tunnel
+ *   Tunnel mode or not.
+ * @param[in, def] value
+ *   Value calculated for this flex parser, either spec or mask.
+ *
+ * @return
+ *   0 on success, -1 for error.
+ */
+int
+mlx5_flex_get_parser_value_per_byte_off(const struct rte_flow_item_flex *item,
+					void *flex, uint32_t byte_off,
+					bool is_mask, bool tunnel, uint32_t *value)
+{
+	struct mlx5_flex_pattern_field *map;
+	struct mlx5_flex_item *tp = flex;
+	uint32_t def, i, pos, val;
+	int id;
+
+	*value = 0;
+	for (i = 0, pos = 0; i < tp->mapnum && pos < item->length * CHAR_BIT; i++) {
+		map = tp->map + i;
+		id = mlx5_flex_get_sample_id(tp, i, &pos, tunnel, &def);
+		if (id == -1)
+			continue;
+		if (id >= (int)tp->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return -1;
+		if (byte_off == tp->devx_fp->sample_ids[id].format_select_dw * sizeof(uint32_t)) {
+			val = mlx5_flex_get_bitfield(item, pos, map->width, map->shift);
+			if (is_mask)
+				val &= RTE_BE32(def);
+			*value |= val;
+		}
+		pos += map->width;
+	}
+	return 0;
+}
+
 /**
  * Translate item pattern into matcher fields according to translation
  * array.
@@ -240,26 +333,17 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
-		uint32_t id = map->reg_id;
-		uint32_t def = (RTE_BIT64(map->width) - 1) << map->shift;
-		uint32_t val, msk;
+		uint32_t val, msk, def;
+		int id = mlx5_flex_get_sample_id(tp, i, &pos, is_inner, &def);
 
-		/* Skip placeholders for DUMMY fields. */
-		if (id == MLX5_INVALID_SAMPLE_REG_ID) {
-			pos += map->width;
+		if (id == -1)
 			continue;
-		}
+		MLX5_ASSERT(id < (int)tp->devx_fp->num_samples);
+		if (id >= (int)tp->devx_fp->num_samples ||
+		    id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
 		val = mlx5_flex_get_bitfield(spec, pos, map->width, map->shift);
 		msk = mlx5_flex_get_bitfield(mask, pos, map->width, map->shift);
-		MLX5_ASSERT(map->width);
-		MLX5_ASSERT(id < tp->devx_fp->num_samples);
-		if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
-			uint32_t num_samples = tp->devx_fp->num_samples / 2;
-
-			MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
-			MLX5_ASSERT(id < num_samples);
-			id += num_samples;
-		}
 		if (attr->ext_sample_id)
 			sample_id = tp->devx_fp->sample_ids[id].sample_id;
 		else
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 9406360a10..c1d4138116 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4335,6 +4335,36 @@ flow_hw_set_vlan_vid_construct(struct rte_eth_dev *dev,
 					      &modify_action);
 }
 
+static int
+flow_hw_flex_item_acquire(struct rte_eth_dev *dev,
+			  struct rte_flow_item_flex_handle *handle,
+			  uint8_t *flex_item)
+{
+	int index = mlx5_flex_acquire_index(dev, handle, false);
+
+	MLX5_ASSERT(index >= 0 && index < (int)(sizeof(uint32_t) * CHAR_BIT));
+	if (index < 0)
+		return -1;
+	if (!(*flex_item & RTE_BIT32(index))) {
+		/* Don't count same flex item again. */
+		if (mlx5_flex_acquire_index(dev, handle, true) != index)
+			MLX5_ASSERT(false);
+		*flex_item |= (uint8_t)RTE_BIT32(index);
+	}
+	return 0;
+}
+
+static void
+flow_hw_flex_item_release(struct rte_eth_dev *dev, uint8_t *flex_item)
+{
+	while (*flex_item) {
+		int index = rte_bsf32(*flex_item);
+
+		mlx5_flex_release_index(dev, index);
+		*flex_item &= ~(uint8_t)RTE_BIT32(index);
+	}
+}
+
 /**
  * Create flow action template.
  *
@@ -4736,6 +4766,7 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_ICMP:
 		case RTE_FLOW_ITEM_TYPE_ICMP6:
 		case RTE_FLOW_ITEM_TYPE_CONNTRACK:
+		case RTE_FLOW_ITEM_TYPE_FLEX:
 			break;
 		case RTE_FLOW_ITEM_TYPE_INTEGRITY:
 			/*
@@ -4813,6 +4844,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 		.mask = &tag_m,
 		.last = NULL
 	};
+	unsigned int i = 0;
 
 	if (flow_hw_pattern_validate(dev, attr, items, error))
 		return NULL;
@@ -4872,6 +4904,19 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 			it->implicit_tag = true;
 		mlx5_free(copied_items);
 	}
+	for (i = 0; items[i].type != RTE_FLOW_ITEM_TYPE_END; ++i) {
+		if (items[i].type == RTE_FLOW_ITEM_TYPE_FLEX) {
+			const struct rte_flow_item_flex *spec =
+				(const struct rte_flow_item_flex *)items[i].spec;
+			struct rte_flow_item_flex_handle *handle = spec->handle;
+
+			if (flow_hw_flex_item_acquire(dev, handle, &it->flex_item)) {
+				claim_zero(mlx5dr_match_template_destroy(it->mt));
+				mlx5_free(it);
+				return NULL;
+			}
+		}
+	}
 	__atomic_fetch_add(&it->refcnt, 1, __ATOMIC_RELAXED);
 	LIST_INSERT_HEAD(&priv->flow_hw_itt, it, next);
 	return it;
@@ -4891,7 +4936,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 			      struct rte_flow_pattern_template *template,
 			      struct rte_flow_error *error __rte_unused)
 {
@@ -4904,6 +4949,7 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "item template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	claim_zero(mlx5dr_match_template_destroy(template->mt));
 	mlx5_free(template);
 	return 0;
-- 
2.27.0

[PATCH v3 07/11] net/mlx5: add flex item modify field implementation

[Rongwei Liu]

Add flex item modify field HWS implementation.
The minimum modify boundary is one byte.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h  |   1 +
 drivers/net/mlx5/mlx5_flow.h    |   3 +
 drivers/net/mlx5/mlx5_flow_dv.c | 165 +++++++++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c |  14 ++-
 4 files changed, 170 insertions(+), 13 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index b32dc735a1..bb3452f74c 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -760,6 +760,7 @@ enum mlx5_modification_field {
 	MLX5_MODI_TUNNEL_HDR_DW_1 = 0x75,
 	MLX5_MODI_GTPU_FIRST_EXT_DW_0 = 0x76,
 	MLX5_MODI_HASH_RESULT = 0x81,
+	MLX5_MODI_INVALID = INT_MAX,
 };
 
 /* Total number of metadata reg_c's. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index c8761c4e5a..c71fa1c0ad 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1080,6 +1080,8 @@ struct field_modify_info {
 	uint32_t size; /* Size of field in protocol header, in bytes. */
 	uint32_t offset; /* Offset of field in protocol header, in bytes. */
 	enum mlx5_modification_field id;
+	uint32_t shift;
+	uint8_t is_flex; /* Temporary indicator for flex item modify filed WA. */
 };
 
 /* HW steering flow attributes. */
@@ -1244,6 +1246,7 @@ struct rte_flow_actions_template {
 	uint16_t mhdr_off; /* Offset of DR modify header action. */
 	uint32_t refcnt; /* Reference counter. */
 	uint16_t rx_cpy_pos; /* Action position of Rx metadata to be copied. */
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Jump action struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 284f18da11..92a5914d4b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -414,10 +414,15 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			++field;
 			continue;
 		}
-		/* Deduce actual data width in bits from mask value. */
-		off_b = rte_bsf32(mask) + carry_b;
-		size_b = sizeof(uint32_t) * CHAR_BIT -
-			 off_b - __builtin_clz(mask);
+		if (type == MLX5_MODIFICATION_TYPE_COPY && field->is_flex) {
+			off_b = 32 - field->shift + carry_b - field->size * CHAR_BIT;
+			size_b = field->size * CHAR_BIT - carry_b;
+		} else {
+			/* Deduce actual data width in bits from mask value. */
+			off_b = rte_bsf32(mask) + carry_b;
+			size_b = sizeof(uint32_t) * CHAR_BIT -
+				 off_b - __builtin_clz(mask);
+		}
 		MLX5_ASSERT(size_b);
 		actions[i] = (struct mlx5_modification_cmd) {
 			.action_type = type,
@@ -437,40 +442,46 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			 * Destination field overflow. Copy leftovers of
 			 * a source field to the next destination field.
 			 */
-			carry_b = 0;
 			if ((size_b > dcopy->size * CHAR_BIT - dcopy->offset) &&
 			    dcopy->size != 0) {
 				actions[i].length =
 					dcopy->size * CHAR_BIT - dcopy->offset;
-				carry_b = actions[i].length;
+				carry_b += actions[i].length;
 				next_field = false;
+			} else {
+				carry_b = 0;
 			}
 			/*
 			 * Not enough bits in a source filed to fill a
 			 * destination field. Switch to the next source.
 			 */
 			if ((size_b < dcopy->size * CHAR_BIT - dcopy->offset) &&
-			    (size_b == field->size * CHAR_BIT - off_b)) {
-				actions[i].length =
-					field->size * CHAR_BIT - off_b;
+			    ((size_b == field->size * CHAR_BIT - off_b) ||
+			     field->is_flex)) {
+				actions[i].length = size_b;
 				dcopy->offset += actions[i].length;
 				next_dcopy = false;
 			}
-			if (next_dcopy)
-				++dcopy;
 		} else {
 			MLX5_ASSERT(item->spec);
 			data = flow_dv_fetch_field((const uint8_t *)item->spec +
 						   field->offset, field->size);
 			/* Shift out the trailing masked bits from data. */
 			data = (data & mask) >> off_b;
+			if (field->is_flex)
+				actions[i].offset = 32 - field->shift - field->size * CHAR_BIT;
 			actions[i].data1 = rte_cpu_to_be_32(data);
 		}
 		/* Convert entire record to expected big-endian format. */
 		actions[i].data0 = rte_cpu_to_be_32(actions[i].data0);
+		if ((type != MLX5_MODIFICATION_TYPE_COPY ||
+		     dcopy->id != (enum mlx5_modification_field)UINT32_MAX) &&
+		    field->id != (enum mlx5_modification_field)UINT32_MAX)
+			++i;
+		if (next_dcopy && type == MLX5_MODIFICATION_TYPE_COPY)
+			++dcopy;
 		if (next_field)
 			++field;
-		++i;
 	} while (field->size);
 	if (resource->actions_num == i)
 		return rte_flow_error_set(error, EINVAL,
@@ -1421,6 +1432,131 @@ flow_modify_info_mask_32_masked(uint32_t length, uint32_t off, uint32_t post_mas
 	return rte_cpu_to_be_32(mask & post_mask);
 }
 
+static void
+mlx5_modify_flex_item(const struct rte_eth_dev *dev,
+		      const struct mlx5_flex_item *flex,
+		      const struct rte_flow_action_modify_data *data,
+		      struct field_modify_info *info,
+		      uint32_t *mask, uint32_t width)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
+	uint32_t i, j;
+	int id = 0;
+	uint32_t pos = 0;
+	const struct mlx5_flex_pattern_field *map;
+	uint32_t offset = data->offset;
+	uint32_t width_left = width;
+	uint32_t def;
+	uint32_t cur_width = 0;
+	uint32_t tmp_ofs;
+	uint32_t idx = 0;
+	struct field_modify_info tmp;
+	int tmp_id;
+
+	if (!attr->ext_sample_id) {
+		DRV_LOG(ERR, "FW doesn't support modify field with flex item.");
+		return;
+	}
+	/*
+	 * search for the mapping instance until Accumulated width is no
+	 * less than data->offset.
+	 */
+	for (i = 0; i < flex->mapnum; i++) {
+		if (flex->map[i].width + pos > data->offset)
+			break;
+		pos += flex->map[i].width;
+	}
+	if (i >= flex->mapnum)
+		return;
+	tmp_ofs = pos < data->offset ? data->offset - pos : 0;
+	for (j = i; i < flex->mapnum && width_left > 0; ) {
+		map = flex->map + i;
+		id = mlx5_flex_get_sample_id(flex, i, &pos, false, &def);
+		if (id == -1) {
+			i++;
+			/* All left length is dummy */
+			if (pos >= data->offset + width)
+				return;
+			cur_width = map->width;
+		/* One mapping instance covers the whole width. */
+		} else if (pos + map->width >= (data->offset + width)) {
+			cur_width = width_left;
+		} else {
+			cur_width = cur_width + map->width - tmp_ofs;
+			pos += map->width;
+			/*
+			 * Continue to search next until:
+			 * 1. Another flex parser ID.
+			 * 2. Width has been covered.
+			 */
+			for (j = i + 1; j < flex->mapnum; j++) {
+				tmp_id = mlx5_flex_get_sample_id(flex, j, &pos, false, &def);
+				if (tmp_id == -1) {
+					i = j;
+					pos -= flex->map[j].width;
+					break;
+				}
+				if (id >= (int)flex->devx_fp->num_samples ||
+				    id >= MLX5_GRAPH_NODE_SAMPLE_NUM ||
+				    tmp_id >= (int)flex->devx_fp->num_samples ||
+				    tmp_id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+					return;
+				if (flex->devx_fp->sample_ids[id].id !=
+						flex->devx_fp->sample_ids[tmp_id].id ||
+				    flex->map[j].shift != flex->map[j - 1].width +
+							  flex->map[j - 1].shift) {
+					i = j;
+					break;
+				}
+				if ((pos + flex->map[j].width) >= (data->offset + width)) {
+					cur_width = width_left;
+					break;
+				}
+				pos += flex->map[j].width;
+				cur_width += flex->map[j].width;
+			}
+		}
+		if (cur_width > width_left)
+			cur_width = width_left;
+		else if (cur_width < width_left && (j == flex->mapnum || i == flex->mapnum))
+			return;
+
+		MLX5_ASSERT(id < (int)flex->devx_fp->num_samples);
+		if (id >= (int)flex->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
+		/* Use invalid entry as placeholder for DUMMY mapping. */
+		info[idx] = (struct field_modify_info){cur_width / CHAR_BIT, offset / CHAR_BIT,
+			     id == -1 ? MLX5_MODI_INVALID :
+			     (enum mlx5_modification_field)
+			     flex->devx_fp->sample_ids[id].modify_field_id,
+			     map->shift + tmp_ofs, 1};
+		offset += cur_width;
+		width_left -= cur_width;
+		if (!mask) {
+			info[idx].offset = (32 - cur_width - map->shift - tmp_ofs);
+			info[idx].size = cur_width / CHAR_BIT + info[idx].offset / CHAR_BIT;
+		}
+		cur_width = 0;
+		tmp_ofs = 0;
+		idx++;
+	}
+	if (unlikely(width_left > 0)) {
+		MLX5_ASSERT(false);
+		return;
+	}
+	if (mask)
+		memset(mask, 0xff, data->offset / CHAR_BIT + width / CHAR_BIT);
+	/* Re-order the info to follow IPv6 address. */
+	for (i = 0; i < idx / 2; i++) {
+		tmp = info[i];
+		MLX5_ASSERT(info[i].id);
+		MLX5_ASSERT(info[idx - 1 - i].id);
+		info[i] = info[idx - 1 - i];
+		info[idx - 1 - i] = tmp;
+	}
+}
+
 void
 mlx5_flow_field_id_to_modify_info
 		(const struct rte_flow_action_modify_data *data,
@@ -1883,6 +2019,11 @@ mlx5_flow_field_id_to_modify_info
 				info[idx].offset = data->offset;
 		}
 		break;
+	case RTE_FLOW_FIELD_FLEX_ITEM:
+		MLX5_ASSERT(data->flex_handle != NULL && !(data->offset & 0x7));
+		mlx5_modify_flex_item(dev, (const struct mlx5_flex_item *)data->flex_handle,
+				      data, info, mask, width);
+		break;
 	case RTE_FLOW_FIELD_POINTER:
 	case RTE_FLOW_FIELD_VALUE:
 	default:
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index c1d4138116..06b302209b 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4554,6 +4554,17 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
 			at->actions[i].conf = actions->conf;
 			at->masks[i].conf = masks->conf;
 		}
+		if (actions->type == RTE_FLOW_ACTION_TYPE_MODIFY_FIELD) {
+			const struct rte_flow_action_modify_field *info = actions->conf;
+
+			if ((info->dst.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			     flow_hw_flex_item_acquire(dev, info->dst.flex_handle,
+						       &at->flex_item)) ||
+			     (info->src.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			      flow_hw_flex_item_acquire(dev, info->src.flex_handle,
+							&at->flex_item)))
+				goto error;
+		}
 	}
 	at->tmpl = flow_hw_dr_actions_template_create(at);
 	if (!at->tmpl)
@@ -4585,7 +4596,7 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_actions_template_destroy(struct rte_eth_dev *dev,
 				 struct rte_flow_actions_template *template,
 				 struct rte_flow_error *error __rte_unused)
 {
@@ -4598,6 +4609,7 @@ flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "action template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	if (template->tmpl)
 		mlx5dr_action_template_destroy(template->tmpl);
 	mlx5_free(template);
-- 
2.27.0

[PATCH v3 08/11] net/mlx5: return error for sws modify field

[Rongwei Liu]

Return unsupported error message when application tries to
modify flex item field.

Validation of packet modifications actions for SW Steering checked
if either source or destination field of MODIFY_FIELD action
was a flex item.
When DEC_TTL action is used, DEC_TTL action does not have any
action configuration and dereferencing source or destination field
is invalid, so validation of source and destination field types
should be moved to MODIFY_FIELD specific validation function, then
field types are validated if and only if action type is MODIFY_FIELD.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 92a5914d4b..a7c0d5bf17 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4828,6 +4828,7 @@ flow_dv_validate_action_modify_hdr(const uint64_t action_flags,
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION_CONF,
 					  NULL, "action configuration not set");
+
 	if (action_flags & MLX5_FLOW_ACTION_ENCAP)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION, NULL,
@@ -5153,17 +5154,21 @@ flow_dv_validate_action_modify_field(struct rte_eth_dev *dev,
 	struct mlx5_hca_attr *hca_attr = &priv->sh->cdev->config.hca_attr;
 	const struct rte_flow_action_modify_field *action_modify_field =
 		action->conf;
-	uint32_t dst_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->dst.field,
-				-1, attr, error);
-	uint32_t src_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->src.field,
-				dst_width, attr, error);
+	uint32_t dst_width, src_width;
 
 	ret = flow_dv_validate_action_modify_hdr(action_flags, action, error);
 	if (ret)
 		return ret;
-
+	if (action_modify_field->src.field == RTE_FLOW_FIELD_FLEX_ITEM ||
+	    action_modify_field->dst.field == RTE_FLOW_FIELD_FLEX_ITEM)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"flex item fields modification"
+				" is not supported");
+	dst_width = mlx5_flow_item_field_width(dev, action_modify_field->dst.field,
+					       -1, attr, error);
+	src_width = mlx5_flow_item_field_width(dev, action_modify_field->src.field,
+					       dst_width, attr, error);
 	if (action_modify_field->width == 0)
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
-- 
2.27.0

[PATCH v3 09/11] app/testpmd: raw encap with flex item support

[Rongwei Liu]

Application should retrieve raw_encap buffer from
spec->pattern if it is flex item.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 9037432cc8..3b2975ee83 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -11193,6 +11193,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 	uint16_t proto = 0;
 	uint16_t idx = in->port; /* We borrow port field as index */
 	int gtp_psc = -1; /* GTP PSC option index. */
+	const void *src_spec;
 
 	if (in->command == SET_SAMPLE_ACTIONS)
 		return cmd_set_raw_parsed_sample(in);
@@ -11216,6 +11217,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 		item = in->args.vc.pattern + i;
 		if (item->spec == NULL)
 			item->spec = flow_item_default_mask(item);
+		src_spec = item->spec;
 		switch (item->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			size = sizeof(struct rte_ether_hdr);
@@ -11343,9 +11345,13 @@ cmd_set_raw_parsed(const struct buffer *in)
 			size = sizeof(struct rte_flow_item_pfcp);
 			break;
 		case RTE_FLOW_ITEM_TYPE_FLEX:
-			size = item->spec ?
-				((const struct rte_flow_item_flex *)
-				item->spec)->length : 0;
+			if (item->spec != NULL) {
+				size = ((const struct rte_flow_item_flex *)item->spec)->length;
+				src_spec = ((const struct rte_flow_item_flex *)item->spec)->pattern;
+			} else {
+				size = 0;
+				src_spec = NULL;
+			}
 			break;
 		case RTE_FLOW_ITEM_TYPE_GRE_OPTION:
 			size = 0;
@@ -11378,12 +11384,14 @@ cmd_set_raw_parsed(const struct buffer *in)
 			fprintf(stderr, "Error - Not supported item\n");
 			goto error;
 		}
-		*total_size += size;
-		rte_memcpy(data_tail - (*total_size), item->spec, size);
-		/* update some fields which cannot be set by cmdline */
-		update_fields((data_tail - (*total_size)), item,
-			      upper_layer);
-		upper_layer = proto;
+		if (size) {
+			*total_size += size;
+			rte_memcpy(data_tail - (*total_size), src_spec, size);
+			/* update some fields which cannot be set by cmdline */
+			update_fields((data_tail - (*total_size)), item,
+				      upper_layer);
+			upper_layer = proto;
+		}
 	}
 	if (verbose_level & 0x1)
 		printf("total data size is %zu\n", (*total_size));
-- 
2.27.0

[PATCH v3 10/11] doc/mlx5: update mlx5 doc

[Rongwei Liu]

Add flex item matching and modify field feature into
mlx5 documents.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 doc/guides/nics/mlx5.rst | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index f137f156f9..aceeeb462c 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -106,6 +106,7 @@ Features
 - Sub-Function representors.
 - Sub-Function.
 - Matching on represented port.
+- Modify flex item field.
 
 
 Limitations
-- 
2.27.0

[PATCH v3 11/11] app/testpmd: adjust cleanup sequence when quitting

[Rongwei Liu]

If flex item is referenced in async flow either by
pattern template or action template, currently testpmd
complains "flex item has flow references". Flex items should
be flushed after async flow resources cleanup.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/testpmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 134d79a555..e35f7a0e7a 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -3298,10 +3298,10 @@ flush_port_owned_resources(portid_t pi)
 {
 	mcast_addr_pool_destroy(pi);
 	port_flow_flush(pi);
-	port_flex_item_flush(pi);
 	port_flow_template_table_flush(pi);
 	port_flow_pattern_template_flush(pi);
 	port_flow_actions_template_flush(pi);
+	port_flex_item_flush(pi);
 	port_action_handle_flush(pi);
 }
 
-- 
2.27.0

Re: [PATCH v3 11/11] app/testpmd: adjust cleanup sequence when quitting

[Singh, Aman Deep]

On 1/30/2023 6:50 PM, Rongwei Liu wrote:
> If flex item is referenced in async flow either by
> pattern template or action template, currently testpmd
> complains "flex item has flow references". Flex items should
> be flushed after async flow resources cleanup.
>
> Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
> Acked-by: Ori Kam <orika@nvidia.com>

Acked-by: Aman Singh <aman.deep.singh@intel.com>

> ---
>   app/test-pmd/testpmd.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index 134d79a555..e35f7a0e7a 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -3298,10 +3298,10 @@ flush_port_owned_resources(portid_t pi)
>   {
>   	mcast_addr_pool_destroy(pi);
>   	port_flow_flush(pi);
> -	port_flex_item_flush(pi);
>   	port_flow_template_table_flush(pi);
>   	port_flow_pattern_template_flush(pi);
>   	port_flow_actions_template_flush(pi);
> +	port_flex_item_flush(pi);
>   	port_action_handle_flush(pi);
>   }
>   

[PATCH v2 02/11] app/testpmd: pass flex handle into matching mask

[Rongwei Liu]

In async flow create API, there is only mask information when
creating flow table but flex item handle is required to parse
the HW sample information.

Pass the flex item handle instead of UINT64/32_MAX to mask.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 323c07253d..f5d7a67def 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -10068,8 +10068,8 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	}
 	if (offset == offsetof(struct rte_flow_item_flex, handle)) {
 		const struct flex_item *fp;
-		struct rte_flow_item_flex *item_flex = ctx->object;
-		handle = (uint16_t)(uintptr_t)item_flex->handle;
+		spec = ctx->object;
+		handle = (uint16_t)(uintptr_t)spec->handle;
 		if (handle >= FLEX_MAX_PARSERS_NUM) {
 			printf("Bad flex item handle\n");
 			return -1;
@@ -10079,7 +10079,9 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 			printf("Bad flex item handle\n");
 			return -1;
 		}
-		item_flex->handle = fp->flex_handle;
+		spec->handle = fp->flex_handle;
+		mask = spec + 2; /* spec, last, mask */
+		mask->handle = fp->flex_handle;
 	} else if (offset == offsetof(struct rte_flow_item_flex, pattern)) {
 		handle = (uint16_t)(uintptr_t)
 			((struct rte_flow_item_flex *)ctx->object)->pattern;
-- 
2.27.0

[PATCH v2 03/11] net/mlx5: enable hws flex item create

[Rongwei Liu]

Enable flex item create and destroy with dv_flow_en=2

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 27 +++++++++++++++------------
 drivers/net/mlx5/mlx5_flow_hw.c  |  2 ++
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a71474c90a..f5b3edea99 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -474,10 +474,20 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 	err = mlx5_alloc_table_hash_list(priv);
 	if (err)
 		goto error;
-	if (priv->sh->config.dv_flow_en == 2)
-		return 0;
 	/* The resources below are only valid with DV support. */
 #ifdef HAVE_IBV_FLOW_DV_SUPPORT
+	/* Init shared flex parsers list, no need lcore_share */
+	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
+	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
+					       mlx5_flex_parser_create_cb,
+					       mlx5_flex_parser_match_cb,
+					       mlx5_flex_parser_remove_cb,
+					       mlx5_flex_parser_clone_cb,
+					       mlx5_flex_parser_clone_free_cb);
+	if (!sh->flex_parsers_dv)
+		goto error;
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 	/* Init port id action list. */
 	snprintf(s, sizeof(s), "%s_port_id_action_list", sh->ibdev_name);
 	sh->port_id_action_list = mlx5_list_create(s, sh, true,
@@ -518,16 +528,9 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 					      flow_dv_dest_array_clone_free_cb);
 	if (!sh->dest_array_list)
 		goto error;
-	/* Init shared flex parsers list, no need lcore_share */
-	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
-	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
-					       mlx5_flex_parser_create_cb,
-					       mlx5_flex_parser_match_cb,
-					       mlx5_flex_parser_remove_cb,
-					       mlx5_flex_parser_clone_cb,
-					       mlx5_flex_parser_clone_free_cb);
-	if (!sh->flex_parsers_dv)
-		goto error;
+#else
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 #endif
 #ifdef HAVE_MLX5DV_DR
 	void *domain;
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 20c71ff7f0..44953451d5 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -8336,6 +8336,8 @@ const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops = {
 	.query = flow_hw_query,
 	.get_aged_flows = flow_hw_get_aged_flows,
 	.get_q_aged_flows = flow_hw_get_q_aged_flows,
+	.item_create = flow_dv_item_create,
+	.item_release = flow_dv_item_release,
 };
 
 /**
-- 
2.27.0

[PATCH v2 04/11] net/mlx5: add IPv6 protocol as flex item input

[Rongwei Liu]

Support IPv6 protocol as new flex item input link.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_flex.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index fb08910ddb..bec07b13c1 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -1043,6 +1043,22 @@ mlx5_flex_arc_in_udp(const struct rte_flow_item *item,
 	return rte_be_to_cpu_16(spec->hdr.dst_port);
 }
 
+static int
+mlx5_flex_arc_in_ipv6(const struct rte_flow_item *item,
+		      struct rte_flow_error *error)
+{
+	const struct rte_flow_item_ipv6 *spec = item->spec;
+	const struct rte_flow_item_ipv6 *mask = item->mask;
+	struct rte_flow_item_ipv6 ip = { .hdr.proto = 0xff };
+
+	if (memcmp(mask, &ip, sizeof(struct rte_flow_item_ipv6))) {
+		return rte_flow_error_set
+			(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item,
+			 "invalid ipv6 item mask, full mask is desired");
+	}
+	return spec->hdr.proto;
+}
+
 static int
 mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 			   const struct rte_flow_item_flex_conf *conf,
@@ -1089,6 +1105,9 @@ mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 		case RTE_FLOW_ITEM_TYPE_UDP:
 			ret = mlx5_flex_arc_in_udp(rte_item, error);
 			break;
+		case RTE_FLOW_ITEM_TYPE_IPV6:
+			ret = mlx5_flex_arc_in_ipv6(rte_item, error);
+			break;
 		default:
 			MLX5_ASSERT(false);
 			return rte_flow_error_set
-- 
2.27.0

[PATCH v2 05/11] net/mlx5: adopt new flex item prm definition

[Rongwei Liu]

Per newest PRM definition, sample_id stands for 3 parts
of information instead of single uint32_t id: sample_id +
modify_filed_id + format_select_dw.

Also new FW capability bits have been introduces to identify
the new capability.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 14 +++++++++++---
 drivers/common/mlx5/mlx5_devx_cmds.h |  7 ++++++-
 drivers/common/mlx5/mlx5_prm.h       | 28 ++++++++++++++++++++++++++--
 drivers/net/mlx5/mlx5.c              | 15 +++++++++++----
 drivers/net/mlx5/mlx5.h              |  3 ++-
 drivers/net/mlx5/mlx5_flow_flex.c    | 14 +++++++++++---
 6 files changed, 67 insertions(+), 14 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index e3a4927d0f..1f65ea7dcb 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -607,7 +607,8 @@ mlx5_devx_cmd_query_hca_vdpa_attr(void *ctx,
 
 int
 mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				  uint32_t ids[], uint32_t num)
+				  struct mlx5_ext_sample_id ids[],
+				  uint32_t num, uint8_t *anchor)
 {
 	uint32_t in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {0};
 	uint32_t out[MLX5_ST_SZ_DW(create_flex_parser_out)] = {0};
@@ -636,6 +637,7 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			(void *)flex_obj);
 		return -rte_errno;
 	}
+	*anchor = MLX5_GET(parse_graph_flex, flex, head_anchor_id);
 	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
 		void *s_off = (void *)((char *)sample + i *
 			      MLX5_ST_SZ_BYTES(parse_graph_flow_match_sample));
@@ -645,8 +647,8 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			      flow_match_sample_en);
 		if (!en)
 			continue;
-		ids[idx++] = MLX5_GET(parse_graph_flow_match_sample, s_off,
-				  flow_match_sample_field_id);
+		ids[idx++].id = MLX5_GET(parse_graph_flow_match_sample, s_off,
+					 flow_match_sample_field_id);
 	}
 	if (num != idx) {
 		rte_errno = EINVAL;
@@ -794,6 +796,12 @@ mlx5_devx_cmd_query_hca_parse_graph_node_cap
 					 max_num_arc_out);
 	attr->max_num_sample = MLX5_GET(parse_graph_node_cap, hcattr,
 					max_num_sample);
+	attr->anchor_en = MLX5_GET(parse_graph_node_cap, hcattr, anchor_en);
+	attr->ext_sample_id = MLX5_GET(parse_graph_node_cap, hcattr, ext_sample_id);
+	attr->sample_tunnel_inner2 = MLX5_GET(parse_graph_node_cap, hcattr,
+					      sample_tunnel_inner2);
+	attr->zero_size_supported = MLX5_GET(parse_graph_node_cap, hcattr,
+					     zero_size_supported);
 	attr->sample_id_in_out = MLX5_GET(parse_graph_node_cap, hcattr,
 					  sample_id_in_out);
 	attr->max_base_header_length = MLX5_GET(parse_graph_node_cap, hcattr,
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index c94b9eac06..5b33010155 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -114,6 +114,10 @@ struct mlx5_hca_flex_attr {
 	uint8_t  max_num_arc_out;
 	uint8_t  max_num_sample;
 	uint8_t  max_num_prog_sample:5;	/* From HCA CAP 2 */
+	uint8_t  anchor_en:1;
+	uint8_t  ext_sample_id:1;
+	uint8_t  sample_tunnel_inner2:1;
+	uint8_t  zero_size_supported:1;
 	uint8_t  sample_id_in_out:1;
 	uint16_t max_base_header_length;
 	uint8_t  max_sample_base_offset;
@@ -706,7 +710,8 @@ int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir,
 			     struct mlx5_devx_modify_tir_attr *tir_attr);
 __rte_internal
 int mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				      uint32_t ids[], uint32_t num);
+				      struct mlx5_ext_sample_id ids[],
+				      uint32_t num, uint8_t *anchor);
 
 __rte_internal
 struct mlx5_devx_obj *
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 3790dc84b8..ce6cd98fd7 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1893,7 +1893,11 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 max_num_arc_in[0x08];
 	u8 max_num_arc_out[0x08];
 	u8 max_num_sample[0x08];
-	u8 reserved_at_78[0x07];
+	u8 reserved_at_78[0x03];
+	u8 anchor_en[0x1];
+	u8 ext_sample_id[0x1];
+	u8 sample_tunnel_inner2[0x1];
+	u8 zero_size_supported[0x1];
 	u8 sample_id_in_out[0x1];
 	u8 max_base_header_length[0x10];
 	u8 reserved_at_90[0x08];
@@ -1903,6 +1907,24 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 header_length_mask_width[0x08];
 };
 
+/* ext_sample_id structure, see PRM Table: Flow Match Sample ID Format. */
+struct mlx5_ext_sample_id {
+	union {
+		struct {
+#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN
+			uint32_t format_select_dw:8;
+			uint32_t modify_field_id:12;
+			uint32_t sample_id:12;
+#else
+			uint32_t sample_id:12;
+			uint32_t modify_field_id:12;
+			uint32_t format_select_dw:8;
+#endif
+		};
+		uint32_t id;
+	};
+};
+
 struct mlx5_ifc_flow_table_prop_layout_bits {
 	u8 ft_support[0x1];
 	u8 flow_tag[0x1];
@@ -4522,7 +4544,9 @@ struct mlx5_ifc_parse_graph_flex_bits {
 	u8 header_length_mode[0x4];
 	u8 header_length_field_offset[0x10];
 	u8 next_header_field_offset[0x10];
-	u8 reserved_at_160[0x1b];
+	u8 reserved_at_160[0x12];
+	u8 head_anchor_id[0x6];
+	u8 reserved_at_178[0x3];
 	u8 next_header_field_size[0x5];
 	u8 header_length_field_mask[0x20];
 	u8 reserved_at_224[0x20];
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index b8643cebdd..0b97c4e78d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -964,11 +964,13 @@ int
 mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
 	struct mlx5_devx_graph_node_attr node = {
 		.modify_field_select = 0,
 	};
-	uint32_t ids[8];
+	struct mlx5_ext_sample_id ids[8];
+	uint8_t anchor_id;
 	int ret;
 
 	if (!priv->sh->cdev->config.hca_attr.parse_graph_flex_node) {
@@ -1004,15 +1006,20 @@ mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->num = 2;
-	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num);
+	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num, &anchor_id);
 	if (ret) {
 		DRV_LOG(ERR, "Failed to query sample IDs.");
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->offset[0] = 0x0;
 	prf->offset[1] = sizeof(uint32_t);
-	prf->ids[0] = ids[0];
-	prf->ids[1] = ids[1];
+	if (attr->ext_sample_id) {
+		prf->ids[0] = ids[0].sample_id;
+		prf->ids[1] = ids[1].sample_id;
+	} else {
+		prf->ids[0] = ids[0].id;
+		prf->ids[1] = ids[1].id;
+	}
 	return 0;
 }
 
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 16b33e1548..83fb316ad8 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1307,9 +1307,10 @@ struct mlx5_lag {
 struct mlx5_flex_parser_devx {
 	struct mlx5_list_entry entry;  /* List element at the beginning. */
 	uint32_t num_samples;
+	uint8_t anchor_id;
 	void *devx_obj;
 	struct mlx5_devx_graph_node_attr devx_conf;
-	uint32_t sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
+	struct mlx5_ext_sample_id sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
 };
 
 /* Pattern field descriptor - how to translate flex pattern into samples. */
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index bec07b13c1..24b7226ee6 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -226,15 +226,18 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	void *misc4_m = MLX5_ADDR_OF(fte_match_param, matcher,
 				     misc_parameters_4);
 	void *misc4_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_4);
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_flex_item *tp;
 	uint32_t i, pos = 0;
+	uint32_t sample_id;
 
 	RTE_SET_USED(dev);
 	MLX5_ASSERT(item->spec && item->mask);
 	spec = item->spec;
 	mask = item->mask;
 	tp = (struct mlx5_flex_item *)spec->handle;
-	MLX5_ASSERT(mlx5_flex_index(dev->data->dev_private, tp) >= 0);
+	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
 		uint32_t id = map->reg_id;
@@ -257,9 +260,13 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 			MLX5_ASSERT(id < num_samples);
 			id += num_samples;
 		}
+		if (attr->ext_sample_id)
+			sample_id = tp->devx_fp->sample_ids[id].sample_id;
+		else
+			sample_id = tp->devx_fp->sample_ids[id].id;
 		mlx5_flex_set_match_sample(misc4_m, misc4_v,
 					   def, msk & def, val & msk & def,
-					   tp->devx_fp->sample_ids[id], id);
+					   sample_id, id);
 		pos += map->width;
 	}
 }
@@ -1317,7 +1324,8 @@ mlx5_flex_parser_create_cb(void *list_ctx, void *ctx)
 	/* Query the firmware assigned sample ids. */
 	ret = mlx5_devx_cmd_query_parse_samples(fp->devx_obj,
 						fp->sample_ids,
-						fp->num_samples);
+						fp->num_samples,
+						&fp->anchor_id);
 	if (ret)
 		goto error;
 	DRV_LOG(DEBUG, "DEVx flex parser %p created, samples num: %u",
-- 
2.27.0

[PATCH v2 06/11] net/mlx5/hws: add hws flex item matching support

[Rongwei Liu]

Support flex item matching in hws and syntax follows
sws exactly.

Flex item should be created in advance and follow current
json mapping logic.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c |  83 ++++++++++++++++++
 drivers/net/mlx5/mlx5.c               |   2 +-
 drivers/net/mlx5/mlx5.h               |   6 ++
 drivers/net/mlx5/mlx5_flow.h          |   1 +
 drivers/net/mlx5/mlx5_flow_dv.c       |   2 +-
 drivers/net/mlx5/mlx5_flow_flex.c     | 116 ++++++++++++++++++++++----
 drivers/net/mlx5/mlx5_flow_hw.c       |  48 ++++++++++-
 7 files changed, 239 insertions(+), 19 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 6b98eb8c96..a6378afb10 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -293,6 +293,43 @@ mlx5dr_definer_integrity_set(struct mlx5dr_definer_fc *fc,
 	DR_SET(tag, ok1_bits, fc->byte_off, fc->bit_off, fc->bit_mask);
 }
 
+static void
+mlx5dr_definer_flex_parser_set(struct mlx5dr_definer_fc *fc,
+			       const void *item,
+			       uint8_t *tag, bool is_inner)
+{
+	const struct rte_flow_item_flex *flex = item;
+	uint32_t byte_off, val, idx;
+	int ret;
+
+	val = 0;
+	byte_off = MLX5_BYTE_OFF(definer_hl, flex_parser.flex_parser_0);
+	idx = fc->fname - MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+	byte_off -= idx * sizeof(uint32_t);
+	ret = mlx5_flex_get_parser_value_per_byte_off(flex, flex->handle, byte_off,
+						      false, is_inner, &val);
+	if (ret == -1 || !val)
+		return;
+
+	DR_SET(tag, val, fc->byte_off, 0, fc->bit_mask);
+}
+
+static void
+mlx5dr_definer_flex_parser_inner_set(struct mlx5dr_definer_fc *fc,
+				     const void *item,
+				     uint8_t *tag)
+{
+	mlx5dr_definer_flex_parser_set(fc, item, tag, true);
+}
+
+static void
+mlx5dr_definer_flex_parser_outer_set(struct mlx5dr_definer_fc *fc,
+				     const void *item,
+				     uint8_t *tag)
+{
+	mlx5dr_definer_flex_parser_set(fc, item, tag, false);
+}
+
 static void
 mlx5dr_definer_gre_key_set(struct mlx5dr_definer_fc *fc,
 			   const void *item_spec,
@@ -1465,6 +1502,47 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_flex_parser(struct mlx5dr_definer_conv_data *cd,
+				     struct rte_flow_item *item,
+				     int item_idx)
+{
+	uint32_t base_off = MLX5_BYTE_OFF(definer_hl, flex_parser.flex_parser_0);
+	const struct rte_flow_item_flex *v, *m;
+	enum mlx5dr_definer_fname fname;
+	struct mlx5dr_definer_fc *fc;
+	uint32_t i, mask, byte_off;
+	bool is_inner = cd->tunnel;
+	int ret;
+
+	m = item->mask;
+	v = item->spec;
+	mask = 0;
+	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
+		byte_off = base_off - i * sizeof(uint32_t);
+		ret = mlx5_flex_get_parser_value_per_byte_off(m, v->handle, byte_off,
+							      true, is_inner, &mask);
+		if (ret == -1) {
+			rte_errno = EINVAL;
+			return rte_errno;
+		}
+
+		if (!mask)
+			continue;
+
+		fname = MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+		fname += (enum mlx5dr_definer_fname)i;
+		fc = &cd->fc[fname];
+		fc->byte_off = byte_off;
+		fc->item_idx = item_idx;
+		fc->tag_set = cd->tunnel ? &mlx5dr_definer_flex_parser_inner_set :
+					   &mlx5dr_definer_flex_parser_outer_set;
+		fc->tag_mask_set = &mlx5dr_definer_ones_set;
+		fc->bit_mask = mask;
+	}
+	return 0;
+}
+
 static int
 mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 				struct mlx5dr_match_template *mt,
@@ -1581,6 +1659,11 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			ret = mlx5dr_definer_conv_item_meter_color(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_METER_COLOR;
 			break;
+		case RTE_FLOW_ITEM_TYPE_FLEX:
+			ret = mlx5dr_definer_conv_item_flex_parser(&cd, items, i);
+			item_flags |= cd.tunnel ? MLX5_FLOW_ITEM_INNER_FLEX :
+						  MLX5_FLOW_ITEM_OUTER_FLEX;
+			break;
 		default:
 			DR_LOG(ERR, "Unsupported item type %d", items->type);
 			rte_errno = ENOTSUP;
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 0b97c4e78d..8297129bd1 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1034,7 +1034,7 @@ static void
 mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
+	struct mlx5_ecpri_parser_profile *prf = &priv->sh->ecpri_parser;
 
 	if (prf->obj)
 		mlx5_devx_cmd_destroy(prf->obj);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 83fb316ad8..2cc3959d01 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -2243,6 +2243,12 @@ void mlx5_flex_item_port_cleanup(struct rte_eth_dev *dev);
 void mlx5_flex_flow_translate_item(struct rte_eth_dev *dev, void *matcher,
 				   void *key, const struct rte_flow_item *item,
 				   bool is_inner);
+int mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			    uint32_t idx, uint32_t *pos,
+			    bool is_inner, uint32_t *def);
+int mlx5_flex_get_parser_value_per_byte_off(const struct rte_flow_item_flex *item,
+					    void *flex, uint32_t byte_off,
+					    bool is_mask, bool tunnel, uint32_t *value);
 int mlx5_flex_acquire_index(struct rte_eth_dev *dev,
 			    struct rte_flow_item_flex_handle *handle,
 			    bool acquire);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index e376dcae93..c8761c4e5a 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1225,6 +1225,7 @@ struct rte_flow_pattern_template {
 	 * tag pattern item for representor matching.
 	 */
 	bool implicit_tag;
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Flow action template struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 7ca909999b..284f18da11 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10588,7 +10588,7 @@ flow_dv_translate_item_flex(struct rte_eth_dev *dev, void *matcher, void *key,
 		(const struct rte_flow_item_flex *)item->spec;
 	int index = mlx5_flex_acquire_index(dev, spec->handle, false);
 
-	MLX5_ASSERT(index >= 0 && index <= (int)(sizeof(uint32_t) * CHAR_BIT));
+	MLX5_ASSERT(index >= 0 && index < (int)(sizeof(uint32_t) * CHAR_BIT));
 	if (index < 0)
 		return;
 	if (!(dev_flow->handle->flex_item & RTE_BIT32(index))) {
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index 24b7226ee6..aa317fc958 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -198,6 +198,99 @@ mlx5_flex_set_match_sample(void *misc4_m, void *misc4_v,
 	}
 #undef SET_FP_MATCH_SAMPLE_ID
 }
+
+/**
+ * Get the flex parser sample id and corresponding mask
+ * per shift and width information.
+ *
+ * @param[in] tp
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] idx
+ *   Mapping index.
+ * @param[in, out] pos
+ *   Where to search the value and mask.
+ * @param[in] is_inner
+ *   For inner matching or not.
+ * @param[in, def] def
+ *   Mask generated by mapping shift and width.
+ *
+ * @return
+ *   0 on success, -1 to ignore.
+ */
+int
+mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			uint32_t idx, uint32_t *pos,
+			bool is_inner, uint32_t *def)
+{
+	const struct mlx5_flex_pattern_field *map = tp->map + idx;
+	uint32_t id = map->reg_id;
+
+	*def = (RTE_BIT64(map->width) - 1) << map->shift;
+	/* Skip placeholders for DUMMY fields. */
+	if (id == MLX5_INVALID_SAMPLE_REG_ID) {
+		*pos += map->width;
+		return -1;
+	}
+	MLX5_ASSERT(map->width);
+	MLX5_ASSERT(id < tp->devx_fp->num_samples);
+	if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
+		uint32_t num_samples = tp->devx_fp->num_samples / 2;
+
+		MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
+		MLX5_ASSERT(id < num_samples);
+		id += num_samples;
+	}
+	return id;
+}
+
+/**
+ * Get the flex parser mapping value per definer format_select_dw.
+ *
+ * @param[in] item
+ *   Rte flex item pointer.
+ * @param[in] flex
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] byte_off
+ *   Mlx5 flex item format_select_dw.
+ * @param[in] is_mask
+ *   Spec or mask.
+ * @param[in] tunnel
+ *   Tunnel mode or not.
+ * @param[in, def] value
+ *   Value calculated for this flex parser, either spec or mask.
+ *
+ * @return
+ *   0 on success, -1 for error.
+ */
+int
+mlx5_flex_get_parser_value_per_byte_off(const struct rte_flow_item_flex *item,
+					void *flex, uint32_t byte_off,
+					bool is_mask, bool tunnel, uint32_t *value)
+{
+	struct mlx5_flex_pattern_field *map;
+	struct mlx5_flex_item *tp = flex;
+	uint32_t def, i, pos, val;
+	int id;
+
+	*value = 0;
+	for (i = 0, pos = 0; i < tp->mapnum && pos < item->length * CHAR_BIT; i++) {
+		map = tp->map + i;
+		id = mlx5_flex_get_sample_id(tp, i, &pos, tunnel, &def);
+		if (id == -1)
+			continue;
+		if (id >= (int)tp->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return -1;
+		if (byte_off == tp->devx_fp->sample_ids[id].format_select_dw * sizeof(uint32_t)) {
+			val = mlx5_flex_get_bitfield(item, pos, map->width, map->shift);
+			if (is_mask)
+				val &= RTE_BE32(def);
+			*value |= val;
+		}
+		pos += map->width;
+	}
+	return 0;
+}
+
 /**
  * Translate item pattern into matcher fields according to translation
  * array.
@@ -240,26 +333,17 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
-		uint32_t id = map->reg_id;
-		uint32_t def = (RTE_BIT64(map->width) - 1) << map->shift;
-		uint32_t val, msk;
+		uint32_t val, msk, def;
+		int id = mlx5_flex_get_sample_id(tp, i, &pos, is_inner, &def);
 
-		/* Skip placeholders for DUMMY fields. */
-		if (id == MLX5_INVALID_SAMPLE_REG_ID) {
-			pos += map->width;
+		if (id == -1)
 			continue;
-		}
+		MLX5_ASSERT(id < (int)tp->devx_fp->num_samples);
+		if (id >= (int)tp->devx_fp->num_samples ||
+		    id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
 		val = mlx5_flex_get_bitfield(spec, pos, map->width, map->shift);
 		msk = mlx5_flex_get_bitfield(mask, pos, map->width, map->shift);
-		MLX5_ASSERT(map->width);
-		MLX5_ASSERT(id < tp->devx_fp->num_samples);
-		if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
-			uint32_t num_samples = tp->devx_fp->num_samples / 2;
-
-			MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
-			MLX5_ASSERT(id < num_samples);
-			id += num_samples;
-		}
 		if (attr->ext_sample_id)
 			sample_id = tp->devx_fp->sample_ids[id].sample_id;
 		else
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 44953451d5..742b0b0d43 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4331,6 +4331,36 @@ flow_hw_set_vlan_vid_construct(struct rte_eth_dev *dev,
 					      &modify_action);
 }
 
+static int
+flow_hw_flex_item_acquire(struct rte_eth_dev *dev,
+			  struct rte_flow_item_flex_handle *handle,
+			  uint8_t *flex_item)
+{
+	int index = mlx5_flex_acquire_index(dev, handle, false);
+
+	MLX5_ASSERT(index >= 0 && index < (int)(sizeof(uint32_t) * CHAR_BIT));
+	if (index < 0)
+		return -1;
+	if (!(*flex_item & RTE_BIT32(index))) {
+		/* Don't count same flex item again. */
+		if (mlx5_flex_acquire_index(dev, handle, true) != index)
+			MLX5_ASSERT(false);
+		*flex_item |= (uint8_t)RTE_BIT32(index);
+	}
+	return 0;
+}
+
+static void
+flow_hw_flex_item_release(struct rte_eth_dev *dev, uint8_t *flex_item)
+{
+	while (*flex_item) {
+		int index = rte_bsf32(*flex_item);
+
+		mlx5_flex_release_index(dev, index);
+		*flex_item &= ~(uint8_t)RTE_BIT32(index);
+	}
+}
+
 /**
  * Create flow action template.
  *
@@ -4732,6 +4762,7 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_ICMP:
 		case RTE_FLOW_ITEM_TYPE_ICMP6:
 		case RTE_FLOW_ITEM_TYPE_CONNTRACK:
+		case RTE_FLOW_ITEM_TYPE_FLEX:
 			break;
 		case RTE_FLOW_ITEM_TYPE_INTEGRITY:
 			/*
@@ -4809,6 +4840,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 		.mask = &tag_m,
 		.last = NULL
 	};
+	unsigned int i = 0;
 
 	if (flow_hw_pattern_validate(dev, attr, items, error))
 		return NULL;
@@ -4868,6 +4900,19 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 			it->implicit_tag = true;
 		mlx5_free(copied_items);
 	}
+	for (i = 0; items[i].type != RTE_FLOW_ITEM_TYPE_END; ++i) {
+		if (items[i].type == RTE_FLOW_ITEM_TYPE_FLEX) {
+			const struct rte_flow_item_flex *spec =
+				(const struct rte_flow_item_flex *)items[i].spec;
+			struct rte_flow_item_flex_handle *handle = spec->handle;
+
+			if (flow_hw_flex_item_acquire(dev, handle, &it->flex_item)) {
+				claim_zero(mlx5dr_match_template_destroy(it->mt));
+				mlx5_free(it);
+				return NULL;
+			}
+		}
+	}
 	__atomic_fetch_add(&it->refcnt, 1, __ATOMIC_RELAXED);
 	LIST_INSERT_HEAD(&priv->flow_hw_itt, it, next);
 	return it;
@@ -4887,7 +4932,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 			      struct rte_flow_pattern_template *template,
 			      struct rte_flow_error *error __rte_unused)
 {
@@ -4900,6 +4945,7 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "item template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	claim_zero(mlx5dr_match_template_destroy(template->mt));
 	mlx5_free(template);
 	return 0;
-- 
2.27.0

[PATCH v2 07/11] net/mlx5: add flex item modify field implementation

[Rongwei Liu]

Add flex item modify field HWS implementation.
The minimum modify boundary is one byte.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h  |   1 +
 drivers/net/mlx5/mlx5_flow.h    |   3 +
 drivers/net/mlx5/mlx5_flow_dv.c | 165 +++++++++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c |  14 ++-
 4 files changed, 170 insertions(+), 13 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index ce6cd98fd7..0c2a516e9d 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -759,6 +759,7 @@ enum mlx5_modification_field {
 	MLX5_MODI_OUT_IP_ECN = 0x73,
 	MLX5_MODI_TUNNEL_HDR_DW_1 = 0x75,
 	MLX5_MODI_GTPU_FIRST_EXT_DW_0 = 0x76,
+	MLX5_MODI_INVALID = INT_MAX,
 };
 
 /* Total number of metadata reg_c's. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index c8761c4e5a..c71fa1c0ad 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1080,6 +1080,8 @@ struct field_modify_info {
 	uint32_t size; /* Size of field in protocol header, in bytes. */
 	uint32_t offset; /* Offset of field in protocol header, in bytes. */
 	enum mlx5_modification_field id;
+	uint32_t shift;
+	uint8_t is_flex; /* Temporary indicator for flex item modify filed WA. */
 };
 
 /* HW steering flow attributes. */
@@ -1244,6 +1246,7 @@ struct rte_flow_actions_template {
 	uint16_t mhdr_off; /* Offset of DR modify header action. */
 	uint32_t refcnt; /* Reference counter. */
 	uint16_t rx_cpy_pos; /* Action position of Rx metadata to be copied. */
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Jump action struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 284f18da11..92a5914d4b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -414,10 +414,15 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			++field;
 			continue;
 		}
-		/* Deduce actual data width in bits from mask value. */
-		off_b = rte_bsf32(mask) + carry_b;
-		size_b = sizeof(uint32_t) * CHAR_BIT -
-			 off_b - __builtin_clz(mask);
+		if (type == MLX5_MODIFICATION_TYPE_COPY && field->is_flex) {
+			off_b = 32 - field->shift + carry_b - field->size * CHAR_BIT;
+			size_b = field->size * CHAR_BIT - carry_b;
+		} else {
+			/* Deduce actual data width in bits from mask value. */
+			off_b = rte_bsf32(mask) + carry_b;
+			size_b = sizeof(uint32_t) * CHAR_BIT -
+				 off_b - __builtin_clz(mask);
+		}
 		MLX5_ASSERT(size_b);
 		actions[i] = (struct mlx5_modification_cmd) {
 			.action_type = type,
@@ -437,40 +442,46 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			 * Destination field overflow. Copy leftovers of
 			 * a source field to the next destination field.
 			 */
-			carry_b = 0;
 			if ((size_b > dcopy->size * CHAR_BIT - dcopy->offset) &&
 			    dcopy->size != 0) {
 				actions[i].length =
 					dcopy->size * CHAR_BIT - dcopy->offset;
-				carry_b = actions[i].length;
+				carry_b += actions[i].length;
 				next_field = false;
+			} else {
+				carry_b = 0;
 			}
 			/*
 			 * Not enough bits in a source filed to fill a
 			 * destination field. Switch to the next source.
 			 */
 			if ((size_b < dcopy->size * CHAR_BIT - dcopy->offset) &&
-			    (size_b == field->size * CHAR_BIT - off_b)) {
-				actions[i].length =
-					field->size * CHAR_BIT - off_b;
+			    ((size_b == field->size * CHAR_BIT - off_b) ||
+			     field->is_flex)) {
+				actions[i].length = size_b;
 				dcopy->offset += actions[i].length;
 				next_dcopy = false;
 			}
-			if (next_dcopy)
-				++dcopy;
 		} else {
 			MLX5_ASSERT(item->spec);
 			data = flow_dv_fetch_field((const uint8_t *)item->spec +
 						   field->offset, field->size);
 			/* Shift out the trailing masked bits from data. */
 			data = (data & mask) >> off_b;
+			if (field->is_flex)
+				actions[i].offset = 32 - field->shift - field->size * CHAR_BIT;
 			actions[i].data1 = rte_cpu_to_be_32(data);
 		}
 		/* Convert entire record to expected big-endian format. */
 		actions[i].data0 = rte_cpu_to_be_32(actions[i].data0);
+		if ((type != MLX5_MODIFICATION_TYPE_COPY ||
+		     dcopy->id != (enum mlx5_modification_field)UINT32_MAX) &&
+		    field->id != (enum mlx5_modification_field)UINT32_MAX)
+			++i;
+		if (next_dcopy && type == MLX5_MODIFICATION_TYPE_COPY)
+			++dcopy;
 		if (next_field)
 			++field;
-		++i;
 	} while (field->size);
 	if (resource->actions_num == i)
 		return rte_flow_error_set(error, EINVAL,
@@ -1421,6 +1432,131 @@ flow_modify_info_mask_32_masked(uint32_t length, uint32_t off, uint32_t post_mas
 	return rte_cpu_to_be_32(mask & post_mask);
 }
 
+static void
+mlx5_modify_flex_item(const struct rte_eth_dev *dev,
+		      const struct mlx5_flex_item *flex,
+		      const struct rte_flow_action_modify_data *data,
+		      struct field_modify_info *info,
+		      uint32_t *mask, uint32_t width)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
+	uint32_t i, j;
+	int id = 0;
+	uint32_t pos = 0;
+	const struct mlx5_flex_pattern_field *map;
+	uint32_t offset = data->offset;
+	uint32_t width_left = width;
+	uint32_t def;
+	uint32_t cur_width = 0;
+	uint32_t tmp_ofs;
+	uint32_t idx = 0;
+	struct field_modify_info tmp;
+	int tmp_id;
+
+	if (!attr->ext_sample_id) {
+		DRV_LOG(ERR, "FW doesn't support modify field with flex item.");
+		return;
+	}
+	/*
+	 * search for the mapping instance until Accumulated width is no
+	 * less than data->offset.
+	 */
+	for (i = 0; i < flex->mapnum; i++) {
+		if (flex->map[i].width + pos > data->offset)
+			break;
+		pos += flex->map[i].width;
+	}
+	if (i >= flex->mapnum)
+		return;
+	tmp_ofs = pos < data->offset ? data->offset - pos : 0;
+	for (j = i; i < flex->mapnum && width_left > 0; ) {
+		map = flex->map + i;
+		id = mlx5_flex_get_sample_id(flex, i, &pos, false, &def);
+		if (id == -1) {
+			i++;
+			/* All left length is dummy */
+			if (pos >= data->offset + width)
+				return;
+			cur_width = map->width;
+		/* One mapping instance covers the whole width. */
+		} else if (pos + map->width >= (data->offset + width)) {
+			cur_width = width_left;
+		} else {
+			cur_width = cur_width + map->width - tmp_ofs;
+			pos += map->width;
+			/*
+			 * Continue to search next until:
+			 * 1. Another flex parser ID.
+			 * 2. Width has been covered.
+			 */
+			for (j = i + 1; j < flex->mapnum; j++) {
+				tmp_id = mlx5_flex_get_sample_id(flex, j, &pos, false, &def);
+				if (tmp_id == -1) {
+					i = j;
+					pos -= flex->map[j].width;
+					break;
+				}
+				if (id >= (int)flex->devx_fp->num_samples ||
+				    id >= MLX5_GRAPH_NODE_SAMPLE_NUM ||
+				    tmp_id >= (int)flex->devx_fp->num_samples ||
+				    tmp_id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+					return;
+				if (flex->devx_fp->sample_ids[id].id !=
+						flex->devx_fp->sample_ids[tmp_id].id ||
+				    flex->map[j].shift != flex->map[j - 1].width +
+							  flex->map[j - 1].shift) {
+					i = j;
+					break;
+				}
+				if ((pos + flex->map[j].width) >= (data->offset + width)) {
+					cur_width = width_left;
+					break;
+				}
+				pos += flex->map[j].width;
+				cur_width += flex->map[j].width;
+			}
+		}
+		if (cur_width > width_left)
+			cur_width = width_left;
+		else if (cur_width < width_left && (j == flex->mapnum || i == flex->mapnum))
+			return;
+
+		MLX5_ASSERT(id < (int)flex->devx_fp->num_samples);
+		if (id >= (int)flex->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
+		/* Use invalid entry as placeholder for DUMMY mapping. */
+		info[idx] = (struct field_modify_info){cur_width / CHAR_BIT, offset / CHAR_BIT,
+			     id == -1 ? MLX5_MODI_INVALID :
+			     (enum mlx5_modification_field)
+			     flex->devx_fp->sample_ids[id].modify_field_id,
+			     map->shift + tmp_ofs, 1};
+		offset += cur_width;
+		width_left -= cur_width;
+		if (!mask) {
+			info[idx].offset = (32 - cur_width - map->shift - tmp_ofs);
+			info[idx].size = cur_width / CHAR_BIT + info[idx].offset / CHAR_BIT;
+		}
+		cur_width = 0;
+		tmp_ofs = 0;
+		idx++;
+	}
+	if (unlikely(width_left > 0)) {
+		MLX5_ASSERT(false);
+		return;
+	}
+	if (mask)
+		memset(mask, 0xff, data->offset / CHAR_BIT + width / CHAR_BIT);
+	/* Re-order the info to follow IPv6 address. */
+	for (i = 0; i < idx / 2; i++) {
+		tmp = info[i];
+		MLX5_ASSERT(info[i].id);
+		MLX5_ASSERT(info[idx - 1 - i].id);
+		info[i] = info[idx - 1 - i];
+		info[idx - 1 - i] = tmp;
+	}
+}
+
 void
 mlx5_flow_field_id_to_modify_info
 		(const struct rte_flow_action_modify_data *data,
@@ -1883,6 +2019,11 @@ mlx5_flow_field_id_to_modify_info
 				info[idx].offset = data->offset;
 		}
 		break;
+	case RTE_FLOW_FIELD_FLEX_ITEM:
+		MLX5_ASSERT(data->flex_handle != NULL && !(data->offset & 0x7));
+		mlx5_modify_flex_item(dev, (const struct mlx5_flex_item *)data->flex_handle,
+				      data, info, mask, width);
+		break;
 	case RTE_FLOW_FIELD_POINTER:
 	case RTE_FLOW_FIELD_VALUE:
 	default:
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 742b0b0d43..2e263880d7 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4550,6 +4550,17 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
 			at->actions[i].conf = actions->conf;
 			at->masks[i].conf = masks->conf;
 		}
+		if (actions->type == RTE_FLOW_ACTION_TYPE_MODIFY_FIELD) {
+			const struct rte_flow_action_modify_field *info = actions->conf;
+
+			if ((info->dst.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			     flow_hw_flex_item_acquire(dev, info->dst.flex_handle,
+						       &at->flex_item)) ||
+			     (info->src.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			      flow_hw_flex_item_acquire(dev, info->src.flex_handle,
+							&at->flex_item)))
+				goto error;
+		}
 	}
 	at->tmpl = flow_hw_dr_actions_template_create(at);
 	if (!at->tmpl)
@@ -4581,7 +4592,7 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_actions_template_destroy(struct rte_eth_dev *dev,
 				 struct rte_flow_actions_template *template,
 				 struct rte_flow_error *error __rte_unused)
 {
@@ -4594,6 +4605,7 @@ flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "action template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	if (template->tmpl)
 		mlx5dr_action_template_destroy(template->tmpl);
 	mlx5_free(template);
-- 
2.27.0

[PATCH v2 08/11] net/mlx5: return error for sws modify field

[Rongwei Liu]

Return unsupported error message when application tries to
modify flex item field.

Validation of packet modifications actions for SW Steering checked
if either source or destination field of MODIFY_FIELD action
was a flex item.
When DEC_TTL action is used, DEC_TTL action does not have any
action configuration and dereferencing source or destination field
is invalid, so validation of source and destination field types
should be moved to MODIFY_FIELD specific validation function, then
field types are validated if and only if action type is MODIFY_FIELD.

Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 92a5914d4b..a7c0d5bf17 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4828,6 +4828,7 @@ flow_dv_validate_action_modify_hdr(const uint64_t action_flags,
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION_CONF,
 					  NULL, "action configuration not set");
+
 	if (action_flags & MLX5_FLOW_ACTION_ENCAP)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION, NULL,
@@ -5153,17 +5154,21 @@ flow_dv_validate_action_modify_field(struct rte_eth_dev *dev,
 	struct mlx5_hca_attr *hca_attr = &priv->sh->cdev->config.hca_attr;
 	const struct rte_flow_action_modify_field *action_modify_field =
 		action->conf;
-	uint32_t dst_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->dst.field,
-				-1, attr, error);
-	uint32_t src_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->src.field,
-				dst_width, attr, error);
+	uint32_t dst_width, src_width;
 
 	ret = flow_dv_validate_action_modify_hdr(action_flags, action, error);
 	if (ret)
 		return ret;
-
+	if (action_modify_field->src.field == RTE_FLOW_FIELD_FLEX_ITEM ||
+	    action_modify_field->dst.field == RTE_FLOW_FIELD_FLEX_ITEM)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"flex item fields modification"
+				" is not supported");
+	dst_width = mlx5_flow_item_field_width(dev, action_modify_field->dst.field,
+					       -1, attr, error);
+	src_width = mlx5_flow_item_field_width(dev, action_modify_field->src.field,
+					       dst_width, attr, error);
 	if (action_modify_field->width == 0)
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
-- 
2.27.0

[PATCH v2 09/11] app/testpmd: raw encap with flex item support

[Rongwei Liu]

Application should retrieve raw_encap buffer from
spec->pattern if it is flex item.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index f5d7a67def..50c8ec5594 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -11193,6 +11193,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 	uint16_t proto = 0;
 	uint16_t idx = in->port; /* We borrow port field as index */
 	int gtp_psc = -1; /* GTP PSC option index. */
+	const void *src_spec;
 
 	if (in->command == SET_SAMPLE_ACTIONS)
 		return cmd_set_raw_parsed_sample(in);
@@ -11216,6 +11217,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 		item = in->args.vc.pattern + i;
 		if (item->spec == NULL)
 			item->spec = flow_item_default_mask(item);
+		src_spec = item->spec;
 		switch (item->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			size = sizeof(struct rte_ether_hdr);
@@ -11343,9 +11345,13 @@ cmd_set_raw_parsed(const struct buffer *in)
 			size = sizeof(struct rte_flow_item_pfcp);
 			break;
 		case RTE_FLOW_ITEM_TYPE_FLEX:
-			size = item->spec ?
-				((const struct rte_flow_item_flex *)
-				item->spec)->length : 0;
+			if (item->spec != NULL) {
+				size = ((const struct rte_flow_item_flex *)item->spec)->length;
+				src_spec = ((const struct rte_flow_item_flex *)item->spec)->pattern;
+			} else {
+				size = 0;
+				src_spec = NULL;
+			}
 			break;
 		case RTE_FLOW_ITEM_TYPE_GRE_OPTION:
 			size = 0;
@@ -11378,12 +11384,14 @@ cmd_set_raw_parsed(const struct buffer *in)
 			fprintf(stderr, "Error - Not supported item\n");
 			goto error;
 		}
-		*total_size += size;
-		rte_memcpy(data_tail - (*total_size), item->spec, size);
-		/* update some fields which cannot be set by cmdline */
-		update_fields((data_tail - (*total_size)), item,
-			      upper_layer);
-		upper_layer = proto;
+		if (size) {
+			*total_size += size;
+			rte_memcpy(data_tail - (*total_size), src_spec, size);
+			/* update some fields which cannot be set by cmdline */
+			update_fields((data_tail - (*total_size)), item,
+				      upper_layer);
+			upper_layer = proto;
+		}
 	}
 	if (verbose_level & 0x1)
 		printf("total data size is %zu\n", (*total_size));
-- 
2.27.0

[PATCH v2 10/11] doc/mlx5: update mlx5 doc

[Rongwei Liu]

Add flex item matching and modify field feature into
mlx5 documents.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 doc/guides/nics/mlx5.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index f137f156f9..5df89c9e64 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -106,7 +106,7 @@ Features
 - Sub-Function representors.
 - Sub-Function.
 - Matching on represented port.
-
+- Modify flex item field.
 
 Limitations
 -----------
-- 
2.27.0

[PATCH v2 11/11] app/testpmd: adjust cleanup sequence when quitting

[Rongwei Liu]

If flex item is referenced in async flow either by
pattern template or action template, currently testpmd
complains "flex item has flow references". Flex items should
be flushed after async flow resources cleanup.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 app/test-pmd/testpmd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 134d79a555..e35f7a0e7a 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -3298,10 +3298,10 @@ flush_port_owned_resources(portid_t pi)
 {
 	mcast_addr_pool_destroy(pi);
 	port_flow_flush(pi);
-	port_flex_item_flush(pi);
 	port_flow_template_table_flush(pi);
 	port_flow_pattern_template_flush(pi);
 	port_flow_actions_template_flush(pi);
+	port_flex_item_flush(pi);
 	port_action_handle_flush(pi);
 }
 
-- 
2.27.0

[RFC 2/9] app/testpmd: add flex item modify field cmdline support

[Rongwei Liu]

Add flex item modify field cmdline support. Now user can use
testpmd cli to specify which flex item to be modified, either
source or destination.

Syntax is as below:
modify_field op set dst_type flex_item dst_level 0
dst_offset 16 src_type value src_value 0x123456781020 width 8

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 89 ++++++++++++++++++++++++++++++++++---
 1 file changed, 82 insertions(+), 7 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 88108498e0..7c12d63cbc 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -601,10 +601,12 @@ enum index {
 	ACTION_MODIFY_FIELD_DST_TYPE,
 	ACTION_MODIFY_FIELD_DST_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_DST_LEVEL,
+	ACTION_MODIFY_FIELD_DST_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_DST_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_TYPE,
 	ACTION_MODIFY_FIELD_SRC_TYPE_VALUE,
 	ACTION_MODIFY_FIELD_SRC_LEVEL,
+	ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE,
 	ACTION_MODIFY_FIELD_SRC_OFFSET,
 	ACTION_MODIFY_FIELD_SRC_VALUE,
 	ACTION_MODIFY_FIELD_SRC_POINTER,
@@ -807,7 +809,8 @@ static const char *const modify_field_ids[] = {
 	"udp_port_src", "udp_port_dst",
 	"vxlan_vni", "geneve_vni", "gtp_teid",
 	"tag", "mark", "meta", "pointer", "value",
-	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color", NULL
+	"ipv4_ecn", "ipv6_ecn", "gtp_psc_qfi", "meter_color",
+	"flex_item", NULL
 };
 
 static const char *const meter_colors[] = {
@@ -2282,6 +2285,10 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 				const char *str, unsigned int len, void *buf,
 				unsigned int size);
 static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+				const char *str, unsigned int len, void *buf,
+				unsigned int size);
+static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
 			 const char *str, unsigned int len, void *buf,
 			 unsigned int size);
@@ -5976,11 +5983,15 @@ static const struct token token_list[] = {
 		.name = "dst_level",
 		.help = "destination field level",
 		.next = NEXT(action_modify_field_dst,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					dst.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_DST_LEVEL_VALUE] = {
+		.name = "{dst_level}",
+		.help = "destination field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_DST_OFFSET] = {
 		.name = "dst_offset",
 		.help = "destination field bit offset",
@@ -6007,11 +6018,15 @@ static const struct token token_list[] = {
 		.name = "src_level",
 		.help = "source field level",
 		.next = NEXT(action_modify_field_src,
-			     NEXT_ENTRY(COMMON_UNSIGNED)),
-		.args = ARGS(ARGS_ENTRY(struct rte_flow_action_modify_field,
-					src.level)),
+			     NEXT_ENTRY(ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)),
 		.call = parse_vc_conf,
 	},
+	[ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE] = {
+		.name = "{src_level}",
+		.help = "source field level value",
+		.call = parse_vc_modify_field_level,
+		.comp = comp_none,
+	},
 	[ACTION_MODIFY_FIELD_SRC_OFFSET] = {
 		.name = "src_offset",
 		.help = "source field bit offset",
@@ -8477,6 +8492,66 @@ parse_vc_modify_field_id(struct context *ctx, const struct token *token,
 	return len;
 }
 
+/** Parse level for modify_field command. */
+static int
+parse_vc_modify_field_level(struct context *ctx, const struct token *token,
+			 const char *str, unsigned int len, void *buf,
+			 unsigned int size)
+{
+	struct rte_flow_action_modify_field *action;
+	struct flex_item *fp;
+	uint32_t val;
+	struct buffer *out = buf;
+	char *end;
+
+	(void)token;
+	(void)size;
+	if (ctx->curr != ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		ctx->curr != ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE)
+		return -1;
+	if (!ctx->object)
+		return len;
+	action = ctx->object;
+	errno = 0;
+	val = strtoumax(str, &end, 0);
+	if (errno || (size_t)(end - str) != len)
+		return -1;
+	/* No need to validate action template mask value */
+	if (out->args.vc.masks) {
+		if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE)
+			action->dst.level = val;
+		else
+			action->src.level = val;
+		return len;
+	}
+	if ((ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE &&
+		action->dst.field == RTE_FLOW_FIELD_FLEX_ITEM) ||
+		(ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE &&
+		action->src.field == RTE_FLOW_FIELD_FLEX_ITEM)) {
+		if (val >= FLEX_MAX_PARSERS_NUM) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+		fp = flex_items[ctx->port][val];
+		if (!fp) {
+			printf("Bad flex item handle\n");
+			return -1;
+		}
+	}
+	if (ctx->curr == ACTION_MODIFY_FIELD_DST_LEVEL_VALUE) {
+		if (action->dst.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->dst.level = val;
+		else
+			action->dst.flex_handle = fp->flex_handle;
+	} else if (ctx->curr == ACTION_MODIFY_FIELD_SRC_LEVEL_VALUE) {
+		if (action->src.field != RTE_FLOW_FIELD_FLEX_ITEM)
+			action->src.level = val;
+		else
+			action->src.flex_handle = fp->flex_handle;
+	}
+	return len;
+}
+
 /** Parse the conntrack update, not a rte_flow_action. */
 static int
 parse_vc_action_conntrack_update(struct context *ctx, const struct token *token,
-- 
2.27.0

[RFC 3/9] app/testpmd: pass flex handle into matching mask

[Rongwei Liu]

In async flow create API, there is only mask information when
creating flow table but flex item handle is required to parse
the HW sample information.

Pass the flex item handle instead of UINT64/32_MAX to mask.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 7c12d63cbc..9037432cc8 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -10068,8 +10068,8 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 	}
 	if (offset == offsetof(struct rte_flow_item_flex, handle)) {
 		const struct flex_item *fp;
-		struct rte_flow_item_flex *item_flex = ctx->object;
-		handle = (uint16_t)(uintptr_t)item_flex->handle;
+		spec = ctx->object;
+		handle = (uint16_t)(uintptr_t)spec->handle;
 		if (handle >= FLEX_MAX_PARSERS_NUM) {
 			printf("Bad flex item handle\n");
 			return -1;
@@ -10079,7 +10079,9 @@ parse_flex_handle(struct context *ctx, const struct token *token,
 			printf("Bad flex item handle\n");
 			return -1;
 		}
-		item_flex->handle = fp->flex_handle;
+		spec->handle = fp->flex_handle;
+		mask = spec + 2; /* spec, last, mask */
+		mask->handle = fp->flex_handle;
 	} else if (offset == offsetof(struct rte_flow_item_flex, pattern)) {
 		handle = (uint16_t)(uintptr_t)
 			((struct rte_flow_item_flex *)ctx->object)->pattern;
-- 
2.27.0

[RFC 4/9] net/mlx5: enable hws flex item create

[Rongwei Liu]

Enable flex item create and destroy with dv_flow_en=2

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 27 +++++++++++++++------------
 drivers/net/mlx5/mlx5_flow_hw.c  |  2 ++
 2 files changed, 17 insertions(+), 12 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a71474c90a..f5b3edea99 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -474,10 +474,20 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 	err = mlx5_alloc_table_hash_list(priv);
 	if (err)
 		goto error;
-	if (priv->sh->config.dv_flow_en == 2)
-		return 0;
 	/* The resources below are only valid with DV support. */
 #ifdef HAVE_IBV_FLOW_DV_SUPPORT
+	/* Init shared flex parsers list, no need lcore_share */
+	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
+	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
+					       mlx5_flex_parser_create_cb,
+					       mlx5_flex_parser_match_cb,
+					       mlx5_flex_parser_remove_cb,
+					       mlx5_flex_parser_clone_cb,
+					       mlx5_flex_parser_clone_free_cb);
+	if (!sh->flex_parsers_dv)
+		goto error;
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 	/* Init port id action list. */
 	snprintf(s, sizeof(s), "%s_port_id_action_list", sh->ibdev_name);
 	sh->port_id_action_list = mlx5_list_create(s, sh, true,
@@ -518,16 +528,9 @@ mlx5_alloc_shared_dr(struct mlx5_priv *priv)
 					      flow_dv_dest_array_clone_free_cb);
 	if (!sh->dest_array_list)
 		goto error;
-	/* Init shared flex parsers list, no need lcore_share */
-	snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
-	sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
-					       mlx5_flex_parser_create_cb,
-					       mlx5_flex_parser_match_cb,
-					       mlx5_flex_parser_remove_cb,
-					       mlx5_flex_parser_clone_cb,
-					       mlx5_flex_parser_clone_free_cb);
-	if (!sh->flex_parsers_dv)
-		goto error;
+#else
+	if (priv->sh->config.dv_flow_en == 2)
+		return 0;
 #endif
 #ifdef HAVE_MLX5DV_DR
 	void *domain;
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 20c71ff7f0..44953451d5 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -8336,6 +8336,8 @@ const struct mlx5_flow_driver_ops mlx5_flow_hw_drv_ops = {
 	.query = flow_hw_query,
 	.get_aged_flows = flow_hw_get_aged_flows,
 	.get_q_aged_flows = flow_hw_get_q_aged_flows,
+	.item_create = flow_dv_item_create,
+	.item_release = flow_dv_item_release,
 };
 
 /**
-- 
2.27.0

[RFC 5/9] net/mlx5: add IPv6 protocol as flex item input

[Rongwei Liu]

Support IPv6 protocol as new flex item input link.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_flex.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index fb08910ddb..bec07b13c1 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -1043,6 +1043,22 @@ mlx5_flex_arc_in_udp(const struct rte_flow_item *item,
 	return rte_be_to_cpu_16(spec->hdr.dst_port);
 }
 
+static int
+mlx5_flex_arc_in_ipv6(const struct rte_flow_item *item,
+		      struct rte_flow_error *error)
+{
+	const struct rte_flow_item_ipv6 *spec = item->spec;
+	const struct rte_flow_item_ipv6 *mask = item->mask;
+	struct rte_flow_item_ipv6 ip = { .hdr.proto = 0xff };
+
+	if (memcmp(mask, &ip, sizeof(struct rte_flow_item_ipv6))) {
+		return rte_flow_error_set
+			(error, EINVAL, RTE_FLOW_ERROR_TYPE_ITEM, item,
+			 "invalid ipv6 item mask, full mask is desired");
+	}
+	return spec->hdr.proto;
+}
+
 static int
 mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 			   const struct rte_flow_item_flex_conf *conf,
@@ -1089,6 +1105,9 @@ mlx5_flex_translate_arc_in(struct mlx5_hca_flex_attr *attr,
 		case RTE_FLOW_ITEM_TYPE_UDP:
 			ret = mlx5_flex_arc_in_udp(rte_item, error);
 			break;
+		case RTE_FLOW_ITEM_TYPE_IPV6:
+			ret = mlx5_flex_arc_in_ipv6(rte_item, error);
+			break;
 		default:
 			MLX5_ASSERT(false);
 			return rte_flow_error_set
-- 
2.27.0

[RFC 6/9] net/mlx5/hws: add hws flex item matching support

[Rongwei Liu]

Support flex item matching in hws and syntax follows
sws exactly.

Flex item should be created in advance and follow current
json mapping logic.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 doc/guides/nics/features/mlx5.ini     |   1 +
 doc/guides/nics/mlx5.rst              |   1 +
 drivers/common/mlx5/mlx5_devx_cmds.c  |  14 ++-
 drivers/common/mlx5/mlx5_devx_cmds.h  |   7 +-
 drivers/common/mlx5/mlx5_prm.h        |  22 +++-
 drivers/net/mlx5/hws/mlx5dr_definer.c | 141 ++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5.c               |  17 +++-
 drivers/net/mlx5/mlx5.h               |   8 +-
 drivers/net/mlx5/mlx5_flow.h          |   1 +
 drivers/net/mlx5/mlx5_flow_flex.c     |  83 +++++++++++----
 drivers/net/mlx5/mlx5_flow_hw.c       |  48 ++++++++-
 11 files changed, 310 insertions(+), 33 deletions(-)

diff --git a/doc/guides/nics/features/mlx5.ini b/doc/guides/nics/features/mlx5.ini
index 62fd330e2b..135b5c035d 100644
--- a/doc/guides/nics/features/mlx5.ini
+++ b/doc/guides/nics/features/mlx5.ini
@@ -87,6 +87,7 @@ vlan                 = Y
 vxlan                = Y
 vxlan_gpe            = Y
 represented_port     = Y
+flex item            = Y
 
 [rte_flow actions]
 age                  = I
diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 51f51259e3..397e04fbed 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -106,6 +106,7 @@ Features
 - Sub-Function representors.
 - Sub-Function.
 - Matching on represented port.
+- Matching on flex item with specific pattern.
 
 
 Limitations
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 59cebb530f..d6d70b600b 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -607,7 +607,8 @@ mlx5_devx_cmd_query_hca_vdpa_attr(void *ctx,
 
 int
 mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				  uint32_t ids[], uint32_t num)
+				  struct mlx5_ext_sample_id ids[],
+				  uint32_t num,  uint8_t *anchor)
 {
 	uint32_t in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {0};
 	uint32_t out[MLX5_ST_SZ_DW(create_flex_parser_out)] = {0};
@@ -636,6 +637,7 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			(void *)flex_obj);
 		return -rte_errno;
 	}
+	*anchor = MLX5_GET(parse_graph_flex, flex, head_anchor_id);
 	for (i = 0; i < MLX5_GRAPH_NODE_SAMPLE_NUM; i++) {
 		void *s_off = (void *)((char *)sample + i *
 			      MLX5_ST_SZ_BYTES(parse_graph_flow_match_sample));
@@ -645,8 +647,8 @@ mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
 			      flow_match_sample_en);
 		if (!en)
 			continue;
-		ids[idx++] = MLX5_GET(parse_graph_flow_match_sample, s_off,
-				  flow_match_sample_field_id);
+		ids[idx++].id = MLX5_GET(parse_graph_flow_match_sample, s_off,
+					 flow_match_sample_field_id);
 	}
 	if (num != idx) {
 		rte_errno = EINVAL;
@@ -794,6 +796,12 @@ mlx5_devx_cmd_query_hca_parse_graph_node_cap
 					 max_num_arc_out);
 	attr->max_num_sample = MLX5_GET(parse_graph_node_cap, hcattr,
 					max_num_sample);
+	attr->anchor_en = MLX5_GET(parse_graph_node_cap, hcattr, anchor_en);
+	attr->ext_sample_id = MLX5_GET(parse_graph_node_cap, hcattr, ext_sample_id);
+	attr->sample_tunnel_inner2 = MLX5_GET(parse_graph_node_cap, hcattr,
+					      sample_tunnel_inner2);
+	attr->zero_size_supported = MLX5_GET(parse_graph_node_cap, hcattr,
+					     zero_size_supported);
 	attr->sample_id_in_out = MLX5_GET(parse_graph_node_cap, hcattr,
 					  sample_id_in_out);
 	attr->max_base_header_length = MLX5_GET(parse_graph_node_cap, hcattr,
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index c94b9eac06..5b33010155 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -114,6 +114,10 @@ struct mlx5_hca_flex_attr {
 	uint8_t  max_num_arc_out;
 	uint8_t  max_num_sample;
 	uint8_t  max_num_prog_sample:5;	/* From HCA CAP 2 */
+	uint8_t  anchor_en:1;
+	uint8_t  ext_sample_id:1;
+	uint8_t  sample_tunnel_inner2:1;
+	uint8_t  zero_size_supported:1;
 	uint8_t  sample_id_in_out:1;
 	uint16_t max_base_header_length;
 	uint8_t  max_sample_base_offset;
@@ -706,7 +710,8 @@ int mlx5_devx_cmd_modify_tir(struct mlx5_devx_obj *tir,
 			     struct mlx5_devx_modify_tir_attr *tir_attr);
 __rte_internal
 int mlx5_devx_cmd_query_parse_samples(struct mlx5_devx_obj *flex_obj,
-				      uint32_t ids[], uint32_t num);
+				      struct mlx5_ext_sample_id ids[],
+				      uint32_t num, uint8_t *anchor);
 
 __rte_internal
 struct mlx5_devx_obj *
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 2b5c43ee6e..b080599570 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1890,7 +1890,11 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 max_num_arc_in[0x08];
 	u8 max_num_arc_out[0x08];
 	u8 max_num_sample[0x08];
-	u8 reserved_at_78[0x07];
+	u8 reserved_at_78[0x3];
+	u8 anchor_en[0x1];
+	u8 ext_sample_id[0x1];
+	u8 sample_tunnel_inner2[0x1];
+	u8 zero_size_supported[0x1];
 	u8 sample_id_in_out[0x1];
 	u8 max_base_header_length[0x10];
 	u8 reserved_at_90[0x08];
@@ -1900,6 +1904,18 @@ struct mlx5_ifc_parse_graph_node_cap_bits {
 	u8 header_length_mask_width[0x08];
 };
 
+/* ext_sample_id structure, see PRM Table 539. */
+struct mlx5_ext_sample_id {
+	union {
+			struct {
+				uint32_t format_select_dw:8;
+				uint32_t modify_field_id:12;
+				uint32_t sample_id:12;
+			};
+			uint32_t id;
+	};
+};
+
 struct mlx5_ifc_flow_table_prop_layout_bits {
 	u8 ft_support[0x1];
 	u8 flow_tag[0x1];
@@ -4455,7 +4471,9 @@ struct mlx5_ifc_parse_graph_flex_bits {
 	u8 header_length_mode[0x4];
 	u8 header_length_field_offset[0x10];
 	u8 next_header_field_offset[0x10];
-	u8 reserved_at_160[0x1b];
+	u8 reserved_at_160[0x12];
+	u8 head_anchor_id[0x6];
+	u8 reserved_at_178[0x3];
 	u8 next_header_field_size[0x5];
 	u8 header_length_field_mask[0x20];
 	u8 reserved_at_224[0x20];
diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 6b98eb8c96..5b933734b7 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -293,6 +293,57 @@ mlx5dr_definer_integrity_set(struct mlx5dr_definer_fc *fc,
 	DR_SET(tag, ok1_bits, fc->byte_off, fc->bit_off, fc->bit_mask);
 }
 
+static uint32_t
+mlx5dr_definer_flex_parser_common(const struct mlx5dr_definer_fc *fc,
+				  const struct rte_flow_item_flex *flex,
+				  bool is_mask)
+{
+	struct mlx5_flex_item *tp = (struct mlx5_flex_item *)flex->handle;
+	struct mlx5_flex_pattern_field *map;
+	uint32_t i, val, pos, def;
+	int id;
+
+	tp = (struct mlx5_flex_item *)flex->handle;
+	for (i = 0, pos = 0, val = 0; i < tp->mapnum && pos < flex->length * CHAR_BIT; i++) {
+		map = tp->map + i;
+		id = mlx5_flex_get_sample_id(tp, i, &pos, fc->bit_off, &def);
+		if (id == -1)
+			continue;
+		MLX5_ASSERT(id < (int)tp->devx_fp->num_samples);
+		if (id >= (int)tp->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return -1;
+		if (tp->devx_fp->sample_ids[id].format_select_dw * 4 == fc->byte_off) {
+			val |= RTE_BE32(mlx5_flex_get_bitfield(flex, pos, map->width, map->shift)) &
+			       (is_mask ? def : UINT32_MAX);
+		}
+		pos += map->width;
+	}
+	return val;
+}
+
+static void
+mlx5dr_definer_flex_parser_set(struct mlx5dr_definer_fc *fc,
+			       const void *item,
+			       uint8_t *tag)
+{
+	uint32_t val;
+
+	val = mlx5dr_definer_flex_parser_common(fc, item, false);
+	DR_SET_BE32(tag, (val & fc->bit_mask), fc->byte_off, 0, fc->bit_mask);
+}
+
+static void
+mlx5dr_definer_flex_parser_mask_set(struct mlx5dr_definer_fc *fc,
+				    const void *item,
+				    uint8_t *tag)
+{
+	uint32_t mask;
+
+	mask = mlx5dr_definer_flex_parser_common(fc, item, true);
+	DR_SET_BE32(tag, (mask), fc->byte_off, 0, UINT32_MAX);
+	fc->bit_mask = mask;
+}
+
 static void
 mlx5dr_definer_gre_key_set(struct mlx5dr_definer_fc *fc,
 			   const void *item_spec,
@@ -1465,6 +1516,91 @@ mlx5dr_definer_conv_item_meter_color(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static struct mlx5dr_definer_fc *
+mlx5dr_definer_get_flex_parser_fc(struct mlx5dr_definer_conv_data *cd,
+				  struct mlx5_ext_sample_id reg, int item_idx)
+{
+	enum mlx5dr_definer_fname i = MLX5DR_DEFINER_FNAME_FLEX_PARSER_0;
+	struct mlx5dr_definer_fc *fc;
+
+	for (; i <= MLX5DR_DEFINER_FNAME_FLEX_PARSER_7; i++) {
+		fc = &cd->fc[i];
+		switch (i) {
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_0:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_0);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_1:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_1);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_2:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_2);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_3:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_3);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_4:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_4);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_5:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_5);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_6:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_6);
+			break;
+		case MLX5DR_DEFINER_FNAME_FLEX_PARSER_7:
+		default:
+			DR_CALC_SET_HDR(fc, flex_parser, flex_parser_7);
+			break;
+		}
+		if (fc->byte_off == reg.format_select_dw * 4)
+			break;
+	}
+	if (i > MLX5DR_DEFINER_FNAME_FLEX_PARSER_7) {
+		rte_errno = ENOTSUP;
+		return NULL;
+	}
+	fc->item_idx = item_idx;
+	fc->tag_set = &mlx5dr_definer_flex_parser_set;
+	fc->fname = i;
+	fc->tag_mask_set = &mlx5dr_definer_flex_parser_mask_set;
+	fc->bit_off = cd->tunnel;
+	return fc;
+}
+
+static int
+mlx5dr_definer_conv_item_flex(struct mlx5dr_definer_conv_data *cd,
+			      struct rte_flow_item *item,
+			      int item_idx)
+{
+	const struct rte_flow_item_flex *v, *m;
+	struct mlx5_flex_pattern_field *map;
+	bool is_inner = cd->tunnel;
+	struct mlx5_flex_item *tp;
+	uint32_t i, mask, def;
+	uint32_t pos;
+	int id;
+
+	MLX5_ASSERT(item->spec && item->mask);
+	m = item->mask;
+	v = item->spec;
+	tp = (struct mlx5_flex_item *)v->handle;
+	for (i = 0, pos = 0; i < tp->mapnum && pos < m->length * CHAR_BIT; i++) {
+		map = tp->map + i;
+		id = mlx5_flex_get_sample_id(tp, i, &pos, is_inner, &def);
+		if (id == -1)
+			continue;
+		MLX5_ASSERT(id < (int)tp->devx_fp->num_samples);
+		if (id >= (int)tp->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return -1;
+		mask = mlx5_flex_get_bitfield(m, pos, map->width, map->shift);
+		if (def & RTE_BE32(mask) &&
+		    !mlx5dr_definer_get_flex_parser_fc(cd, tp->devx_fp->sample_ids[id], item_idx))
+			return rte_errno;
+		pos += map->width;
+	}
+	return 0;
+}
+
 static int
 mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 				struct mlx5dr_match_template *mt,
@@ -1581,6 +1717,11 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			ret = mlx5dr_definer_conv_item_meter_color(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_METER_COLOR;
 			break;
+		case RTE_FLOW_ITEM_TYPE_FLEX:
+			ret = mlx5dr_definer_conv_item_flex(&cd, items, i);
+			item_flags |= cd.tunnel ? MLX5_FLOW_ITEM_INNER_FLEX :
+				      MLX5_FLOW_ITEM_OUTER_FLEX;
+			break;
 		default:
 			DR_LOG(ERR, "Unsupported item type %d", items->type);
 			rte_errno = ENOTSUP;
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index e55be8720e..667d90405a 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -964,11 +964,13 @@ int
 mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
 	struct mlx5_devx_graph_node_attr node = {
 		.modify_field_select = 0,
 	};
-	uint32_t ids[8];
+	struct mlx5_ext_sample_id ids[8];
+	uint8_t anchor_id;
 	int ret;
 
 	if (!priv->sh->cdev->config.hca_attr.parse_graph_flex_node) {
@@ -1004,15 +1006,20 @@ mlx5_flex_parser_ecpri_alloc(struct rte_eth_dev *dev)
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->num = 2;
-	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num);
+	ret = mlx5_devx_cmd_query_parse_samples(prf->obj, ids, prf->num, &anchor_id);
 	if (ret) {
 		DRV_LOG(ERR, "Failed to query sample IDs.");
 		return (rte_errno == 0) ? -ENODEV : -rte_errno;
 	}
 	prf->offset[0] = 0x0;
 	prf->offset[1] = sizeof(uint32_t);
-	prf->ids[0] = ids[0];
-	prf->ids[1] = ids[1];
+	if (attr->ext_sample_id) {
+		prf->ids[0] = ids[0].sample_id;
+		prf->ids[1] = ids[1].sample_id;
+	} else {
+		prf->ids[0] = ids[0].id;
+		prf->ids[1] = ids[1].id;
+	}
 	return 0;
 }
 
@@ -1027,7 +1034,7 @@ static void
 mlx5_flex_parser_ecpri_release(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_ecpri_parser_profile *prf =	&priv->sh->ecpri_parser;
+	struct mlx5_ecpri_parser_profile *prf = &priv->sh->ecpri_parser;
 
 	if (prf->obj)
 		mlx5_devx_cmd_destroy(prf->obj);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 31982002ee..20b2529cc4 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1307,9 +1307,10 @@ struct mlx5_lag {
 struct mlx5_flex_parser_devx {
 	struct mlx5_list_entry entry;  /* List element at the beginning. */
 	uint32_t num_samples;
+	uint8_t anchor_id;
 	void *devx_obj;
 	struct mlx5_devx_graph_node_attr devx_conf;
-	uint32_t sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
+	struct mlx5_ext_sample_id sample_ids[MLX5_GRAPH_NODE_SAMPLE_NUM];
 };
 
 /* Pattern field descriptor - how to translate flex pattern into samples. */
@@ -2238,6 +2239,11 @@ void mlx5_flex_item_port_cleanup(struct rte_eth_dev *dev);
 void mlx5_flex_flow_translate_item(struct rte_eth_dev *dev, void *matcher,
 				   void *key, const struct rte_flow_item *item,
 				   bool is_inner);
+int mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			    uint32_t idx, uint32_t *pos,
+			    bool is_inner, uint32_t *def);
+uint32_t mlx5_flex_get_bitfield(const struct rte_flow_item_flex *item,
+				uint32_t pos, uint32_t width, uint32_t shift);
 int mlx5_flex_acquire_index(struct rte_eth_dev *dev,
 			    struct rte_flow_item_flex_handle *handle,
 			    bool acquire);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 1f57ecd6e1..c2f656e702 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1225,6 +1225,7 @@ struct rte_flow_pattern_template {
 	 * tag pattern item for representor matching.
 	 */
 	bool implicit_tag;
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Flow action template struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_flex.c b/drivers/net/mlx5/mlx5_flow_flex.c
index bec07b13c1..affec62384 100644
--- a/drivers/net/mlx5/mlx5_flow_flex.c
+++ b/drivers/net/mlx5/mlx5_flow_flex.c
@@ -113,7 +113,7 @@ mlx5_flex_free(struct mlx5_priv *priv, struct mlx5_flex_item *item)
 	}
 }
 
-static uint32_t
+uint32_t
 mlx5_flex_get_bitfield(const struct rte_flow_item_flex *item,
 		       uint32_t pos, uint32_t width, uint32_t shift)
 {
@@ -198,6 +198,50 @@ mlx5_flex_set_match_sample(void *misc4_m, void *misc4_v,
 	}
 #undef SET_FP_MATCH_SAMPLE_ID
 }
+
+/**
+ * Get the flex parser sample id and corresponding mask
+ * per shift and width information.
+ *
+ * @param[in] tp
+ *   Mlx5 flex item sample mapping handle.
+ * @param[in] idx
+ *   Mapping index.
+ * @param[in, out] pos
+ *   Where to search the value and mask.
+ * @param[in] is_inner
+ *   For inner matching or not.
+ * @param[in, def] def
+ *   Mask generated by mapping shift and width.
+ *
+ * @return
+ *   0 on success, -1 to ignore.
+ */
+int
+mlx5_flex_get_sample_id(const struct mlx5_flex_item *tp,
+			uint32_t idx, uint32_t *pos,
+			bool is_inner, uint32_t *def)
+{
+	const struct mlx5_flex_pattern_field *map = tp->map + idx;
+	uint32_t id = map->reg_id;
+
+	*def = (RTE_BIT64(map->width) - 1) << map->shift;
+	/* Skip placeholders for DUMMY fields. */
+	if (id == MLX5_INVALID_SAMPLE_REG_ID) {
+		*pos += map->width;
+		return -1;
+	}
+	MLX5_ASSERT(map->width);
+	MLX5_ASSERT(id < tp->devx_fp->num_samples);
+	if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
+		uint32_t num_samples = tp->devx_fp->num_samples / 2;
+
+		MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
+		MLX5_ASSERT(id < num_samples);
+		id += num_samples;
+	}
+	return id;
+}
 /**
  * Translate item pattern into matcher fields according to translation
  * array.
@@ -226,40 +270,38 @@ mlx5_flex_flow_translate_item(struct rte_eth_dev *dev,
 	void *misc4_m = MLX5_ADDR_OF(fte_match_param, matcher,
 				     misc_parameters_4);
 	void *misc4_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_4);
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
 	struct mlx5_flex_item *tp;
 	uint32_t i, pos = 0;
+	uint32_t sample_id;
 
 	RTE_SET_USED(dev);
 	MLX5_ASSERT(item->spec && item->mask);
 	spec = item->spec;
 	mask = item->mask;
 	tp = (struct mlx5_flex_item *)spec->handle;
-	MLX5_ASSERT(mlx5_flex_index(dev->data->dev_private, tp) >= 0);
+	MLX5_ASSERT(mlx5_flex_index(priv, tp) >= 0);
 	for (i = 0; i < tp->mapnum; i++) {
 		struct mlx5_flex_pattern_field *map = tp->map + i;
-		uint32_t id = map->reg_id;
-		uint32_t def = (RTE_BIT64(map->width) - 1) << map->shift;
-		uint32_t val, msk;
+		uint32_t val, msk, def;
+		int id = mlx5_flex_get_sample_id(tp, i, &pos, is_inner, &def);
 
-		/* Skip placeholders for DUMMY fields. */
-		if (id == MLX5_INVALID_SAMPLE_REG_ID) {
-			pos += map->width;
+		if (id == -1)
 			continue;
-		}
+		MLX5_ASSERT(id < (int)tp->devx_fp->num_samples);
+		if (id >= (int)tp->devx_fp->num_samples ||
+		    id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
 		val = mlx5_flex_get_bitfield(spec, pos, map->width, map->shift);
 		msk = mlx5_flex_get_bitfield(mask, pos, map->width, map->shift);
-		MLX5_ASSERT(map->width);
-		MLX5_ASSERT(id < tp->devx_fp->num_samples);
-		if (tp->tunnel_mode == FLEX_TUNNEL_MODE_MULTI && is_inner) {
-			uint32_t num_samples = tp->devx_fp->num_samples / 2;
-
-			MLX5_ASSERT(tp->devx_fp->num_samples % 2 == 0);
-			MLX5_ASSERT(id < num_samples);
-			id += num_samples;
-		}
+		if (attr->ext_sample_id)
+			sample_id = tp->devx_fp->sample_ids[id].sample_id;
+		else
+			sample_id = tp->devx_fp->sample_ids[id].id;
 		mlx5_flex_set_match_sample(misc4_m, misc4_v,
 					   def, msk & def, val & msk & def,
-					   tp->devx_fp->sample_ids[id], id);
+					   sample_id, id);
 		pos += map->width;
 	}
 }
@@ -1317,7 +1359,8 @@ mlx5_flex_parser_create_cb(void *list_ctx, void *ctx)
 	/* Query the firmware assigned sample ids. */
 	ret = mlx5_devx_cmd_query_parse_samples(fp->devx_obj,
 						fp->sample_ids,
-						fp->num_samples);
+						fp->num_samples,
+						&fp->anchor_id);
 	if (ret)
 		goto error;
 	DRV_LOG(DEBUG, "DEVx flex parser %p created, samples num: %u",
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 44953451d5..abd0ebbb23 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4331,6 +4331,36 @@ flow_hw_set_vlan_vid_construct(struct rte_eth_dev *dev,
 					      &modify_action);
 }
 
+static int
+flow_hw_flex_item_acquire(struct rte_eth_dev *dev,
+			  struct rte_flow_item_flex_handle *handle,
+			  uint8_t *flex_item)
+{
+	int index = mlx5_flex_acquire_index(dev, handle, false);
+
+	MLX5_ASSERT(index >= 0 && index <= (int)(sizeof(uint32_t) * CHAR_BIT));
+	if (index < 0)
+		return -1;
+	if (!(*flex_item & RTE_BIT32(index))) {
+		/* Don't count same flex item again. */
+		if (mlx5_flex_acquire_index(dev, handle, true) != index)
+			MLX5_ASSERT(false);
+		*flex_item |= (uint8_t)RTE_BIT32(index);
+	}
+	return 0;
+}
+
+static void
+flow_hw_flex_item_release(struct rte_eth_dev *dev, uint8_t *flex_item)
+{
+	while (*flex_item) {
+		int index = rte_bsf32(*flex_item);
+
+		mlx5_flex_release_index(dev, index);
+		*flex_item &= ~(uint8_t)RTE_BIT32(index);
+	}
+}
+
 /**
  * Create flow action template.
  *
@@ -4732,6 +4762,7 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_ICMP:
 		case RTE_FLOW_ITEM_TYPE_ICMP6:
 		case RTE_FLOW_ITEM_TYPE_CONNTRACK:
+		case RTE_FLOW_ITEM_TYPE_FLEX:
 			break;
 		case RTE_FLOW_ITEM_TYPE_INTEGRITY:
 			/*
@@ -4809,6 +4840,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 		.mask = &tag_m,
 		.last = NULL
 	};
+	unsigned int i = 0;
 
 	if (flow_hw_pattern_validate(dev, attr, items, error))
 		return NULL;
@@ -4868,6 +4900,19 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 			it->implicit_tag = true;
 		mlx5_free(copied_items);
 	}
+	for (i = 0; items[i].type != RTE_FLOW_ITEM_TYPE_END; ++i) {
+		if (items[i].type == RTE_FLOW_ITEM_TYPE_FLEX) {
+			const struct rte_flow_item_flex *spec =
+				(const struct rte_flow_item_flex *)items[i].spec;
+			struct rte_flow_item_flex_handle *handle = spec->handle;
+
+			if (flow_hw_flex_item_acquire(dev, handle, &it->flex_item)) {
+				claim_zero(mlx5dr_match_template_destroy(it->mt));
+				mlx5_free(it);
+				return NULL;
+			}
+		}
+	}
 	__atomic_fetch_add(&it->refcnt, 1, __ATOMIC_RELAXED);
 	LIST_INSERT_HEAD(&priv->flow_hw_itt, it, next);
 	return it;
@@ -4887,7 +4932,7 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 			      struct rte_flow_pattern_template *template,
 			      struct rte_flow_error *error __rte_unused)
 {
@@ -4900,6 +4945,7 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "item template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	claim_zero(mlx5dr_match_template_destroy(template->mt));
 	mlx5_free(template);
 	return 0;
-- 
2.27.0

[RFC 7/9] net/mlx5/hws: add flex item modify field implementation

[Rongwei Liu]

Add flex item modify field HWS implementation.
The minimum modify boundary is one byte.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 doc/guides/nics/mlx5.rst        |   1 +
 drivers/common/mlx5/mlx5_prm.h  |   1 +
 drivers/net/mlx5/mlx5_flow.h    |   3 +
 drivers/net/mlx5/mlx5_flow_dv.c | 165 +++++++++++++++++++++++++++++---
 drivers/net/mlx5/mlx5_flow_hw.c |  14 ++-
 5 files changed, 171 insertions(+), 13 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 397e04fbed..26d68d86f4 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -107,6 +107,7 @@ Features
 - Sub-Function.
 - Matching on represented port.
 - Matching on flex item with specific pattern.
+- Modify flex item field.
 
 
 Limitations
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index b080599570..74fc8d9936 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -753,6 +753,7 @@ enum mlx5_modification_field {
 	MLX5_MODI_OUT_IP_ECN = 0x73,
 	MLX5_MODI_TUNNEL_HDR_DW_1 = 0x75,
 	MLX5_MODI_GTPU_FIRST_EXT_DW_0 = 0x76,
+	MLX5_MODI_INVALID = INT_MAX,
 };
 
 /* Total number of metadata reg_c's. */
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index c2f656e702..b54eaf64f4 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1080,6 +1080,8 @@ struct field_modify_info {
 	uint32_t size; /* Size of field in protocol header, in bytes. */
 	uint32_t offset; /* Offset of field in protocol header, in bytes. */
 	enum mlx5_modification_field id;
+	uint32_t shift;
+	uint8_t is_flex; /* Temporary indicator for flex item modify filed WA. */
 };
 
 /* HW steering flow attributes. */
@@ -1244,6 +1246,7 @@ struct rte_flow_actions_template {
 	uint16_t mhdr_off; /* Offset of DR modify header action. */
 	uint32_t refcnt; /* Reference counter. */
 	uint16_t rx_cpy_pos; /* Action position of Rx metadata to be copied. */
+	uint8_t flex_item; /* flex item index. */
 };
 
 /* Jump action struct. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 7ca909999b..110d08b271 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -414,10 +414,15 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			++field;
 			continue;
 		}
-		/* Deduce actual data width in bits from mask value. */
-		off_b = rte_bsf32(mask) + carry_b;
-		size_b = sizeof(uint32_t) * CHAR_BIT -
-			 off_b - __builtin_clz(mask);
+		if (type == MLX5_MODIFICATION_TYPE_COPY && field->is_flex) {
+			off_b = 32 - field->shift + carry_b - field->size * CHAR_BIT;
+			size_b = field->size * CHAR_BIT - carry_b;
+		} else {
+			/* Deduce actual data width in bits from mask value. */
+			off_b = rte_bsf32(mask) + carry_b;
+			size_b = sizeof(uint32_t) * CHAR_BIT -
+				 off_b - __builtin_clz(mask);
+		}
 		MLX5_ASSERT(size_b);
 		actions[i] = (struct mlx5_modification_cmd) {
 			.action_type = type,
@@ -437,40 +442,46 @@ flow_dv_convert_modify_action(struct rte_flow_item *item,
 			 * Destination field overflow. Copy leftovers of
 			 * a source field to the next destination field.
 			 */
-			carry_b = 0;
 			if ((size_b > dcopy->size * CHAR_BIT - dcopy->offset) &&
 			    dcopy->size != 0) {
 				actions[i].length =
 					dcopy->size * CHAR_BIT - dcopy->offset;
-				carry_b = actions[i].length;
+				carry_b += actions[i].length;
 				next_field = false;
+			} else {
+				carry_b = 0;
 			}
 			/*
 			 * Not enough bits in a source filed to fill a
 			 * destination field. Switch to the next source.
 			 */
 			if ((size_b < dcopy->size * CHAR_BIT - dcopy->offset) &&
-			    (size_b == field->size * CHAR_BIT - off_b)) {
-				actions[i].length =
-					field->size * CHAR_BIT - off_b;
+			    ((size_b == field->size * CHAR_BIT - off_b) ||
+			     field->is_flex)) {
+				actions[i].length = size_b;
 				dcopy->offset += actions[i].length;
 				next_dcopy = false;
 			}
-			if (next_dcopy)
-				++dcopy;
 		} else {
 			MLX5_ASSERT(item->spec);
 			data = flow_dv_fetch_field((const uint8_t *)item->spec +
 						   field->offset, field->size);
 			/* Shift out the trailing masked bits from data. */
 			data = (data & mask) >> off_b;
+			if (field->is_flex)
+				actions[i].offset = 32 - field->shift - field->size * CHAR_BIT;
 			actions[i].data1 = rte_cpu_to_be_32(data);
 		}
 		/* Convert entire record to expected big-endian format. */
 		actions[i].data0 = rte_cpu_to_be_32(actions[i].data0);
+		if ((type != MLX5_MODIFICATION_TYPE_COPY ||
+		     dcopy->id != (enum mlx5_modification_field)UINT32_MAX) &&
+		    field->id != (enum mlx5_modification_field)UINT32_MAX)
+			++i;
+		if (next_dcopy && type == MLX5_MODIFICATION_TYPE_COPY)
+			++dcopy;
 		if (next_field)
 			++field;
-		++i;
 	} while (field->size);
 	if (resource->actions_num == i)
 		return rte_flow_error_set(error, EINVAL,
@@ -1421,6 +1432,131 @@ flow_modify_info_mask_32_masked(uint32_t length, uint32_t off, uint32_t post_mas
 	return rte_cpu_to_be_32(mask & post_mask);
 }
 
+static void
+mlx5_modify_flex_item(const struct rte_eth_dev *dev,
+		      const struct mlx5_flex_item *flex,
+		      const struct rte_flow_action_modify_data *data,
+		      struct field_modify_info *info,
+		      uint32_t *mask, uint32_t width)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_flex_attr *attr = &priv->sh->cdev->config.hca_attr.flex;
+	uint32_t i, j;
+	int id = 0;
+	uint32_t pos = 0;
+	const struct mlx5_flex_pattern_field *map;
+	uint32_t offset = data->offset;
+	uint32_t width_left = width;
+	uint32_t def;
+	uint32_t cur_width = 0;
+	uint32_t tmp_ofs;
+	uint32_t idx = 0;
+	struct field_modify_info tmp;
+	int tmp_id;
+
+	if (!attr->ext_sample_id) {
+		DRV_LOG(ERR, "FW doesn't support modify field with flex item.");
+		return;
+	}
+	/*
+	 * search for the mapping instance until Accumulated width is no
+	 * less than data->offset.
+	 */
+	for (i = 0; i < flex->mapnum; i++) {
+		if (flex->map[i].width + pos > data->offset)
+			break;
+		pos += flex->map[i].width;
+	}
+	if (i >= flex->mapnum)
+		return;
+	tmp_ofs = pos < data->offset ? data->offset - pos : 0;
+	for (j = i; i < flex->mapnum && width_left > 0; ) {
+		map = flex->map + i;
+		id = mlx5_flex_get_sample_id(flex, i, &pos, false, &def);
+		if (id == -1) {
+			i++;
+			/* All left length is dummy */
+			if (pos >= data->offset + width)
+				return;
+			cur_width = map->width;
+		/* One mapping instance covers the whole width. */
+		} else if (pos + map->width >= (data->offset + width)) {
+			cur_width = width_left;
+		} else {
+			cur_width = cur_width + map->width - tmp_ofs;
+			pos += map->width;
+			/*
+			 * Continue to search next until:
+			 * 1. Another flex parser ID.
+			 * 2. Width has been covered.
+			 */
+			for (j = i + 1; j < flex->mapnum; j++) {
+				tmp_id = mlx5_flex_get_sample_id(flex, j, &pos, false, &def);
+				if (tmp_id == -1) {
+					i = j;
+					pos -= flex->map[j].width;
+					break;
+				}
+				if (id >= (int)flex->devx_fp->num_samples ||
+				    id >= MLX5_GRAPH_NODE_SAMPLE_NUM ||
+				    tmp_id >= (int)flex->devx_fp->num_samples ||
+				    tmp_id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+					return;
+				if (flex->devx_fp->sample_ids[id].id !=
+						flex->devx_fp->sample_ids[tmp_id].id ||
+				    flex->map[j].shift != flex->map[j - 1].width +
+							  flex->map[j - 1].shift) {
+					i = j;
+					break;
+				}
+				if ((pos + flex->map[j].width) >= (data->offset + width)) {
+					cur_width = width_left;
+					break;
+				}
+				pos += flex->map[j].width;
+				cur_width += flex->map[j].width;
+			}
+		}
+		if (cur_width > width_left)
+			cur_width = width_left;
+		else if (cur_width < width_left && (j == flex->mapnum || i == flex->mapnum))
+			return;
+
+		MLX5_ASSERT(id < (int)flex->devx_fp->num_samples);
+		if (id >= (int)flex->devx_fp->num_samples || id >= MLX5_GRAPH_NODE_SAMPLE_NUM)
+			return;
+		/* Use invalid entry as placeholder for DUMMY mapping. */
+		info[idx] = (struct field_modify_info){cur_width / CHAR_BIT, offset / CHAR_BIT,
+			     id == -1 ? MLX5_MODI_INVALID :
+			     (enum mlx5_modification_field)
+			     flex->devx_fp->sample_ids[id].modify_field_id,
+			     map->shift + tmp_ofs, 1};
+		offset += cur_width;
+		width_left -= cur_width;
+		if (!mask) {
+			info[idx].offset = (32 - cur_width - map->shift - tmp_ofs);
+			info[idx].size = cur_width / CHAR_BIT + info[idx].offset / CHAR_BIT;
+		}
+		cur_width = 0;
+		tmp_ofs = 0;
+		idx++;
+	}
+	if (unlikely(width_left > 0)) {
+		MLX5_ASSERT(false);
+		return;
+	}
+	if (mask)
+		memset(mask, 0xff, data->offset / CHAR_BIT + width / CHAR_BIT);
+	/* Re-order the info to follow IPv6 address. */
+	for (i = 0; i < idx / 2; i++) {
+		tmp = info[i];
+		MLX5_ASSERT(info[i].id);
+		MLX5_ASSERT(info[idx - 1 - i].id);
+		info[i] = info[idx - 1 - i];
+		info[idx - 1 - i] = tmp;
+	}
+}
+
 void
 mlx5_flow_field_id_to_modify_info
 		(const struct rte_flow_action_modify_data *data,
@@ -1883,6 +2019,11 @@ mlx5_flow_field_id_to_modify_info
 				info[idx].offset = data->offset;
 		}
 		break;
+	case RTE_FLOW_FIELD_FLEX_ITEM:
+		MLX5_ASSERT(data->flex_handle != NULL && !(data->offset & 0x7));
+		mlx5_modify_flex_item(dev, (const struct mlx5_flex_item *)data->flex_handle,
+				      data, info, mask, width);
+		break;
 	case RTE_FLOW_FIELD_POINTER:
 	case RTE_FLOW_FIELD_VALUE:
 	default:
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index abd0ebbb23..a023c2051b 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4550,6 +4550,17 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
 			at->actions[i].conf = actions->conf;
 			at->masks[i].conf = masks->conf;
 		}
+		if (actions->type == RTE_FLOW_ACTION_TYPE_MODIFY_FIELD) {
+			const struct rte_flow_action_modify_field *info = actions->conf;
+
+			if ((info->dst.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			     flow_hw_flex_item_acquire(dev, info->dst.flex_handle,
+						       &at->flex_item)) ||
+			     (info->src.field == RTE_FLOW_FIELD_FLEX_ITEM &&
+			      flow_hw_flex_item_acquire(dev, info->src.flex_handle,
+							&at->flex_item)))
+				goto error;
+		}
 	}
 	at->tmpl = flow_hw_dr_actions_template_create(at);
 	if (!at->tmpl)
@@ -4581,7 +4592,7 @@ flow_hw_actions_template_create(struct rte_eth_dev *dev,
  *   0 on success, a negative errno value otherwise and rte_errno is set.
  */
 static int
-flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
+flow_hw_actions_template_destroy(struct rte_eth_dev *dev,
 				 struct rte_flow_actions_template *template,
 				 struct rte_flow_error *error __rte_unused)
 {
@@ -4594,6 +4605,7 @@ flow_hw_actions_template_destroy(struct rte_eth_dev *dev __rte_unused,
 				   "action template in using");
 	}
 	LIST_REMOVE(template, next);
+	flow_hw_flex_item_release(dev, &template->flex_item);
 	if (template->tmpl)
 		mlx5dr_action_template_destroy(template->tmpl);
 	mlx5_free(template);
-- 
2.27.0

[RFC 8/9] net/mlx5: return error for sws modify field

[Rongwei Liu]

Return unsupported error message when application tries to
modify flex item field.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 110d08b271..f9419463eb 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -4824,10 +4824,18 @@ flow_dv_validate_action_modify_hdr(const uint64_t action_flags,
 				   const struct rte_flow_action *action,
 				   struct rte_flow_error *error)
 {
+	const struct rte_flow_action_modify_field *action_modify_field = action->conf;
+
 	if (action->type != RTE_FLOW_ACTION_TYPE_DEC_TTL && !action->conf)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION_CONF,
 					  NULL, "action configuration not set");
+	if (action_modify_field->src.field == RTE_FLOW_FIELD_FLEX_ITEM ||
+	    action_modify_field->dst.field == RTE_FLOW_FIELD_FLEX_ITEM)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"flex item fields modification"
+				" is not supported");
 	if (action_flags & MLX5_FLOW_ACTION_ENCAP)
 		return rte_flow_error_set(error, EINVAL,
 					  RTE_FLOW_ERROR_TYPE_ACTION, NULL,
@@ -5153,17 +5161,16 @@ flow_dv_validate_action_modify_field(struct rte_eth_dev *dev,
 	struct mlx5_hca_attr *hca_attr = &priv->sh->cdev->config.hca_attr;
 	const struct rte_flow_action_modify_field *action_modify_field =
 		action->conf;
-	uint32_t dst_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->dst.field,
-				-1, attr, error);
-	uint32_t src_width = mlx5_flow_item_field_width(dev,
-				action_modify_field->src.field,
-				dst_width, attr, error);
+	uint32_t dst_width, src_width;
 
 	ret = flow_dv_validate_action_modify_hdr(action_flags, action, error);
 	if (ret)
 		return ret;
 
+	dst_width = mlx5_flow_item_field_width(dev, action_modify_field->dst.field,
+					       -1, attr, error);
+	src_width = mlx5_flow_item_field_width(dev, action_modify_field->src.field,
+					       dst_width, attr, error);
 	if (action_modify_field->width == 0)
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
-- 
2.27.0

[RFC 9/9] app/testpmd: raw encap with flex item support

[Rongwei Liu]

Application should retrieve raw_encap buffer from
spec->pattern if it is flex item.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
---
 app/test-pmd/cmdline_flow.c | 26 +++++++++++++++++---------
 1 file changed, 17 insertions(+), 9 deletions(-)

diff --git a/app/test-pmd/cmdline_flow.c b/app/test-pmd/cmdline_flow.c
index 9037432cc8..3b2975ee83 100644
--- a/app/test-pmd/cmdline_flow.c
+++ b/app/test-pmd/cmdline_flow.c
@@ -11193,6 +11193,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 	uint16_t proto = 0;
 	uint16_t idx = in->port; /* We borrow port field as index */
 	int gtp_psc = -1; /* GTP PSC option index. */
+	const void *src_spec;
 
 	if (in->command == SET_SAMPLE_ACTIONS)
 		return cmd_set_raw_parsed_sample(in);
@@ -11216,6 +11217,7 @@ cmd_set_raw_parsed(const struct buffer *in)
 		item = in->args.vc.pattern + i;
 		if (item->spec == NULL)
 			item->spec = flow_item_default_mask(item);
+		src_spec = item->spec;
 		switch (item->type) {
 		case RTE_FLOW_ITEM_TYPE_ETH:
 			size = sizeof(struct rte_ether_hdr);
@@ -11343,9 +11345,13 @@ cmd_set_raw_parsed(const struct buffer *in)
 			size = sizeof(struct rte_flow_item_pfcp);
 			break;
 		case RTE_FLOW_ITEM_TYPE_FLEX:
-			size = item->spec ?
-				((const struct rte_flow_item_flex *)
-				item->spec)->length : 0;
+			if (item->spec != NULL) {
+				size = ((const struct rte_flow_item_flex *)item->spec)->length;
+				src_spec = ((const struct rte_flow_item_flex *)item->spec)->pattern;
+			} else {
+				size = 0;
+				src_spec = NULL;
+			}
 			break;
 		case RTE_FLOW_ITEM_TYPE_GRE_OPTION:
 			size = 0;
@@ -11378,12 +11384,14 @@ cmd_set_raw_parsed(const struct buffer *in)
 			fprintf(stderr, "Error - Not supported item\n");
 			goto error;
 		}
-		*total_size += size;
-		rte_memcpy(data_tail - (*total_size), item->spec, size);
-		/* update some fields which cannot be set by cmdline */
-		update_fields((data_tail - (*total_size)), item,
-			      upper_layer);
-		upper_layer = proto;
+		if (size) {
+			*total_size += size;
+			rte_memcpy(data_tail - (*total_size), src_spec, size);
+			/* update some fields which cannot be set by cmdline */
+			update_fields((data_tail - (*total_size)), item,
+				      upper_layer);
+			upper_layer = proto;
+		}
 	}
 	if (verbose_level & 0x1)
 		printf("total data size is %zu\n", (*total_size));
-- 
2.27.0