[PATCH v1 00/23] net/mlx5: support Geneve and options for HWS

[PATCH v1 00/23] net/mlx5: support Geneve and options for HWS

[Michael Baum]

Add HWS support for both GENEVE and GENEVE TLV option headers.
This patchset supports:

 - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE" flow item.
 - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE_OPT" flow item.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_VNI" for modify field flow
   action.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_TYPE" for modify field
   flow action.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_CLASS" for modify field
   flow action.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_DATA" for modify field
   flow action.

The GENEVE TLV options support using flex parser.
The profile should be specified to either 8 for multiple option or 0 for
single option.
A new API is added to create the GENEVE option parser before using it in
templates API.

Alex Vesker (4):
  net/mlx5/hws: fix tunnel protocol checks
  net/mlx5/hws: increase hl size for future compatibility
  net/mlx5/hws: support GENEVE matching
  net/mlx5/hws: support GENEVE options header

Michael Baum (19):
  common/mlx5: fix duplicate read of general capabilities
  common/mlx5: fix query sample info capability
  net/mlx5: remove GENEVE options length limitation
  net/mlx5: fix GENEVE option item translation
  common/mlx5: add system image GUID attribute
  common/mlx5: add GENEVE TLV option attribute structure
  common/mlx5: add PRM attribute for TLV sample
  common/mlx5: add sample info query syndrome into error log
  common/mlx5: query GENEVE option sample ID from HCA attr
  common/mlx5: add function to query GENEVE TLV option
  net/mlx5: add physical device handle
  net/mlx5: add GENEVE TLV options parser API
  net/mlx5: add API to expose GENEVE option FW information
  net/mlx5: add testpmd support for GENEVE TLV parser
  net/mlx5: add support for GENEVE and option item in HWS
  net/mlx5: add GENEVE option support for profile 0
  net/mlx5: add GENEVE option support for group 0
  net/mlx5: add support for GENEVE VNI modify field
  net/mlx5: add support for modify GENEVE option header

 doc/guides/nics/mlx5.rst               |  251 +++++-
 doc/guides/platform/mlx5.rst           |    2 +
 doc/guides/rel_notes/release_24_03.rst |    9 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  139 +++-
 drivers/common/mlx5/mlx5_devx_cmds.h   |   29 +-
 drivers/common/mlx5/mlx5_prm.h         |   20 +-
 drivers/common/mlx5/version.map        |    1 +
 drivers/net/mlx5/hws/mlx5dr_definer.c  |  277 ++++++-
 drivers/net/mlx5/hws/mlx5dr_definer.h  |   49 +-
 drivers/net/mlx5/meson.build           |    1 +
 drivers/net/mlx5/mlx5.c                |  115 ++-
 drivers/net/mlx5/mlx5.h                |   21 +
 drivers/net/mlx5/mlx5_flow.c           |   30 +
 drivers/net/mlx5/mlx5_flow.h           |   92 ++-
 drivers/net/mlx5/mlx5_flow_dv.c        |  158 ++--
 drivers/net/mlx5/mlx5_flow_geneve.c    | 1011 ++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_flow_hw.c        |  127 ++-
 drivers/net/mlx5/mlx5_testpmd.c        |  556 ++++++++++++-
 drivers/net/mlx5/rte_pmd_mlx5.h        |  102 +++
 drivers/net/mlx5/version.map           |    3 +
 20 files changed, 2809 insertions(+), 184 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_flow_geneve.c

-- 
2.25.1

[PATCH v1 01/23] common/mlx5: fix duplicate read of general capabilities

[Michael Baum]

General object types support is indicated in bitmap general_obj_types,
which is part of HCA capabilities list.
This bitmap was read multiple times, and each time a different bit was
extracted.

Previous patch optimized the code, reading the bitmap once into a local
variable, and then extracting the required bits.
However, it missed few of them which still read the bitmap for
themselves. In addition, for other readings, it moved them to use local
variable without removing the old reading, and they are read twice.

This patch moves them all to use the local variable and removes all
duplications.

Fixes: 876d4702b141 ("common/mlx5: optimize read of general capabilities")
Cc: dekelp@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 18 ++++--------------
 1 file changed, 4 insertions(+), 14 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 4d8818924a..41f6e0a727 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -966,18 +966,6 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->max_geneve_tlv_option_data_len = MLX5_GET(cmd_hca_cap, hcattr,
 			max_geneve_tlv_option_data_len);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
-	attr->qos.flow_meter_aso_sup = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
-			      MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_METER_ASO);
-	attr->vdpa.valid = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
-			      MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q);
-	attr->vdpa.queue_counters_valid = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-							general_obj_types) &
-				  MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS);
-	attr->parse_graph_flex_node = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
-			      MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE);
 	attr->wqe_index_ignore = MLX5_GET(cmd_hca_cap, hcattr,
 					  wqe_index_ignore_cap);
 	attr->cross_channel = MLX5_GET(cmd_hca_cap, hcattr, cd);
@@ -1001,6 +989,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	/* Read the general_obj_types bitmap and extract the relevant bits. */
 	general_obj_types_supported = MLX5_GET64(cmd_hca_cap, hcattr,
 						 general_obj_types);
+	attr->qos.flow_meter_aso_sup =
+			!!(general_obj_types_supported &
+			   MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_METER_ASO);
 	attr->vdpa.valid = !!(general_obj_types_supported &
 			      MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q);
 	attr->vdpa.queue_counters_valid =
@@ -1074,8 +1065,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 		MLX5_GET(cmd_hca_cap, hcattr, umr_modify_entity_size_disabled);
 	attr->wait_on_time = MLX5_GET(cmd_hca_cap, hcattr, wait_on_time);
 	attr->crypto = MLX5_GET(cmd_hca_cap, hcattr, crypto);
-	attr->ct_offload = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
+	attr->ct_offload = !!(general_obj_types_supported &
 			      MLX5_GENERAL_OBJ_TYPES_CAP_CONN_TRACK_OFFLOAD);
 	attr->rq_delay_drop = MLX5_GET(cmd_hca_cap, hcattr, rq_delay_drop);
 	attr->nic_flow_table = MLX5_GET(cmd_hca_cap, hcattr, nic_flow_table);
-- 
2.25.1

[PATCH v1 02/23] common/mlx5: fix query sample info capability

[Michael Baum]

Query sample info operation might be used by either Geneve TLV option or
parse graph. Each operations can be supported regardless to another
according the configured profile.

In current implementation, the query sample info capability is turn on
only when parse graph operation is supported adding unnecessary
requirement for Geneve TLV option.

This patch adds different cap for Geneve TLV option.

Fixes: bc0a9303ed6a ("net/mlx5: adopt new sample ID")
Cc: rongweil@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 6 ++++--
 drivers/common/mlx5/mlx5_devx_cmds.h | 1 +
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 41f6e0a727..3eeb27fc3f 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -965,6 +965,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			max_geneve_tlv_options);
 	attr->max_geneve_tlv_option_data_len = MLX5_GET(cmd_hca_cap, hcattr,
 			max_geneve_tlv_option_data_len);
+	attr->query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
+						 query_match_sample_info);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
 	attr->wqe_index_ignore = MLX5_GET(cmd_hca_cap, hcattr,
 					  wqe_index_ignore_cap);
@@ -1094,8 +1096,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			(ctx, &attr->flex);
 		if (rc)
 			return -1;
-		attr->flex.query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
-							      query_match_sample_info);
+		attr->flex.query_match_sample_info =
+						attr->query_match_sample_info;
 	}
 	if (attr->crypto) {
 		attr->aes_xts = MLX5_GET(cmd_hca_cap, hcattr, aes_xts) ||
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 7f23e925a5..b814c8becc 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -315,6 +315,7 @@ struct mlx5_hca_attr {
 	uint32_t flow_counter_bulk_log_granularity:5;
 	uint32_t alloc_flow_counter_pd:1;
 	uint32_t flow_counter_access_aso:1;
+	uint32_t query_match_sample_info:1;
 	uint32_t flow_access_aso_opc_mod:8;
 	uint32_t cross_vhca:1;
 	uint32_t lag_rx_port_affinity:1;
-- 
2.25.1

[PATCH v1 03/23] net/mlx5/hws: fix tunnel protocol checks

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

Align GRE, GTPU and VXLAN tunnel protocols to fail
in case the packet is already tunneled. Also use local
defines for protocol UDP ports for better layering of
mlx5dr API.

Fixes: c55c2bf35333 ("net/mlx5/hws: add definer layer")
Fixes: 5bf14a4beb1a ("net/mlx5/hws: support matching on MPLSoUDP")
Cc: valex@nvidia.com
Cc: erezsh@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Alex Vesker <valex@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 39 +++++++++++++--------------
 1 file changed, 19 insertions(+), 20 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 0b60479406..bab1869369 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -8,8 +8,9 @@
 #define BAD_PORT	0xBAD
 #define ETH_TYPE_IPV4_VXLAN	0x0800
 #define ETH_TYPE_IPV6_VXLAN	0x86DD
-#define ETH_VXLAN_DEFAULT_PORT	4789
-#define IP_UDP_PORT_MPLS	6635
+#define UDP_GTPU_PORT	2152
+#define UDP_VXLAN_PORT	4789
+#define UDP_PORT_MPLS	6635
 #define UDP_ROCEV2_PORT	4791
 #define DR_FLOW_LAYER_TUNNEL_NO_MPLS (MLX5_FLOW_LAYER_TUNNEL & ~MLX5_FLOW_LAYER_MPLS)
 
@@ -158,7 +159,7 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		tcp_protocol,		STE_TCP,		rte_flow_item_tcp) \
 	X(SET_BE16,	tcp_src_port,		v->hdr.src_port,	rte_flow_item_tcp) \
 	X(SET_BE16,	tcp_dst_port,		v->hdr.dst_port,	rte_flow_item_tcp) \
-	X(SET,		gtp_udp_port,		RTE_GTPU_UDP_PORT,	rte_flow_item_gtp) \
+	X(SET,		gtp_udp_port,		UDP_GTPU_PORT,		rte_flow_item_gtp) \
 	X(SET_BE32,	gtp_teid,		v->hdr.teid,		rte_flow_item_gtp) \
 	X(SET,		gtp_msg_type,		v->hdr.msg_type,	rte_flow_item_gtp) \
 	X(SET,		gtp_ext_flag,		!!v->hdr.gtp_hdr_info,	rte_flow_item_gtp) \
@@ -166,8 +167,8 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		gtp_ext_hdr_pdu,	v->hdr.type,		rte_flow_item_gtp_psc) \
 	X(SET,		gtp_ext_hdr_qfi,	v->hdr.qfi,		rte_flow_item_gtp_psc) \
 	X(SET,		vxlan_flags,		v->flags,		rte_flow_item_vxlan) \
-	X(SET,		vxlan_udp_port,		ETH_VXLAN_DEFAULT_PORT,	rte_flow_item_vxlan) \
-	X(SET,		mpls_udp_port,		IP_UDP_PORT_MPLS,	rte_flow_item_mpls) \
+	X(SET,		vxlan_udp_port,		UDP_VXLAN_PORT,		rte_flow_item_vxlan) \
+	X(SET,		mpls_udp_port,		UDP_PORT_MPLS,		rte_flow_item_mpls) \
 	X(SET,		source_qp,		v->queue,		mlx5_rte_flow_item_sq) \
 	X(SET,		tag,			v->data,		rte_flow_item_tag) \
 	X(SET,		metadata,		v->data,		rte_flow_item_meta) \
@@ -1170,6 +1171,12 @@ mlx5dr_definer_conv_item_gtp(struct mlx5dr_definer_conv_data *cd,
 	const struct rte_flow_item_gtp *m = item->mask;
 	struct mlx5dr_definer_fc *fc;
 
+	if (cd->tunnel) {
+		DR_LOG(ERR, "Inner GTPU item not supported");
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
 	/* Overwrite GTPU dest port if not present */
 	fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, false)];
 	if (!fc->tag_set && !cd->relaxed) {
@@ -1344,9 +1351,13 @@ mlx5dr_definer_conv_item_vxlan(struct mlx5dr_definer_conv_data *cd,
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
-	/* In order to match on VXLAN we must match on ether_type, ip_protocol
-	 * and l4_dport.
-	 */
+	if (inner) {
+		DR_LOG(ERR, "Inner VXLAN item not supported");
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
+	/* In order to match on VXLAN we must match on ip_protocol and l4_dport */
 	if (!cd->relaxed) {
 		fc = &cd->fc[DR_CALC_FNAME(IP_PROTOCOL, inner)];
 		if (!fc->tag_set) {
@@ -1369,12 +1380,6 @@ mlx5dr_definer_conv_item_vxlan(struct mlx5dr_definer_conv_data *cd,
 		return 0;
 
 	if (m->flags) {
-		if (inner) {
-			DR_LOG(ERR, "Inner VXLAN flags item not supported");
-			rte_errno = ENOTSUP;
-			return rte_errno;
-		}
-
 		fc = &cd->fc[MLX5DR_DEFINER_FNAME_VXLAN_FLAGS];
 		fc->item_idx = item_idx;
 		fc->tag_set = &mlx5dr_definer_vxlan_flags_set;
@@ -1384,12 +1389,6 @@ mlx5dr_definer_conv_item_vxlan(struct mlx5dr_definer_conv_data *cd,
 	}
 
 	if (!is_mem_zero(m->vni, 3)) {
-		if (inner) {
-			DR_LOG(ERR, "Inner VXLAN vni item not supported");
-			rte_errno = ENOTSUP;
-			return rte_errno;
-		}
-
 		fc = &cd->fc[MLX5DR_DEFINER_FNAME_VXLAN_VNI];
 		fc->item_idx = item_idx;
 		fc->tag_set = &mlx5dr_definer_vxlan_vni_set;
-- 
2.25.1

[PATCH v1 04/23] net/mlx5: remove GENEVE options length limitation

[Michael Baum]

GENEVE header has field named "opt_len" describing the total length of
all GENEVE options in 4-byte granularity.

In SW sreering implementation, only single option with single DW data is
supported. When matching on GENEVE option data is requested, matching on
"opt_len" field is added according to given option length.

This behaveior assumes that only packets with single option can be
matched, but it is wrong, packet with a few option can be matched but
only one of them can match its value.

This patch removes the "opt_len" matching unless user ask it explicitly.

Fixes: e440d6cf589e ("net/mlx5: add GENEVE TLV option flow translation")
Cc: shirik@nvidia.com

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 16 ----------------
 1 file changed, 16 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 115d730317..62ca742654 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -9986,7 +9986,6 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 	const struct rte_flow_item_geneve_opt *geneve_opt_m;
 	const struct rte_flow_item_geneve_opt *geneve_opt_v;
 	const struct rte_flow_item_geneve_opt *geneve_opt_vv = item->spec;
-	void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
 	void *misc3_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_3);
 	rte_be32_t opt_data_key = 0, opt_data_mask = 0;
 	uint32_t *data;
@@ -10005,21 +10004,6 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 			return ret;
 		}
 	}
-	/*
-	 * Set the option length in GENEVE header if not requested.
-	 * The GENEVE TLV option length is expressed by the option length field
-	 * in the GENEVE header.
-	 * If the option length was not requested but the GENEVE TLV option item
-	 * is present we set the option length field implicitly.
-	 */
-	if (!MLX5_GET16(fte_match_set_misc, misc_v, geneve_opt_len)) {
-		if (key_type & MLX5_SET_MATCHER_M)
-			MLX5_SET(fte_match_set_misc, misc_v, geneve_opt_len,
-				 MLX5_GENEVE_OPTLEN_MASK);
-		else
-			MLX5_SET(fte_match_set_misc, misc_v, geneve_opt_len,
-				 geneve_opt_v->option_len + 1);
-	}
 	/* Set the data. */
 	if (key_type == MLX5_SET_MATCHER_SW_V)
 		data = geneve_opt_vv->data;
-- 
2.25.1

[PATCH v1 05/23] net/mlx5: fix GENEVE option item translation

[Michael Baum]

The "flow_dv_translate_item_geneve_opt()" function is called twice per
flow rule, for either matcher focusing the mask or value focusing the
spec.
The spec is always provided and its field "option_len" indicates the
data size for both spec and mask. For using it, function has another
pointer "geneve_opt_vv" representing the spec regardless to focusing
while the "geneve_opt_v" pointer represents the mask for matcher and
spec for rule creation.

The current implementation has 2 issues:
1. geneve_opt_v get the spec in rule creation as sane as geneve_opt_vv,
   but function use if-else which is bacicly has same value.
2. function uses "option_len" from "geneve_opt_v" instead of
   "geneve_opt_v" even when the focus is on mask, for HWS the mask value
   may be 0 even data is valid.

This patch refactors the function implementation to avoid those issues.

Fixes: cd4ab742064a ("net/mlx5: split flow item matcher and value translation")
Cc: suanmingm@nvidia.com

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 28 +++++++++++-----------------
 1 file changed, 11 insertions(+), 17 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 62ca742654..f8e364dfdb 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -9985,13 +9985,13 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 {
 	const struct rte_flow_item_geneve_opt *geneve_opt_m;
 	const struct rte_flow_item_geneve_opt *geneve_opt_v;
-	const struct rte_flow_item_geneve_opt *geneve_opt_vv = item->spec;
+	const struct rte_flow_item_geneve_opt *orig_spec = item->spec;
 	void *misc3_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_3);
 	rte_be32_t opt_data_key = 0, opt_data_mask = 0;
-	uint32_t *data;
+	size_t option_byte_len;
 	int ret = 0;
 
-	if (MLX5_ITEM_VALID(item, key_type))
+	if (MLX5_ITEM_VALID(item, key_type) || !orig_spec)
 		return -1;
 	MLX5_ITEM_UPDATE(item, key_type, geneve_opt_v, geneve_opt_m,
 			 &rte_flow_item_geneve_opt_mask);
@@ -10004,21 +10004,15 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 			return ret;
 		}
 	}
-	/* Set the data. */
-	if (key_type == MLX5_SET_MATCHER_SW_V)
-		data = geneve_opt_vv->data;
-	else
-		data = geneve_opt_v->data;
-	if (data) {
-		memcpy(&opt_data_key, data,
-			RTE_MIN((uint32_t)(geneve_opt_v->option_len * 4),
-				sizeof(opt_data_key)));
-		memcpy(&opt_data_mask, geneve_opt_m->data,
-			RTE_MIN((uint32_t)(geneve_opt_v->option_len * 4),
-				sizeof(opt_data_mask)));
+	/* Convert the option length from DW to bytes for using memcpy. */
+	option_byte_len = RTE_MIN((size_t)(orig_spec->option_len * 4),
+				  sizeof(rte_be32_t));
+	if (geneve_opt_v->data) {
+		memcpy(&opt_data_key, geneve_opt_v->data, option_byte_len);
+		memcpy(&opt_data_mask, geneve_opt_m->data, option_byte_len);
 		MLX5_SET(fte_match_set_misc3, misc3_v,
-				geneve_tlv_option_0_data,
-			rte_be_to_cpu_32(opt_data_key & opt_data_mask));
+			 geneve_tlv_option_0_data,
+			 rte_be_to_cpu_32(opt_data_key & opt_data_mask));
 	}
 	return ret;
 }
-- 
2.25.1

[PATCH v1 06/23] common/mlx5: add system image GUID attribute

[Michael Baum]

Add to the "system_image_guid" filed describing uniquely the physical
device into "mlx5_hca_attr" structure.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 10 ++++++----
 drivers/common/mlx5/mlx5_devx_cmds.h |  1 +
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 3eeb27fc3f..74609e7cb2 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -518,8 +518,11 @@ mlx5_devx_cmd_query_nic_vport_context(void *ctx,
 	}
 	vctx = MLX5_ADDR_OF(query_nic_vport_context_out, out,
 			    nic_vport_context);
-	attr->vport_inline_mode = MLX5_GET(nic_vport_context, vctx,
-					   min_wqe_inline_mode);
+	if (attr->wqe_inline_mode == MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
+		attr->vport_inline_mode = MLX5_GET(nic_vport_context, vctx,
+						   min_wqe_inline_mode);
+	attr->system_image_guid = MLX5_GET64(nic_vport_context, vctx,
+					     system_image_guid);
 	return 0;
 }
 
@@ -1348,8 +1351,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 		}
 		attr->qp_ts_format = MLX5_GET(roce_caps, hcattr, qp_ts_format);
 	}
-	if (attr->eth_virt &&
-	    attr->wqe_inline_mode == MLX5_CAP_INLINE_MODE_VPORT_CONTEXT) {
+	if (attr->eth_virt) {
 		rc = mlx5_devx_cmd_query_nic_vport_context(ctx, 0, attr);
 		if (rc) {
 			attr->eth_virt = 0;
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index b814c8becc..56ed911c2a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -321,6 +321,7 @@ struct mlx5_hca_attr {
 	uint32_t lag_rx_port_affinity:1;
 	uint32_t wqe_based_flow_table_sup:1;
 	uint8_t max_header_modify_pattern_length;
+	uint64_t system_image_guid;
 };
 
 /* LAG Context. */
-- 
2.25.1

[PATCH v1 07/23] common/mlx5: add GENEVE TLV option attribute structure

[Michael Baum]

Add a new structure "mlx5_devx_geneve_tlv_option_attr" to use in GENEVE
TLV option creation.
Later this structure will be used by GENEVE TLV option query operation
as well.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 28 +++++++++++++---------------
 drivers/common/mlx5/mlx5_devx_cmds.h | 11 ++++++++++-
 drivers/net/mlx5/mlx5_flow_dv.c      | 10 +++++++---
 3 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 74609e7cb2..9855a97bf4 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2855,19 +2855,15 @@ mlx5_devx_cmd_create_conn_track_offload_obj(void *ctx, uint32_t pd,
  *
  * @param[in] ctx
  *   Context returned from mlx5 open_device() glue function.
- * @param [in] class
- *   TLV option variable value of class
- * @param [in] type
- *   TLV option variable value of type
- * @param [in] len
- *   TLV option variable value of len
+ * @param[in] attr
+ *   Pointer to GENEVE TLV option attributes structure.
  *
  * @return
  *   The DevX object created, NULL otherwise and rte_errno is set.
  */
 struct mlx5_devx_obj *
 mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
-		uint16_t class, uint8_t type, uint8_t len)
+				  struct mlx5_devx_geneve_tlv_option_attr *attr)
 {
 	uint32_t in[MLX5_ST_SZ_DW(create_geneve_tlv_option_in)] = {0};
 	uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
@@ -2876,25 +2872,27 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 						   0, SOCKET_ID_ANY);
 
 	if (!geneve_tlv_opt_obj) {
-		DRV_LOG(ERR, "Failed to allocate geneve tlv option object.");
+		DRV_LOG(ERR, "Failed to allocate GENEVE TLV option object.");
 		rte_errno = ENOMEM;
 		return NULL;
 	}
 	void *hdr = MLX5_ADDR_OF(create_geneve_tlv_option_in, in, hdr);
 	void *opt = MLX5_ADDR_OF(create_geneve_tlv_option_in, in,
-			geneve_tlv_opt);
+				 geneve_tlv_opt);
 	MLX5_SET(general_obj_in_cmd_hdr, hdr, opcode,
-			MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+		 MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
 	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
 		 MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
 	MLX5_SET(geneve_tlv_option, opt, option_class,
-			rte_be_to_cpu_16(class));
-	MLX5_SET(geneve_tlv_option, opt, option_type, type);
-	MLX5_SET(geneve_tlv_option, opt, option_data_length, len);
+		 rte_be_to_cpu_16(attr->option_class));
+	MLX5_SET(geneve_tlv_option, opt, option_type, attr->option_type);
+	MLX5_SET(geneve_tlv_option, opt, option_data_length,
+		 attr->option_data_len);
 	geneve_tlv_opt_obj->obj = mlx5_glue->devx_obj_create(ctx, in,
-					sizeof(in), out, sizeof(out));
+							     sizeof(in), out,
+							     sizeof(out));
 	if (!geneve_tlv_opt_obj->obj) {
-		DEVX_DRV_LOG(ERR, out, "create GENEVE TLV", NULL, 0);
+		DEVX_DRV_LOG(ERR, out, "create GENEVE TLV option", NULL, 0);
 		mlx5_free(geneve_tlv_opt_obj);
 		return NULL;
 	}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 56ed911c2a..78337dff17 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -667,6 +667,15 @@ struct mlx5_devx_crypto_login_attr {
 	uint8_t credential[MLX5_CRYPTO_CREDENTIAL_SIZE];
 };
 
+/*
+ * GENEVE TLV option attributes structure, used by GENEVE TLV option create.
+ */
+struct mlx5_devx_geneve_tlv_option_attr {
+	uint32_t option_class:16;
+	uint32_t option_type:8;
+	uint32_t option_data_len:5;
+};
+
 /* mlx5_devx_cmds.c */
 
 __rte_internal
@@ -777,7 +786,7 @@ int mlx5_devx_cmd_register_write(void *ctx, uint16_t reg_id,
 __rte_internal
 struct mlx5_devx_obj *
 mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
-		uint16_t class, uint8_t type, uint8_t len);
+				 struct mlx5_devx_geneve_tlv_option_attr *attr);
 
 /**
  * Create virtio queue counters object DevX API.
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index f8e364dfdb..8894f51f4c 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -9928,11 +9928,15 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 			goto exit;
 		}
 	} else {
+		struct mlx5_devx_geneve_tlv_option_attr attr = {
+			.option_class = geneve_opt_v->option_class,
+			.option_type = geneve_opt_v->option_type,
+			.option_data_len = geneve_opt_v->option_len,
+		};
+
 		/* Create a GENEVE TLV object and resource. */
 		obj = mlx5_devx_cmd_create_geneve_tlv_option(sh->cdev->ctx,
-				geneve_opt_v->option_class,
-				geneve_opt_v->option_type,
-				geneve_opt_v->option_len);
+							     &attr);
 		if (!obj) {
 			ret = rte_flow_error_set(error, ENODATA,
 				RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
-- 
2.25.1

[PATCH v1 08/23] common/mlx5: add PRM attribute for TLV sample

[Michael Baum]

Add GENEVE TLV sample fields in 2 places:
1. New HCA capabilities indicating GENEVE TLV sample is supported.
2. New fields in "mlx5_ifc_geneve_tlv_option_bits" structure.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 18 ++++++++++++++++--
 drivers/common/mlx5/mlx5_devx_cmds.h |  9 +++++++--
 drivers/common/mlx5/mlx5_prm.h       | 15 +++++++++++----
 3 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 9855a97bf4..674130c11f 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -968,6 +968,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			max_geneve_tlv_options);
 	attr->max_geneve_tlv_option_data_len = MLX5_GET(cmd_hca_cap, hcattr,
 			max_geneve_tlv_option_data_len);
+	attr->geneve_tlv_option_offset = MLX5_GET(cmd_hca_cap, hcattr,
+						  geneve_tlv_option_offset);
+	attr->geneve_tlv_sample = MLX5_GET(cmd_hca_cap, hcattr,
+					   geneve_tlv_sample);
 	attr->query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
 						 query_match_sample_info);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
@@ -2883,11 +2887,21 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 		 MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
 	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
 		 MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
-	MLX5_SET(geneve_tlv_option, opt, option_class,
-		 rte_be_to_cpu_16(attr->option_class));
 	MLX5_SET(geneve_tlv_option, opt, option_type, attr->option_type);
 	MLX5_SET(geneve_tlv_option, opt, option_data_length,
 		 attr->option_data_len);
+	if (attr->option_class_ignore)
+		MLX5_SET(geneve_tlv_option, opt, option_class_ignore,
+			 attr->option_class_ignore);
+	else
+		MLX5_SET(geneve_tlv_option, opt, option_class,
+			 rte_be_to_cpu_16(attr->option_class));
+	if (attr->offset_valid) {
+		MLX5_SET(geneve_tlv_option, opt, sample_offset_valid,
+			 attr->offset_valid);
+		MLX5_SET(geneve_tlv_option, opt, sample_offset,
+			 attr->sample_offset);
+	}
 	geneve_tlv_opt_obj->obj = mlx5_glue->devx_obj_create(ctx, in,
 							     sizeof(in), out,
 							     sizeof(out));
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 78337dff17..3f294e8f04 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -212,8 +212,10 @@ struct mlx5_hca_attr {
 	uint32_t lro_timer_supported_periods[MLX5_LRO_NUM_SUPP_PERIODS];
 	uint16_t lro_min_mss_size;
 	uint32_t flex_parser_protocols;
-	uint32_t max_geneve_tlv_options;
-	uint32_t max_geneve_tlv_option_data_len;
+	uint32_t max_geneve_tlv_options:8;
+	uint32_t max_geneve_tlv_option_data_len:5;
+	uint32_t geneve_tlv_sample:1;
+	uint32_t geneve_tlv_option_offset:1;
 	uint32_t hairpin:1;
 	uint32_t log_max_hairpin_queues:5;
 	uint32_t log_max_hairpin_wq_data_sz:5;
@@ -674,6 +676,9 @@ struct mlx5_devx_geneve_tlv_option_attr {
 	uint32_t option_class:16;
 	uint32_t option_type:8;
 	uint32_t option_data_len:5;
+	uint32_t option_class_ignore:1;
+	uint32_t offset_valid:1;
+	uint32_t sample_offset:8;
 };
 
 /* mlx5_devx_cmds.c */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 9e22dce6da..59643a8788 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1849,7 +1849,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8 num_of_uars_per_page[0x20];
 	u8 flex_parser_protocols[0x20];
 	u8 max_geneve_tlv_options[0x8];
-	u8 reserved_at_568[0x3];
+	u8 geneve_tlv_sample[0x1];
+	u8 geneve_tlv_option_offset[0x1];
+	u8 reserved_at_56a[0x1];
 	u8 max_geneve_tlv_option_data_len[0x5];
 	u8 flex_parser_header_modify[0x1];
 	u8 reserved_at_571[0x2];
@@ -3416,16 +3418,21 @@ struct mlx5_ifc_virtio_q_counters_bits {
 
 struct mlx5_ifc_geneve_tlv_option_bits {
 	u8 modify_field_select[0x40];
-	u8 reserved_at_40[0x18];
+	u8 reserved_at_40[0x8];
+	u8 sample_offset[0x8];
+	u8 sample_id_valid[0x1];
+	u8 sample_offset_valid[0x1];
+	u8 option_class_ignore[0x1];
+	u8 reserved_at_53[0x5];
 	u8 geneve_option_fte_index[0x8];
 	u8 option_class[0x10];
 	u8 option_type[0x8];
 	u8 reserved_at_78[0x3];
 	u8 option_data_length[0x5];
-	u8 reserved_at_80[0x180];
+	u8 geneve_sample_field_id[0x20];
+	u8 reserved_at_a0[0x160];
 };
 
-
 enum mlx5_ifc_rtc_update_mode {
 	MLX5_IFC_RTC_STE_UPDATE_MODE_BY_HASH = 0x0,
 	MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET = 0x1,
-- 
2.25.1

[PATCH v1 09/23] common/mlx5: add sample info query syndrome into error log

[Michael Baum]

Move "mlx5_devx_cmd_match_sample_info_query()" function to use
"DEVX_DRV_LOG" in case of "devx_general_cmd" failure.
This macro contains syndrome report and used by all other function
calling "devx_general_cmd".

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 674130c11f..b30f54ab1c 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -641,11 +641,10 @@ mlx5_devx_cmd_match_sample_info_query(void *ctx, uint32_t sample_field_id,
 	MLX5_SET(query_match_sample_info_in, in, sample_field_id,
 		 sample_field_id);
 	rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out));
-	if (rc) {
-		DRV_LOG(ERR, "Failed to query match sample info using DevX: %s",
-			strerror(rc));
-		rte_errno = rc;
-		return -rc;
+	if (rc || MLX5_FW_STATUS(out)) {
+		DEVX_DRV_LOG(ERR, out, "query match sample info",
+			     "sample_field_id", sample_field_id);
+		return MLX5_DEVX_ERR_RC(rc);
 	}
 	attr->modify_field_id = MLX5_GET(query_match_sample_info_out, out,
 					 modify_field_id);
-- 
2.25.1

[PATCH v1 10/23] common/mlx5: query GENEVE option sample ID from HCA attr

[Michael Baum]

This patch adds the GENEVE option sample ID into HCA attribute
structure.
This sample ID is used as the input of
"mlx5_devx_cmd_match_sample_info_query" function when flex parser
profile is 1.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 2 ++
 drivers/common/mlx5/mlx5_devx_cmds.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index b30f54ab1c..332aebbe57 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -973,6 +973,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 					   geneve_tlv_sample);
 	attr->query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
 						 query_match_sample_info);
+	attr->geneve_tlv_option_sample_id = MLX5_GET(cmd_hca_cap, hcattr,
+						     flex_parser_id_geneve_opt_0);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
 	attr->wqe_index_ignore = MLX5_GET(cmd_hca_cap, hcattr,
 					  wqe_index_ignore_cap);
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 3f294e8f04..1daf2fcca4 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -216,6 +216,7 @@ struct mlx5_hca_attr {
 	uint32_t max_geneve_tlv_option_data_len:5;
 	uint32_t geneve_tlv_sample:1;
 	uint32_t geneve_tlv_option_offset:1;
+	uint32_t geneve_tlv_option_sample_id:4;
 	uint32_t hairpin:1;
 	uint32_t log_max_hairpin_queues:5;
 	uint32_t log_max_hairpin_wq_data_sz:5;
-- 
2.25.1

[PATCH v1 11/23] common/mlx5: add function to query GENEVE TLV option

[Michael Baum]

Add a new function to query information about GENEVE TLV option parser.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 50 ++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h |  6 ++++
 drivers/common/mlx5/mlx5_prm.h       |  5 +++
 drivers/common/mlx5/version.map      |  1 +
 4 files changed, 62 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 332aebbe57..1fa75cd964 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2915,6 +2915,56 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 	return geneve_tlv_opt_obj;
 }
 
+/**
+ * Query GENEVE TLV option using DevX API.
+ *
+ * @param[in] ctx
+ *   Context used to create GENEVE TLV option object.
+ * @param[in] geneve_tlv_opt_obj
+ *   DevX object of the GENEVE TLV option.
+ * @param[out] attr
+ *   Pointer to match sample info attributes structure.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_devx_cmd_query_geneve_tlv_option(void *ctx,
+				      struct mlx5_devx_obj *geneve_tlv_opt_obj,
+				      struct mlx5_devx_match_sample_info_query_attr *attr)
+{
+	uint32_t in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {0};
+	uint32_t out[MLX5_ST_SZ_DW(query_geneve_tlv_option_out)] = {0};
+	void *hdr = MLX5_ADDR_OF(query_geneve_tlv_option_out, in, hdr);
+	void *opt = MLX5_ADDR_OF(query_geneve_tlv_option_out, out,
+				 geneve_tlv_opt);
+	int ret;
+
+	MLX5_SET(general_obj_in_cmd_hdr, hdr, opcode,
+		 MLX5_CMD_OP_QUERY_GENERAL_OBJECT);
+	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
+		 MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
+	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_id, geneve_tlv_opt_obj->id);
+	/* Call first query to get sample handle. */
+	ret = mlx5_glue->devx_obj_query(geneve_tlv_opt_obj->obj, in, sizeof(in),
+					out, sizeof(out));
+	if (ret) {
+		DRV_LOG(ERR, "Failed to query GENEVE TLV option using DevX.");
+		rte_errno = errno;
+		return -errno;
+	}
+	/* Call second query to get sample information. */
+	if (MLX5_GET(geneve_tlv_option, opt, sample_id_valid)) {
+		uint32_t sample_id = MLX5_GET(geneve_tlv_option, opt,
+					      geneve_sample_field_id);
+
+		return mlx5_devx_cmd_match_sample_info_query(ctx, sample_id,
+							     attr);
+	}
+	DRV_LOG(DEBUG, "GENEVE TLV option sample isn't valid.");
+	return 0;
+}
+
 int
 mlx5_devx_cmd_wq_query(void *wq, uint32_t *counter_set_id)
 {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 1daf2fcca4..6161c275da 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -794,6 +794,12 @@ struct mlx5_devx_obj *
 mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 				 struct mlx5_devx_geneve_tlv_option_attr *attr);
 
+__rte_internal
+int
+mlx5_devx_cmd_query_geneve_tlv_option(void *ctx,
+				      struct mlx5_devx_obj *geneve_tlv_opt_obj,
+				      struct mlx5_devx_match_sample_info_query_attr *attr);
+
 /**
  * Create virtio queue counters object DevX API.
  *
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 59643a8788..7c36961564 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -3721,6 +3721,11 @@ struct mlx5_ifc_create_geneve_tlv_option_in_bits {
 	struct mlx5_ifc_geneve_tlv_option_bits geneve_tlv_opt;
 };
 
+struct mlx5_ifc_query_geneve_tlv_option_out_bits {
+	struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr;
+	struct mlx5_ifc_geneve_tlv_option_bits geneve_tlv_opt;
+};
+
 struct mlx5_ifc_create_rtc_in_bits {
 	struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr;
 	struct mlx5_ifc_rtc_bits rtc;
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 074eed46fd..589a450145 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -54,6 +54,7 @@ INTERNAL {
 	mlx5_devx_cmd_modify_tir;
 	mlx5_devx_cmd_modify_virtq;
 	mlx5_devx_cmd_qp_query_tis_td;
+	mlx5_devx_cmd_query_geneve_tlv_option;
 	mlx5_devx_cmd_query_hca_attr;
 	mlx5_devx_cmd_query_lag;
 	mlx5_devx_cmd_query_parse_samples;
-- 
2.25.1

[PATCH v1 12/23] net/mlx5: add physical device handle

[Michael Baum]

Add structure describing physical device, and manage physical device
global list.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/net/mlx5/mlx5.c | 77 ++++++++++++++++++++++++++++++++++++-----
 drivers/net/mlx5/mlx5.h | 13 +++++++
 2 files changed, 82 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 3a182de248..f9fc652136 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -190,9 +190,10 @@ struct mlx5_shared_data *mlx5_shared_data;
 /** Driver-specific log messages type. */
 int mlx5_logtype;
 
-static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list =
-						LIST_HEAD_INITIALIZER();
+static LIST_HEAD(mlx5_dev_ctx_list, mlx5_dev_ctx_shared) dev_ctx_list = LIST_HEAD_INITIALIZER();
+static LIST_HEAD(mlx5_phdev_list, mlx5_physical_device) phdev_list = LIST_HEAD_INITIALIZER();
 static pthread_mutex_t mlx5_dev_ctx_list_mutex;
+
 static const struct mlx5_indexed_pool_config mlx5_ipool_cfg[] = {
 #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	[MLX5_IPOOL_DECAP_ENCAP] = {
@@ -1692,6 +1693,60 @@ mlx5_init_shared_dev_registers(struct mlx5_dev_ctx_shared *sh)
 	mlx5_init_hws_flow_tags_registers(sh);
 }
 
+static struct mlx5_physical_device *
+mlx5_get_physical_device(struct mlx5_common_device *cdev)
+{
+	struct mlx5_physical_device *phdev;
+	struct mlx5_hca_attr *attr = &cdev->config.hca_attr;
+
+	/* Search for physical device by system_image_guid. */
+	LIST_FOREACH(phdev, &phdev_list, next) {
+		if (phdev->guid == attr->system_image_guid) {
+			phdev->refcnt++;
+			return phdev;
+		}
+	}
+	phdev = mlx5_malloc(MLX5_MEM_ZERO | MLX5_MEM_RTE,
+			    sizeof(struct mlx5_physical_device),
+			    RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (!phdev) {
+		DRV_LOG(ERR, "Physical device allocation failure.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	phdev->guid = attr->system_image_guid;
+	phdev->refcnt = 1;
+	LIST_INSERT_HEAD(&phdev_list, phdev, next);
+	DRV_LOG(DEBUG, "Physical device is created, guid=%" PRIu64 ".",
+		phdev->guid);
+	return phdev;
+}
+
+static void
+mlx5_physical_device_destroy(struct mlx5_physical_device *phdev)
+{
+#ifdef RTE_LIBRTE_MLX5_DEBUG
+	/* Check the object presence in the list. */
+	struct mlx5_physical_device *lphdev;
+
+	LIST_FOREACH(lphdev, &phdev_list, next)
+		if (lphdev == phdev)
+			break;
+	MLX5_ASSERT(lphdev);
+	if (lphdev != phdev) {
+		DRV_LOG(ERR, "Freeing non-existing physical device");
+		return;
+	}
+#endif
+	MLX5_ASSERT(phdev);
+	MLX5_ASSERT(phdev->refcnt);
+	if (--phdev->refcnt)
+		return;
+	/* Remove physical device from the global device list. */
+	LIST_REMOVE(phdev, next);
+	mlx5_free(phdev);
+}
+
 /**
  * Allocate shared device context. If there is multiport device the
  * master and representors will share this context, if there is single
@@ -1725,7 +1780,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
 	pthread_mutex_lock(&mlx5_dev_ctx_list_mutex);
 	/* Search for IB context by device name. */
-	LIST_FOREACH(sh, &mlx5_dev_ctx_list, next) {
+	LIST_FOREACH(sh, &dev_ctx_list, next) {
 		if (!strcmp(sh->ibdev_name, spawn->phys_dev_name)) {
 			sh->refcnt++;
 			goto exit;
@@ -1765,6 +1820,9 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sizeof(sh->ibdev_name) - 1);
 	strncpy(sh->ibdev_path, mlx5_os_get_ctx_device_path(sh->cdev->ctx),
 		sizeof(sh->ibdev_path) - 1);
+	sh->phdev = mlx5_get_physical_device(sh->cdev);
+	if (!sh->phdev)
+		goto error;
 	/*
 	 * Setting port_id to max unallowed value means there is no interrupt
 	 * subhandler installed for the given port index i.
@@ -1798,7 +1856,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 #endif
 	}
 	mlx5_os_dev_shared_handler_install(sh);
-	if (LIST_EMPTY(&mlx5_dev_ctx_list)) {
+	if (LIST_EMPTY(&dev_ctx_list)) {
 		err = mlx5_flow_os_init_workspace_once();
 		if (err)
 			goto error;
@@ -1811,7 +1869,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	mlx5_flow_aging_init(sh);
 	mlx5_flow_ipool_create(sh);
 	/* Add context to the global device list. */
-	LIST_INSERT_HEAD(&mlx5_dev_ctx_list, sh, next);
+	LIST_INSERT_HEAD(&dev_ctx_list, sh, next);
 	rte_spinlock_init(&sh->geneve_tlv_opt_sl);
 	mlx5_init_shared_dev_registers(sh);
 	/* Init counter pool list header and lock. */
@@ -1833,6 +1891,8 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	} while (++i <= (uint32_t)sh->bond.n_port);
 	if (sh->td)
 		claim_zero(mlx5_devx_cmd_destroy(sh->td));
+	if (sh->phdev)
+		mlx5_physical_device_destroy(sh->phdev);
 	mlx5_free(sh);
 	rte_errno = err;
 	return NULL;
@@ -1919,7 +1979,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 	/* Check the object presence in the list. */
 	struct mlx5_dev_ctx_shared *lctx;
 
-	LIST_FOREACH(lctx, &mlx5_dev_ctx_list, next)
+	LIST_FOREACH(lctx, &dev_ctx_list, next)
 		if (lctx == sh)
 			break;
 	MLX5_ASSERT(lctx);
@@ -1945,7 +2005,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 	/* Remove context from the global device list. */
 	LIST_REMOVE(sh, next);
 	/* Release resources on the last device removal. */
-	if (LIST_EMPTY(&mlx5_dev_ctx_list)) {
+	if (LIST_EMPTY(&dev_ctx_list)) {
 		mlx5_os_net_cleanup();
 		mlx5_flow_os_release_workspace();
 	}
@@ -1985,6 +2045,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL);
 	pthread_mutex_destroy(&sh->txpp.mutex);
 	mlx5_lwm_unset(sh);
+	mlx5_physical_device_destroy(sh->phdev);
 	mlx5_free(sh);
 	return;
 exit:
@@ -2929,7 +2990,7 @@ mlx5_probe_again_args_validate(struct mlx5_common_device *cdev,
 		return 0;
 	pthread_mutex_lock(&mlx5_dev_ctx_list_mutex);
 	/* Search for IB context by common device pointer. */
-	LIST_FOREACH(sh, &mlx5_dev_ctx_list, next)
+	LIST_FOREACH(sh, &dev_ctx_list, next)
 		if (sh->cdev == cdev)
 			break;
 	pthread_mutex_unlock(&mlx5_dev_ctx_list_mutex);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 263ebead7f..6a82c38cf4 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1419,6 +1419,18 @@ struct mlx5_dev_registers {
 #define HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT
 #endif
 
+/**
+ * Physical device structure.
+ * This device is created once per NIC to manage recourses shared by all ports
+ * under same physical device.
+ */
+struct mlx5_physical_device {
+	LIST_ENTRY(mlx5_physical_device) next;
+	struct mlx5_dev_ctx_shared *sh; /* Created on sherd context. */
+	uint64_t guid; /* System image guid, the uniq ID of physical device. */
+	uint32_t refcnt;
+};
+
 /*
  * Shared Infiniband device context for Master/Representors
  * which belong to same IB device with multiple IB ports.
@@ -1449,6 +1461,7 @@ struct mlx5_dev_ctx_shared {
 	uint32_t max_port; /* Maximal IB device port index. */
 	struct mlx5_bond_info bond; /* Bonding information. */
 	struct mlx5_common_device *cdev; /* Backend mlx5 device. */
+	struct mlx5_physical_device *phdev; /* Backend physical device. */
 	uint32_t tdn; /* Transport Domain number. */
 	char ibdev_name[MLX5_FS_NAME_MAX]; /* SYSFS dev name. */
 	char ibdev_path[MLX5_FS_PATH_MAX]; /* SYSFS dev path for secondary */
-- 
2.25.1

[PATCH v1 13/23] net/mlx5: add GENEVE TLV options parser API

[Michael Baum]

Add a new private API to create/destroy parser for GENEVE TLV options.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
---
 doc/guides/nics/mlx5.rst            | 122 ++++++
 doc/guides/platform/mlx5.rst        |   6 +-
 drivers/net/mlx5/meson.build        |   1 +
 drivers/net/mlx5/mlx5.c             |  30 +-
 drivers/net/mlx5/mlx5.h             |   8 +
 drivers/net/mlx5/mlx5_flow.c        |  30 ++
 drivers/net/mlx5/mlx5_flow.h        |  18 +
 drivers/net/mlx5/mlx5_flow_geneve.c | 627 ++++++++++++++++++++++++++++
 drivers/net/mlx5/rte_pmd_mlx5.h     | 102 +++++
 drivers/net/mlx5/version.map        |   3 +
 10 files changed, 945 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_flow_geneve.c

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 6b52fb93c5..80446d8d82 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -2298,6 +2298,128 @@ and disables ``avail_thresh_triggered``.
    testpmd> mlx5 set port 1 host_shaper avail_thresh_triggered 0 rate 50
 
 
+.. _geneve_parser_api:
+
+GENEVE TLV options parser
+-------------------------
+
+NVIDIA ConnectX and BlueField devices support configure flex parser for
+`GENEVE TLV options <https://www.rfc-editor.org/rfc/rfc8926.html#name-tunnel-options>`_.
+
+Each physical device has 7 DWs for GENEVE TLV options.
+Partial option configuration is supported, mask for data is provided in parser
+creation indicating which DWs configuration is requested. Only masked data DWs
+can be matched later as item field using flow API.
+
+Matching of ``type`` field is supported for each configured option.
+However, for matching ``class` field, the option should be configured with
+``match_on_class_mode=2``. Matching on ``length`` field is not supported.
+When ``match_on_class_mode=2`` is requested, one extra DW is consumed for it.
+
+Parser API
+~~~~~~~~~~
+
+An API to create/destroy GENEVE TLV parser is added.
+Although the parser is created per physical device, this API is port oriented.
+Each port should call this API before using GENEVE OPT item,
+but its configuration must use the same options list with same internal order
+configured by first port.
+
+Calling this API for different ports under same physical device doesn't consume
+more DWs, the first one creates the parser and the rest use same configuration.
+
+``struct rte_pmd_mlx5_geneve_tlv`` is used for single option configuration:
+
+.. _table_rte_pmd_mlx5_geneve_tlv:
+
+.. table:: GENEVE TLV
+
+   +-------------------------+-------------------------------------------------+
+   | Field                   | Value                                           |
+   +=========================+=================================================+
+   | ``option_class``        | class                                           |
+   +-------------------------+-------------------------------------------------+
+   | ``option_type``         | type                                            |
+   +-------------------------+-------------------------------------------------+
+   | ``option_len``          | data length in DW granularity                   |
+   +-------------------------+-------------------------------------------------+
+   | ``match_on_class_mode`` | indicator about class field role in this option |
+   +-------------------------+-------------------------------------------------+
+   | ``offset``              | offset of the first sample in DW granularity    |
+   +-------------------------+-------------------------------------------------+
+   | ``sample_len``          | number of DW to sample                          |
+   +-------------------------+-------------------------------------------------+
+   | ``match_data_mask``     | array of DWs which each bit marks if this bit   |
+   |                         | should be sampled                               |
+   +-------------------------+-------------------------------------------------+
+
+Creation
+^^^^^^^^
+
+Creates GENEVE TLV parser for the selected port.
+This function must be called before first use of GENEVE option.
+
+.. code-block:: c
+
+   void *
+   rte_pmd_mlx5_create_geneve_tlv_parser(uint16_t port_id,
+                                         const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+                                         uint8_t nb_options);
+
+The parser creation is done once for all GENEVE TLV options.
+For adding a new option, the exist parser should be destroyed first.
+
+Arguments:
+
+- ``port_id``: port identifier of Ethernet device.
+- ``tlv_list``: list of GENEVE TLV options to create parser for them.
+- ``nb_options``: number of options in TLV list.
+
+Return values:
+
+- A valid handle in case of success, NULL otherwise (``rte_errno`` is also set),
+  the following errors are defined.
+- ``ENODEV``: there is no Ethernet device for this port id.
+- ``EINVAL``: invalid GENEVE TLV option requested.
+- ``ENOTSUP``: the port doesn't support GENEVE TLV parsing.
+- ``EEXIST``: this port already has GENEVE TLV parser or another port under same
+  physical device has already prepared a different parser.
+- ``ENOMEM``: not enough memory to execute the function, or resource limitation
+  on the device.
+
+
+Destruction
+^^^^^^^^^^^
+
+Destroy GENEVE TLV parser created by ``rte_pmd_mlx5_create_geneve_tlv_parser()``.
+This function must be called after last use of GENEVE option and before port
+closing.
+
+.. code-block:: c
+
+   int
+   rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle);
+
+Failure to destroy a parser handle may occur when one of the options is used by
+valid template table.
+
+Arguments:
+
+- ``handle``: handle for the GENEVE TLV parser object to be destroyed.
+
+Return values:
+
+- 0 on success, a negative errno value otherwise and ``rte_errno`` is set.
+
+
+Limitations
+~~~~~~~~~~~
+
+* Supported only in HW steering (``dv_flow_en`` = 2).
+* Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+* Supported for FW version **xx.37.0142** and above.
+
+
 Testpmd driver specific commands
 --------------------------------
 
diff --git a/doc/guides/platform/mlx5.rst b/doc/guides/platform/mlx5.rst
index 400000e284..d16508d0da 100644
--- a/doc/guides/platform/mlx5.rst
+++ b/doc/guides/platform/mlx5.rst
@@ -536,10 +536,14 @@ Below are some firmware configurations listed.
    or
    FLEX_PARSER_PROFILE_ENABLE=1
 
-- enable Geneve TLV option flow matching::
+- enable Geneve TLV option flow matching in SW steering::
 
    FLEX_PARSER_PROFILE_ENABLE=0
 
+- enable Geneve TLV option flow matching in HW steering::
+
+   FLEX_PARSER_PROFILE_ENABLE=8
+
 - enable GTP flow matching::
 
    FLEX_PARSER_PROFILE_ENABLE=3
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index 69771c63ab..d705fe21bb 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -46,6 +46,7 @@ sources = files(
 
 if is_linux
     sources += files(
+            'mlx5_flow_geneve.c',
             'mlx5_flow_hw.c',
             'mlx5_hws_cnt.c',
             'mlx5_flow_quota.c',
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index f9fc652136..5f8af31aea 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1722,6 +1722,19 @@ mlx5_get_physical_device(struct mlx5_common_device *cdev)
 	return phdev;
 }
 
+struct mlx5_physical_device *
+mlx5_get_locked_physical_device(struct mlx5_priv *priv)
+{
+	pthread_mutex_lock(&mlx5_dev_ctx_list_mutex);
+	return priv->sh->phdev;
+}
+
+void
+mlx5_unlock_physical_device(void)
+{
+	pthread_mutex_unlock(&mlx5_dev_ctx_list_mutex);
+}
+
 static void
 mlx5_physical_device_destroy(struct mlx5_physical_device *phdev)
 {
@@ -2278,6 +2291,7 @@ int
 mlx5_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
 	unsigned int i;
 	int ret;
 
@@ -2290,7 +2304,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 		rte_eth_dev_release_port(dev);
 		return 0;
 	}
-	if (!priv->sh)
+	if (!sh)
 		return 0;
 	if (priv->shared_refcnt) {
 		DRV_LOG(ERR, "port %u is shared host in use (%u)",
@@ -2298,6 +2312,15 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 		rte_errno = EBUSY;
 		return -EBUSY;
 	}
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	/* Check if shared GENEVE options created on context being closed. */
+	ret = mlx5_geneve_tlv_options_check_busy(priv);
+	if (ret) {
+		DRV_LOG(ERR, "port %u maintains shared GENEVE TLV options",
+			dev->data->port_id);
+		return ret;
+	}
+#endif
 	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
 		dev->data->port_id,
 		((priv->sh->cdev->ctx != NULL) ?
@@ -2330,6 +2353,11 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	flow_hw_destroy_vport_action(dev);
 	flow_hw_resource_release(dev);
 	flow_hw_clear_port_info(dev);
+	if (priv->tlv_options != NULL) {
+		/* Free the GENEVE TLV parser resource. */
+		claim_zero(mlx5_geneve_tlv_options_destroy(priv->tlv_options, sh->phdev));
+		priv->tlv_options = NULL;
+	}
 #endif
 	if (priv->rxq_privs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 6a82c38cf4..2e0d71a12c 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1419,6 +1419,8 @@ struct mlx5_dev_registers {
 #define HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT
 #endif
 
+struct mlx5_geneve_tlv_options;
+
 /**
  * Physical device structure.
  * This device is created once per NIC to manage recourses shared by all ports
@@ -1428,6 +1430,7 @@ struct mlx5_physical_device {
 	LIST_ENTRY(mlx5_physical_device) next;
 	struct mlx5_dev_ctx_shared *sh; /* Created on sherd context. */
 	uint64_t guid; /* System image guid, the uniq ID of physical device. */
+	struct mlx5_geneve_tlv_options *tlv_options;
 	uint32_t refcnt;
 };
 
@@ -1949,6 +1952,8 @@ struct mlx5_priv {
 	/* Action template list. */
 	LIST_HEAD(flow_hw_at, rte_flow_actions_template) flow_hw_at;
 	struct mlx5dr_context *dr_ctx; /**< HW steering DR context. */
+	/* Pointer to the GENEVE TLV options. */
+	struct mlx5_geneve_tlv_options *tlv_options;
 	/* HW steering queue polling mechanism job descriptor LIFO. */
 	uint32_t hws_strict_queue:1;
 	/**< Whether all operations strictly happen on the same HWS queue. */
@@ -2087,6 +2092,9 @@ void mlx5_flow_counter_mode_config(struct rte_eth_dev *dev);
 int mlx5_flow_aso_age_mng_init(struct mlx5_dev_ctx_shared *sh);
 int mlx5_aso_flow_mtrs_mng_init(struct mlx5_dev_ctx_shared *sh);
 int mlx5_flow_aso_ct_mng_init(struct mlx5_dev_ctx_shared *sh);
+struct mlx5_physical_device *
+mlx5_get_locked_physical_device(struct mlx5_priv *priv);
+void mlx5_unlock_physical_device(void);
 
 /* mlx5_ethdev.c */
 
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 85e8c77c81..ae53cc8e74 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -12476,3 +12476,33 @@ mlx5_flow_pick_transfer_proxy(struct rte_eth_dev *dev,
 				  RTE_FLOW_ERROR_TYPE_UNSPECIFIED,
 				  NULL, "unable to find a proxy port");
 }
+
+void *
+rte_pmd_mlx5_create_geneve_tlv_parser(uint16_t port_id,
+				      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+				      uint8_t nb_options)
+{
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	return mlx5_geneve_tlv_parser_create(port_id, tlv_list, nb_options);
+#else
+	(void)port_id;
+	(void)tlv_list;
+	(void)nb_options;
+	DRV_LOG(ERR, "%s is not supported.", __func__);
+	rte_errno = ENOTSUP;
+	return NULL;
+#endif
+}
+
+int
+rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle)
+{
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	return mlx5_geneve_tlv_parser_destroy(handle);
+#else
+	(void)handle;
+	DRV_LOG(ERR, "%s is not supported.", __func__);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+#endif
+}
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 6dde9de688..4bfc218175 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1330,6 +1330,8 @@ struct mlx5_action_construct_data {
 	};
 };
 
+#define MAX_GENEVE_OPTIONS_RESOURCES 7
+
 /* Flow item template struct. */
 struct rte_flow_pattern_template {
 	LIST_ENTRY(rte_flow_pattern_template) next;
@@ -1644,6 +1646,11 @@ struct mlx5_flow_split_info {
 	uint64_t prefix_layers; /**< Prefix subflow layers. */
 };
 
+struct mlx5_hl_data {
+	uint8_t dw_offset;
+	uint32_t dw_mask;
+};
+
 struct flow_hw_port_info {
 	uint32_t regc_mask;
 	uint32_t regc_value;
@@ -1759,6 +1766,12 @@ flow_hw_get_reg_id_from_ctx(void *dr_ctx,
 	return REG_NON;
 }
 
+void *
+mlx5_geneve_tlv_parser_create(uint16_t port_id,
+			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+			      uint8_t nb_options);
+int mlx5_geneve_tlv_parser_destroy(void *handle);
+
 void flow_hw_set_port_info(struct rte_eth_dev *dev);
 void flow_hw_clear_port_info(struct rte_eth_dev *dev);
 int flow_hw_create_vport_action(struct rte_eth_dev *dev);
@@ -2803,6 +2816,11 @@ mlx5_get_tof(const struct rte_flow_item *items,
 	     enum mlx5_tof_rule_type *rule_type);
 void
 flow_hw_resource_release(struct rte_eth_dev *dev);
+int
+mlx5_geneve_tlv_options_destroy(struct mlx5_geneve_tlv_options *options,
+				struct mlx5_physical_device *phdev);
+int
+mlx5_geneve_tlv_options_check_busy(struct mlx5_priv *priv);
 void
 flow_hw_rxq_flag_set(struct rte_eth_dev *dev, bool enable);
 int flow_dv_action_validate(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
new file mode 100644
index 0000000000..f23fb31aa0
--- /dev/null
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -0,0 +1,627 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 NVIDIA Corporation & Affiliates
+ */
+
+#include <rte_flow.h>
+
+#include <mlx5_malloc.h>
+#include <stdint.h>
+
+#include "generic/rte_byteorder.h"
+#include "mlx5.h"
+#include "mlx5_flow.h"
+#include "rte_pmd_mlx5.h"
+
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
+
+#define MAX_GENEVE_OPTION_DATA_SIZE 32
+#define MAX_GENEVE_OPTION_TOTAL_DATA_SIZE \
+		(MAX_GENEVE_OPTION_DATA_SIZE * MAX_GENEVE_OPTIONS_RESOURCES)
+
+/**
+ * Single DW inside GENEVE TLV option.
+ */
+struct mlx5_geneve_tlv_resource {
+	struct mlx5_devx_obj *obj; /* FW object returned in parser creation. */
+	uint32_t modify_field; /* Modify field ID for this DW. */
+	uint8_t offset; /* Offset used in obj creation, from option start. */
+};
+
+/**
+ * Single GENEVE TLV option context.
+ * May include some FW objects for different DWs in same option.
+ */
+struct mlx5_geneve_tlv_option {
+	uint8_t type;
+	uint16_t class;
+	uint8_t class_mode;
+	struct mlx5_hl_data match_data[MAX_GENEVE_OPTION_DATA_SIZE];
+	uint32_t match_data_size;
+	struct mlx5_hl_data hl_ok_bit;
+	struct mlx5_geneve_tlv_resource resources[MAX_GENEVE_OPTIONS_RESOURCES];
+	RTE_ATOMIC(uint32_t) refcnt;
+};
+
+/**
+ * List of GENEVE TLV options.
+ */
+struct mlx5_geneve_tlv_options {
+	/* List of configured GENEVE TLV options. */
+	struct mlx5_geneve_tlv_option options[MAX_GENEVE_OPTIONS_RESOURCES];
+	/*
+	 * Copy of list given in parser creation, use to compare with new
+	 * configuration.
+	 */
+	struct rte_pmd_mlx5_geneve_tlv spec[MAX_GENEVE_OPTIONS_RESOURCES];
+	rte_be32_t buffer[MAX_GENEVE_OPTION_TOTAL_DATA_SIZE];
+	uint8_t nb_options; /* Number entries in above lists. */
+	RTE_ATOMIC(uint32_t) refcnt;
+};
+
+/**
+ * Create single GENEVE TLV option sample.
+ *
+ * @param ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param attr
+ *   Pointer to GENEVE TLV option attributes structure.
+ * @param query_attr
+ *   Pointer to match sample info attributes structure.
+ * @param match_data
+ *   Pointer to header layout structure to update.
+ * @param resource
+ *   Pointer to single sample context to fill.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_create_sample(void *ctx,
+		      struct mlx5_devx_geneve_tlv_option_attr *attr,
+		      struct mlx5_devx_match_sample_info_query_attr *query_attr,
+		      struct mlx5_hl_data *match_data,
+		      struct mlx5_geneve_tlv_resource *resource)
+{
+	struct mlx5_devx_obj *obj;
+	int ret;
+
+	obj = mlx5_devx_cmd_create_geneve_tlv_option(ctx, attr);
+	if (obj == NULL)
+		return -rte_errno;
+	ret = mlx5_devx_cmd_query_geneve_tlv_option(ctx, obj, query_attr);
+	if (ret) {
+		claim_zero(mlx5_devx_cmd_destroy(obj));
+		return ret;
+	}
+	resource->obj = obj;
+	resource->offset = attr->sample_offset;
+	resource->modify_field = query_attr->modify_field_id;
+	match_data->dw_offset = query_attr->sample_dw_data;
+	match_data->dw_mask = 0xffffffff;
+	return 0;
+}
+
+/**
+ * Destroy single GENEVE TLV option sample.
+ *
+ * @param resource
+ *   Pointer to single sample context to clean.
+ */
+static void
+mlx5_geneve_tlv_option_destroy_sample(struct mlx5_geneve_tlv_resource *resource)
+{
+	claim_zero(mlx5_devx_cmd_destroy(resource->obj));
+	resource->obj = NULL;
+}
+
+/**
+ * Create single GENEVE TLV option.
+ *
+ * @param ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param spec
+ *   Pointer to user configuration.
+ * @param option
+ *   Pointer to single GENEVE TLV option to fill.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *spec,
+			      struct mlx5_geneve_tlv_option *option)
+{
+	struct mlx5_devx_geneve_tlv_option_attr attr = {
+		.option_class = spec->option_class,
+		.option_type = spec->option_type,
+		.option_data_len = spec->option_len,
+		.option_class_ignore = spec->match_on_class_mode == 1 ? 0 : 1,
+		.offset_valid = 1,
+	};
+	struct mlx5_devx_match_sample_info_query_attr query_attr = {0};
+	struct mlx5_geneve_tlv_resource *resource;
+	uint8_t i, resource_id = 0;
+	int ret;
+
+	if (spec->match_on_class_mode == 2) {
+		/* Header is matchable, create sample for DW0. */
+		attr.sample_offset = 0;
+		resource = &option->resources[resource_id];
+		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
+							   &query_attr,
+							   &option->match_data[0],
+							   resource);
+		if (ret)
+			return ret;
+		resource_id++;
+	}
+	/*
+	 * Create FW object for each DW request by user.
+	 * Starting from 1 since FW offset starts from header.
+	 */
+	for (i = 1; i <= spec->sample_len; ++i) {
+		if (spec->match_data_mask[i - 1] == 0)
+			continue;
+		/* offset of data + offset inside data = specific DW offset. */
+		attr.sample_offset = spec->offset + i;
+		resource = &option->resources[resource_id];
+		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
+							   &query_attr,
+							   &option->match_data[i],
+							   resource);
+		if (ret)
+			goto error;
+		resource_id++;
+	}
+	/*
+	 * Update the OK bit information according to last query.
+	 * It should be same for each query under same option.
+	 */
+	option->hl_ok_bit.dw_offset = query_attr.sample_dw_ok_bit;
+	option->hl_ok_bit.dw_mask = 1 << query_attr.sample_dw_ok_bit_offset;
+	option->match_data_size = spec->sample_len + 1;
+	option->type = spec->option_type;
+	option->class = spec->option_class;
+	option->class_mode = spec->match_on_class_mode;
+	rte_atomic_store_explicit(&option->refcnt, 0, rte_memory_order_relaxed);
+	return 0;
+error:
+	for (i = 0; i < resource_id; ++i) {
+		resource = &option->resources[i];
+		mlx5_geneve_tlv_option_destroy_sample(resource);
+	}
+	return ret;
+}
+
+/**
+ * Destroy single GENEVE TLV option.
+ *
+ * @param option
+ *   Pointer to single GENEVE TLV option to destroy.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_destroy(struct mlx5_geneve_tlv_option *option)
+{
+	uint8_t i;
+
+	if (rte_atomic_load_explicit(&option->refcnt, rte_memory_order_relaxed)) {
+		DRV_LOG(ERR,
+			"Option type %u class %u is still in used by %u tables.",
+			option->type, option->class, option->refcnt);
+		rte_errno = EBUSY;
+		return -rte_errno;
+	}
+	for (i = 0; option->resources[i].obj != NULL; ++i)
+		mlx5_geneve_tlv_option_destroy_sample(&option->resources[i]);
+	return 0;
+}
+
+/**
+ * Copy the GENEVE TLV option user configuration for future comparing.
+ *
+ * @param dst
+ *   Pointer to internal user configuration copy.
+ * @param src
+ *   Pointer to user configuration.
+ * @param match_data_mask
+ *   Pointer to allocated data array.
+ */
+static void
+mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst,
+			    const struct rte_pmd_mlx5_geneve_tlv *src,
+			    rte_be32_t *match_data_mask)
+{
+	uint8_t i;
+
+	dst->option_type = src->option_type;
+	dst->option_class = src->option_class;
+	dst->option_len = src->option_len;
+	dst->offset = src->offset;
+	dst->match_on_class_mode = src->match_on_class_mode;
+	dst->sample_len = src->sample_len;
+	for (i = 0; i < dst->sample_len; ++i)
+		match_data_mask[i] = src->match_data_mask[i];
+	dst->match_data_mask = match_data_mask;
+}
+
+/**
+ * Create list of GENEVE TLV options according to user configuration list.
+ *
+ * @param sh
+ *   Shared context the options are being created on.
+ * @param tlv_list
+ *   A list of GENEVE TLV options to create parser for them.
+ * @param nb_options
+ *   The number of options in TLV list.
+ *
+ * @return
+ *   A pointer to GENEVE TLV options parser structure on success,
+ *   NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_geneve_tlv_options *
+mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh,
+			       const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+			       uint8_t nb_options)
+{
+	struct mlx5_geneve_tlv_options *options;
+	const struct rte_pmd_mlx5_geneve_tlv *spec;
+	rte_be32_t *data_mask;
+	uint8_t i, j;
+	int ret;
+
+	options = mlx5_malloc(MLX5_MEM_ZERO | MLX5_MEM_RTE,
+			      sizeof(struct mlx5_geneve_tlv_options),
+			      RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (options == NULL) {
+		DRV_LOG(ERR,
+			"Failed to allocate memory for GENEVE TLV options.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	for (i = 0; i < nb_options; ++i) {
+		spec = &tlv_list[i];
+		ret = mlx5_geneve_tlv_option_create(sh->cdev->ctx, spec,
+						    &options->options[i]);
+		if (ret < 0)
+			goto error;
+		/* Copy the user list for comparing future configuration. */
+		data_mask = options->buffer + i * MAX_GENEVE_OPTION_DATA_SIZE;
+		mlx5_geneve_tlv_option_copy(&options->spec[i], spec, data_mask);
+	}
+	MLX5_ASSERT(sh->phdev->sh == NULL);
+	sh->phdev->sh = sh;
+	options->nb_options = nb_options;
+	options->refcnt = 1;
+	return options;
+error:
+	for (j = 0; j < i; ++j)
+		mlx5_geneve_tlv_option_destroy(&options->options[j]);
+	mlx5_free(options);
+	return NULL;
+}
+
+/**
+ * Destroy GENEVE TLV options structure.
+ *
+ * @param options
+ *   Pointer to GENEVE TLV options structure to destroy.
+ * @param phdev
+ *   Pointer physical device options were created on.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_geneve_tlv_options_destroy(struct mlx5_geneve_tlv_options *options,
+				struct mlx5_physical_device *phdev)
+{
+	uint8_t i;
+	int ret;
+
+	if (--options->refcnt)
+		return 0;
+	for (i = 0; i < options->nb_options; ++i) {
+		ret = mlx5_geneve_tlv_option_destroy(&options->options[i]);
+		if (ret < 0) {
+			DRV_LOG(ERR,
+				"Failed to destroy option %u, %u/%u is already destroyed.",
+				i, i, options->nb_options);
+			return ret;
+		}
+	}
+	mlx5_free(options);
+	phdev->tlv_options = NULL;
+	phdev->sh = NULL;
+	return 0;
+}
+
+/**
+ * Check if GENEVE TLV options are hosted on the current port
+ * and the port can be closed
+ *
+ * @param priv
+ *   Device private data.
+ *
+ * @return
+ *   0 on success, a negative EBUSY and rte_errno is set.
+ */
+int
+mlx5_geneve_tlv_options_check_busy(struct mlx5_priv *priv)
+{
+	struct mlx5_physical_device *phdev = mlx5_get_locked_physical_device(priv);
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
+
+	if (!phdev || phdev->sh != sh) {
+		mlx5_unlock_physical_device();
+		return 0;
+	}
+	if (!sh->phdev->tlv_options || sh->phdev->tlv_options->refcnt == 1) {
+		/* Mark port as being closed one */
+		sh->phdev->sh = NULL;
+		mlx5_unlock_physical_device();
+		return 0;
+	}
+	mlx5_unlock_physical_device();
+	rte_errno = EBUSY;
+	return -EBUSY;
+}
+
+/**
+ * Validate GENEVE TLV option user request structure.
+ *
+ * @param attr
+ *   Pointer to HCA attribute structure.
+ * @param option
+ *   Pointer to user configuration.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_validate(struct mlx5_hca_attr *attr,
+				const struct rte_pmd_mlx5_geneve_tlv *option)
+{
+	if (option->option_len > attr->max_geneve_tlv_option_data_len) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option length (%u) exceeds the limit (%u).",
+			option->option_len,
+			attr->max_geneve_tlv_option_data_len);
+		rte_errno = ENOTSUP;
+		return -rte_errno;
+	}
+	if (option->option_len < option->offset + option->sample_len) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option length is smaller than (offset + sample_len).");
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	if (option->match_on_class_mode > 2) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option match_on_class_mode is invalid.");
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	return 0;
+}
+
+/**
+ * Get the number of requested DWs in given GENEVE TLV option.
+ *
+ * @param option
+ *   Pointer to user configuration.
+ *
+ * @return
+ *   Number of requested DWs for given GENEVE TLV option.
+ */
+static uint8_t
+mlx5_geneve_tlv_option_get_nb_dws(const struct rte_pmd_mlx5_geneve_tlv *option)
+{
+	uint8_t nb_dws = 0;
+	uint8_t i;
+
+	if (option->match_on_class_mode == 2)
+		nb_dws++;
+	for (i = 0; i < option->sample_len; ++i) {
+		if (option->match_data_mask[i] == 0xffffffff)
+			nb_dws++;
+	}
+	return nb_dws;
+}
+
+/**
+ * Compare GENEVE TLV option user request structure.
+ *
+ * @param option1
+ *   Pointer to first user configuration.
+ * @param option2
+ *   Pointer to second user configuration.
+ *
+ * @return
+ *   True if the options are equal, false otherwise.
+ */
+static bool
+mlx5_geneve_tlv_option_compare(const struct rte_pmd_mlx5_geneve_tlv *option1,
+			       const struct rte_pmd_mlx5_geneve_tlv *option2)
+{
+	uint8_t i;
+
+	if (option1->option_type != option2->option_type ||
+	    option1->option_class != option2->option_class ||
+	    option1->option_len != option2->option_len ||
+	    option1->offset != option2->offset ||
+	    option1->match_on_class_mode != option2->match_on_class_mode ||
+	    option1->sample_len != option2->sample_len)
+		return false;
+	for (i = 0; i < option1->sample_len; ++i) {
+		if (option1->match_data_mask[i] != option2->match_data_mask[i])
+			return false;
+	}
+	return true;
+}
+
+/**
+ * Check whether the given GENEVE TLV option list is equal to internal list.
+ * The lists are equal when they have same size and same options in the same
+ * order inside the list.
+ *
+ * @param options
+ *   Pointer to GENEVE TLV options structure.
+ * @param tlv_list
+ *   A list of GENEVE TLV options to compare.
+ * @param nb_options
+ *   The number of options in TLV list.
+ *
+ * @return
+ *   True if the lists are equal, false otherwise.
+ */
+static bool
+mlx5_is_same_geneve_tlv_options(const struct mlx5_geneve_tlv_options *options,
+				const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+				uint8_t nb_options)
+{
+	const struct rte_pmd_mlx5_geneve_tlv *spec = options->spec;
+	uint8_t i;
+
+	if (options->nb_options != nb_options)
+		return false;
+	for (i = 0; i < nb_options; ++i) {
+		if (!mlx5_geneve_tlv_option_compare(&spec[i], &tlv_list[i]))
+			return false;
+	}
+	return true;
+}
+
+void *
+mlx5_geneve_tlv_parser_create(uint16_t port_id,
+			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+			      uint8_t nb_options)
+{
+	struct mlx5_geneve_tlv_options *options = NULL;
+	struct mlx5_physical_device *phdev;
+	struct rte_eth_dev *dev;
+	struct mlx5_priv *priv;
+	struct mlx5_hca_attr *attr;
+	uint8_t total_dws = 0;
+	uint8_t i;
+
+	/*
+	 * Validate the input before taking a lock and before any memory
+	 * allocation.
+	 */
+	if (rte_eth_dev_is_valid_port(port_id) < 0) {
+		DRV_LOG(ERR, "There is no Ethernet device for port %u.",
+			port_id);
+		rte_errno = ENODEV;
+		return NULL;
+	}
+	dev = &rte_eth_devices[port_id];
+	priv = dev->data->dev_private;
+	if (priv->tlv_options) {
+		DRV_LOG(ERR, "Port %u already has GENEVE TLV parser.", port_id);
+		rte_errno = EEXIST;
+		return NULL;
+	}
+	if (priv->sh->config.dv_flow_en < 2) {
+		DRV_LOG(ERR,
+			"GENEVE TLV parser is only supported for HW steering.");
+		rte_errno = ENOTSUP;
+		return NULL;
+	}
+	attr = &priv->sh->cdev->config.hca_attr;
+	MLX5_ASSERT(MAX_GENEVE_OPTIONS_RESOURCES <=
+		    attr->max_geneve_tlv_options);
+	if (!attr->geneve_tlv_option_offset || !attr->geneve_tlv_sample ||
+	    !attr->query_match_sample_info || !attr->geneve_tlv_opt) {
+		DRV_LOG(ERR, "Not enough capabilities to support GENEVE TLV parser, maybe old FW version");
+		rte_errno = ENOTSUP;
+		return NULL;
+	}
+	if (nb_options > MAX_GENEVE_OPTIONS_RESOURCES) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option number (%u) exceeds the limit (%u).",
+			nb_options, MAX_GENEVE_OPTIONS_RESOURCES);
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	for (i = 0; i < nb_options; ++i) {
+		if (mlx5_geneve_tlv_option_validate(attr, &tlv_list[i]) < 0) {
+			DRV_LOG(ERR, "GENEVE TLV option %u is invalid.", i);
+			return NULL;
+		}
+		total_dws += mlx5_geneve_tlv_option_get_nb_dws(&tlv_list[i]);
+	}
+	if (total_dws > MAX_GENEVE_OPTIONS_RESOURCES) {
+		DRV_LOG(ERR,
+			"Total requested DWs (%u) exceeds the limit (%u).",
+			total_dws, MAX_GENEVE_OPTIONS_RESOURCES);
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	/* Take lock for this physical device and manage the options. */
+	phdev = mlx5_get_locked_physical_device(priv);
+	options = priv->sh->phdev->tlv_options;
+	if (options) {
+		if (!mlx5_is_same_geneve_tlv_options(options, tlv_list,
+						     nb_options)) {
+			mlx5_unlock_physical_device();
+			DRV_LOG(ERR, "Another port has already prepared different GENEVE TLV parser.");
+			rte_errno = EEXIST;
+			return NULL;
+		}
+		if (phdev->sh == NULL) {
+			mlx5_unlock_physical_device();
+			DRV_LOG(ERR, "GENEVE TLV options are hosted on port being closed.");
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		/* Use existing options. */
+		options->refcnt++;
+		goto exit;
+	}
+	/* Create GENEVE TLV options for this physical device. */
+	options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options);
+	if (!options) {
+		mlx5_unlock_physical_device();
+		return NULL;
+	}
+	phdev->tlv_options = options;
+exit:
+	mlx5_unlock_physical_device();
+	priv->tlv_options = options;
+	return priv;
+}
+
+int
+mlx5_geneve_tlv_parser_destroy(void *handle)
+{
+	struct mlx5_priv *priv = (struct mlx5_priv *)handle;
+	struct mlx5_physical_device *phdev;
+	int ret;
+
+	if (priv == NULL) {
+		DRV_LOG(ERR, "Handle input is invalid (NULL).");
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	if (priv->tlv_options == NULL) {
+		DRV_LOG(ERR, "This parser has been already released.");
+		rte_errno = ENOENT;
+		return -rte_errno;
+	}
+	/* Take lock for this physical device and manage the options. */
+	phdev = mlx5_get_locked_physical_device(priv);
+	/* Destroy the options */
+	ret = mlx5_geneve_tlv_options_destroy(phdev->tlv_options, phdev);
+	if (ret < 0) {
+		mlx5_unlock_physical_device();
+		return ret;
+	}
+	priv->tlv_options = NULL;
+	mlx5_unlock_physical_device();
+	return 0;
+}
+
+#endif /* defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) */
diff --git a/drivers/net/mlx5/rte_pmd_mlx5.h b/drivers/net/mlx5/rte_pmd_mlx5.h
index 654dd3cff3..004be0eea1 100644
--- a/drivers/net/mlx5/rte_pmd_mlx5.h
+++ b/drivers/net/mlx5/rte_pmd_mlx5.h
@@ -229,6 +229,108 @@ enum rte_pmd_mlx5_flow_engine_mode {
 __rte_experimental
 int rte_pmd_mlx5_flow_engine_set_mode(enum rte_pmd_mlx5_flow_engine_mode mode, uint32_t flags);
 
+/**
+ * User configuration structure using to create parser for single GENEVE TLV option.
+ */
+struct rte_pmd_mlx5_geneve_tlv {
+	/**
+	 * The class of the GENEVE TLV option.
+	 * Relevant only when 'match_on_class_mode' is 1.
+	 */
+	rte_be16_t option_class;
+	/**
+	 * The type of the GENEVE TLV option.
+	 * This field is the identifier of the option.
+	 */
+	uint8_t option_type;
+	/**
+	 * The length of the GENEVE TLV option data excluding the option header
+	 * in DW granularity.
+	 */
+	uint8_t option_len;
+	/**
+	 * Indicator about class field role in this option:
+	 *  0 - class is ignored.
+	 *  1 - class is fixed (the class defines the option along with the type).
+	 *  2 - class matching per flow.
+	 */
+	uint8_t match_on_class_mode;
+	/**
+	 * The offset of the first sample in DW granularity.
+	 * This offset is relative to first of option data.
+	 * The 'match_data_mask' corresponds to option data since this offset.
+	 */
+	uint8_t offset;
+	/**
+	 * The number of DW to sample.
+	 * This field describes the length of 'match_data_mask' in DW
+	 * granularity.
+	 */
+	uint8_t sample_len;
+	/**
+	 * Array of DWs which each bit marks if this bit should be sampled.
+	 * Each nonzero DW consumes one DW from maximum 7 DW in total.
+	 */
+	rte_be32_t *match_data_mask;
+};
+
+/**
+ * Creates GENEVE TLV parser for the selected port.
+ * This function must be called before first use of GENEVE option.
+ *
+ * This API is port oriented, but the configuration is done once for all ports
+ * under the same physical device. Each port should call this API before using
+ * GENEVE OPT item, but it must use the same options in the same order inside
+ * the list.
+ *
+ * Each physical device has 7 DWs for GENEVE TLV options. Each nonzero element
+ * in 'match_data_mask' array consumes one DW, and choosing matchable mode for
+ * class consumes additional one.
+ * Calling this API for second port under same physical device doesn't consume
+ * more DW, it uses same configuration.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] tlv_list
+ *   A list of GENEVE TLV options to create parser for them.
+ * @param[in] nb_options
+ *   The number of options in TLV list.
+ *
+ * @return
+ *   A pointer to TLV handle on success, NULL otherwise and rte_errno is set.
+ *   Possible values for rte_errno:
+ *   - ENOMEM - not enough memory to create GENEVE TLV parser.
+ *   - EEXIST - this port already has GENEVE TLV parser or another port under
+ *              same physical device has already prepared a different parser.
+ *   - EINVAL - invalid GENEVE TLV requested.
+ *   - ENODEV - there is no Ethernet device for this port id.
+ *   - ENOTSUP - the port doesn't support GENEVE TLV parsing.
+ */
+__rte_experimental
+void *
+rte_pmd_mlx5_create_geneve_tlv_parser(uint16_t port_id,
+				      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+				      uint8_t nb_options);
+
+/**
+ * Destroy GENEVE TLV parser for the selected port.
+ * This function must be called after last use of GENEVE option and before port
+ * closing.
+ *
+ * @param[in] handle
+ *   Handle for the GENEVE TLV parser object to be destroyed.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ *   Possible values for rte_errno:
+ *   - EINVAL - invalid handle.
+ *   - ENOENT - there is no valid GENEVE TLV parser in this handle.
+ *   - EBUSY - one of options is in used by template table.
+ */
+__rte_experimental
+int
+rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/drivers/net/mlx5/version.map b/drivers/net/mlx5/version.map
index 99f5ab754a..8fb0e07303 100644
--- a/drivers/net/mlx5/version.map
+++ b/drivers/net/mlx5/version.map
@@ -17,4 +17,7 @@ EXPERIMENTAL {
 	rte_pmd_mlx5_external_sq_enable;
 	# added in 23.03
 	rte_pmd_mlx5_flow_engine_set_mode;
+	# added in 24.03
+	rte_pmd_mlx5_create_geneve_tlv_parser;
+	rte_pmd_mlx5_destroy_geneve_tlv_parser;
 };
-- 
2.25.1

[PATCH v1 14/23] net/mlx5: add API to expose GENEVE option FW information

[Michael Baum]

Add a new API to expose GENEVE option FW information to DR layer.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.h        | 28 +++++++++
 drivers/net/mlx5/mlx5_flow_geneve.c | 94 +++++++++++++++++++++++++++++
 2 files changed, 122 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4bfc218175..dca3cacb65 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1766,6 +1766,34 @@ flow_hw_get_reg_id_from_ctx(void *dr_ctx,
 	return REG_NON;
 }
 
+/**
+ * Get GENEVE TLV option FW information according type and class.
+ *
+ * @param[in] dr_ctx
+ *   Pointer to HW steering DR context.
+ * @param[in] type
+ *   GENEVE TLV option type.
+ * @param[in] class
+ *   GENEVE TLV option class.
+ * @param[out] hl_ok_bit
+ *   Pointer to header layout structure describing OK bit FW information.
+ * @param[out] num_of_dws
+ *   Pointer to fill inside the size of 'hl_dws' array.
+ * @param[out] hl_dws
+ *   Pointer to header layout array describing data DWs FW information.
+ * @param[out] ok_bit_on_class
+ *   Pointer to an indicator whether OK bit includes class along with type.
+ *
+ * @return
+ *   0 on success, negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
+			struct mlx5_hl_data ** const hl_ok_bit,
+			uint8_t *num_of_dws,
+			struct mlx5_hl_data ** const hl_dws,
+			bool *ok_bit_on_class);
+
 void *
 mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index f23fb31aa0..2d593b70ba 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -58,6 +58,100 @@ struct mlx5_geneve_tlv_options {
 	RTE_ATOMIC(uint32_t) refcnt;
 };
 
+/**
+ * Check if type and class is matching to given GENEVE TLV option.
+ *
+ * @param type
+ *   GENEVE option type.
+ * @param class
+ *   GENEVE option class.
+ * @param option
+ *   Pointer to GENEVE TLV option structure.
+ *
+ * @return
+ *   True if this type and class match to this option, false otherwise.
+ */
+static inline bool
+option_match_type_and_class(uint8_t type, uint16_t class,
+			    struct mlx5_geneve_tlv_option *option)
+{
+	if (type != option->type)
+		return false;
+	if (option->class_mode == 1 && option->class != class)
+		return false;
+	return true;
+}
+
+/**
+ * Get GENEVE TLV option matching to given type and class.
+ *
+ * @param priv
+ *   Pointer to port's private data.
+ * @param type
+ *   GENEVE option type.
+ * @param class
+ *   GENEVE option class.
+ *
+ * @return
+ *   Pointer to option structure if exist, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_geneve_tlv_option *
+mlx5_geneve_tlv_option_get(const struct mlx5_priv *priv, uint8_t type,
+			   uint16_t class)
+{
+	struct mlx5_geneve_tlv_options *options;
+	uint8_t i;
+
+	if (priv->tlv_options == NULL) {
+		DRV_LOG(ERR,
+			"Port %u doesn't have configured GENEVE TLV options.",
+			priv->dev_data->port_id);
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	options = priv->tlv_options;
+	MLX5_ASSERT(options != NULL);
+	for (i = 0; i < options->nb_options; ++i) {
+		struct mlx5_geneve_tlv_option *option = &options->options[i];
+
+		if (option_match_type_and_class(type, class, option))
+			return option;
+	}
+	DRV_LOG(ERR, "TLV option type %u class %u doesn't exist.", type, class);
+	rte_errno = ENOENT;
+	return NULL;
+}
+
+int
+mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
+			struct mlx5_hl_data ** const hl_ok_bit,
+			uint8_t *num_of_dws,
+			struct mlx5_hl_data ** const hl_dws,
+			bool *ok_bit_on_class)
+{
+	uint16_t port_id;
+
+	MLX5_ETH_FOREACH_DEV(port_id, NULL) {
+		struct mlx5_priv *priv;
+		struct mlx5_geneve_tlv_option *option;
+
+		priv = rte_eth_devices[port_id].data->dev_private;
+		if (priv->dr_ctx != dr_ctx)
+			continue;
+		/* Find specific option inside list. */
+		option = mlx5_geneve_tlv_option_get(priv, type, class);
+		if (option == NULL)
+			return -rte_errno;
+		*hl_ok_bit = &option->hl_ok_bit;
+		*hl_dws = option->match_data;
+		*num_of_dws = option->match_data_size;
+		*ok_bit_on_class = !!(option->class_mode == 1);
+		return 0;
+	}
+	DRV_LOG(ERR, "DR CTX %p doesn't belong to any DPDK port.", dr_ctx);
+	return -EINVAL;
+}
+
 /**
  * Create single GENEVE TLV option sample.
  *
-- 
2.25.1

[PATCH v1 15/23] net/mlx5: add testpmd support for GENEVE TLV parser

[Michael Baum]

Add GENEVE TLV parser support for mlx5 testpmd using following commands:

1. Add single option to the global option list:

   testpmd> mlx5 set tlv_option class (class) type (type) len (length) \
            offset (sample_offset) sample_len (sample_len) \
            class_mode (ignore|fixed|matchable) \
            data (0xffffffff|0x0 [0xffffffff|0x0]*)

2. Remove several options from the global option list:

   testpmd> mlx5 flush tlv_options max (nb_option)

3. Print all options which are set in the global option list so far:

   testpmd> mlx5 list tlv_options

4. Create GENEVE TLV parser for specific port using option list which
   are set so far:

   testpmd> mlx5 port (port_id) apply tlv_options

5. Destroy GENEVE TLV parser for specific port:

   testpmd> mlx5 port (port_id) destroy tlv_options

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 doc/guides/nics/mlx5.rst        |  97 ++++++
 drivers/net/mlx5/mlx5_testpmd.c | 556 +++++++++++++++++++++++++++++++-
 2 files changed, 652 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 80446d8d82..b0f2cdcd62 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -2499,3 +2499,100 @@ This command is used for testing live migration,
 and works for software steering only.
 Default FDB jump should be disabled if switchdev is enabled.
 The mode will propagate to all the probed ports.
+
+GENEVE TLV options parser
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+GENEVE TLV options parser management.
+See :ref:`options parser API <geneve_parser_api>` for more information.
+
+Setting Option
+^^^^^^^^^^^^^^
+
+Add single option to the global option list::
+
+   testpmd> mlx5 set tlv_option class (class) type (type) len (length) \
+            offset (sample_offset) sample_len (sample_len) \
+            class_mode (ignore|fixed|matchable) data (0xffffffff|0x0 [0xffffffff|0x0]*)
+
+where:
+
+* ``class``: option class.
+* ``type``: option type.
+* ``length``: option data length in 4 bytes granularity.
+* ``sample_offset``: offset to data list related to option data start.
+  The offset is in 4 bytes granularity.
+* ``sample_len``: length data list in 4 bytes granularity.
+* ``ignore``: ignore ``class`` field.
+* ``fixed``: option class is fixed and defines the option along with the type.
+* ``matchable``: ``class`` field is matchable.
+* ``data``: list of masks indicating which DW should be configure.
+  The size of list should be equal to ``sample_len``.
+* ``0xffffffff``: this DW should be configure.
+* ``0x0``: this DW shouldn't be configure.
+
+
+Flushing Options
+^^^^^^^^^^^^^^^^
+
+Remove several options from the global option list::
+
+   testpmd> mlx5 flush tlv_options max (nb_option)
+
+where:
+
+* ``nb_option``: maximum number of option to remove from list. The order is LIFO.
+
+
+Listing Options
+^^^^^^^^^^^^^^^
+
+Print all options which are set in the global option list so far::
+
+   testpmd> mlx5 list tlv_options
+
+Output contains the values of each option, one per line.
+There is no output at all when no options are configured on the global list::
+
+   ID      Type    Class   Class_mode   Len     Offset  Sample_len   Data
+   [...]   [...]   [...]   [...]        [...]   [...]   [...]        [...]
+
+Setting several options and listing them::
+
+   testpmd> mlx5 set tlv_option class 1 type 1 len 4 offset 1 sample_len 3
+            class_mode fixed data 0xffffffff 0x0 0xffffffff
+   testpmd: set new option in global list, now it has 1 options
+   testpmd> mlx5 set tlv_option class 1 type 2 len 2 offset 0 sample_len 2
+            class_mode fixed data 0xffffffff 0xffffffff
+   testpmd: set new option in global list, now it has 2 options
+   testpmd> mlx5 set tlv_option class 1 type 3 len 5 offset 4 sample_len 1
+            class_mode fixed data 0xffffffff
+   testpmd: set new option in global list, now it has 3 options
+   testpmd> mlx5 list tlv_options
+   ID      Type    Class   Class_mode   Len    Offset  Sample_len  Data
+   0       1       1       fixed        4      1       3           0xffffffff 0x0 0xffffffff
+   1       2       1       fixed        2      0       2           0xffffffff 0xffffffff
+   2       3       1       fixed        5      4       1           0xffffffff
+   testpmd>
+
+
+Applying Options
+^^^^^^^^^^^^^^^^
+
+Create GENEVE TLV parser for specific port using option list which are set so
+far::
+
+   testpmd> mlx5 port (port_id) apply tlv_options
+
+The same global option list can used by several ports.
+
+
+Destroying Options
+^^^^^^^^^^^^^^^^^^
+
+Destroy GENEVE TLV parser for specific port::
+
+   testpmd> mlx5 port (port_id) destroy tlv_options
+
+This command doesn't destroy the global list,
+For releasing options, ``flush`` command should be used.
diff --git a/drivers/net/mlx5/mlx5_testpmd.c b/drivers/net/mlx5/mlx5_testpmd.c
index 403f3a8f83..5bc4dd0551 100644
--- a/drivers/net/mlx5/mlx5_testpmd.c
+++ b/drivers/net/mlx5/mlx5_testpmd.c
@@ -23,9 +23,25 @@
 #include "mlx5_testpmd.h"
 #include "testpmd.h"
 
-static uint8_t host_shaper_avail_thresh_triggered[RTE_MAX_ETHPORTS];
 #define SHAPER_DISABLE_DELAY_US 100000 /* 100ms */
+#define MAX_GENEVE_OPTIONS_RESOURCES 7
 #define PARSE_DELIMITER " \f\n\r\t\v"
+#define SPACE_DELIMITER (" ")
+
+static uint8_t host_shaper_avail_thresh_triggered[RTE_MAX_ETHPORTS];
+
+struct mlx5_port {
+	void *geneve_tlv_parser_handle;
+};
+
+static struct mlx5_port private_port[RTE_MAX_ETHPORTS] = {{0}};
+
+struct tlv_list_manager {
+	uint8_t nb_options;
+	struct rte_pmd_mlx5_geneve_tlv tlv_list[MAX_GENEVE_OPTIONS_RESOURCES];
+};
+
+static struct tlv_list_manager tlv_mng = {.nb_options = 0};
 
 static int
 parse_uint(uint64_t *value, const char *str)
@@ -304,6 +320,88 @@ mlx5_test_attach_port_extend_devargs(char *identifier)
 }
 #endif
 
+static inline const char *
+mode2string(uint8_t mode)
+{
+	switch (mode) {
+	case 0:
+		return "ignored\t";
+	case 1:
+		return "fixed\t";
+	case 2:
+		return "matchable";
+	default:
+		break;
+	}
+	return "unknown";
+}
+
+static inline uint8_t
+string2mode(const char *mode)
+{
+	if (strcmp(mode, "ignored") == 0)
+		return 0;
+	if (strcmp(mode, "fixed") == 0)
+		return 1;
+	if (strcmp(mode, "matchable") == 0)
+		return 2;
+	return UINT8_MAX;
+}
+
+static int
+mlx5_test_parse_geneve_option_data(const char *buff, uint8_t data_len,
+				   rte_be32_t **match_data_mask)
+{
+	rte_be32_t *data;
+	char *buff2;
+	char *token;
+	uint8_t i = 0;
+
+	if (data_len == 0) {
+		*match_data_mask = NULL;
+		return 0;
+	}
+
+	data = calloc(data_len, sizeof(rte_be32_t));
+	if (data == NULL) {
+		TESTPMD_LOG(ERR, "Fail to allocate memory for GENEVE TLV option data\n");
+		return -ENOMEM;
+	}
+
+	buff2 = strdup(buff);
+	if (buff2 == NULL) {
+		TESTPMD_LOG(ERR,
+			    "Fail to duplicate GENEVE TLV option data string (%s)\n",
+			    buff);
+		free(data);
+		return -ENOMEM;
+	}
+
+	token = strtok(buff2, SPACE_DELIMITER);
+	while (token != NULL) {
+		if (i == data_len) {
+			TESTPMD_LOG(ERR,
+				    "GENEVE TLV option has more data then given data length %u\n",
+				    data_len);
+			free(buff2);
+			free(data);
+			return -EINVAL;
+		}
+
+		if (strcmp(token, "0xffffffff") == 0)
+			data[i] = 0xffffffff;
+		else
+			data[i] = 0x0;
+
+		token = strtok(NULL, SPACE_DELIMITER);
+		i++;
+	}
+
+	free(buff2);
+	*match_data_mask = data;
+	return 0;
+}
+
 /* *** SET HOST_SHAPER FOR A PORT *** */
 struct cmd_port_host_shaper_result {
 	cmdline_fixed_string_t mlx5;
@@ -680,6 +778,429 @@ cmdline_parse_inst_t mlx5_cmd_set_flow_engine_mode = {
 	}
 };
 
+/* Prepare single GENEVE TLV option and add it into global option list. */
+struct mlx5_cmd_set_tlv_option {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t set;
+	cmdline_fixed_string_t tlv_option;
+	cmdline_fixed_string_t class;
+	uint16_t class_id;
+	cmdline_fixed_string_t type;
+	uint8_t type_id;
+	cmdline_fixed_string_t len;
+	uint8_t option_len;
+	cmdline_fixed_string_t offset;
+	uint8_t off;
+	cmdline_fixed_string_t sample_len;
+	uint8_t length;
+	cmdline_fixed_string_t class_mode;
+	cmdline_fixed_string_t cmode;
+	cmdline_fixed_string_t data;
+	cmdline_fixed_string_t data_mask;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, mlx5, "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_set =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, set, "set");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_tlv_option =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, tlv_option,
+				 "tlv_option");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_class =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, class,
+				 "class");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_class_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, class_id,
+			      RTE_UINT16);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_type =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, type, "type");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_type_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, type_id,
+			      RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_len =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, len, "len");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_option_len =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, option_len,
+			      RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_offset =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, offset,
+				 "offset");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_off =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, off, RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_sample_len =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, sample_len,
+				 "sample_len");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_length =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, length,
+			      RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_class_mode =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, class_mode,
+				 "class_mode");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_cmode =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, cmode,
+				 "ignored#fixed#matchable");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_data =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, data, "data");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_data_mask =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, data_mask, "");
+
+static void
+mlx5_cmd_set_tlv_option_parsed(void *parsed_result,
+			       __rte_unused struct cmdline *cl,
+			       __rte_unused void *data)
+{
+	struct mlx5_cmd_set_tlv_option *res = parsed_result;
+	struct rte_pmd_mlx5_geneve_tlv *option;
+	uint8_t class_mode;
+	int ret;
+
+	if (tlv_mng.nb_options == MAX_GENEVE_OPTIONS_RESOURCES) {
+		fprintf(stderr, "GENEVE TLV option list is full\n");
+		return;
+	}
+
+	if (res->option_len < res->length + res->off) {
+		fprintf(stderr,
+			"GENEVE TLV option length (%u) cannot be less than offset (%u) + sample_len (%u)\n",
+			res->option_len, res->length, res->off);
+		return;
+	}
+
+	if (res->option_len > 32) {
+		fprintf(stderr,
+			"GENEVE TLV option length (%u) must be less than 32\n",
+			res->option_len);
+		return;
+	}
+
+	class_mode = string2mode(res->cmode);
+	if (class_mode == UINT8_MAX) {
+		fprintf(stderr, "Invalid class mode \"%s\"\n", res->cmode);
+		return;
+	}
+
+	if (res->length > 0) {
+		if (strcmp(res->data, "data") || !strcmp(res->data_mask, "")) {
+			fprintf(stderr,
+				"sample_len is %u but any data isn't provided\n",
+				res->length);
+			return;
+		}
+	} else {
+		if (!strcmp(res->data, "data") && strcmp(res->data_mask, "")) {
+			fprintf(stderr,
+				"sample_len is 0 but data is provided (%s)\n",
+				res->data_mask);
+			return;
+		}
+	}
+
+	option = &tlv_mng.tlv_list[tlv_mng.nb_options];
+	ret = mlx5_test_parse_geneve_option_data(res->data_mask, res->length,
+						 &option->match_data_mask);
+	if (ret < 0)
+		return;
+
+	option->match_on_class_mode = class_mode;
+	option->option_class = rte_cpu_to_be_16(res->class_id);
+	option->option_type = res->type_id;
+	option->option_len = res->option_len;
+	option->offset = res->off;
+	option->sample_len = res->length;
+	tlv_mng.nb_options++;
+
+	TESTPMD_LOG(DEBUG,
+		    "set new option in global list, now it has %u options\n",
+		    tlv_mng.nb_options);
+}
+
+cmdline_parse_inst_t mlx5_cmd_set_tlv_option = {
+	.f = mlx5_cmd_set_tlv_option_parsed,
+	.data = NULL,
+	.help_str = "mlx5 set tlv_option class <class_id> type <type_id> len "
+		"<option_len> offset <sample_offset> sample_len "
+		"<sample_length> class_mode <ignored|fixed|matchable> data <mask1 [mask2 [...]>",
+	.tokens = {
+		(void *)&mlx5_cmd_set_tlv_option_mlx5,
+		(void *)&mlx5_cmd_set_tlv_option_set,
+		(void *)&mlx5_cmd_set_tlv_option_tlv_option,
+		(void *)&mlx5_cmd_set_tlv_option_class,
+		(void *)&mlx5_cmd_set_tlv_option_class_id,
+		(void *)&mlx5_cmd_set_tlv_option_type,
+		(void *)&mlx5_cmd_set_tlv_option_type_id,
+		(void *)&mlx5_cmd_set_tlv_option_len,
+		(void *)&mlx5_cmd_set_tlv_option_option_len,
+		(void *)&mlx5_cmd_set_tlv_option_offset,
+		(void *)&mlx5_cmd_set_tlv_option_off,
+		(void *)&mlx5_cmd_set_tlv_option_sample_len,
+		(void *)&mlx5_cmd_set_tlv_option_length,
+		(void *)&mlx5_cmd_set_tlv_option_class_mode,
+		(void *)&mlx5_cmd_set_tlv_option_cmode,
+		(void *)&mlx5_cmd_set_tlv_option_data,
+		(void *)&mlx5_cmd_set_tlv_option_data_mask,
+		NULL,
+	}
+};
+
+/* Print all GENEVE TLV options which are configured so far. */
+struct mlx5_cmd_list_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t list;
+	cmdline_fixed_string_t tlv_options;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_list_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_list_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_list_tlv_options_list =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_list_tlv_options, list,
+				 "list");
+cmdline_parse_token_string_t mlx5_cmd_list_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_list_tlv_options, tlv_options,
+				 "tlv_options");
+
+static void
+mlx5_cmd_list_tlv_options_parsed(__rte_unused void *parsed_result,
+				 __rte_unused struct cmdline *cl,
+				 __rte_unused void *data)
+{
+	struct rte_pmd_mlx5_geneve_tlv *option;
+	uint8_t i, j;
+
+	printf("ID\tType\tClass\tClass_mode\tLen\tOffset\tSample_len\tData\n");
+	for (i = 0; i < tlv_mng.nb_options; ++i) {
+		option = &tlv_mng.tlv_list[i];
+		printf("%u\t%u\t%u\t%s\t%u\t%u\t%u\t\t", i,
+		       option->option_type, rte_be_to_cpu_16(option->option_class),
+		       mode2string(option->match_on_class_mode),
+		       option->option_len,
+		       option->offset, option->sample_len);
+		for (j = 0; j < option->sample_len; ++j)
+			printf("0x%x ", option->match_data_mask[j]);
+		printf("\n");
+	}
+}
+
+cmdline_parse_inst_t mlx5_cmd_list_tlv_options = {
+	.f = mlx5_cmd_list_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 list tlv_options",
+	.tokens = {
+		(void *)&mlx5_cmd_list_tlv_options_mlx5,
+		(void *)&mlx5_cmd_list_tlv_options_list,
+		(void *)&mlx5_cmd_list_tlv_options_tlv_options,
+		NULL,
+	}
+};
+
+/* Clear all GENEVE TLV options which are configured so far. */
+struct mlx5_cmd_flush_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t flush;
+	cmdline_fixed_string_t tlv_options;
+	cmdline_fixed_string_t max;
+	uint8_t number;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_flush =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, flush,
+				 "flush");
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, tlv_options,
+				 "tlv_options");
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_max =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, max, "max");
+cmdline_parse_token_num_t mlx5_cmd_flush_tlv_options_number =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_flush_tlv_options, number,
+			      RTE_UINT8);
+
+static void
+mlx5_cmd_flush_tlv_options_parsed(void *parsed_result,
+				  __rte_unused struct cmdline *cl,
+				  __rte_unused void *data)
+{
+	struct mlx5_cmd_flush_tlv_options *res = parsed_result;
+	struct rte_pmd_mlx5_geneve_tlv *option;
+	uint8_t nb_options_flush = tlv_mng.nb_options;
+	uint8_t nb_options_left = 0;
+
+	if (strcmp(res->max, "max") == 0 && res->number < tlv_mng.nb_options) {
+		nb_options_left = tlv_mng.nb_options - res->number;
+		nb_options_flush = RTE_MIN(res->number, nb_options_flush);
+	}
+
+	while (tlv_mng.nb_options > nb_options_left) {
+		tlv_mng.nb_options--;
+		option = &tlv_mng.tlv_list[tlv_mng.nb_options];
+		if (option->match_data_mask) {
+			free(option->match_data_mask);
+			option->match_data_mask = NULL;
+		}
+	}
+
+	TESTPMD_LOG(DEBUG, "Flush %u latest configured GENEVE TLV options, "
+		    "current number of options in the list is %u\n",
+		    nb_options_flush, nb_options_left);
+}
+
+cmdline_parse_inst_t mlx5_cmd_flush_tlv_options = {
+	.f = mlx5_cmd_flush_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 flush tlv_options max <nb_options>",
+	.tokens = {
+		(void *)&mlx5_cmd_flush_tlv_options_mlx5,
+		(void *)&mlx5_cmd_flush_tlv_options_flush,
+		(void *)&mlx5_cmd_flush_tlv_options_tlv_options,
+		(void *)&mlx5_cmd_flush_tlv_options_max,
+		(void *)&mlx5_cmd_flush_tlv_options_number,
+		NULL,
+	}
+};
+
+/* Create GENEVE TLV parser using option list which is configured before. */
+struct mlx5_cmd_apply_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t port;
+	portid_t port_id;
+	cmdline_fixed_string_t apply;
+	cmdline_fixed_string_t tlv_options;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_port =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, port,
+				 "port");
+cmdline_parse_token_num_t mlx5_cmd_apply_tlv_options_port_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_apply_tlv_options, port_id,
+			      RTE_UINT16);
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_apply =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, apply,
+				 "apply");
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, tlv_options,
+				 "tlv_options");
+
+static void
+mlx5_cmd_apply_tlv_options_parsed(void *parsed_result,
+				  __rte_unused struct cmdline *cl,
+				  __rte_unused void *data)
+{
+	struct mlx5_cmd_apply_tlv_options *res = parsed_result;
+	struct mlx5_port *port;
+	void *handle;
+
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
+		return;
+
+	if (tlv_mng.nb_options == 0) {
+		fprintf(stderr, "The option list is empty, please set options\n");
+		return;
+	}
+
+	handle = rte_pmd_mlx5_create_geneve_tlv_parser(res->port_id,
+						       tlv_mng.tlv_list,
+						       tlv_mng.nb_options);
+	if (handle == NULL) {
+		fprintf(stderr,
+			"Fail to create GENEVE TLV parser, nb_option=%u: %s\n",
+			tlv_mng.nb_options, strerror(rte_errno));
+		return;
+	}
+
+	TESTPMD_LOG(DEBUG, "GENEVE TLV options parser is successfully created:"
+		    " nb_option=%u, handle=%p\n", tlv_mng.nb_options, handle);
+
+	port = &private_port[res->port_id];
+	port->geneve_tlv_parser_handle = handle;
+}
+
+cmdline_parse_inst_t mlx5_cmd_apply_tlv_options = {
+	.f = mlx5_cmd_apply_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 port <port_id> apply tlv_options",
+	.tokens = {
+		(void *)&mlx5_cmd_apply_tlv_options_mlx5,
+		(void *)&mlx5_cmd_apply_tlv_options_port,
+		(void *)&mlx5_cmd_apply_tlv_options_port_id,
+		(void *)&mlx5_cmd_apply_tlv_options_apply,
+		(void *)&mlx5_cmd_apply_tlv_options_tlv_options,
+		NULL,
+	}
+};
+
+/* Destroy GENEVE TLV parser created by apply command. */
+struct mlx5_cmd_destroy_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t port;
+	portid_t port_id;
+	cmdline_fixed_string_t destroy;
+	cmdline_fixed_string_t tlv_options;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_port =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, port,
+				 "port");
+cmdline_parse_token_num_t mlx5_cmd_destroy_tlv_options_port_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, port_id,
+			      RTE_UINT16);
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_destroy =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, destroy,
+				 "destroy");
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, tlv_options,
+				 "tlv_options");
+
+static void
+mlx5_cmd_destroy_tlv_options_parsed(void *parsed_result,
+				    __rte_unused struct cmdline *cl,
+				    __rte_unused void *data)
+{
+	struct mlx5_cmd_destroy_tlv_options *res = parsed_result;
+	struct mlx5_port *port;
+	int ret;
+
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
+		return;
+
+	port = &private_port[res->port_id];
+	if (!port->geneve_tlv_parser_handle)
+		return;
+
+	ret = rte_pmd_mlx5_destroy_geneve_tlv_parser(port->geneve_tlv_parser_handle);
+	if (ret < 0) {
+		fprintf(stderr, "Fail to destroy GENEVE TLV parser: %s\n",
+			strerror(-ret));
+		return;
+	}
+
+	TESTPMD_LOG(DEBUG, "GENEVE TLV options parser is successfully released:"
+		    " handle=%p\n", port->geneve_tlv_parser_handle);
+
+	port->geneve_tlv_parser_handle = NULL;
+}
+
+cmdline_parse_inst_t mlx5_cmd_destroy_tlv_options = {
+	.f = mlx5_cmd_destroy_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 port <port_id> destroy tlv_options",
+	.tokens = {
+		(void *)&mlx5_cmd_destroy_tlv_options_mlx5,
+		(void *)&mlx5_cmd_destroy_tlv_options_port,
+		(void *)&mlx5_cmd_destroy_tlv_options_port_id,
+		(void *)&mlx5_cmd_destroy_tlv_options_destroy,
+		(void *)&mlx5_cmd_destroy_tlv_options_tlv_options,
+		NULL,
+	}
+};
+
 static struct testpmd_driver_commands mlx5_driver_cmds = {
 	.commands = {
 		{
@@ -712,6 +1233,39 @@ static struct testpmd_driver_commands mlx5_driver_cmds = {
 			.help = "mlx5 set flow_engine (active|standby) [(flag)]\n"
 				"    Set flow_engine to the specific mode with flag.\n\n"
 		},
+		{
+			.ctx = &mlx5_cmd_set_tlv_option,
+			.help = "mlx5 set tlv_option class (class_id) type "
+				"(type_id) len (option_length) offset "
+				"(sample_offset) sample_len (sample_length) "
+				"class_mode (ignored|fixed|matchable) "
+				"data (mask1) [(mask2) [...]]\n"
+				"    Set single GENEVE TLV option inside global list "
+				"using later by apply command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_list_tlv_options,
+			.help = "mlx5 list tlv_options\n"
+				"    Print all GENEVE TLV options which are configured "
+				"so far by TLV option set command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_flush_tlv_options,
+			.help = "mlx5 flush tlv_options [max (number options)]\n"
+				"    Clear all GENEVE TLV options which are configured "
+				"so far by TLV option set command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_apply_tlv_options,
+			.help = "mlx5 port (port_id) apply tlv_options\n"
+				"    Create GENEVE TLV parser using option list which is "
+				"configured before by TLV option set command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_destroy_tlv_options,
+			.help = "mlx5 port (port_id) destroy tlv_options\n"
+				"    Destroy GENEVE TLV parser\n\n",
+		},
 		{
 			.ctx = NULL,
 		},
-- 
2.25.1

[PATCH v1 16/23] net/mlx5/hws: increase hl size for future compatibility

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

In some cases we rely on header layout DW offset from FW caps,
this is done in case of future HW which may support current
flex fields natively, for this we must increase header layout to
255 DWs, which is the limit in current definer creation.

Signed-off-by: Alex Vesker <valex@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.h | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h b/drivers/net/mlx5/hws/mlx5dr_definer.h
index 6f1c99e37a..e2be579303 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.h
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
@@ -523,10 +523,8 @@ struct mlx5_ifc_definer_hl_bits {
 	u8 unsupported_free_running_timestamp[0x40];
 	struct mlx5_ifc_definer_hl_flex_parser_bits flex_parser;
 	struct mlx5_ifc_definer_hl_registers_bits registers;
-	/* struct x ib_l3_extended; */
-	/* struct x rwh */
-	/* struct x dcceth */
-	/* struct x dceth */
+	/* Reserved in case header layout on future HW */
+	u8 unsupported_reserved[0xd40];
 };
 
 enum mlx5dr_definer_gtp {
-- 
2.25.1

[PATCH v1 17/23] net/mlx5/hws: support GENEVE matching

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

Add matching for GENEVE tunnel header.

Signed-off-by: Alex Vesker <valex@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 91 +++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_definer.h | 19 ++++++
 2 files changed, 110 insertions(+)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index bab1869369..141941c309 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -11,6 +11,7 @@
 #define UDP_GTPU_PORT	2152
 #define UDP_VXLAN_PORT	4789
 #define UDP_PORT_MPLS	6635
+#define UDP_GENEVE_PORT 6081
 #define UDP_ROCEV2_PORT	4791
 #define DR_FLOW_LAYER_TUNNEL_NO_MPLS (MLX5_FLOW_LAYER_TUNNEL & ~MLX5_FLOW_LAYER_MPLS)
 
@@ -172,6 +173,9 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		source_qp,		v->queue,		mlx5_rte_flow_item_sq) \
 	X(SET,		tag,			v->data,		rte_flow_item_tag) \
 	X(SET,		metadata,		v->data,		rte_flow_item_meta) \
+	X(SET_BE16,	geneve_protocol,	v->protocol,		rte_flow_item_geneve) \
+	X(SET,		geneve_udp_port,	UDP_GENEVE_PORT,	rte_flow_item_geneve) \
+	X(SET_BE16,	geneve_ctrl,		v->ver_opt_len_o_c_rsvd0,	rte_flow_item_geneve) \
 	X(SET_BE16,	gre_c_ver,		v->c_rsvd0_ver,		rte_flow_item_gre) \
 	X(SET_BE16,	gre_protocol_type,	v->protocol,		rte_flow_item_gre) \
 	X(SET,		ipv4_protocol_gre,	IPPROTO_GRE,		rte_flow_item_gre) \
@@ -682,6 +686,16 @@ mlx5dr_definer_mpls_label_set(struct mlx5dr_definer_fc *fc,
 	memcpy(tag + fc->byte_off + sizeof(v->label_tc_s), &v->ttl, sizeof(v->ttl));
 }
 
+static void
+mlx5dr_definer_geneve_vni_set(struct mlx5dr_definer_fc *fc,
+			      const void *item_spec,
+			      uint8_t *tag)
+{
+	const struct rte_flow_item_geneve *v = item_spec;
+
+	memcpy(tag + fc->byte_off, v->vni, sizeof(v->vni));
+}
+
 static void
 mlx5dr_definer_ib_l4_qp_set(struct mlx5dr_definer_fc *fc,
 			    const void *item_spec,
@@ -2172,6 +2186,79 @@ mlx5dr_definer_conv_item_ipv6_routing_ext(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_geneve(struct mlx5dr_definer_conv_data *cd,
+				struct rte_flow_item *item,
+				int item_idx)
+{
+	const struct rte_flow_item_geneve *m = item->mask;
+	struct mlx5dr_definer_fc *fc;
+	bool inner = cd->tunnel;
+
+	if (inner) {
+		DR_LOG(ERR, "Inner GENEVE item not supported");
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
+	/* In order to match on Geneve we must match on ip_protocol and l4_dport */
+	if (!cd->relaxed) {
+		fc = &cd->fc[DR_CALC_FNAME(IP_PROTOCOL, inner)];
+		if (!fc->tag_set) {
+			fc->item_idx = item_idx;
+			fc->tag_mask_set = &mlx5dr_definer_ones_set;
+			fc->tag_set = &mlx5dr_definer_udp_protocol_set;
+			DR_CALC_SET(fc, eth_l2, l4_type_bwc, inner);
+		}
+
+		fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, inner)];
+		if (!fc->tag_set) {
+			fc->item_idx = item_idx;
+			fc->tag_mask_set = &mlx5dr_definer_ones_set;
+			fc->tag_set = &mlx5dr_definer_geneve_udp_port_set;
+			DR_CALC_SET(fc, eth_l4, destination_port, inner);
+		}
+	}
+
+	if (!m)
+		return 0;
+
+	if (m->rsvd1) {
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
+	if (m->ver_opt_len_o_c_rsvd0) {
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_CTRL];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_ctrl_set;
+		DR_CALC_SET_HDR(fc, tunnel_header, tunnel_header_0);
+		fc->bit_mask = __mlx5_mask(header_geneve, ver_opt_len_o_c_rsvd);
+		fc->bit_off = __mlx5_dw_bit_off(header_geneve, ver_opt_len_o_c_rsvd);
+	}
+
+	if (m->protocol) {
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_PROTO];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_protocol_set;
+		DR_CALC_SET_HDR(fc, tunnel_header, tunnel_header_0);
+		fc->byte_off += MLX5_BYTE_OFF(header_geneve, protocol_type);
+		fc->bit_mask = __mlx5_mask(header_geneve, protocol_type);
+		fc->bit_off = __mlx5_dw_bit_off(header_geneve, protocol_type);
+	}
+
+	if (!is_mem_zero(m->vni, 3)) {
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_VNI];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_vni_set;
+		DR_CALC_SET_HDR(fc, tunnel_header, tunnel_header_1);
+		fc->bit_mask = __mlx5_mask(header_geneve, vni);
+		fc->bit_off = __mlx5_dw_bit_off(header_geneve, vni);
+	}
+
+	return 0;
+}
+
 static int
 mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
 			 struct mlx5dr_definer_fc *fc,
@@ -2528,6 +2615,10 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			item_flags |= MLX5_FLOW_LAYER_MPLS;
 			cd.mpls_idx++;
 			break;
+		case RTE_FLOW_ITEM_TYPE_GENEVE:
+			ret = mlx5dr_definer_conv_item_geneve(&cd, items, i);
+			item_flags |= MLX5_FLOW_LAYER_GENEVE;
+			break;
 		case RTE_FLOW_ITEM_TYPE_IB_BTH:
 			ret = mlx5dr_definer_conv_item_ib_l4(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_IB_BTH;
diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h b/drivers/net/mlx5/hws/mlx5dr_definer.h
index e2be579303..c09c0be62e 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.h
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
@@ -91,6 +91,9 @@ enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_VPORT_REG_C_0,
 	MLX5DR_DEFINER_FNAME_VXLAN_FLAGS,
 	MLX5DR_DEFINER_FNAME_VXLAN_VNI,
+	MLX5DR_DEFINER_FNAME_GENEVE_CTRL,
+	MLX5DR_DEFINER_FNAME_GENEVE_PROTO,
+	MLX5DR_DEFINER_FNAME_GENEVE_VNI,
 	MLX5DR_DEFINER_FNAME_SOURCE_QP,
 	MLX5DR_DEFINER_FNAME_REG_0,
 	MLX5DR_DEFINER_FNAME_REG_1,
@@ -608,6 +611,22 @@ struct mlx5_ifc_header_gre_bits {
 	u8 reserved_at_30[0x10];
 };
 
+struct mlx5_ifc_header_geneve_bits {
+	union {
+		u8 ver_opt_len_o_c_rsvd[0x10];
+		struct {
+			u8 version[0x2];
+			u8 opt_len[0x6];
+			u8 o_flag[0x1];
+			u8 c_flag[0x1];
+			u8 reserved_at_a[0x6];
+		};
+	};
+	u8 protocol_type[0x10];
+	u8 vni[0x18];
+	u8 reserved_at_38[0x8];
+};
+
 struct mlx5_ifc_header_icmp_bits {
 	union {
 		u8 icmp_dw1[0x20];
-- 
2.25.1

[PATCH v1 18/23] net/mlx5/hws: support GENEVE options header

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

Add support for matching multiple GENEVE options. Options
header introduces new complexities since there can be more
than one GENEVE option. This requires us to track the total
DWs used for matching. Current code supports 8DWs for data
including type, class, length. There is also an optimization
to use a special OK bit to reduce the use of limited data DWs.

Signed-off-by: Alex Vesker <valex@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 147 ++++++++++++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_definer.h |  24 +++++
 2 files changed, 165 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 141941c309..126e522235 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -117,6 +117,8 @@ struct mlx5dr_definer_conv_data {
 	uint8_t relaxed;
 	uint8_t tunnel;
 	uint8_t mpls_idx;
+	uint8_t geneve_opt_ok_idx;
+	uint8_t geneve_opt_data_idx;
 	enum rte_flow_item_type last_item;
 };
 
@@ -696,6 +698,29 @@ mlx5dr_definer_geneve_vni_set(struct mlx5dr_definer_fc *fc,
 	memcpy(tag + fc->byte_off, v->vni, sizeof(v->vni));
 }
 
+static void
+mlx5dr_definer_geneve_opt_ctrl_set(struct mlx5dr_definer_fc *fc,
+				   const void *item_spec,
+				   uint8_t *tag)
+{
+	const struct rte_flow_item_geneve_opt *v = item_spec;
+	uint32_t dw0 = 0;
+
+	dw0 |= v->option_type << __mlx5_dw_bit_off(header_geneve_opt, type);
+	dw0 |= rte_cpu_to_be_16(v->option_class) << __mlx5_dw_bit_off(header_geneve_opt, class);
+	DR_SET(tag, dw0, fc->byte_off, fc->bit_off, fc->bit_mask);
+}
+
+static void
+mlx5dr_definer_geneve_opt_data_set(struct mlx5dr_definer_fc *fc,
+				   const void *item_spec,
+				   uint8_t *tag)
+{
+	const struct rte_flow_item_geneve_opt *v = item_spec;
+
+	DR_SET_BE32(tag, v->data[fc->extra_data], fc->byte_off, fc->bit_off, fc->bit_mask);
+}
+
 static void
 mlx5dr_definer_ib_l4_qp_set(struct mlx5dr_definer_fc *fc,
 			    const void *item_spec,
@@ -1328,7 +1353,6 @@ mlx5dr_definer_conv_item_port(struct mlx5dr_definer_conv_data *cd,
 	struct mlx5dr_cmd_query_caps *caps = cd->ctx->caps;
 	const struct rte_flow_item_ethdev *m = item->mask;
 	struct mlx5dr_definer_fc *fc;
-	uint8_t bit_offset = 0;
 
 	if (m->port_id) {
 		if (!caps->wire_regc_mask) {
@@ -1337,16 +1361,13 @@ mlx5dr_definer_conv_item_port(struct mlx5dr_definer_conv_data *cd,
 			return rte_errno;
 		}
 
-		while (!(caps->wire_regc_mask & (1 << bit_offset)))
-			bit_offset++;
-
 		fc = &cd->fc[MLX5DR_DEFINER_FNAME_VPORT_REG_C_0];
 		fc->item_idx = item_idx;
 		fc->tag_set = &mlx5dr_definer_vport_set;
 		fc->tag_mask_set = &mlx5dr_definer_ones_set;
 		DR_CALC_SET_HDR(fc, registers, register_c_0);
-		fc->bit_off = bit_offset;
-		fc->bit_mask = caps->wire_regc_mask >> bit_offset;
+		fc->bit_off = __builtin_ctz(caps->wire_regc_mask);
+		fc->bit_mask = caps->wire_regc_mask >> fc->bit_off;
 	} else {
 		DR_LOG(ERR, "Pord ID item mask must specify ID mask");
 		rte_errno = EINVAL;
@@ -2259,6 +2280,116 @@ mlx5dr_definer_conv_item_geneve(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_geneve_opt(struct mlx5dr_definer_conv_data *cd,
+				    struct rte_flow_item *item,
+				    int item_idx)
+{
+	const struct rte_flow_item_geneve_opt *m = item->mask;
+	const struct rte_flow_item_geneve_opt *v = item->spec;
+	struct mlx5_hl_data *hl_ok_bit, *hl_dws;
+	struct mlx5dr_definer_fc *fc;
+	uint8_t num_of_dws, i;
+	bool ok_bit_on_class;
+	int ret;
+
+	if (!m || !(m->option_class || m->option_type || m->data))
+		return 0;
+
+	if (!v || m->option_type != 0xff) {
+		DR_LOG(ERR, "Cannot match geneve opt without valid opt type");
+		goto out_not_supp;
+	}
+
+	if (m->option_class && m->option_class != RTE_BE16(UINT16_MAX)) {
+		DR_LOG(ERR, "Geneve option class has invalid mask");
+		goto out_not_supp;
+	}
+
+	ret = mlx5_get_geneve_hl_data(cd->ctx,
+				      v->option_type,
+				      v->option_class,
+				      &hl_ok_bit,
+				      &num_of_dws,
+				      &hl_dws,
+				      &ok_bit_on_class);
+	if (ret) {
+		DR_LOG(ERR, "Geneve opt type and class %d not supported", v->option_type);
+		goto out_not_supp;
+	}
+
+	if (!ok_bit_on_class && m->option_class) {
+		/* DW0 is used, we will match type, class */
+		if (!num_of_dws || hl_dws[0].dw_mask != UINT32_MAX) {
+			DR_LOG(ERR, "Geneve opt type %d DW0 not supported", v->option_type);
+			goto out_not_supp;
+		}
+
+		if (MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx >
+		    MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_7) {
+			DR_LOG(ERR, "Max match geneve opt DWs reached");
+			goto out_not_supp;
+		}
+
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx++];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_opt_ctrl_set;
+		fc->byte_off = hl_dws[0].dw_offset * DW_SIZE;
+		fc->bit_mask = UINT32_MAX;
+	} else {
+		/* DW0 is not used, we must verify geneve opt type exists in packet */
+		if (!hl_ok_bit->dw_mask) {
+			DR_LOG(ERR, "Geneve opt OK bits not supported");
+			goto out_not_supp;
+		}
+
+		if (MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_0 + cd->geneve_opt_ok_idx >
+		    MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_7) {
+			DR_LOG(ERR, "Max match geneve opt reached");
+			goto out_not_supp;
+		}
+
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_0 + cd->geneve_opt_ok_idx++];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_ones_set;
+		fc->byte_off = hl_ok_bit->dw_offset * DW_SIZE +
+				__builtin_clz(hl_ok_bit->dw_mask) / 8;
+		fc->bit_off = __builtin_ctz(hl_ok_bit->dw_mask);
+		fc->bit_mask = 0x1;
+	}
+
+	for (i = 1; i < num_of_dws; i++) {
+		/* Process each valid geneve option data DW1..N */
+		if (!m->data[i - 1])
+			continue;
+
+		if (hl_dws[i].dw_mask != UINT32_MAX) {
+			DR_LOG(ERR, "Matching Geneve opt data[%d] not supported", i - 1);
+			goto out_not_supp;
+		}
+
+		if (MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx >
+		    MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_7) {
+			DR_LOG(ERR, "Max match geneve options DWs reached");
+			goto out_not_supp;
+		}
+
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx++];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_opt_data_set;
+		fc->byte_off = hl_dws[i].dw_offset * DW_SIZE;
+		fc->bit_mask = m->data[i - 1];
+		/* Use extra_data for data[] set offset */
+		fc->extra_data = i - 1;
+	}
+
+	return 0;
+
+out_not_supp:
+	rte_errno = ENOTSUP;
+	return rte_errno;
+}
+
 static int
 mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
 			 struct mlx5dr_definer_fc *fc,
@@ -2619,6 +2750,10 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			ret = mlx5dr_definer_conv_item_geneve(&cd, items, i);
 			item_flags |= MLX5_FLOW_LAYER_GENEVE;
 			break;
+		case RTE_FLOW_ITEM_TYPE_GENEVE_OPT:
+			ret = mlx5dr_definer_conv_item_geneve_opt(&cd, items, i);
+			item_flags |= RTE_FLOW_ITEM_TYPE_GENEVE_OPT;
+			break;
 		case RTE_FLOW_ITEM_TYPE_IB_BTH:
 			ret = mlx5dr_definer_conv_item_ib_l4(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_IB_BTH;
diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h b/drivers/net/mlx5/hws/mlx5dr_definer.h
index c09c0be62e..0aaafe46de 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.h
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
@@ -141,6 +141,22 @@ enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_OKS2_MPLS2_I,
 	MLX5DR_DEFINER_FNAME_OKS2_MPLS3_I,
 	MLX5DR_DEFINER_FNAME_OKS2_MPLS4_I,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_0,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_1,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_2,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_3,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_4,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_5,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_6,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_7,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_1,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_2,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_3,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_4,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_5,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_6,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_7,
 	MLX5DR_DEFINER_FNAME_IB_L4_OPCODE,
 	MLX5DR_DEFINER_FNAME_IB_L4_QPN,
 	MLX5DR_DEFINER_FNAME_IB_L4_A,
@@ -165,6 +181,7 @@ enum mlx5dr_definer_type {
 struct mlx5dr_definer_fc {
 	uint8_t item_idx;
 	uint8_t is_range;
+	uint16_t extra_data;
 	uint32_t byte_off;
 	int bit_off;
 	uint32_t bit_mask;
@@ -627,6 +644,13 @@ struct mlx5_ifc_header_geneve_bits {
 	u8 reserved_at_38[0x8];
 };
 
+struct mlx5_ifc_header_geneve_opt_bits {
+	u8 class[0x10];
+	u8 type[0x8];
+	u8 reserved[0x3];
+	u8 len[0x5];
+};
+
 struct mlx5_ifc_header_icmp_bits {
 	union {
 		u8 icmp_dw1[0x20];
-- 
2.25.1

[PATCH v1 19/23] net/mlx5: add support for GENEVE and option item in HWS

[Michael Baum]

Add HW steering support for both "RTE_FLOW_ITEM_TYPE_GENEVE" and
"RTE_FLOW_ITEM_TYPE_GENEVE_OPT".

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 doc/guides/nics/mlx5.rst               |  15 ++-
 doc/guides/rel_notes/release_24_03.rst |   5 +
 drivers/net/mlx5/mlx5_flow.h           |  21 +++++
 drivers/net/mlx5/mlx5_flow_geneve.c    | 121 ++++++++++++++++++++++++-
 drivers/net/mlx5/mlx5_flow_hw.c        |  44 ++++++++-
 5 files changed, 199 insertions(+), 7 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index b0f2cdcd62..645b566d80 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -329,12 +329,25 @@ Limitations
      - Length
      - Data
 
-  Only one Class/Type/Length Geneve TLV option is supported per shared device.
   Class/Type/Length fields must be specified as well as masks.
   Class/Type/Length specified masks must be full.
   Matching Geneve TLV option without specifying data is not supported.
   Matching Geneve TLV option with ``data & mask == 0`` is not supported.
 
+  In SW steering (``dv_flow_en`` = 1):
+
+     - Only one Class/Type/Length Geneve TLV option is supported per shared
+       device.
+     - Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 0.
+
+  In HW steering (``dv_flow_en`` = 2):
+
+     - Multiple Class/Type/Length Geneve TLV option are supported per physical
+       device. See :ref:`geneve_parser_api` for more information.
+     - Multiple of same Geneve TLV option isn't supported at the same pattern
+       template.
+     - Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+
 - VF: flow rules created on VF devices can only match traffic targeted at the
   configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``).
 
diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
index e9c9717706..bedef2a4c0 100644
--- a/doc/guides/rel_notes/release_24_03.rst
+++ b/doc/guides/rel_notes/release_24_03.rst
@@ -55,6 +55,11 @@ New Features
      Also, make sure to start the actual text at the margin.
      =======================================================
 
+* **Updated NVIDIA mlx5 net driver.**
+
+  * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE`` flow item.
+  * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE_OPT`` flow item.
+
 
 Removed Items
 -------------
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index dca3cacb65..04a2eb0b0c 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1332,6 +1332,15 @@ struct mlx5_action_construct_data {
 
 #define MAX_GENEVE_OPTIONS_RESOURCES 7
 
+/* GENEVE TLV options manager structure. */
+struct mlx5_geneve_tlv_options_mng {
+	uint8_t nb_options; /* Number of options inside the template. */
+	struct {
+		uint8_t opt_type;
+		uint16_t opt_class;
+	} options[MAX_GENEVE_OPTIONS_RESOURCES];
+};
+
 /* Flow item template struct. */
 struct rte_flow_pattern_template {
 	LIST_ENTRY(rte_flow_pattern_template) next;
@@ -1351,6 +1360,8 @@ struct rte_flow_pattern_template {
 	 * tag pattern item for representor matching.
 	 */
 	bool implicit_tag;
+	/* Manages all GENEVE TLV options used by this pattern template. */
+	struct mlx5_geneve_tlv_options_mng geneve_opt_mng;
 	uint8_t flex_item; /* flex item index. */
 };
 
@@ -1799,6 +1810,16 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
 			      uint8_t nb_options);
 int mlx5_geneve_tlv_parser_destroy(void *handle);
+int mlx5_flow_geneve_tlv_option_validate(struct mlx5_priv *priv,
+					 const struct rte_flow_item *geneve_opt,
+					 struct rte_flow_error *error);
+
+struct mlx5_geneve_tlv_options_mng;
+int mlx5_geneve_tlv_option_register(struct mlx5_priv *priv,
+				    const struct rte_flow_item_geneve_opt *spec,
+				    struct mlx5_geneve_tlv_options_mng *mng);
+void mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
+					struct mlx5_geneve_tlv_options_mng *mng);
 
 void flow_hw_set_port_info(struct rte_eth_dev *dev);
 void flow_hw_clear_port_info(struct rte_eth_dev *dev);
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index 2d593b70ba..2c8dc39e74 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -152,6 +152,106 @@ mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
 	return -EINVAL;
 }
 
+/**
+ * Calculate total data size.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] geneve_opt
+ *   Pointer to GENEVE option item structure.
+ * @param[out] error
+ *   Pointer to error structure.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_flow_geneve_tlv_option_validate(struct mlx5_priv *priv,
+				     const struct rte_flow_item *geneve_opt,
+				     struct rte_flow_error *error)
+{
+	const struct rte_flow_item_geneve_opt *spec = geneve_opt->spec;
+	const struct rte_flow_item_geneve_opt *mask = geneve_opt->mask;
+	struct mlx5_geneve_tlv_option *option;
+
+	option = mlx5_geneve_tlv_option_get(priv, spec->option_type, spec->option_class);
+	if (option == NULL)
+		return rte_flow_error_set(error, rte_errno,
+					  RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+					  "Unregistered GENEVE option");
+	if (mask->option_type != UINT8_MAX)
+		return rte_flow_error_set(error, EINVAL,
+					  RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+					  "GENEVE option type must be fully masked");
+	if (option->class_mode == 1 && mask->option_class != UINT16_MAX)
+		return rte_flow_error_set(error, EINVAL,
+					  RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+					  "GENEVE option class must be fully masked");
+	return 0;
+}
+
+/**
+ * Register single GENEVE TLV option as used by pattern template.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] spec
+ *   Pointer to GENEVE option item structure.
+ * @param[out] mng
+ *   Pointer to GENEVE option manager.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_geneve_tlv_option_register(struct mlx5_priv *priv,
+				const struct rte_flow_item_geneve_opt *spec,
+				struct mlx5_geneve_tlv_options_mng *mng)
+{
+	struct mlx5_geneve_tlv_option *option;
+
+	option = mlx5_geneve_tlv_option_get(priv, spec->option_type, spec->option_class);
+	if (option == NULL)
+		return -rte_errno;
+	/* Increase the option reference counter. */
+	rte_atomic_fetch_add_explicit(&option->refcnt, 1,
+				      rte_memory_order_relaxed);
+	/* Update the manager with option information. */
+	mng->options[mng->nb_options].opt_type = spec->option_type;
+	mng->options[mng->nb_options].opt_class = spec->option_class;
+	mng->nb_options++;
+	return 0;
+}
+
+/**
+ * Unregister all GENEVE TLV options used by pattern template.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] mng
+ *   Pointer to GENEVE option manager.
+ */
+void
+mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
+				   struct mlx5_geneve_tlv_options_mng *mng)
+{
+	struct mlx5_geneve_tlv_option *option;
+	uint8_t i;
+
+	for (i = 0; i < mng->nb_options; ++i) {
+		option = mlx5_geneve_tlv_option_get(priv,
+						    mng->options[i].opt_type,
+						    mng->options[i].opt_class);
+		MLX5_ASSERT(option != NULL);
+		/* Decrease the option reference counter. */
+		rte_atomic_fetch_sub_explicit(&option->refcnt, 1,
+					      rte_memory_order_relaxed);
+		mng->options[i].opt_type = 0;
+		mng->options[i].opt_class = 0;
+	}
+	mng->nb_options = 0;
+}
+
 /**
  * Create single GENEVE TLV option sample.
  *
@@ -208,6 +308,24 @@ mlx5_geneve_tlv_option_destroy_sample(struct mlx5_geneve_tlv_resource *resource)
 	resource->obj = NULL;
 }
 
+/*
+ * Sample for DW0 are created when one of two conditions is met:
+ * 1. Header is matchable.
+ * 2. This option doesn't configure any data DW.
+ */
+static bool
+should_configure_sample_for_dw0(const struct rte_pmd_mlx5_geneve_tlv *spec)
+{
+	uint8_t i;
+
+	if (spec->match_on_class_mode == 2)
+		return true;
+	for (i = 0; i < spec->sample_len; ++i)
+		if (spec->match_data_mask[i] != 0)
+			return false;
+	return true;
+}
+
 /**
  * Create single GENEVE TLV option.
  *
@@ -237,8 +355,7 @@ mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *s
 	uint8_t i, resource_id = 0;
 	int ret;
 
-	if (spec->match_on_class_mode == 2) {
-		/* Header is matchable, create sample for DW0. */
+	if (should_configure_sample_for_dw0(spec)) {
 		attr.sample_offset = 0;
 		resource = &option->resources[resource_id];
 		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index da873ae2e2..7c786c432f 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -6781,6 +6781,17 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 							  " attribute");
 			break;
 		}
+		case RTE_FLOW_ITEM_TYPE_GENEVE_OPT:
+		{
+			int ret;
+
+			ret = mlx5_flow_geneve_tlv_option_validate(priv,
+								   &items[i],
+								   error);
+			if (ret < 0)
+				return ret;
+			break;
+		}
 		case RTE_FLOW_ITEM_TYPE_VOID:
 		case RTE_FLOW_ITEM_TYPE_ETH:
 		case RTE_FLOW_ITEM_TYPE_VLAN:
@@ -6792,6 +6803,7 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_GTP_PSC:
 		case RTE_FLOW_ITEM_TYPE_VXLAN:
 		case RTE_FLOW_ITEM_TYPE_MPLS:
+		case RTE_FLOW_ITEM_TYPE_GENEVE:
 		case MLX5_RTE_FLOW_ITEM_TYPE_SQ:
 		case RTE_FLOW_ITEM_TYPE_GRE:
 		case RTE_FLOW_ITEM_TYPE_GRE_KEY:
@@ -6959,24 +6971,45 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 		}
 	}
 	for (i = 0; items[i].type != RTE_FLOW_ITEM_TYPE_END; ++i) {
-		if (items[i].type == RTE_FLOW_ITEM_TYPE_FLEX) {
+		switch (items[i].type) {
+		case RTE_FLOW_ITEM_TYPE_FLEX: {
 			const struct rte_flow_item_flex *spec =
 				(const struct rte_flow_item_flex *)items[i].spec;
 			struct rte_flow_item_flex_handle *handle = spec->handle;
 
 			if (flow_hw_flex_item_acquire(dev, handle, &it->flex_item)) {
-				claim_zero(mlx5dr_match_template_destroy(it->mt));
-				mlx5_free(it);
 				rte_flow_error_set(error, rte_errno,
 						   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
 						   "Failed to acquire flex item");
-				return NULL;
+				goto error;
 			}
+			break;
+		}
+		case RTE_FLOW_ITEM_TYPE_GENEVE_OPT: {
+			const struct rte_flow_item_geneve_opt *spec = items[i].spec;
+
+			if (mlx5_geneve_tlv_option_register(priv, spec,
+							    &it->geneve_opt_mng)) {
+				rte_flow_error_set(error, rte_errno,
+						   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+						   "Failed to register GENEVE TLV option");
+				goto error;
+			}
+			break;
+		}
+		default:
+			break;
 		}
 	}
 	__atomic_fetch_add(&it->refcnt, 1, __ATOMIC_RELAXED);
 	LIST_INSERT_HEAD(&priv->flow_hw_itt, it, next);
 	return it;
+error:
+	flow_hw_flex_item_release(dev, &it->flex_item);
+	mlx5_geneve_tlv_options_unregister(priv, &it->geneve_opt_mng);
+	claim_zero(mlx5dr_match_template_destroy(it->mt));
+	mlx5_free(it);
+	return NULL;
 }
 
 /**
@@ -6997,6 +7030,8 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 			      struct rte_flow_pattern_template *template,
 			      struct rte_flow_error *error __rte_unused)
 {
+	struct mlx5_priv *priv = dev->data->dev_private;
+
 	if (__atomic_load_n(&template->refcnt, __ATOMIC_RELAXED) > 1) {
 		DRV_LOG(WARNING, "Item template %p is still in use.",
 			(void *)template);
@@ -7010,6 +7045,7 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 		mlx5_free_srh_flex_parser(dev);
 	LIST_REMOVE(template, next);
 	flow_hw_flex_item_release(dev, &template->flex_item);
+	mlx5_geneve_tlv_options_unregister(priv, &template->geneve_opt_mng);
 	claim_zero(mlx5dr_match_template_destroy(template->mt));
 	mlx5_free(template);
 	return 0;
-- 
2.25.1

[PATCH v1 20/23] net/mlx5: add GENEVE option support for profile 0

[Michael Baum]

Add support for matching and modifying GENEVE option for
FLEX_PARSER_PROFILE_ENABLE=0.
Before this patch it is supported when FLEX_PARSER_PROFILE_ENABLE=8 in
HW steering and when FLEX_PARSER_PROFILE_ENABLE=0 in SW steering.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 doc/guides/nics/mlx5.rst            |   9 ++-
 doc/guides/platform/mlx5.rst        |   6 +-
 drivers/net/mlx5/mlx5_flow_geneve.c | 114 +++++++++++++++++++++-------
 3 files changed, 95 insertions(+), 34 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 645b566d80..b946ce00c2 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -347,6 +347,7 @@ Limitations
      - Multiple of same Geneve TLV option isn't supported at the same pattern
        template.
      - Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+     - Supported also when ``FLEX_PARSER_PROFILE_ENABLE`` = 0 for single DW only.
 
 - VF: flow rules created on VF devices can only match traffic targeted at the
   configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``).
@@ -2429,8 +2430,14 @@ Limitations
 ~~~~~~~~~~~
 
 * Supported only in HW steering (``dv_flow_en`` = 2).
-* Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
 * Supported for FW version **xx.37.0142** and above.
+* Parser creation can be done only for E-Switch manager.
+* Supported for multiple DW only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+* Supported for single DW also when ``FLEX_PARSER_PROFILE_ENABLE`` = 0 with some limitations:
+
+   - ``sample_len`` must be equal to ``option_len`` and not bigger than 1.
+   - ``match_on_class_mode`` different than 1 is not supported.
+   - ``offset`` must be 0.
 
 
 Testpmd driver specific commands
diff --git a/doc/guides/platform/mlx5.rst b/doc/guides/platform/mlx5.rst
index d16508d0da..a66cf778d1 100644
--- a/doc/guides/platform/mlx5.rst
+++ b/doc/guides/platform/mlx5.rst
@@ -536,12 +536,10 @@ Below are some firmware configurations listed.
    or
    FLEX_PARSER_PROFILE_ENABLE=1
 
-- enable Geneve TLV option flow matching in SW steering::
+- enable Geneve TLV option flow matching::
 
    FLEX_PARSER_PROFILE_ENABLE=0
-
-- enable Geneve TLV option flow matching in HW steering::
-
+   or
    FLEX_PARSER_PROFILE_ENABLE=8
 
 - enable GTP flow matching::
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index 2c8dc39e74..f3ee414d02 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -18,6 +18,8 @@
 #define MAX_GENEVE_OPTION_TOTAL_DATA_SIZE \
 		(MAX_GENEVE_OPTION_DATA_SIZE * MAX_GENEVE_OPTIONS_RESOURCES)
 
+#define INVALID_SAMPLE_ID (UINT8_MAX)
+
 /**
  * Single DW inside GENEVE TLV option.
  */
@@ -265,6 +267,8 @@ mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
  *   Pointer to header layout structure to update.
  * @param resource
  *   Pointer to single sample context to fill.
+ * @param sample_id
+ *   The flex parser id for single DW or UINT8_MAX for multiple DWs.
  *
  * @return
  *   0 on success, a negative errno otherwise and rte_errno is set.
@@ -274,7 +278,7 @@ mlx5_geneve_tlv_option_create_sample(void *ctx,
 		      struct mlx5_devx_geneve_tlv_option_attr *attr,
 		      struct mlx5_devx_match_sample_info_query_attr *query_attr,
 		      struct mlx5_hl_data *match_data,
-		      struct mlx5_geneve_tlv_resource *resource)
+		      struct mlx5_geneve_tlv_resource *resource, uint8_t sample_id)
 {
 	struct mlx5_devx_obj *obj;
 	int ret;
@@ -282,7 +286,10 @@ mlx5_geneve_tlv_option_create_sample(void *ctx,
 	obj = mlx5_devx_cmd_create_geneve_tlv_option(ctx, attr);
 	if (obj == NULL)
 		return -rte_errno;
-	ret = mlx5_devx_cmd_query_geneve_tlv_option(ctx, obj, query_attr);
+	if (sample_id == INVALID_SAMPLE_ID)
+		ret = mlx5_devx_cmd_query_geneve_tlv_option(ctx, obj, query_attr);
+	else
+		ret = mlx5_devx_cmd_match_sample_info_query(ctx, sample_id, query_attr);
 	if (ret) {
 		claim_zero(mlx5_devx_cmd_destroy(obj));
 		return ret;
@@ -335,20 +342,22 @@ should_configure_sample_for_dw0(const struct rte_pmd_mlx5_geneve_tlv *spec)
  *   Pointer to user configuration.
  * @param option
  *   Pointer to single GENEVE TLV option to fill.
+ * @param sample_id
+ *   The flex parser id for single DW or UINT8_MAX for multiple DWs.
  *
  * @return
  *   0 on success, a negative errno otherwise and rte_errno is set.
  */
 static int
 mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *spec,
-			      struct mlx5_geneve_tlv_option *option)
+			      struct mlx5_geneve_tlv_option *option, uint8_t sample_id)
 {
 	struct mlx5_devx_geneve_tlv_option_attr attr = {
 		.option_class = spec->option_class,
 		.option_type = spec->option_type,
 		.option_data_len = spec->option_len,
 		.option_class_ignore = spec->match_on_class_mode == 1 ? 0 : 1,
-		.offset_valid = 1,
+		.offset_valid = sample_id == INVALID_SAMPLE_ID ? 1 : 0,
 	};
 	struct mlx5_devx_match_sample_info_query_attr query_attr = {0};
 	struct mlx5_geneve_tlv_resource *resource;
@@ -356,12 +365,14 @@ mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *s
 	int ret;
 
 	if (should_configure_sample_for_dw0(spec)) {
+		MLX5_ASSERT(sample_id == INVALID_SAMPLE_ID);
 		attr.sample_offset = 0;
 		resource = &option->resources[resource_id];
 		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
 							   &query_attr,
 							   &option->match_data[0],
-							   resource);
+							   resource,
+							   INVALID_SAMPLE_ID);
 		if (ret)
 			return ret;
 		resource_id++;
@@ -379,7 +390,8 @@ mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *s
 		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
 							   &query_attr,
 							   &option->match_data[i],
-							   resource);
+							   resource,
+							   sample_id);
 		if (ret)
 			goto error;
 		resource_id++;
@@ -467,6 +479,8 @@ mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst,
  *   A list of GENEVE TLV options to create parser for them.
  * @param nb_options
  *   The number of options in TLV list.
+ * @param sample_id
+ *   The flex parser id for single DW or UINT8_MAX for multiple DWs.
  *
  * @return
  *   A pointer to GENEVE TLV options parser structure on success,
@@ -475,7 +489,7 @@ mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst,
 static struct mlx5_geneve_tlv_options *
 mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh,
 			       const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
-			       uint8_t nb_options)
+			       uint8_t nb_options, uint8_t sample_id)
 {
 	struct mlx5_geneve_tlv_options *options;
 	const struct rte_pmd_mlx5_geneve_tlv *spec;
@@ -495,7 +509,7 @@ mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh,
 	for (i = 0; i < nb_options; ++i) {
 		spec = &tlv_list[i];
 		ret = mlx5_geneve_tlv_option_create(sh->cdev->ctx, spec,
-						    &options->options[i]);
+						    &options->options[i], sample_id);
 		if (ret < 0)
 			goto error;
 		/* Copy the user list for comparing future configuration. */
@@ -705,6 +719,12 @@ mlx5_is_same_geneve_tlv_options(const struct mlx5_geneve_tlv_options *options,
 	return true;
 }
 
+static inline bool
+multiple_dws_supported(struct mlx5_hca_attr *attr)
+{
+	return attr->geneve_tlv_option_offset && attr->geneve_tlv_sample;
+}
+
 void *
 mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
@@ -715,8 +735,7 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 	struct rte_eth_dev *dev;
 	struct mlx5_priv *priv;
 	struct mlx5_hca_attr *attr;
-	uint8_t total_dws = 0;
-	uint8_t i;
+	uint8_t sample_id;
 
 	/*
 	 * Validate the input before taking a lock and before any memory
@@ -742,34 +761,71 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 		return NULL;
 	}
 	attr = &priv->sh->cdev->config.hca_attr;
-	MLX5_ASSERT(MAX_GENEVE_OPTIONS_RESOURCES <=
-		    attr->max_geneve_tlv_options);
-	if (!attr->geneve_tlv_option_offset || !attr->geneve_tlv_sample ||
-	    !attr->query_match_sample_info || !attr->geneve_tlv_opt) {
-		DRV_LOG(ERR, "Not enough capabilities to support GENEVE TLV parser, maybe old FW version");
+	if (!attr->query_match_sample_info || !attr->geneve_tlv_opt) {
+		DRV_LOG(ERR, "Not enough capabilities to support GENEVE TLV parser, is this device eswitch manager?");
 		rte_errno = ENOTSUP;
 		return NULL;
 	}
-	if (nb_options > MAX_GENEVE_OPTIONS_RESOURCES) {
+	DRV_LOG(DEBUG, "Max DWs supported for GENEVE TLV option is %u",
+		attr->max_geneve_tlv_options);
+	if (nb_options > attr->max_geneve_tlv_options) {
 		DRV_LOG(ERR,
 			"GENEVE TLV option number (%u) exceeds the limit (%u).",
-			nb_options, MAX_GENEVE_OPTIONS_RESOURCES);
+			nb_options, attr->max_geneve_tlv_options);
 		rte_errno = EINVAL;
 		return NULL;
 	}
-	for (i = 0; i < nb_options; ++i) {
-		if (mlx5_geneve_tlv_option_validate(attr, &tlv_list[i]) < 0) {
-			DRV_LOG(ERR, "GENEVE TLV option %u is invalid.", i);
+	if (multiple_dws_supported(attr)) {
+		uint8_t total_dws = 0;
+		uint8_t i;
+
+		MLX5_ASSERT(attr->max_geneve_tlv_options >= MAX_GENEVE_OPTIONS_RESOURCES);
+		for (i = 0; i < nb_options; ++i) {
+			if (mlx5_geneve_tlv_option_validate(attr, &tlv_list[i]) < 0) {
+				DRV_LOG(ERR, "GENEVE TLV option %u is invalid.", i);
+				return NULL;
+			}
+			total_dws += mlx5_geneve_tlv_option_get_nb_dws(&tlv_list[i]);
+		}
+		if (total_dws > MAX_GENEVE_OPTIONS_RESOURCES) {
+			DRV_LOG(ERR,
+				"Total requested DWs (%u) exceeds the limit (%u).",
+				total_dws, MAX_GENEVE_OPTIONS_RESOURCES);
+			rte_errno = EINVAL;
 			return NULL;
 		}
-		total_dws += mlx5_geneve_tlv_option_get_nb_dws(&tlv_list[i]);
-	}
-	if (total_dws > MAX_GENEVE_OPTIONS_RESOURCES) {
-		DRV_LOG(ERR,
-			"Total requested DWs (%u) exceeds the limit (%u).",
-			total_dws, MAX_GENEVE_OPTIONS_RESOURCES);
-		rte_errno = EINVAL;
-		return NULL;
+		/* Multiple DWs is supported, each of the has sample ID given later. */
+		sample_id = INVALID_SAMPLE_ID;
+		DRV_LOG(DEBUG, "GENEVE TLV parser supports multiple DWs, FLEX_PARSER_PROFILE_ENABLE == 8");
+	} else {
+		const struct rte_pmd_mlx5_geneve_tlv *option = &tlv_list[0];
+
+		if (option->offset != 0) {
+			DRV_LOG(ERR,
+				"GENEVE TLV option offset %u is required but not supported.",
+				option->offset);
+			rte_errno = ENOTSUP;
+			return NULL;
+		}
+		if (option->sample_len != option->option_len) {
+			DRV_LOG(ERR,
+				"GENEVE TLV option length (%u) should be equal to sample length (%u).",
+				option->option_len, option->sample_len);
+			rte_errno = ENOTSUP;
+			return NULL;
+		}
+		if (option->match_on_class_mode != 1) {
+			DRV_LOG(ERR,
+				"GENEVE TLV option match_on_class_mode %u is invalid for flex parser profile 0.",
+				option->match_on_class_mode);
+			rte_errno = EINVAL;
+			return NULL;
+		}
+		if (mlx5_geneve_tlv_option_validate(attr, option) < 0)
+			return NULL;
+		/* Single DW is supported, its sample ID is given. */
+		sample_id = attr->geneve_tlv_option_sample_id;
+		DRV_LOG(DEBUG, "GENEVE TLV parser supports only single DW, FLEX_PARSER_PROFILE_ENABLE == 0");
 	}
 	/* Take lock for this physical device and manage the options. */
 	phdev = mlx5_get_locked_physical_device(priv);
@@ -793,7 +849,7 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 		goto exit;
 	}
 	/* Create GENEVE TLV options for this physical device. */
-	options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options);
+	options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options, sample_id);
 	if (!options) {
 		mlx5_unlock_physical_device();
 		return NULL;
-- 
2.25.1

[PATCH v1 21/23] net/mlx5: add GENEVE option support for group 0

[Michael Baum]

Add support for HWS GENEVE options for flex parser profile 0 and group
0.

This patch avoids parser creation during matcher/flow preparation for HW
steering (MLX5_SET_MATCHER_HS) and removes some logic done in
"flow_dev_geneve_tlv_option_resource_*()" functions when dv_flow_en=2.

After this change, those functions became static and they were removed
from header file.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         |  8 +-------
 drivers/net/mlx5/mlx5_flow.h    |  4 ----
 drivers/net/mlx5/mlx5_flow_dv.c | 24 +++++++++++-------------
 3 files changed, 12 insertions(+), 24 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 5f8af31aea..881c42a97a 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2049,13 +2049,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 	} while (++i <= sh->bond.n_port);
 	if (sh->td)
 		claim_zero(mlx5_devx_cmd_destroy(sh->td));
-#ifdef HAVE_MLX5_HWS_SUPPORT
-	/* HWS manages geneve_tlv_option resource as global. */
-	if (sh->config.dv_flow_en == 2)
-		flow_dev_geneve_tlv_option_resource_release(sh);
-	else
-#endif
-		MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL);
+	MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL);
 	pthread_mutex_destroy(&sh->txpp.mutex);
 	mlx5_lwm_unset(sh);
 	mlx5_physical_device_destroy(sh->phdev);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 04a2eb0b0c..808f364c6c 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -2825,10 +2825,6 @@ void flow_hw_grp_clone_free_cb(void *tool_ctx, struct mlx5_list_entry *entry);
 
 struct mlx5_aso_age_action *flow_aso_age_get_by_idx(struct rte_eth_dev *dev,
 						    uint32_t age_idx);
-int flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
-					     const struct rte_flow_item *item,
-					     struct rte_flow_error *error);
-void flow_dev_geneve_tlv_option_resource_release(struct mlx5_dev_ctx_shared *sh);
 
 void flow_release_workspace(void *data);
 int mlx5_flow_os_init_workspace_once(void);
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 8894f51f4c..72e0d82e7b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -9880,7 +9880,7 @@ flow_dv_translate_item_geneve(void *key, const struct rte_flow_item *item,
 /**
  * Create Geneve TLV option resource.
  *
- * @param dev[in, out]
+ * @param[in, out] dev
  *   Pointer to rte_eth_dev structure.
  * @param[in] item
  *   Flow pattern to translate.
@@ -9890,8 +9890,7 @@ flow_dv_translate_item_geneve(void *key, const struct rte_flow_item *item,
  * @return
  *   0 on success otherwise -errno and errno is set.
  */
-
-int
+static int
 flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 					     const struct rte_flow_item *item,
 					     struct rte_flow_error *error)
@@ -9904,6 +9903,7 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 	const struct rte_flow_item_geneve_opt *geneve_opt_v = item->spec;
 	int ret = 0;
 
+	MLX5_ASSERT(sh->config.dv_flow_en == 1);
 	if (!geneve_opt_v)
 		return -1;
 	rte_spinlock_lock(&sh->geneve_tlv_opt_sl);
@@ -9914,13 +9914,8 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 			geneve_opt_v->option_type &&
 			geneve_opt_resource->length ==
 			geneve_opt_v->option_len) {
-			/*
-			 * We already have GENEVE TLV option obj allocated.
-			 * Increasing refcnt only in SWS. HWS uses it as global.
-			 */
-			if (priv->sh->config.dv_flow_en == 1)
-				__atomic_fetch_add(&geneve_opt_resource->refcnt, 1,
-						   __ATOMIC_RELAXED);
+			__atomic_fetch_add(&geneve_opt_resource->refcnt, 1,
+					   __ATOMIC_RELAXED);
 		} else {
 			ret = rte_flow_error_set(error, ENOMEM,
 				RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
@@ -9999,8 +9994,11 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 		return -1;
 	MLX5_ITEM_UPDATE(item, key_type, geneve_opt_v, geneve_opt_m,
 			 &rte_flow_item_geneve_opt_mask);
-	/* Register resource requires item spec. */
-	if (key_type & MLX5_SET_MATCHER_V) {
+	/*
+	 * Register resource requires item spec for SW steering,
+	 * for HW steering resources is registered explicitly by user.
+	 */
+	if (key_type & MLX5_SET_MATCHER_SW_V) {
 		ret = flow_dev_geneve_tlv_option_resource_register(dev, item,
 								   error);
 		if (ret) {
@@ -15777,7 +15775,7 @@ flow_dv_dest_array_resource_release(struct rte_eth_dev *dev,
 				    &resource->entry);
 }
 
-void
+static void
 flow_dev_geneve_tlv_option_resource_release(struct mlx5_dev_ctx_shared *sh)
 {
 	struct mlx5_geneve_tlv_option_resource *geneve_opt_resource =
-- 
2.25.1

[PATCH v1 22/23] net/mlx5: add support for GENEVE VNI modify field

[Michael Baum]

Add support for GENEVE VNI field modification.
The support is only using HW steering.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 doc/guides/nics/mlx5.rst               |  6 +++++-
 doc/guides/rel_notes/release_24_03.rst |  1 +
 drivers/net/mlx5/mlx5_flow_dv.c        |  4 +---
 drivers/net/mlx5/mlx5_flow_hw.c        | 12 ++++++++++--
 4 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index b946ce00c2..fceb5bd58b 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -577,7 +577,11 @@ Limitations
   - Modification of an arbitrary place in a packet via the special ``RTE_FLOW_FIELD_START`` Field ID is not supported.
   - Modification of the MPLS header is supported only in HWS and only to copy from,
     the encapsulation level is always 0.
-  - Modification of the 802.1Q Tag, VXLAN Network or GENEVE Network ID's is not supported.
+  - Modification of the 802.1Q Tag is not supported.
+  - Modification of VXLAN Network or GENEVE Network ID's is supported only for HW steering.
+  - Modification of GENEVE Network ID's is not supported when configured
+    ``FLEX_PARSER_PROFILE_ENABLE`` supports Geneve TLV options.
+    See :ref:`mlx5_firmware_config` for more flex parser information.
   - Encapsulation levels are not supported, can modify outermost header fields only.
   - Offsets cannot skip past the boundary of a field.
   - If the field type is ``RTE_FLOW_FIELD_MAC_TYPE``
diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
index bedef2a4c0..8a99d6bfa4 100644
--- a/doc/guides/rel_notes/release_24_03.rst
+++ b/doc/guides/rel_notes/release_24_03.rst
@@ -59,6 +59,7 @@ New Features
 
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE`` flow item.
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE_OPT`` flow item.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_VNI`` flow action.
 
 
 Removed Items
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 72e0d82e7b..bb3d7ddc3c 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1881,6 +1881,7 @@ mlx5_flow_field_id_to_modify_info
 			info[idx].offset = off_be;
 		break;
 	case RTE_FLOW_FIELD_VXLAN_VNI:
+	case RTE_FLOW_FIELD_GENEVE_VNI:
 		MLX5_ASSERT(data->offset + width <= 24);
 		/* VNI is on bits 31-8 of TUNNEL_HDR_DW_1. */
 		off_be = 24 - (data->offset + width) + 8;
@@ -1891,9 +1892,6 @@ mlx5_flow_field_id_to_modify_info
 		else
 			info[idx].offset = off_be;
 		break;
-	case RTE_FLOW_FIELD_GENEVE_VNI:
-		/* not supported yet*/
-		break;
 	case RTE_FLOW_FIELD_GTP_TEID:
 		MLX5_ASSERT(data->offset + width <= 32);
 		off_be = 32 - (data->offset + width);
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 7c786c432f..22ac4e0a7c 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4952,6 +4952,8 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 {
 	const struct rte_flow_action_modify_field *action_conf = action->conf;
 	const struct rte_flow_action_modify_field *mask_conf = mask->conf;
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_attr *attr = &priv->sh->cdev->config.hca_attr;
 	int ret;
 
 	if (!mask_conf)
@@ -5047,10 +5049,16 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
 				"modifying vlan_type is not supported");
-	if (flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_VNI))
+	/**
+	 * Geneve VNI modification is supported only when Geneve header is
+	 * parsed natively. When GENEVE options are supported, they both Geneve
+	 * and options headers are parsed as a flex parser.
+	 */
+	if (flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_VNI) &&
+	    attr->geneve_tlv_opt)
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"modifying Geneve VNI is not supported");
+				"modifying Geneve VNI is not supported when GENEVE opt is supported");
 	/* Due to HW bug, tunnel MPLS header is read only. */
 	if (action_conf->dst.field == RTE_FLOW_FIELD_MPLS)
 		return rte_flow_error_set(error, EINVAL,
-- 
2.25.1

[PATCH v1 23/23] net/mlx5: add support for modify GENEVE option header

[Michael Baum]

Add support for GENEVE option fields modification.
Only fields configured in parser creation can be modified.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
 doc/guides/nics/mlx5.rst               |   4 +
 doc/guides/rel_notes/release_24_03.rst |   3 +
 drivers/net/mlx5/mlx5_flow.h           |  21 +++++
 drivers/net/mlx5/mlx5_flow_dv.c        |  78 ++++++++++++++++-
 drivers/net/mlx5/mlx5_flow_geneve.c    | 117 +++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_flow_hw.c        |  71 ++++++++++-----
 6 files changed, 268 insertions(+), 26 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index fceb5bd58b..85820d7931 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -582,6 +582,10 @@ Limitations
   - Modification of GENEVE Network ID's is not supported when configured
     ``FLEX_PARSER_PROFILE_ENABLE`` supports Geneve TLV options.
     See :ref:`mlx5_firmware_config` for more flex parser information.
+  - Modification of GENEVE TLV option fields is supported only for HW steering.
+    Only DWs configured in :ref:`parser creation <geneve_parser_api>` can be modified,
+    'type' and 'class' fields can be modified when ``match_on_class_mode=2``.
+  - Modification of GENEVE TLV option data supports one DW per action.
   - Encapsulation levels are not supported, can modify outermost header fields only.
   - Offsets cannot skip past the boundary of a field.
   - If the field type is ``RTE_FLOW_FIELD_MAC_TYPE``
diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
index 8a99d6bfa4..f8d87c8a3c 100644
--- a/doc/guides/rel_notes/release_24_03.rst
+++ b/doc/guides/rel_notes/release_24_03.rst
@@ -60,6 +60,9 @@ New Features
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE`` flow item.
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE_OPT`` flow item.
   * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_VNI`` flow action.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_OPT_TYPE`` flow action.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_OPT_CLASS`` flow action.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_OPT_DATA`` flow action.
 
 
 Removed Items
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 808f364c6c..65fe5be2fd 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1805,6 +1805,25 @@ mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
 			struct mlx5_hl_data ** const hl_dws,
 			bool *ok_bit_on_class);
 
+/**
+ * Get modify field ID for single DW inside configured GENEVE TLV option.
+ *
+ * @param[in] dr_ctx
+ *   Pointer to HW steering DR context.
+ * @param[in] type
+ *   GENEVE TLV option type.
+ * @param[in] class
+ *   GENEVE TLV option class.
+ * @param[in] dw_offset
+ *   Offset of DW inside the option.
+ *
+ * @return
+ *   Modify field ID on success, negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_get_geneve_option_modify_field_id(const void *dr_ctx, uint8_t type,
+				       uint16_t class, uint8_t dw_offset);
+
 void *
 mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
@@ -1813,6 +1832,8 @@ int mlx5_geneve_tlv_parser_destroy(void *handle);
 int mlx5_flow_geneve_tlv_option_validate(struct mlx5_priv *priv,
 					 const struct rte_flow_item *geneve_opt,
 					 struct rte_flow_error *error);
+int mlx5_geneve_opt_modi_field_get(struct mlx5_priv *priv,
+				   const struct rte_flow_action_modify_data *data);
 
 struct mlx5_geneve_tlv_options_mng;
 int mlx5_geneve_tlv_option_register(struct mlx5_priv *priv,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index bb3d7ddc3c..2a7ee4e91f 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1446,6 +1446,21 @@ mlx5_mpls_modi_field_get(const struct rte_flow_action_modify_data *data)
 	return MLX5_MODI_IN_MPLS_LABEL_0 + data->tag_index;
 }
 
+static __rte_always_inline int
+flow_geneve_opt_modi_field_get(struct mlx5_priv *priv,
+			       const struct rte_flow_action_modify_data *data)
+{
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	return mlx5_geneve_opt_modi_field_get(priv, data);
+#else
+	(void)priv;
+	(void)data;
+	DRV_LOG(ERR, "GENEVE option modification is not supported.");
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+#endif
+}
+
 static void
 mlx5_modify_flex_item(const struct rte_eth_dev *dev,
 		      const struct mlx5_flex_item *flex,
@@ -1579,9 +1594,11 @@ mlx5_flow_field_id_to_modify_info
 		 const struct rte_flow_attr *attr, struct rte_flow_error *error)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	enum mlx5_modification_field modi_id;
 	uint32_t idx = 0;
 	uint32_t off_be = 0;
 	uint32_t length = 0;
+
 	switch ((int)data->field) {
 	case RTE_FLOW_FIELD_START:
 		/* not supported yet */
@@ -1892,6 +1909,48 @@ mlx5_flow_field_id_to_modify_info
 		else
 			info[idx].offset = off_be;
 		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_TYPE:
+		MLX5_ASSERT(data->offset + width <= 8);
+		modi_id = flow_geneve_opt_modi_field_get(priv, data);
+		if (modi_id < 0)
+			return;
+		/* Type is on bits 16-8 of GENEVE option header (DW0). */
+		off_be = 32 - (16 + data->offset + width);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
+		if (mask)
+			mask[idx] = flow_modify_info_mask_32(width, off_be);
+		else
+			info[idx].offset = off_be;
+		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_CLASS:
+		MLX5_ASSERT(data->offset + width <= 16);
+		modi_id = flow_geneve_opt_modi_field_get(priv, data);
+		if (modi_id < 0)
+			return;
+		/* Class is on bits 31-16 of GENEVE option header (DW0). */
+		off_be = 32 - (data->offset + width);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
+		if (mask)
+			mask[idx] = flow_modify_info_mask_32(width, off_be);
+		else
+			info[idx].offset = off_be;
+		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_DATA:
+		if ((data->offset % 32) + width > 32) {
+			DRV_LOG(ERR, "Geneve TLV option data is per DW.");
+			return;
+		}
+		modi_id = flow_geneve_opt_modi_field_get(priv, data);
+		if (modi_id < 0)
+			return;
+		/* Use offset inside DW. */
+		off_be = 32 - ((data->offset % 32) + width);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
+		if (mask)
+			mask[idx] = flow_modify_info_mask_32(width, off_be);
+		else
+			info[idx].offset = off_be;
+		break;
 	case RTE_FLOW_FIELD_GTP_TEID:
 		MLX5_ASSERT(data->offset + width <= 32);
 		off_be = 32 - (data->offset + width);
@@ -1905,8 +1964,8 @@ mlx5_flow_field_id_to_modify_info
 	case RTE_FLOW_FIELD_MPLS:
 		MLX5_ASSERT(data->offset + width <= 32);
 		off_be = 32 - (data->offset + width);
-		info[idx] = (struct field_modify_info){4, 0,
-					mlx5_mpls_modi_field_get(data)};
+		modi_id = mlx5_mpls_modi_field_get(data);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
 		if (mask)
 			mask[idx] = flow_modify_info_mask_32(width, off_be);
 		else
@@ -5388,6 +5447,21 @@ flow_dv_validate_action_modify_field(struct rte_eth_dev *dev,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
 				"modifications of the GENEVE Network"
 				" Identifier is not supported");
+	if (dst_data->field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE ||
+	    src_data->field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifications of the GENEVE option type is not supported");
+	if (dst_data->field == RTE_FLOW_FIELD_GENEVE_OPT_CLASS ||
+	    src_data->field == RTE_FLOW_FIELD_GENEVE_OPT_CLASS)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifications of the GENEVE option class is not supported");
+	if (dst_data->field == RTE_FLOW_FIELD_GENEVE_OPT_DATA ||
+	    src_data->field == RTE_FLOW_FIELD_GENEVE_OPT_DATA)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifications of the GENEVE option data is not supported");
 	if (dst_data->field == RTE_FLOW_FIELD_MPLS ||
 	    src_data->field == RTE_FLOW_FIELD_MPLS)
 		return rte_flow_error_set(error, ENOTSUP,
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index f3ee414d02..d5847a60e9 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -254,6 +254,123 @@ mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
 	mng->nb_options = 0;
 }
 
+/**
+ * Get single DW resource from given option.
+ *
+ * @param option
+ *   Pointer to single GENEVE TLV option.
+ * @param offset
+ *   Offset of DW related to option start.
+ *
+ * @return
+ *   DW resource on success, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_geneve_tlv_resource *
+mlx5_geneve_tlv_option_get_resource_by_offset(struct mlx5_geneve_tlv_option *option,
+					      uint8_t offset)
+{
+	uint8_t i;
+
+	for (i = 0; option->resources[i].obj != NULL; ++i) {
+		if (option->resources[i].offset < offset)
+			continue;
+		if (option->resources[i].offset == offset)
+			return &option->resources[i];
+		break;
+	}
+	DRV_LOG(ERR, "The DW in offset %u wasn't configured.", offset);
+	rte_errno = EINVAL;
+	return NULL;
+}
+
+int
+mlx5_get_geneve_option_modify_field_id(const void *dr_ctx, uint8_t type,
+				       uint16_t class, uint8_t dw_offset)
+{
+	uint16_t port_id;
+
+	MLX5_ETH_FOREACH_DEV(port_id, NULL) {
+		struct mlx5_priv *priv;
+		struct mlx5_geneve_tlv_option *option;
+		struct mlx5_geneve_tlv_resource *resource;
+
+		priv = rte_eth_devices[port_id].data->dev_private;
+		if (priv->dr_ctx != dr_ctx)
+			continue;
+		/* Find specific option inside list. */
+		option = mlx5_geneve_tlv_option_get(priv, type, class);
+		if (option == NULL)
+			return -rte_errno;
+		/* Find specific FW object inside option resources. */
+		resource = mlx5_geneve_tlv_option_get_resource_by_offset(option,
+									 dw_offset);
+		if (resource == NULL)
+			return -rte_errno;
+		return resource->modify_field;
+	}
+	DRV_LOG(ERR, "DR CTX %p doesn't belong to any DPDK port.", dr_ctx);
+	rte_errno = EINVAL;
+	return -rte_errno;
+}
+
+/**
+ * Get modify field ID for single DW inside configured GENEVE TLV option.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] data
+ *   Pointer to modify field data structure.
+ *
+ * @return
+ *   Modify field ID on success, negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_geneve_opt_modi_field_get(struct mlx5_priv *priv,
+			       const struct rte_flow_action_modify_data *data)
+{
+	uint16_t class = data->class_id;
+	uint8_t type = data->type;
+	struct mlx5_geneve_tlv_option *option;
+	struct mlx5_geneve_tlv_resource *resource;
+	uint8_t offset;
+
+	option = mlx5_geneve_tlv_option_get(priv, type, class);
+	if (option == NULL)
+		return -rte_errno;
+	switch (data->field) {
+	case RTE_FLOW_FIELD_GENEVE_OPT_TYPE:
+	case RTE_FLOW_FIELD_GENEVE_OPT_CLASS:
+		if (!option->match_data[0].dw_mask) {
+			DRV_LOG(ERR, "DW0 isn't configured");
+			rte_errno = EINVAL;
+			return -rte_errno;
+		}
+		resource = &option->resources[0];
+		MLX5_ASSERT(resource->offset == 0);
+		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_DATA:
+		/*
+		 * Convert offset twice:
+		 *  - First conversion from bit offset to DW offset.
+		 *  - Second conversion is to be related to data start instead
+		 *    of option start.
+		 */
+		offset = (data->offset >> 5) + 1;
+		resource = mlx5_geneve_tlv_option_get_resource_by_offset(option,
+									 offset);
+		break;
+	default:
+		DRV_LOG(ERR,
+			"Field ID %u doesn't describe GENEVE option header.",
+			data->field);
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	if (resource == NULL)
+		return -rte_errno;
+	return resource->modify_field;
+}
+
 /**
  * Create single GENEVE TLV option sample.
  *
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 22ac4e0a7c..692bbe063e 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -1254,10 +1254,12 @@ flow_hw_modify_field_compile(struct rte_eth_dev *dev,
 			else
 				value = rte_cpu_to_be_32(value);
 			item.spec = &value;
-		} else if (conf->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI) {
+		} else if (conf->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI ||
+			   conf->dst.field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE) {
 			/*
-			 * QFI is passed as an uint8_t integer, but it is accessed through
-			 * a 2nd least significant byte of a 32-bit field in modify header command.
+			 * Both QFI and Geneve option type are passed as an uint8_t integer,
+			 * but it is accessed through a 2nd least significant byte of a 32-bit
+			 * field in modify header command.
 			 */
 			value = *(const uint8_t *)item.spec;
 			value = rte_cpu_to_be_32(value << 8);
@@ -2825,12 +2827,14 @@ flow_hw_modify_field_construct(struct mlx5_hw_q_job *job,
 			*value_p = rte_cpu_to_be_32(*value_p << 16);
 		else
 			*value_p = rte_cpu_to_be_32(*value_p);
-	} else if (mhdr_action->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI) {
+	} else if (mhdr_action->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI ||
+		   mhdr_action->dst.field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE) {
 		uint32_t tmp;
 
 		/*
-		 * QFI is passed as an uint8_t integer, but it is accessed through
-		 * a 2nd least significant byte of a 32-bit field in modify header command.
+		 * Both QFI and Geneve option type are passed as an uint8_t integer,
+		 * but it is accessed through a 2nd least significant byte of a 32-bit
+		 * field in modify header command.
 		 */
 		tmp = values[0];
 		value_p = (unaligned_uint32_t *)values;
@@ -4944,6 +4948,14 @@ flow_hw_modify_field_is_used(const struct rte_flow_action_modify_field *action,
 	return action->src.field == field || action->dst.field == field;
 }
 
+static bool
+flow_hw_modify_field_is_geneve_opt(enum rte_flow_field_id field)
+{
+	return field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE ||
+	       field == RTE_FLOW_FIELD_GENEVE_OPT_CLASS ||
+	       field == RTE_FLOW_FIELD_GENEVE_OPT_DATA;
+}
+
 static int
 flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 				     const struct rte_flow_action *action,
@@ -4977,15 +4989,17 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 	ret = flow_validate_modify_field_level(&action_conf->dst, error);
 	if (ret)
 		return ret;
-	if (action_conf->dst.tag_index &&
-	    !flow_modify_field_support_tag_array(action_conf->dst.field))
-		return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"destination tag index is not supported");
-	if (action_conf->dst.class_id)
-		return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"destination class id is not supported");
+	if (!flow_hw_modify_field_is_geneve_opt(action_conf->dst.field)) {
+		if (action_conf->dst.tag_index &&
+		    !flow_modify_field_support_tag_array(action_conf->dst.field))
+			return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"destination tag index is not supported");
+		if (action_conf->dst.class_id)
+			return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"destination class id is not supported");
+	}
 	if (mask_conf->dst.level != UINT8_MAX)
 		return rte_flow_error_set(error, EINVAL,
 			RTE_FLOW_ERROR_TYPE_ACTION, action,
@@ -5000,15 +5014,17 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 				"destination field mask and template are not equal");
 	if (action_conf->src.field != RTE_FLOW_FIELD_POINTER &&
 	    action_conf->src.field != RTE_FLOW_FIELD_VALUE) {
-		if (action_conf->src.tag_index &&
-		    !flow_modify_field_support_tag_array(action_conf->src.field))
-			return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"source tag index is not supported");
-		if (action_conf->src.class_id)
-			return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"source class id is not supported");
+		if (!flow_hw_modify_field_is_geneve_opt(action_conf->src.field)) {
+			if (action_conf->src.tag_index &&
+			    !flow_modify_field_support_tag_array(action_conf->src.field))
+				return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"source tag index is not supported");
+			if (action_conf->src.class_id)
+				return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"source class id is not supported");
+		}
 		if (mask_conf->src.level != UINT8_MAX)
 			return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
@@ -5059,6 +5075,13 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
 				"modifying Geneve VNI is not supported when GENEVE opt is supported");
+	if (priv->tlv_options == NULL &&
+	    (flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_OPT_TYPE) ||
+	     flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_OPT_CLASS) ||
+	     flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_OPT_DATA)))
+		return rte_flow_error_set(error, EINVAL,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifying Geneve TLV option is supported only after parser configuration");
 	/* Due to HW bug, tunnel MPLS header is read only. */
 	if (action_conf->dst.field == RTE_FLOW_FIELD_MPLS)
 		return rte_flow_error_set(error, EINVAL,
-- 
2.25.1

RE: [PATCH v1 00/23] net/mlx5: support Geneve and options for HWS

[Suanming Mou]

Hi,

> -----Original Message-----
> From: Michael Baum <michaelba@nvidia.com>
> Sent: Sunday, December 3, 2023 7:25 PM
> To: dev@dpdk.org
> Cc: Matan Azrad <matan@nvidia.com>; Raslan Darawsheh
> <rasland@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam
> <orika@nvidia.com>; Suanming Mou <suanmingm@nvidia.com>
> Subject: [PATCH v1 00/23] net/mlx5: support Geneve and options for HWS
> 
> Add HWS support for both GENEVE and GENEVE TLV option headers.
> This patchset supports:
> 
>  - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE" flow item.
>  - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE_OPT" flow item.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_VNI" for modify field flow
>    action.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_TYPE" for modify field
>    flow action.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_CLASS" for modify field
>    flow action.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_DATA" for modify field
>    flow action.
> 
> The GENEVE TLV options support using flex parser.
> The profile should be specified to either 8 for multiple option or 0 for single
> option.
> A new API is added to create the GENEVE option parser before using it in
> templates API.

Series Acked-by: Suanming Mou <suanmingm@nvidia.com>

Thank you!

> 
> Alex Vesker (4):
>   net/mlx5/hws: fix tunnel protocol checks
>   net/mlx5/hws: increase hl size for future compatibility
>   net/mlx5/hws: support GENEVE matching
>   net/mlx5/hws: support GENEVE options header
> 
> Michael Baum (19):
>   common/mlx5: fix duplicate read of general capabilities
>   common/mlx5: fix query sample info capability
>   net/mlx5: remove GENEVE options length limitation
>   net/mlx5: fix GENEVE option item translation
>   common/mlx5: add system image GUID attribute
>   common/mlx5: add GENEVE TLV option attribute structure
>   common/mlx5: add PRM attribute for TLV sample
>   common/mlx5: add sample info query syndrome into error log
>   common/mlx5: query GENEVE option sample ID from HCA attr
>   common/mlx5: add function to query GENEVE TLV option
>   net/mlx5: add physical device handle
>   net/mlx5: add GENEVE TLV options parser API
>   net/mlx5: add API to expose GENEVE option FW information
>   net/mlx5: add testpmd support for GENEVE TLV parser
>   net/mlx5: add support for GENEVE and option item in HWS
>   net/mlx5: add GENEVE option support for profile 0
>   net/mlx5: add GENEVE option support for group 0
>   net/mlx5: add support for GENEVE VNI modify field
>   net/mlx5: add support for modify GENEVE option header
> 
>  doc/guides/nics/mlx5.rst               |  251 +++++-
>  doc/guides/platform/mlx5.rst           |    2 +
>  doc/guides/rel_notes/release_24_03.rst |    9 +
>  drivers/common/mlx5/mlx5_devx_cmds.c   |  139 +++-
>  drivers/common/mlx5/mlx5_devx_cmds.h   |   29 +-
>  drivers/common/mlx5/mlx5_prm.h         |   20 +-
>  drivers/common/mlx5/version.map        |    1 +
>  drivers/net/mlx5/hws/mlx5dr_definer.c  |  277 ++++++-
>  drivers/net/mlx5/hws/mlx5dr_definer.h  |   49 +-
>  drivers/net/mlx5/meson.build           |    1 +
>  drivers/net/mlx5/mlx5.c                |  115 ++-
>  drivers/net/mlx5/mlx5.h                |   21 +
>  drivers/net/mlx5/mlx5_flow.c           |   30 +
>  drivers/net/mlx5/mlx5_flow.h           |   92 ++-
>  drivers/net/mlx5/mlx5_flow_dv.c        |  158 ++--
>  drivers/net/mlx5/mlx5_flow_geneve.c    | 1011 ++++++++++++++++++++++++
>  drivers/net/mlx5/mlx5_flow_hw.c        |  127 ++-
>  drivers/net/mlx5/mlx5_testpmd.c        |  556 ++++++++++++-
>  drivers/net/mlx5/rte_pmd_mlx5.h        |  102 +++
>  drivers/net/mlx5/version.map           |    3 +
>  20 files changed, 2809 insertions(+), 184 deletions(-)  create mode 100644
> drivers/net/mlx5/mlx5_flow_geneve.c
> 
> --
> 2.25.1

[PATCH v2 00/23] net/mlx5: support Geneve and options for HWS

[Michael Baum]

Add HWS support for both GENEVE and GENEVE TLV option headers.
This patchset supports:

 - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE" flow item.
 - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE_OPT" flow item.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_VNI" for modify field flow
   action.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_TYPE" for modify field
   flow action.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_CLASS" for modify field
   flow action.
 - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_DATA" for modify field
   flow action.

The GENEVE TLV options support using flex parser.
The profile should be specified to either 8 for multiple option or 0 for
single option.
A new API is added to create the GENEVE option parser before using it in
templates API.

v2:
 - Rebase.
 - Add "Acked-by" from v1.

Alex Vesker (4):
  net/mlx5/hws: fix tunnel protocol checks
  net/mlx5/hws: increase hl size for future compatibility
  net/mlx5/hws: support GENEVE matching
  net/mlx5/hws: support GENEVE options header

Michael Baum (19):
  common/mlx5: fix duplicate read of general capabilities
  common/mlx5: fix query sample info capability
  net/mlx5: remove GENEVE options length limitation
  net/mlx5: fix GENEVE option item translation
  common/mlx5: add system image GUID attribute
  common/mlx5: add GENEVE TLV option attribute structure
  common/mlx5: add PRM attribute for TLV sample
  common/mlx5: add sample info query syndrome into error log
  common/mlx5: query GENEVE option sample ID from HCA attr
  common/mlx5: add function to query GENEVE TLV option
  net/mlx5: add physical device handle
  net/mlx5: add GENEVE TLV options parser API
  net/mlx5: add API to expose GENEVE option FW information
  net/mlx5: add testpmd support for GENEVE TLV parser
  net/mlx5: add support for GENEVE and option item in HWS
  net/mlx5: add GENEVE option support for profile 0
  net/mlx5: add GENEVE option support for group 0
  net/mlx5: add support for GENEVE VNI modify field
  net/mlx5: add support for modify GENEVE option header

 doc/guides/nics/mlx5.rst               |  251 +++++-
 doc/guides/platform/mlx5.rst           |    2 +
 doc/guides/rel_notes/release_24_03.rst |    9 +
 drivers/common/mlx5/mlx5_devx_cmds.c   |  139 +++-
 drivers/common/mlx5/mlx5_devx_cmds.h   |   29 +-
 drivers/common/mlx5/mlx5_prm.h         |   20 +-
 drivers/common/mlx5/version.map        |    1 +
 drivers/net/mlx5/hws/mlx5dr_definer.c  |  281 ++++++-
 drivers/net/mlx5/hws/mlx5dr_definer.h  |   49 +-
 drivers/net/mlx5/meson.build           |    1 +
 drivers/net/mlx5/mlx5.c                |  115 ++-
 drivers/net/mlx5/mlx5.h                |   21 +
 drivers/net/mlx5/mlx5_flow.c           |   30 +
 drivers/net/mlx5/mlx5_flow.h           |   92 ++-
 drivers/net/mlx5/mlx5_flow_dv.c        |  158 ++--
 drivers/net/mlx5/mlx5_flow_geneve.c    | 1011 ++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_flow_hw.c        |  127 ++-
 drivers/net/mlx5/mlx5_testpmd.c        |  556 ++++++++++++-
 drivers/net/mlx5/rte_pmd_mlx5.h        |  102 +++
 drivers/net/mlx5/version.map           |    3 +
 20 files changed, 2811 insertions(+), 186 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_flow_geneve.c

-- 
2.25.1

[PATCH v2 01/23] common/mlx5: fix duplicate read of general capabilities

[Michael Baum]

General object types support is indicated in bitmap general_obj_types,
which is part of HCA capabilities list.
This bitmap was read multiple times, and each time a different bit was
extracted.

Previous patch optimized the code, reading the bitmap once into a local
variable, and then extracting the required bits.
However, it missed few of them which still read the bitmap for
themselves. In addition, for other readings, it moved them to use local
variable without removing the old reading, and they are read twice.

This patch moves them all to use the local variable and removes all
duplications.

Fixes: 876d4702b141 ("common/mlx5: optimize read of general capabilities")
Cc: dekelp@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 18 ++++--------------
 1 file changed, 4 insertions(+), 14 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 3a894f894a..faa38a9f95 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -966,18 +966,6 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->max_geneve_tlv_option_data_len = MLX5_GET(cmd_hca_cap, hcattr,
 			max_geneve_tlv_option_data_len);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
-	attr->qos.flow_meter_aso_sup = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
-			      MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_METER_ASO);
-	attr->vdpa.valid = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
-			      MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q);
-	attr->vdpa.queue_counters_valid = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-							general_obj_types) &
-				  MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS);
-	attr->parse_graph_flex_node = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
-			      MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE);
 	attr->wqe_index_ignore = MLX5_GET(cmd_hca_cap, hcattr,
 					  wqe_index_ignore_cap);
 	attr->cross_channel = MLX5_GET(cmd_hca_cap, hcattr, cd);
@@ -1001,6 +989,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	/* Read the general_obj_types bitmap and extract the relevant bits. */
 	general_obj_types_supported = MLX5_GET64(cmd_hca_cap, hcattr,
 						 general_obj_types);
+	attr->qos.flow_meter_aso_sup =
+			!!(general_obj_types_supported &
+			   MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_METER_ASO);
 	attr->vdpa.valid = !!(general_obj_types_supported &
 			      MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q);
 	attr->vdpa.queue_counters_valid =
@@ -1074,8 +1065,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 		MLX5_GET(cmd_hca_cap, hcattr, umr_modify_entity_size_disabled);
 	attr->wait_on_time = MLX5_GET(cmd_hca_cap, hcattr, wait_on_time);
 	attr->crypto = MLX5_GET(cmd_hca_cap, hcattr, crypto);
-	attr->ct_offload = !!(MLX5_GET64(cmd_hca_cap, hcattr,
-					 general_obj_types) &
+	attr->ct_offload = !!(general_obj_types_supported &
 			      MLX5_GENERAL_OBJ_TYPES_CAP_CONN_TRACK_OFFLOAD);
 	attr->rq_delay_drop = MLX5_GET(cmd_hca_cap, hcattr, rq_delay_drop);
 	attr->nic_flow_table = MLX5_GET(cmd_hca_cap, hcattr, nic_flow_table);
-- 
2.25.1

[PATCH v2 02/23] common/mlx5: fix query sample info capability

[Michael Baum]

Query sample info operation might be used by either Geneve TLV option or
parse graph. Each operations can be supported regardless to another
according the configured profile.

In current implementation, the query sample info capability is turn on
only when parse graph operation is supported adding unnecessary
requirement for Geneve TLV option.

This patch adds different cap for Geneve TLV option.

Fixes: bc0a9303ed6a ("net/mlx5: adopt new sample ID")
Cc: rongweil@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 6 ++++--
 drivers/common/mlx5/mlx5_devx_cmds.h | 1 +
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index faa38a9f95..9b1cfcc135 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -965,6 +965,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			max_geneve_tlv_options);
 	attr->max_geneve_tlv_option_data_len = MLX5_GET(cmd_hca_cap, hcattr,
 			max_geneve_tlv_option_data_len);
+	attr->query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
+						 query_match_sample_info);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
 	attr->wqe_index_ignore = MLX5_GET(cmd_hca_cap, hcattr,
 					  wqe_index_ignore_cap);
@@ -1094,8 +1096,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			(ctx, &attr->flex);
 		if (rc)
 			return -1;
-		attr->flex.query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
-							      query_match_sample_info);
+		attr->flex.query_match_sample_info =
+						attr->query_match_sample_info;
 	}
 	if (attr->crypto) {
 		attr->aes_xts = MLX5_GET(cmd_hca_cap, hcattr, aes_xts) ||
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 4a6008dc1a..0c5727b669 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -316,6 +316,7 @@ struct mlx5_hca_attr {
 	uint32_t flow_counter_bulk_log_granularity:5;
 	uint32_t alloc_flow_counter_pd:1;
 	uint32_t flow_counter_access_aso:1;
+	uint32_t query_match_sample_info:1;
 	uint32_t flow_access_aso_opc_mod:8;
 	uint32_t cross_vhca:1;
 	uint32_t lag_rx_port_affinity:1;
-- 
2.25.1

[PATCH v2 03/23] net/mlx5/hws: fix tunnel protocol checks

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

Align GRE, GTPU and VXLAN tunnel protocols to fail
in case the packet is already tunneled. Also use local
defines for protocol UDP ports for better layering of
mlx5dr API.

Fixes: c55c2bf35333 ("net/mlx5/hws: add definer layer")
Fixes: 5bf14a4beb1a ("net/mlx5/hws: support matching on MPLSoUDP")
Cc: valex@nvidia.com
Cc: erezsh@nvidia.com
Cc: stable@dpdk.org

Signed-off-by: Alex Vesker <valex@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 43 +++++++++++++--------------
 1 file changed, 21 insertions(+), 22 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 750eb9c7c6..219bffd3b5 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -8,9 +8,10 @@
 #define BAD_PORT	0xBAD
 #define ETH_TYPE_IPV4_VXLAN	0x0800
 #define ETH_TYPE_IPV6_VXLAN	0x86DD
-#define ETH_VXLAN_DEFAULT_PORT	4789
-#define ETH_VXLAN_GPE_DEFAULT_PORT	4790
-#define IP_UDP_PORT_MPLS	6635
+#define UDP_VXLAN_PORT	4789
+#define UDP_VXLAN_GPE_PORT	4790
+#define UDP_GTPU_PORT	2152
+#define UDP_PORT_MPLS	6635
 #define UDP_ROCEV2_PORT	4791
 #define DR_FLOW_LAYER_TUNNEL_NO_MPLS (MLX5_FLOW_LAYER_TUNNEL & ~MLX5_FLOW_LAYER_MPLS)
 
@@ -159,7 +160,7 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		tcp_protocol,		STE_TCP,		rte_flow_item_tcp) \
 	X(SET_BE16,	tcp_src_port,		v->hdr.src_port,	rte_flow_item_tcp) \
 	X(SET_BE16,	tcp_dst_port,		v->hdr.dst_port,	rte_flow_item_tcp) \
-	X(SET,		gtp_udp_port,		RTE_GTPU_UDP_PORT,	rte_flow_item_gtp) \
+	X(SET,		gtp_udp_port,		UDP_GTPU_PORT,		rte_flow_item_gtp) \
 	X(SET_BE32,	gtp_teid,		v->hdr.teid,		rte_flow_item_gtp) \
 	X(SET,		gtp_msg_type,		v->hdr.msg_type,	rte_flow_item_gtp) \
 	X(SET,		gtp_ext_flag,		!!v->hdr.gtp_hdr_info,	rte_flow_item_gtp) \
@@ -167,12 +168,12 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		gtp_ext_hdr_pdu,	v->hdr.type,		rte_flow_item_gtp_psc) \
 	X(SET,		gtp_ext_hdr_qfi,	v->hdr.qfi,		rte_flow_item_gtp_psc) \
 	X(SET,		vxlan_flags,		v->flags,		rte_flow_item_vxlan) \
-	X(SET,		vxlan_udp_port,		ETH_VXLAN_DEFAULT_PORT,	rte_flow_item_vxlan) \
-	X(SET,		vxlan_gpe_udp_port,	ETH_VXLAN_GPE_DEFAULT_PORT,	rte_flow_item_vxlan_gpe) \
+	X(SET,		vxlan_udp_port,		UDP_VXLAN_PORT,		rte_flow_item_vxlan) \
+	X(SET,		vxlan_gpe_udp_port,	UDP_VXLAN_GPE_PORT,	rte_flow_item_vxlan_gpe) \
 	X(SET,		vxlan_gpe_flags,	v->flags,		rte_flow_item_vxlan_gpe) \
 	X(SET,		vxlan_gpe_protocol,	v->protocol,		rte_flow_item_vxlan_gpe) \
 	X(SET,		vxlan_gpe_rsvd1,	v->rsvd1,		rte_flow_item_vxlan_gpe) \
-	X(SET,		mpls_udp_port,		IP_UDP_PORT_MPLS,	rte_flow_item_mpls) \
+	X(SET,		mpls_udp_port,		UDP_PORT_MPLS,		rte_flow_item_mpls) \
 	X(SET,		source_qp,		v->queue,		mlx5_rte_flow_item_sq) \
 	X(SET,		tag,			v->data,		rte_flow_item_tag) \
 	X(SET,		metadata,		v->data,		rte_flow_item_meta) \
@@ -1198,6 +1199,12 @@ mlx5dr_definer_conv_item_gtp(struct mlx5dr_definer_conv_data *cd,
 	const struct rte_flow_item_gtp *m = item->mask;
 	struct mlx5dr_definer_fc *fc;
 
+	if (cd->tunnel) {
+		DR_LOG(ERR, "Inner GTPU item not supported");
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
 	/* Overwrite GTPU dest port if not present */
 	fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, false)];
 	if (!fc->tag_set && !cd->relaxed) {
@@ -1372,9 +1379,13 @@ mlx5dr_definer_conv_item_vxlan(struct mlx5dr_definer_conv_data *cd,
 	struct mlx5dr_definer_fc *fc;
 	bool inner = cd->tunnel;
 
-	/* In order to match on VXLAN we must match on ether_type, ip_protocol
-	 * and l4_dport.
-	 */
+	if (inner) {
+		DR_LOG(ERR, "Inner VXLAN item not supported");
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
+	/* In order to match on VXLAN we must match on ip_protocol and l4_dport */
 	if (!cd->relaxed) {
 		fc = &cd->fc[DR_CALC_FNAME(IP_PROTOCOL, inner)];
 		if (!fc->tag_set) {
@@ -1397,12 +1408,6 @@ mlx5dr_definer_conv_item_vxlan(struct mlx5dr_definer_conv_data *cd,
 		return 0;
 
 	if (m->flags) {
-		if (inner) {
-			DR_LOG(ERR, "Inner VXLAN flags item not supported");
-			rte_errno = ENOTSUP;
-			return rte_errno;
-		}
-
 		fc = &cd->fc[MLX5DR_DEFINER_FNAME_VXLAN_FLAGS];
 		fc->item_idx = item_idx;
 		fc->tag_set = &mlx5dr_definer_vxlan_flags_set;
@@ -1412,12 +1417,6 @@ mlx5dr_definer_conv_item_vxlan(struct mlx5dr_definer_conv_data *cd,
 	}
 
 	if (!is_mem_zero(m->vni, 3)) {
-		if (inner) {
-			DR_LOG(ERR, "Inner VXLAN vni item not supported");
-			rte_errno = ENOTSUP;
-			return rte_errno;
-		}
-
 		fc = &cd->fc[MLX5DR_DEFINER_FNAME_VXLAN_VNI];
 		fc->item_idx = item_idx;
 		fc->tag_set = &mlx5dr_definer_vxlan_vni_set;
-- 
2.25.1

[PATCH v2 04/23] net/mlx5: remove GENEVE options length limitation

[Michael Baum]

GENEVE header has field named "opt_len" describing the total length of
all GENEVE options in 4-byte granularity.

In SW sreering implementation, only single option with single DW data is
supported. When matching on GENEVE option data is requested, matching on
"opt_len" field is added according to given option length.

This behaveior assumes that only packets with single option can be
matched, but it is wrong, packet with a few option can be matched but
only one of them can match its value.

This patch removes the "opt_len" matching unless user ask it explicitly.

Fixes: e440d6cf589e ("net/mlx5: add GENEVE TLV option flow translation")
Cc: shirik@nvidia.com

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 16 ----------------
 1 file changed, 16 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 88b5c20758..b091eb9d11 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10104,7 +10104,6 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 	const struct rte_flow_item_geneve_opt *geneve_opt_m;
 	const struct rte_flow_item_geneve_opt *geneve_opt_v;
 	const struct rte_flow_item_geneve_opt *geneve_opt_vv = item->spec;
-	void *misc_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters);
 	void *misc3_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_3);
 	rte_be32_t opt_data_key = 0, opt_data_mask = 0;
 	uint32_t *data;
@@ -10123,21 +10122,6 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 			return ret;
 		}
 	}
-	/*
-	 * Set the option length in GENEVE header if not requested.
-	 * The GENEVE TLV option length is expressed by the option length field
-	 * in the GENEVE header.
-	 * If the option length was not requested but the GENEVE TLV option item
-	 * is present we set the option length field implicitly.
-	 */
-	if (!MLX5_GET16(fte_match_set_misc, misc_v, geneve_opt_len)) {
-		if (key_type & MLX5_SET_MATCHER_M)
-			MLX5_SET(fte_match_set_misc, misc_v, geneve_opt_len,
-				 MLX5_GENEVE_OPTLEN_MASK);
-		else
-			MLX5_SET(fte_match_set_misc, misc_v, geneve_opt_len,
-				 geneve_opt_v->option_len + 1);
-	}
 	/* Set the data. */
 	if (key_type == MLX5_SET_MATCHER_SW_V)
 		data = geneve_opt_vv->data;
-- 
2.25.1

[PATCH v2 05/23] net/mlx5: fix GENEVE option item translation

[Michael Baum]

The "flow_dv_translate_item_geneve_opt()" function is called twice per
flow rule, for either matcher focusing the mask or value focusing the
spec.
The spec is always provided and its field "option_len" indicates the
data size for both spec and mask. For using it, function has another
pointer "geneve_opt_vv" representing the spec regardless to focusing
while the "geneve_opt_v" pointer represents the mask for matcher and
spec for rule creation.

The current implementation has 2 issues:
1. geneve_opt_v get the spec in rule creation as sane as geneve_opt_vv,
   but function use if-else which is bacicly has same value.
2. function uses "option_len" from "geneve_opt_v" instead of
   "geneve_opt_v" even when the focus is on mask, for HWS the mask value
   may be 0 even data is valid.

This patch refactors the function implementation to avoid those issues.

Fixes: cd4ab742064a ("net/mlx5: split flow item matcher and value translation")
Cc: suanmingm@nvidia.com

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 28 +++++++++++-----------------
 1 file changed, 11 insertions(+), 17 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index b091eb9d11..cc5549f9ce 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10103,13 +10103,13 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 {
 	const struct rte_flow_item_geneve_opt *geneve_opt_m;
 	const struct rte_flow_item_geneve_opt *geneve_opt_v;
-	const struct rte_flow_item_geneve_opt *geneve_opt_vv = item->spec;
+	const struct rte_flow_item_geneve_opt *orig_spec = item->spec;
 	void *misc3_v = MLX5_ADDR_OF(fte_match_param, key, misc_parameters_3);
 	rte_be32_t opt_data_key = 0, opt_data_mask = 0;
-	uint32_t *data;
+	size_t option_byte_len;
 	int ret = 0;
 
-	if (MLX5_ITEM_VALID(item, key_type))
+	if (MLX5_ITEM_VALID(item, key_type) || !orig_spec)
 		return -1;
 	MLX5_ITEM_UPDATE(item, key_type, geneve_opt_v, geneve_opt_m,
 			 &rte_flow_item_geneve_opt_mask);
@@ -10122,21 +10122,15 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 			return ret;
 		}
 	}
-	/* Set the data. */
-	if (key_type == MLX5_SET_MATCHER_SW_V)
-		data = geneve_opt_vv->data;
-	else
-		data = geneve_opt_v->data;
-	if (data) {
-		memcpy(&opt_data_key, data,
-			RTE_MIN((uint32_t)(geneve_opt_v->option_len * 4),
-				sizeof(opt_data_key)));
-		memcpy(&opt_data_mask, geneve_opt_m->data,
-			RTE_MIN((uint32_t)(geneve_opt_v->option_len * 4),
-				sizeof(opt_data_mask)));
+	/* Convert the option length from DW to bytes for using memcpy. */
+	option_byte_len = RTE_MIN((size_t)(orig_spec->option_len * 4),
+				  sizeof(rte_be32_t));
+	if (geneve_opt_v->data) {
+		memcpy(&opt_data_key, geneve_opt_v->data, option_byte_len);
+		memcpy(&opt_data_mask, geneve_opt_m->data, option_byte_len);
 		MLX5_SET(fte_match_set_misc3, misc3_v,
-				geneve_tlv_option_0_data,
-			rte_be_to_cpu_32(opt_data_key & opt_data_mask));
+			 geneve_tlv_option_0_data,
+			 rte_be_to_cpu_32(opt_data_key & opt_data_mask));
 	}
 	return ret;
 }
-- 
2.25.1

[PATCH v2 06/23] common/mlx5: add system image GUID attribute

[Michael Baum]

Add to the "system_image_guid" filed describing uniquely the physical
device into "mlx5_hca_attr" structure.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 10 ++++++----
 drivers/common/mlx5/mlx5_devx_cmds.h |  1 +
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 9b1cfcc135..d3ecdfece7 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -518,8 +518,11 @@ mlx5_devx_cmd_query_nic_vport_context(void *ctx,
 	}
 	vctx = MLX5_ADDR_OF(query_nic_vport_context_out, out,
 			    nic_vport_context);
-	attr->vport_inline_mode = MLX5_GET(nic_vport_context, vctx,
-					   min_wqe_inline_mode);
+	if (attr->wqe_inline_mode == MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
+		attr->vport_inline_mode = MLX5_GET(nic_vport_context, vctx,
+						   min_wqe_inline_mode);
+	attr->system_image_guid = MLX5_GET64(nic_vport_context, vctx,
+					     system_image_guid);
 	return 0;
 }
 
@@ -1351,8 +1354,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 		}
 		attr->qp_ts_format = MLX5_GET(roce_caps, hcattr, qp_ts_format);
 	}
-	if (attr->eth_virt &&
-	    attr->wqe_inline_mode == MLX5_CAP_INLINE_MODE_VPORT_CONTEXT) {
+	if (attr->eth_virt) {
 		rc = mlx5_devx_cmd_query_nic_vport_context(ctx, 0, attr);
 		if (rc) {
 			attr->eth_virt = 0;
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 0c5727b669..dfb1148b84 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -322,6 +322,7 @@ struct mlx5_hca_attr {
 	uint32_t lag_rx_port_affinity:1;
 	uint32_t wqe_based_flow_table_sup:1;
 	uint8_t max_header_modify_pattern_length;
+	uint64_t system_image_guid;
 };
 
 /* LAG Context. */
-- 
2.25.1

[PATCH v2 07/23] common/mlx5: add GENEVE TLV option attribute structure

[Michael Baum]

Add a new structure "mlx5_devx_geneve_tlv_option_attr" to use in GENEVE
TLV option creation.
Later this structure will be used by GENEVE TLV option query operation
as well.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 28 +++++++++++++---------------
 drivers/common/mlx5/mlx5_devx_cmds.h | 11 ++++++++++-
 drivers/net/mlx5/mlx5_flow_dv.c      | 10 +++++++---
 3 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index d3ecdfece7..c783fc0e10 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2858,19 +2858,15 @@ mlx5_devx_cmd_create_conn_track_offload_obj(void *ctx, uint32_t pd,
  *
  * @param[in] ctx
  *   Context returned from mlx5 open_device() glue function.
- * @param [in] class
- *   TLV option variable value of class
- * @param [in] type
- *   TLV option variable value of type
- * @param [in] len
- *   TLV option variable value of len
+ * @param[in] attr
+ *   Pointer to GENEVE TLV option attributes structure.
  *
  * @return
  *   The DevX object created, NULL otherwise and rte_errno is set.
  */
 struct mlx5_devx_obj *
 mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
-		uint16_t class, uint8_t type, uint8_t len)
+				  struct mlx5_devx_geneve_tlv_option_attr *attr)
 {
 	uint32_t in[MLX5_ST_SZ_DW(create_geneve_tlv_option_in)] = {0};
 	uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
@@ -2879,25 +2875,27 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 						   0, SOCKET_ID_ANY);
 
 	if (!geneve_tlv_opt_obj) {
-		DRV_LOG(ERR, "Failed to allocate geneve tlv option object.");
+		DRV_LOG(ERR, "Failed to allocate GENEVE TLV option object.");
 		rte_errno = ENOMEM;
 		return NULL;
 	}
 	void *hdr = MLX5_ADDR_OF(create_geneve_tlv_option_in, in, hdr);
 	void *opt = MLX5_ADDR_OF(create_geneve_tlv_option_in, in,
-			geneve_tlv_opt);
+				 geneve_tlv_opt);
 	MLX5_SET(general_obj_in_cmd_hdr, hdr, opcode,
-			MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+		 MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
 	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
 		 MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
 	MLX5_SET(geneve_tlv_option, opt, option_class,
-			rte_be_to_cpu_16(class));
-	MLX5_SET(geneve_tlv_option, opt, option_type, type);
-	MLX5_SET(geneve_tlv_option, opt, option_data_length, len);
+		 rte_be_to_cpu_16(attr->option_class));
+	MLX5_SET(geneve_tlv_option, opt, option_type, attr->option_type);
+	MLX5_SET(geneve_tlv_option, opt, option_data_length,
+		 attr->option_data_len);
 	geneve_tlv_opt_obj->obj = mlx5_glue->devx_obj_create(ctx, in,
-					sizeof(in), out, sizeof(out));
+							     sizeof(in), out,
+							     sizeof(out));
 	if (!geneve_tlv_opt_obj->obj) {
-		DEVX_DRV_LOG(ERR, out, "create GENEVE TLV", NULL, 0);
+		DEVX_DRV_LOG(ERR, out, "create GENEVE TLV option", NULL, 0);
 		mlx5_free(geneve_tlv_opt_obj);
 		return NULL;
 	}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index dfb1148b84..d11f1d650f 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -668,6 +668,15 @@ struct mlx5_devx_crypto_login_attr {
 	uint8_t credential[MLX5_CRYPTO_CREDENTIAL_SIZE];
 };
 
+/*
+ * GENEVE TLV option attributes structure, used by GENEVE TLV option create.
+ */
+struct mlx5_devx_geneve_tlv_option_attr {
+	uint32_t option_class:16;
+	uint32_t option_type:8;
+	uint32_t option_data_len:5;
+};
+
 /* mlx5_devx_cmds.c */
 
 __rte_internal
@@ -778,7 +787,7 @@ int mlx5_devx_cmd_register_write(void *ctx, uint16_t reg_id,
 __rte_internal
 struct mlx5_devx_obj *
 mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
-		uint16_t class, uint8_t type, uint8_t len);
+				 struct mlx5_devx_geneve_tlv_option_attr *attr);
 
 /**
  * Create virtio queue counters object DevX API.
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index cc5549f9ce..ae10981165 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10046,11 +10046,15 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 			goto exit;
 		}
 	} else {
+		struct mlx5_devx_geneve_tlv_option_attr attr = {
+			.option_class = geneve_opt_v->option_class,
+			.option_type = geneve_opt_v->option_type,
+			.option_data_len = geneve_opt_v->option_len,
+		};
+
 		/* Create a GENEVE TLV object and resource. */
 		obj = mlx5_devx_cmd_create_geneve_tlv_option(sh->cdev->ctx,
-				geneve_opt_v->option_class,
-				geneve_opt_v->option_type,
-				geneve_opt_v->option_len);
+							     &attr);
 		if (!obj) {
 			ret = rte_flow_error_set(error, ENODATA,
 				RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
-- 
2.25.1

[PATCH v2 08/23] common/mlx5: add PRM attribute for TLV sample

[Michael Baum]

Add GENEVE TLV sample fields in 2 places:
1. New HCA capabilities indicating GENEVE TLV sample is supported.
2. New fields in "mlx5_ifc_geneve_tlv_option_bits" structure.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 18 ++++++++++++++++--
 drivers/common/mlx5/mlx5_devx_cmds.h |  9 +++++++--
 drivers/common/mlx5/mlx5_prm.h       | 15 +++++++++++----
 3 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index c783fc0e10..68137dc535 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -968,6 +968,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 			max_geneve_tlv_options);
 	attr->max_geneve_tlv_option_data_len = MLX5_GET(cmd_hca_cap, hcattr,
 			max_geneve_tlv_option_data_len);
+	attr->geneve_tlv_option_offset = MLX5_GET(cmd_hca_cap, hcattr,
+						  geneve_tlv_option_offset);
+	attr->geneve_tlv_sample = MLX5_GET(cmd_hca_cap, hcattr,
+					   geneve_tlv_sample);
 	attr->query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
 						 query_match_sample_info);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
@@ -2886,11 +2890,21 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 		 MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
 	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
 		 MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
-	MLX5_SET(geneve_tlv_option, opt, option_class,
-		 rte_be_to_cpu_16(attr->option_class));
 	MLX5_SET(geneve_tlv_option, opt, option_type, attr->option_type);
 	MLX5_SET(geneve_tlv_option, opt, option_data_length,
 		 attr->option_data_len);
+	if (attr->option_class_ignore)
+		MLX5_SET(geneve_tlv_option, opt, option_class_ignore,
+			 attr->option_class_ignore);
+	else
+		MLX5_SET(geneve_tlv_option, opt, option_class,
+			 rte_be_to_cpu_16(attr->option_class));
+	if (attr->offset_valid) {
+		MLX5_SET(geneve_tlv_option, opt, sample_offset_valid,
+			 attr->offset_valid);
+		MLX5_SET(geneve_tlv_option, opt, sample_offset,
+			 attr->sample_offset);
+	}
 	geneve_tlv_opt_obj->obj = mlx5_glue->devx_obj_create(ctx, in,
 							     sizeof(in), out,
 							     sizeof(out));
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index d11f1d650f..4f264560a9 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -212,8 +212,10 @@ struct mlx5_hca_attr {
 	uint32_t lro_timer_supported_periods[MLX5_LRO_NUM_SUPP_PERIODS];
 	uint16_t lro_min_mss_size;
 	uint32_t flex_parser_protocols;
-	uint32_t max_geneve_tlv_options;
-	uint32_t max_geneve_tlv_option_data_len;
+	uint32_t max_geneve_tlv_options:8;
+	uint32_t max_geneve_tlv_option_data_len:5;
+	uint32_t geneve_tlv_sample:1;
+	uint32_t geneve_tlv_option_offset:1;
 	uint32_t hairpin:1;
 	uint32_t log_max_hairpin_queues:5;
 	uint32_t log_max_hairpin_wq_data_sz:5;
@@ -675,6 +677,9 @@ struct mlx5_devx_geneve_tlv_option_attr {
 	uint32_t option_class:16;
 	uint32_t option_type:8;
 	uint32_t option_data_len:5;
+	uint32_t option_class_ignore:1;
+	uint32_t offset_valid:1;
+	uint32_t sample_offset:8;
 };
 
 /* mlx5_devx_cmds.c */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 69404b5ed8..f15e3c2bd7 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1854,7 +1854,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8 num_of_uars_per_page[0x20];
 	u8 flex_parser_protocols[0x20];
 	u8 max_geneve_tlv_options[0x8];
-	u8 reserved_at_568[0x3];
+	u8 geneve_tlv_sample[0x1];
+	u8 geneve_tlv_option_offset[0x1];
+	u8 reserved_at_56a[0x1];
 	u8 max_geneve_tlv_option_data_len[0x5];
 	u8 flex_parser_header_modify[0x1];
 	u8 reserved_at_571[0x2];
@@ -3424,16 +3426,21 @@ struct mlx5_ifc_virtio_q_counters_bits {
 
 struct mlx5_ifc_geneve_tlv_option_bits {
 	u8 modify_field_select[0x40];
-	u8 reserved_at_40[0x18];
+	u8 reserved_at_40[0x8];
+	u8 sample_offset[0x8];
+	u8 sample_id_valid[0x1];
+	u8 sample_offset_valid[0x1];
+	u8 option_class_ignore[0x1];
+	u8 reserved_at_53[0x5];
 	u8 geneve_option_fte_index[0x8];
 	u8 option_class[0x10];
 	u8 option_type[0x8];
 	u8 reserved_at_78[0x3];
 	u8 option_data_length[0x5];
-	u8 reserved_at_80[0x180];
+	u8 geneve_sample_field_id[0x20];
+	u8 reserved_at_a0[0x160];
 };
 
-
 enum mlx5_ifc_rtc_update_mode {
 	MLX5_IFC_RTC_STE_UPDATE_MODE_BY_HASH = 0x0,
 	MLX5_IFC_RTC_STE_UPDATE_MODE_BY_OFFSET = 0x1,
-- 
2.25.1

[PATCH v2 09/23] common/mlx5: add sample info query syndrome into error log

[Michael Baum]

Move "mlx5_devx_cmd_match_sample_info_query()" function to use
"DEVX_DRV_LOG" in case of "devx_general_cmd" failure.
This macro contains syndrome report and used by all other function
calling "devx_general_cmd".

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 68137dc535..b8f4a840e7 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -641,11 +641,10 @@ mlx5_devx_cmd_match_sample_info_query(void *ctx, uint32_t sample_field_id,
 	MLX5_SET(query_match_sample_info_in, in, sample_field_id,
 		 sample_field_id);
 	rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out));
-	if (rc) {
-		DRV_LOG(ERR, "Failed to query match sample info using DevX: %s",
-			strerror(rc));
-		rte_errno = rc;
-		return -rc;
+	if (rc || MLX5_FW_STATUS(out)) {
+		DEVX_DRV_LOG(ERR, out, "query match sample info",
+			     "sample_field_id", sample_field_id);
+		return MLX5_DEVX_ERR_RC(rc);
 	}
 	attr->modify_field_id = MLX5_GET(query_match_sample_info_out, out,
 					 modify_field_id);
-- 
2.25.1

[PATCH v2 10/23] common/mlx5: query GENEVE option sample ID from HCA attr

[Michael Baum]

This patch adds the GENEVE option sample ID into HCA attribute
structure.
This sample ID is used as the input of
"mlx5_devx_cmd_match_sample_info_query" function when flex parser
profile is 0.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 2 ++
 drivers/common/mlx5/mlx5_devx_cmds.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index b8f4a840e7..394149d542 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -973,6 +973,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 					   geneve_tlv_sample);
 	attr->query_match_sample_info = MLX5_GET(cmd_hca_cap, hcattr,
 						 query_match_sample_info);
+	attr->geneve_tlv_option_sample_id = MLX5_GET(cmd_hca_cap, hcattr,
+						     flex_parser_id_geneve_opt_0);
 	attr->qos.sup = MLX5_GET(cmd_hca_cap, hcattr, qos);
 	attr->wqe_index_ignore = MLX5_GET(cmd_hca_cap, hcattr,
 					  wqe_index_ignore_cap);
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 4f264560a9..7747c230de 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -216,6 +216,7 @@ struct mlx5_hca_attr {
 	uint32_t max_geneve_tlv_option_data_len:5;
 	uint32_t geneve_tlv_sample:1;
 	uint32_t geneve_tlv_option_offset:1;
+	uint32_t geneve_tlv_option_sample_id:4;
 	uint32_t hairpin:1;
 	uint32_t log_max_hairpin_queues:5;
 	uint32_t log_max_hairpin_wq_data_sz:5;
-- 
2.25.1

[PATCH v2 11/23] common/mlx5: add function to query GENEVE TLV option

[Michael Baum]

Add a new function to query information about GENEVE TLV option parser.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 50 ++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h |  6 ++++
 drivers/common/mlx5/mlx5_prm.h       |  5 +++
 drivers/common/mlx5/version.map      |  1 +
 4 files changed, 62 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 394149d542..9b7ababae7 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2918,6 +2918,56 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 	return geneve_tlv_opt_obj;
 }
 
+/**
+ * Query GENEVE TLV option using DevX API.
+ *
+ * @param[in] ctx
+ *   Context used to create GENEVE TLV option object.
+ * @param[in] geneve_tlv_opt_obj
+ *   DevX object of the GENEVE TLV option.
+ * @param[out] attr
+ *   Pointer to match sample info attributes structure.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_devx_cmd_query_geneve_tlv_option(void *ctx,
+				      struct mlx5_devx_obj *geneve_tlv_opt_obj,
+				      struct mlx5_devx_match_sample_info_query_attr *attr)
+{
+	uint32_t in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {0};
+	uint32_t out[MLX5_ST_SZ_DW(query_geneve_tlv_option_out)] = {0};
+	void *hdr = MLX5_ADDR_OF(query_geneve_tlv_option_out, in, hdr);
+	void *opt = MLX5_ADDR_OF(query_geneve_tlv_option_out, out,
+				 geneve_tlv_opt);
+	int ret;
+
+	MLX5_SET(general_obj_in_cmd_hdr, hdr, opcode,
+		 MLX5_CMD_OP_QUERY_GENERAL_OBJECT);
+	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
+		 MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
+	MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_id, geneve_tlv_opt_obj->id);
+	/* Call first query to get sample handle. */
+	ret = mlx5_glue->devx_obj_query(geneve_tlv_opt_obj->obj, in, sizeof(in),
+					out, sizeof(out));
+	if (ret) {
+		DRV_LOG(ERR, "Failed to query GENEVE TLV option using DevX.");
+		rte_errno = errno;
+		return -errno;
+	}
+	/* Call second query to get sample information. */
+	if (MLX5_GET(geneve_tlv_option, opt, sample_id_valid)) {
+		uint32_t sample_id = MLX5_GET(geneve_tlv_option, opt,
+					      geneve_sample_field_id);
+
+		return mlx5_devx_cmd_match_sample_info_query(ctx, sample_id,
+							     attr);
+	}
+	DRV_LOG(DEBUG, "GENEVE TLV option sample isn't valid.");
+	return 0;
+}
+
 int
 mlx5_devx_cmd_wq_query(void *wq, uint32_t *counter_set_id)
 {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 7747c230de..c79f8dc48d 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -795,6 +795,12 @@ struct mlx5_devx_obj *
 mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
 				 struct mlx5_devx_geneve_tlv_option_attr *attr);
 
+__rte_internal
+int
+mlx5_devx_cmd_query_geneve_tlv_option(void *ctx,
+				      struct mlx5_devx_obj *geneve_tlv_opt_obj,
+				      struct mlx5_devx_match_sample_info_query_attr *attr);
+
 /**
  * Create virtio queue counters object DevX API.
  *
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index f15e3c2bd7..f64f25dbb7 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -3729,6 +3729,11 @@ struct mlx5_ifc_create_geneve_tlv_option_in_bits {
 	struct mlx5_ifc_geneve_tlv_option_bits geneve_tlv_opt;
 };
 
+struct mlx5_ifc_query_geneve_tlv_option_out_bits {
+	struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr;
+	struct mlx5_ifc_geneve_tlv_option_bits geneve_tlv_opt;
+};
+
 struct mlx5_ifc_create_rtc_in_bits {
 	struct mlx5_ifc_general_obj_in_cmd_hdr_bits hdr;
 	struct mlx5_ifc_rtc_bits rtc;
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 074eed46fd..589a450145 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -54,6 +54,7 @@ INTERNAL {
 	mlx5_devx_cmd_modify_tir;
 	mlx5_devx_cmd_modify_virtq;
 	mlx5_devx_cmd_qp_query_tis_td;
+	mlx5_devx_cmd_query_geneve_tlv_option;
 	mlx5_devx_cmd_query_hca_attr;
 	mlx5_devx_cmd_query_lag;
 	mlx5_devx_cmd_query_parse_samples;
-- 
2.25.1

[PATCH v2 12/23] net/mlx5: add physical device handle

[Michael Baum]

Add structure describing physical device, and manage physical device
global list.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/mlx5.c | 77 ++++++++++++++++++++++++++++++++++++-----
 drivers/net/mlx5/mlx5.h | 13 +++++++
 2 files changed, 82 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 3a182de248..f9fc652136 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -190,9 +190,10 @@ struct mlx5_shared_data *mlx5_shared_data;
 /** Driver-specific log messages type. */
 int mlx5_logtype;
 
-static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list =
-						LIST_HEAD_INITIALIZER();
+static LIST_HEAD(mlx5_dev_ctx_list, mlx5_dev_ctx_shared) dev_ctx_list = LIST_HEAD_INITIALIZER();
+static LIST_HEAD(mlx5_phdev_list, mlx5_physical_device) phdev_list = LIST_HEAD_INITIALIZER();
 static pthread_mutex_t mlx5_dev_ctx_list_mutex;
+
 static const struct mlx5_indexed_pool_config mlx5_ipool_cfg[] = {
 #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	[MLX5_IPOOL_DECAP_ENCAP] = {
@@ -1692,6 +1693,60 @@ mlx5_init_shared_dev_registers(struct mlx5_dev_ctx_shared *sh)
 	mlx5_init_hws_flow_tags_registers(sh);
 }
 
+static struct mlx5_physical_device *
+mlx5_get_physical_device(struct mlx5_common_device *cdev)
+{
+	struct mlx5_physical_device *phdev;
+	struct mlx5_hca_attr *attr = &cdev->config.hca_attr;
+
+	/* Search for physical device by system_image_guid. */
+	LIST_FOREACH(phdev, &phdev_list, next) {
+		if (phdev->guid == attr->system_image_guid) {
+			phdev->refcnt++;
+			return phdev;
+		}
+	}
+	phdev = mlx5_malloc(MLX5_MEM_ZERO | MLX5_MEM_RTE,
+			    sizeof(struct mlx5_physical_device),
+			    RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (!phdev) {
+		DRV_LOG(ERR, "Physical device allocation failure.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	phdev->guid = attr->system_image_guid;
+	phdev->refcnt = 1;
+	LIST_INSERT_HEAD(&phdev_list, phdev, next);
+	DRV_LOG(DEBUG, "Physical device is created, guid=%" PRIu64 ".",
+		phdev->guid);
+	return phdev;
+}
+
+static void
+mlx5_physical_device_destroy(struct mlx5_physical_device *phdev)
+{
+#ifdef RTE_LIBRTE_MLX5_DEBUG
+	/* Check the object presence in the list. */
+	struct mlx5_physical_device *lphdev;
+
+	LIST_FOREACH(lphdev, &phdev_list, next)
+		if (lphdev == phdev)
+			break;
+	MLX5_ASSERT(lphdev);
+	if (lphdev != phdev) {
+		DRV_LOG(ERR, "Freeing non-existing physical device");
+		return;
+	}
+#endif
+	MLX5_ASSERT(phdev);
+	MLX5_ASSERT(phdev->refcnt);
+	if (--phdev->refcnt)
+		return;
+	/* Remove physical device from the global device list. */
+	LIST_REMOVE(phdev, next);
+	mlx5_free(phdev);
+}
+
 /**
  * Allocate shared device context. If there is multiport device the
  * master and representors will share this context, if there is single
@@ -1725,7 +1780,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
 	pthread_mutex_lock(&mlx5_dev_ctx_list_mutex);
 	/* Search for IB context by device name. */
-	LIST_FOREACH(sh, &mlx5_dev_ctx_list, next) {
+	LIST_FOREACH(sh, &dev_ctx_list, next) {
 		if (!strcmp(sh->ibdev_name, spawn->phys_dev_name)) {
 			sh->refcnt++;
 			goto exit;
@@ -1765,6 +1820,9 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sizeof(sh->ibdev_name) - 1);
 	strncpy(sh->ibdev_path, mlx5_os_get_ctx_device_path(sh->cdev->ctx),
 		sizeof(sh->ibdev_path) - 1);
+	sh->phdev = mlx5_get_physical_device(sh->cdev);
+	if (!sh->phdev)
+		goto error;
 	/*
 	 * Setting port_id to max unallowed value means there is no interrupt
 	 * subhandler installed for the given port index i.
@@ -1798,7 +1856,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 #endif
 	}
 	mlx5_os_dev_shared_handler_install(sh);
-	if (LIST_EMPTY(&mlx5_dev_ctx_list)) {
+	if (LIST_EMPTY(&dev_ctx_list)) {
 		err = mlx5_flow_os_init_workspace_once();
 		if (err)
 			goto error;
@@ -1811,7 +1869,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	mlx5_flow_aging_init(sh);
 	mlx5_flow_ipool_create(sh);
 	/* Add context to the global device list. */
-	LIST_INSERT_HEAD(&mlx5_dev_ctx_list, sh, next);
+	LIST_INSERT_HEAD(&dev_ctx_list, sh, next);
 	rte_spinlock_init(&sh->geneve_tlv_opt_sl);
 	mlx5_init_shared_dev_registers(sh);
 	/* Init counter pool list header and lock. */
@@ -1833,6 +1891,8 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	} while (++i <= (uint32_t)sh->bond.n_port);
 	if (sh->td)
 		claim_zero(mlx5_devx_cmd_destroy(sh->td));
+	if (sh->phdev)
+		mlx5_physical_device_destroy(sh->phdev);
 	mlx5_free(sh);
 	rte_errno = err;
 	return NULL;
@@ -1919,7 +1979,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 	/* Check the object presence in the list. */
 	struct mlx5_dev_ctx_shared *lctx;
 
-	LIST_FOREACH(lctx, &mlx5_dev_ctx_list, next)
+	LIST_FOREACH(lctx, &dev_ctx_list, next)
 		if (lctx == sh)
 			break;
 	MLX5_ASSERT(lctx);
@@ -1945,7 +2005,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 	/* Remove context from the global device list. */
 	LIST_REMOVE(sh, next);
 	/* Release resources on the last device removal. */
-	if (LIST_EMPTY(&mlx5_dev_ctx_list)) {
+	if (LIST_EMPTY(&dev_ctx_list)) {
 		mlx5_os_net_cleanup();
 		mlx5_flow_os_release_workspace();
 	}
@@ -1985,6 +2045,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL);
 	pthread_mutex_destroy(&sh->txpp.mutex);
 	mlx5_lwm_unset(sh);
+	mlx5_physical_device_destroy(sh->phdev);
 	mlx5_free(sh);
 	return;
 exit:
@@ -2929,7 +2990,7 @@ mlx5_probe_again_args_validate(struct mlx5_common_device *cdev,
 		return 0;
 	pthread_mutex_lock(&mlx5_dev_ctx_list_mutex);
 	/* Search for IB context by common device pointer. */
-	LIST_FOREACH(sh, &mlx5_dev_ctx_list, next)
+	LIST_FOREACH(sh, &dev_ctx_list, next)
 		if (sh->cdev == cdev)
 			break;
 	pthread_mutex_unlock(&mlx5_dev_ctx_list_mutex);
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 779805bcd8..8bf7f86416 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1419,6 +1419,18 @@ struct mlx5_dev_registers {
 #define HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT
 #endif
 
+/**
+ * Physical device structure.
+ * This device is created once per NIC to manage recourses shared by all ports
+ * under same physical device.
+ */
+struct mlx5_physical_device {
+	LIST_ENTRY(mlx5_physical_device) next;
+	struct mlx5_dev_ctx_shared *sh; /* Created on sherd context. */
+	uint64_t guid; /* System image guid, the uniq ID of physical device. */
+	uint32_t refcnt;
+};
+
 /*
  * Shared Infiniband device context for Master/Representors
  * which belong to same IB device with multiple IB ports.
@@ -1450,6 +1462,7 @@ struct mlx5_dev_ctx_shared {
 	uint32_t max_port; /* Maximal IB device port index. */
 	struct mlx5_bond_info bond; /* Bonding information. */
 	struct mlx5_common_device *cdev; /* Backend mlx5 device. */
+	struct mlx5_physical_device *phdev; /* Backend physical device. */
 	uint32_t tdn; /* Transport Domain number. */
 	char ibdev_name[MLX5_FS_NAME_MAX]; /* SYSFS dev name. */
 	char ibdev_path[MLX5_FS_PATH_MAX]; /* SYSFS dev path for secondary */
-- 
2.25.1

[PATCH v2 13/23] net/mlx5: add GENEVE TLV options parser API

[Michael Baum]

Add a new private API to create/destroy parser for GENEVE TLV options.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 doc/guides/nics/mlx5.rst            | 122 ++++++
 doc/guides/platform/mlx5.rst        |   6 +-
 drivers/net/mlx5/meson.build        |   1 +
 drivers/net/mlx5/mlx5.c             |  30 +-
 drivers/net/mlx5/mlx5.h             |   8 +
 drivers/net/mlx5/mlx5_flow.c        |  30 ++
 drivers/net/mlx5/mlx5_flow.h        |  18 +
 drivers/net/mlx5/mlx5_flow_geneve.c | 627 ++++++++++++++++++++++++++++
 drivers/net/mlx5/rte_pmd_mlx5.h     | 102 +++++
 drivers/net/mlx5/version.map        |   3 +
 10 files changed, 945 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/mlx5/mlx5_flow_geneve.c

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index f8930cb902..e82f7034aa 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -2314,6 +2314,128 @@ and disables ``avail_thresh_triggered``.
    testpmd> mlx5 set port 1 host_shaper avail_thresh_triggered 0 rate 50
 
 
+.. _geneve_parser_api:
+
+GENEVE TLV options parser
+-------------------------
+
+NVIDIA ConnectX and BlueField devices support configure flex parser for
+`GENEVE TLV options <https://www.rfc-editor.org/rfc/rfc8926.html#name-tunnel-options>`_.
+
+Each physical device has 7 DWs for GENEVE TLV options.
+Partial option configuration is supported, mask for data is provided in parser
+creation indicating which DWs configuration is requested. Only masked data DWs
+can be matched later as item field using flow API.
+
+Matching of ``type`` field is supported for each configured option.
+However, for matching ``class` field, the option should be configured with
+``match_on_class_mode=2``. Matching on ``length`` field is not supported.
+When ``match_on_class_mode=2`` is requested, one extra DW is consumed for it.
+
+Parser API
+~~~~~~~~~~
+
+An API to create/destroy GENEVE TLV parser is added.
+Although the parser is created per physical device, this API is port oriented.
+Each port should call this API before using GENEVE OPT item,
+but its configuration must use the same options list with same internal order
+configured by first port.
+
+Calling this API for different ports under same physical device doesn't consume
+more DWs, the first one creates the parser and the rest use same configuration.
+
+``struct rte_pmd_mlx5_geneve_tlv`` is used for single option configuration:
+
+.. _table_rte_pmd_mlx5_geneve_tlv:
+
+.. table:: GENEVE TLV
+
+   +-------------------------+-------------------------------------------------+
+   | Field                   | Value                                           |
+   +=========================+=================================================+
+   | ``option_class``        | class                                           |
+   +-------------------------+-------------------------------------------------+
+   | ``option_type``         | type                                            |
+   +-------------------------+-------------------------------------------------+
+   | ``option_len``          | data length in DW granularity                   |
+   +-------------------------+-------------------------------------------------+
+   | ``match_on_class_mode`` | indicator about class field role in this option |
+   +-------------------------+-------------------------------------------------+
+   | ``offset``              | offset of the first sample in DW granularity    |
+   +-------------------------+-------------------------------------------------+
+   | ``sample_len``          | number of DW to sample                          |
+   +-------------------------+-------------------------------------------------+
+   | ``match_data_mask``     | array of DWs which each bit marks if this bit   |
+   |                         | should be sampled                               |
+   +-------------------------+-------------------------------------------------+
+
+Creation
+^^^^^^^^
+
+Creates GENEVE TLV parser for the selected port.
+This function must be called before first use of GENEVE option.
+
+.. code-block:: c
+
+   void *
+   rte_pmd_mlx5_create_geneve_tlv_parser(uint16_t port_id,
+                                         const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+                                         uint8_t nb_options);
+
+The parser creation is done once for all GENEVE TLV options.
+For adding a new option, the exist parser should be destroyed first.
+
+Arguments:
+
+- ``port_id``: port identifier of Ethernet device.
+- ``tlv_list``: list of GENEVE TLV options to create parser for them.
+- ``nb_options``: number of options in TLV list.
+
+Return values:
+
+- A valid handle in case of success, NULL otherwise (``rte_errno`` is also set),
+  the following errors are defined.
+- ``ENODEV``: there is no Ethernet device for this port id.
+- ``EINVAL``: invalid GENEVE TLV option requested.
+- ``ENOTSUP``: the port doesn't support GENEVE TLV parsing.
+- ``EEXIST``: this port already has GENEVE TLV parser or another port under same
+  physical device has already prepared a different parser.
+- ``ENOMEM``: not enough memory to execute the function, or resource limitation
+  on the device.
+
+
+Destruction
+^^^^^^^^^^^
+
+Destroy GENEVE TLV parser created by ``rte_pmd_mlx5_create_geneve_tlv_parser()``.
+This function must be called after last use of GENEVE option and before port
+closing.
+
+.. code-block:: c
+
+   int
+   rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle);
+
+Failure to destroy a parser handle may occur when one of the options is used by
+valid template table.
+
+Arguments:
+
+- ``handle``: handle for the GENEVE TLV parser object to be destroyed.
+
+Return values:
+
+- 0 on success, a negative errno value otherwise and ``rte_errno`` is set.
+
+
+Limitations
+~~~~~~~~~~~
+
+* Supported only in HW steering (``dv_flow_en`` = 2).
+* Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+* Supported for FW version **xx.37.0142** and above.
+
+
 Testpmd driver specific commands
 --------------------------------
 
diff --git a/doc/guides/platform/mlx5.rst b/doc/guides/platform/mlx5.rst
index 400000e284..d16508d0da 100644
--- a/doc/guides/platform/mlx5.rst
+++ b/doc/guides/platform/mlx5.rst
@@ -536,10 +536,14 @@ Below are some firmware configurations listed.
    or
    FLEX_PARSER_PROFILE_ENABLE=1
 
-- enable Geneve TLV option flow matching::
+- enable Geneve TLV option flow matching in SW steering::
 
    FLEX_PARSER_PROFILE_ENABLE=0
 
+- enable Geneve TLV option flow matching in HW steering::
+
+   FLEX_PARSER_PROFILE_ENABLE=8
+
 - enable GTP flow matching::
 
    FLEX_PARSER_PROFILE_ENABLE=3
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index 69771c63ab..d705fe21bb 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -46,6 +46,7 @@ sources = files(
 
 if is_linux
     sources += files(
+            'mlx5_flow_geneve.c',
             'mlx5_flow_hw.c',
             'mlx5_hws_cnt.c',
             'mlx5_flow_quota.c',
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index f9fc652136..5f8af31aea 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1722,6 +1722,19 @@ mlx5_get_physical_device(struct mlx5_common_device *cdev)
 	return phdev;
 }
 
+struct mlx5_physical_device *
+mlx5_get_locked_physical_device(struct mlx5_priv *priv)
+{
+	pthread_mutex_lock(&mlx5_dev_ctx_list_mutex);
+	return priv->sh->phdev;
+}
+
+void
+mlx5_unlock_physical_device(void)
+{
+	pthread_mutex_unlock(&mlx5_dev_ctx_list_mutex);
+}
+
 static void
 mlx5_physical_device_destroy(struct mlx5_physical_device *phdev)
 {
@@ -2278,6 +2291,7 @@ int
 mlx5_dev_close(struct rte_eth_dev *dev)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
 	unsigned int i;
 	int ret;
 
@@ -2290,7 +2304,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 		rte_eth_dev_release_port(dev);
 		return 0;
 	}
-	if (!priv->sh)
+	if (!sh)
 		return 0;
 	if (priv->shared_refcnt) {
 		DRV_LOG(ERR, "port %u is shared host in use (%u)",
@@ -2298,6 +2312,15 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 		rte_errno = EBUSY;
 		return -EBUSY;
 	}
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	/* Check if shared GENEVE options created on context being closed. */
+	ret = mlx5_geneve_tlv_options_check_busy(priv);
+	if (ret) {
+		DRV_LOG(ERR, "port %u maintains shared GENEVE TLV options",
+			dev->data->port_id);
+		return ret;
+	}
+#endif
 	DRV_LOG(DEBUG, "port %u closing device \"%s\"",
 		dev->data->port_id,
 		((priv->sh->cdev->ctx != NULL) ?
@@ -2330,6 +2353,11 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	flow_hw_destroy_vport_action(dev);
 	flow_hw_resource_release(dev);
 	flow_hw_clear_port_info(dev);
+	if (priv->tlv_options != NULL) {
+		/* Free the GENEVE TLV parser resource. */
+		claim_zero(mlx5_geneve_tlv_options_destroy(priv->tlv_options, sh->phdev));
+		priv->tlv_options = NULL;
+	}
 #endif
 	if (priv->rxq_privs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 8bf7f86416..683029023e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1419,6 +1419,8 @@ struct mlx5_dev_registers {
 #define HAVE_MLX5_DR_CREATE_ACTION_ASO_EXT
 #endif
 
+struct mlx5_geneve_tlv_options;
+
 /**
  * Physical device structure.
  * This device is created once per NIC to manage recourses shared by all ports
@@ -1428,6 +1430,7 @@ struct mlx5_physical_device {
 	LIST_ENTRY(mlx5_physical_device) next;
 	struct mlx5_dev_ctx_shared *sh; /* Created on sherd context. */
 	uint64_t guid; /* System image guid, the uniq ID of physical device. */
+	struct mlx5_geneve_tlv_options *tlv_options;
 	uint32_t refcnt;
 };
 
@@ -1950,6 +1953,8 @@ struct mlx5_priv {
 	/* Action template list. */
 	LIST_HEAD(flow_hw_at, rte_flow_actions_template) flow_hw_at;
 	struct mlx5dr_context *dr_ctx; /**< HW steering DR context. */
+	/* Pointer to the GENEVE TLV options. */
+	struct mlx5_geneve_tlv_options *tlv_options;
 	/* HW steering queue polling mechanism job descriptor LIFO. */
 	uint32_t hws_strict_queue:1;
 	/**< Whether all operations strictly happen on the same HWS queue. */
@@ -2088,6 +2093,9 @@ void mlx5_flow_counter_mode_config(struct rte_eth_dev *dev);
 int mlx5_flow_aso_age_mng_init(struct mlx5_dev_ctx_shared *sh);
 int mlx5_aso_flow_mtrs_mng_init(struct mlx5_dev_ctx_shared *sh);
 int mlx5_flow_aso_ct_mng_init(struct mlx5_dev_ctx_shared *sh);
+struct mlx5_physical_device *
+mlx5_get_locked_physical_device(struct mlx5_priv *priv);
+void mlx5_unlock_physical_device(void);
 
 /* mlx5_ethdev.c */
 
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index acaf34ce52..5159e8e773 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -12524,3 +12524,33 @@ mlx5_flow_discover_ipv6_tc_support(struct rte_eth_dev *dev)
 	flow_list_destroy(dev, MLX5_FLOW_TYPE_GEN, flow_idx);
 	return 0;
 }
+
+void *
+rte_pmd_mlx5_create_geneve_tlv_parser(uint16_t port_id,
+				      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+				      uint8_t nb_options)
+{
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	return mlx5_geneve_tlv_parser_create(port_id, tlv_list, nb_options);
+#else
+	(void)port_id;
+	(void)tlv_list;
+	(void)nb_options;
+	DRV_LOG(ERR, "%s is not supported.", __func__);
+	rte_errno = ENOTSUP;
+	return NULL;
+#endif
+}
+
+int
+rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle)
+{
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	return mlx5_geneve_tlv_parser_destroy(handle);
+#else
+	(void)handle;
+	DRV_LOG(ERR, "%s is not supported.", __func__);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+#endif
+}
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 6f720de14d..4bf9ed7e4d 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1336,6 +1336,8 @@ struct mlx5_action_construct_data {
 	};
 };
 
+#define MAX_GENEVE_OPTIONS_RESOURCES 7
+
 /* Flow item template struct. */
 struct rte_flow_pattern_template {
 	LIST_ENTRY(rte_flow_pattern_template) next;
@@ -1650,6 +1652,11 @@ struct mlx5_flow_split_info {
 	uint64_t prefix_layers; /**< Prefix subflow layers. */
 };
 
+struct mlx5_hl_data {
+	uint8_t dw_offset;
+	uint32_t dw_mask;
+};
+
 struct flow_hw_port_info {
 	uint32_t regc_mask;
 	uint32_t regc_value;
@@ -1765,6 +1772,12 @@ flow_hw_get_reg_id_from_ctx(void *dr_ctx,
 	return REG_NON;
 }
 
+void *
+mlx5_geneve_tlv_parser_create(uint16_t port_id,
+			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+			      uint8_t nb_options);
+int mlx5_geneve_tlv_parser_destroy(void *handle);
+
 void flow_hw_set_port_info(struct rte_eth_dev *dev);
 void flow_hw_clear_port_info(struct rte_eth_dev *dev);
 int flow_hw_create_vport_action(struct rte_eth_dev *dev);
@@ -2810,6 +2823,11 @@ mlx5_get_tof(const struct rte_flow_item *items,
 	     enum mlx5_tof_rule_type *rule_type);
 void
 flow_hw_resource_release(struct rte_eth_dev *dev);
+int
+mlx5_geneve_tlv_options_destroy(struct mlx5_geneve_tlv_options *options,
+				struct mlx5_physical_device *phdev);
+int
+mlx5_geneve_tlv_options_check_busy(struct mlx5_priv *priv);
 void
 flow_hw_rxq_flag_set(struct rte_eth_dev *dev, bool enable);
 int flow_dv_action_validate(struct rte_eth_dev *dev,
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
new file mode 100644
index 0000000000..f23fb31aa0
--- /dev/null
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -0,0 +1,627 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2022 NVIDIA Corporation & Affiliates
+ */
+
+#include <rte_flow.h>
+
+#include <mlx5_malloc.h>
+#include <stdint.h>
+
+#include "generic/rte_byteorder.h"
+#include "mlx5.h"
+#include "mlx5_flow.h"
+#include "rte_pmd_mlx5.h"
+
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
+
+#define MAX_GENEVE_OPTION_DATA_SIZE 32
+#define MAX_GENEVE_OPTION_TOTAL_DATA_SIZE \
+		(MAX_GENEVE_OPTION_DATA_SIZE * MAX_GENEVE_OPTIONS_RESOURCES)
+
+/**
+ * Single DW inside GENEVE TLV option.
+ */
+struct mlx5_geneve_tlv_resource {
+	struct mlx5_devx_obj *obj; /* FW object returned in parser creation. */
+	uint32_t modify_field; /* Modify field ID for this DW. */
+	uint8_t offset; /* Offset used in obj creation, from option start. */
+};
+
+/**
+ * Single GENEVE TLV option context.
+ * May include some FW objects for different DWs in same option.
+ */
+struct mlx5_geneve_tlv_option {
+	uint8_t type;
+	uint16_t class;
+	uint8_t class_mode;
+	struct mlx5_hl_data match_data[MAX_GENEVE_OPTION_DATA_SIZE];
+	uint32_t match_data_size;
+	struct mlx5_hl_data hl_ok_bit;
+	struct mlx5_geneve_tlv_resource resources[MAX_GENEVE_OPTIONS_RESOURCES];
+	RTE_ATOMIC(uint32_t) refcnt;
+};
+
+/**
+ * List of GENEVE TLV options.
+ */
+struct mlx5_geneve_tlv_options {
+	/* List of configured GENEVE TLV options. */
+	struct mlx5_geneve_tlv_option options[MAX_GENEVE_OPTIONS_RESOURCES];
+	/*
+	 * Copy of list given in parser creation, use to compare with new
+	 * configuration.
+	 */
+	struct rte_pmd_mlx5_geneve_tlv spec[MAX_GENEVE_OPTIONS_RESOURCES];
+	rte_be32_t buffer[MAX_GENEVE_OPTION_TOTAL_DATA_SIZE];
+	uint8_t nb_options; /* Number entries in above lists. */
+	RTE_ATOMIC(uint32_t) refcnt;
+};
+
+/**
+ * Create single GENEVE TLV option sample.
+ *
+ * @param ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param attr
+ *   Pointer to GENEVE TLV option attributes structure.
+ * @param query_attr
+ *   Pointer to match sample info attributes structure.
+ * @param match_data
+ *   Pointer to header layout structure to update.
+ * @param resource
+ *   Pointer to single sample context to fill.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_create_sample(void *ctx,
+		      struct mlx5_devx_geneve_tlv_option_attr *attr,
+		      struct mlx5_devx_match_sample_info_query_attr *query_attr,
+		      struct mlx5_hl_data *match_data,
+		      struct mlx5_geneve_tlv_resource *resource)
+{
+	struct mlx5_devx_obj *obj;
+	int ret;
+
+	obj = mlx5_devx_cmd_create_geneve_tlv_option(ctx, attr);
+	if (obj == NULL)
+		return -rte_errno;
+	ret = mlx5_devx_cmd_query_geneve_tlv_option(ctx, obj, query_attr);
+	if (ret) {
+		claim_zero(mlx5_devx_cmd_destroy(obj));
+		return ret;
+	}
+	resource->obj = obj;
+	resource->offset = attr->sample_offset;
+	resource->modify_field = query_attr->modify_field_id;
+	match_data->dw_offset = query_attr->sample_dw_data;
+	match_data->dw_mask = 0xffffffff;
+	return 0;
+}
+
+/**
+ * Destroy single GENEVE TLV option sample.
+ *
+ * @param resource
+ *   Pointer to single sample context to clean.
+ */
+static void
+mlx5_geneve_tlv_option_destroy_sample(struct mlx5_geneve_tlv_resource *resource)
+{
+	claim_zero(mlx5_devx_cmd_destroy(resource->obj));
+	resource->obj = NULL;
+}
+
+/**
+ * Create single GENEVE TLV option.
+ *
+ * @param ctx
+ *   Context returned from mlx5 open_device() glue function.
+ * @param spec
+ *   Pointer to user configuration.
+ * @param option
+ *   Pointer to single GENEVE TLV option to fill.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *spec,
+			      struct mlx5_geneve_tlv_option *option)
+{
+	struct mlx5_devx_geneve_tlv_option_attr attr = {
+		.option_class = spec->option_class,
+		.option_type = spec->option_type,
+		.option_data_len = spec->option_len,
+		.option_class_ignore = spec->match_on_class_mode == 1 ? 0 : 1,
+		.offset_valid = 1,
+	};
+	struct mlx5_devx_match_sample_info_query_attr query_attr = {0};
+	struct mlx5_geneve_tlv_resource *resource;
+	uint8_t i, resource_id = 0;
+	int ret;
+
+	if (spec->match_on_class_mode == 2) {
+		/* Header is matchable, create sample for DW0. */
+		attr.sample_offset = 0;
+		resource = &option->resources[resource_id];
+		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
+							   &query_attr,
+							   &option->match_data[0],
+							   resource);
+		if (ret)
+			return ret;
+		resource_id++;
+	}
+	/*
+	 * Create FW object for each DW request by user.
+	 * Starting from 1 since FW offset starts from header.
+	 */
+	for (i = 1; i <= spec->sample_len; ++i) {
+		if (spec->match_data_mask[i - 1] == 0)
+			continue;
+		/* offset of data + offset inside data = specific DW offset. */
+		attr.sample_offset = spec->offset + i;
+		resource = &option->resources[resource_id];
+		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
+							   &query_attr,
+							   &option->match_data[i],
+							   resource);
+		if (ret)
+			goto error;
+		resource_id++;
+	}
+	/*
+	 * Update the OK bit information according to last query.
+	 * It should be same for each query under same option.
+	 */
+	option->hl_ok_bit.dw_offset = query_attr.sample_dw_ok_bit;
+	option->hl_ok_bit.dw_mask = 1 << query_attr.sample_dw_ok_bit_offset;
+	option->match_data_size = spec->sample_len + 1;
+	option->type = spec->option_type;
+	option->class = spec->option_class;
+	option->class_mode = spec->match_on_class_mode;
+	rte_atomic_store_explicit(&option->refcnt, 0, rte_memory_order_relaxed);
+	return 0;
+error:
+	for (i = 0; i < resource_id; ++i) {
+		resource = &option->resources[i];
+		mlx5_geneve_tlv_option_destroy_sample(resource);
+	}
+	return ret;
+}
+
+/**
+ * Destroy single GENEVE TLV option.
+ *
+ * @param option
+ *   Pointer to single GENEVE TLV option to destroy.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_destroy(struct mlx5_geneve_tlv_option *option)
+{
+	uint8_t i;
+
+	if (rte_atomic_load_explicit(&option->refcnt, rte_memory_order_relaxed)) {
+		DRV_LOG(ERR,
+			"Option type %u class %u is still in used by %u tables.",
+			option->type, option->class, option->refcnt);
+		rte_errno = EBUSY;
+		return -rte_errno;
+	}
+	for (i = 0; option->resources[i].obj != NULL; ++i)
+		mlx5_geneve_tlv_option_destroy_sample(&option->resources[i]);
+	return 0;
+}
+
+/**
+ * Copy the GENEVE TLV option user configuration for future comparing.
+ *
+ * @param dst
+ *   Pointer to internal user configuration copy.
+ * @param src
+ *   Pointer to user configuration.
+ * @param match_data_mask
+ *   Pointer to allocated data array.
+ */
+static void
+mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst,
+			    const struct rte_pmd_mlx5_geneve_tlv *src,
+			    rte_be32_t *match_data_mask)
+{
+	uint8_t i;
+
+	dst->option_type = src->option_type;
+	dst->option_class = src->option_class;
+	dst->option_len = src->option_len;
+	dst->offset = src->offset;
+	dst->match_on_class_mode = src->match_on_class_mode;
+	dst->sample_len = src->sample_len;
+	for (i = 0; i < dst->sample_len; ++i)
+		match_data_mask[i] = src->match_data_mask[i];
+	dst->match_data_mask = match_data_mask;
+}
+
+/**
+ * Create list of GENEVE TLV options according to user configuration list.
+ *
+ * @param sh
+ *   Shared context the options are being created on.
+ * @param tlv_list
+ *   A list of GENEVE TLV options to create parser for them.
+ * @param nb_options
+ *   The number of options in TLV list.
+ *
+ * @return
+ *   A pointer to GENEVE TLV options parser structure on success,
+ *   NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_geneve_tlv_options *
+mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh,
+			       const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+			       uint8_t nb_options)
+{
+	struct mlx5_geneve_tlv_options *options;
+	const struct rte_pmd_mlx5_geneve_tlv *spec;
+	rte_be32_t *data_mask;
+	uint8_t i, j;
+	int ret;
+
+	options = mlx5_malloc(MLX5_MEM_ZERO | MLX5_MEM_RTE,
+			      sizeof(struct mlx5_geneve_tlv_options),
+			      RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (options == NULL) {
+		DRV_LOG(ERR,
+			"Failed to allocate memory for GENEVE TLV options.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	for (i = 0; i < nb_options; ++i) {
+		spec = &tlv_list[i];
+		ret = mlx5_geneve_tlv_option_create(sh->cdev->ctx, spec,
+						    &options->options[i]);
+		if (ret < 0)
+			goto error;
+		/* Copy the user list for comparing future configuration. */
+		data_mask = options->buffer + i * MAX_GENEVE_OPTION_DATA_SIZE;
+		mlx5_geneve_tlv_option_copy(&options->spec[i], spec, data_mask);
+	}
+	MLX5_ASSERT(sh->phdev->sh == NULL);
+	sh->phdev->sh = sh;
+	options->nb_options = nb_options;
+	options->refcnt = 1;
+	return options;
+error:
+	for (j = 0; j < i; ++j)
+		mlx5_geneve_tlv_option_destroy(&options->options[j]);
+	mlx5_free(options);
+	return NULL;
+}
+
+/**
+ * Destroy GENEVE TLV options structure.
+ *
+ * @param options
+ *   Pointer to GENEVE TLV options structure to destroy.
+ * @param phdev
+ *   Pointer physical device options were created on.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_geneve_tlv_options_destroy(struct mlx5_geneve_tlv_options *options,
+				struct mlx5_physical_device *phdev)
+{
+	uint8_t i;
+	int ret;
+
+	if (--options->refcnt)
+		return 0;
+	for (i = 0; i < options->nb_options; ++i) {
+		ret = mlx5_geneve_tlv_option_destroy(&options->options[i]);
+		if (ret < 0) {
+			DRV_LOG(ERR,
+				"Failed to destroy option %u, %u/%u is already destroyed.",
+				i, i, options->nb_options);
+			return ret;
+		}
+	}
+	mlx5_free(options);
+	phdev->tlv_options = NULL;
+	phdev->sh = NULL;
+	return 0;
+}
+
+/**
+ * Check if GENEVE TLV options are hosted on the current port
+ * and the port can be closed
+ *
+ * @param priv
+ *   Device private data.
+ *
+ * @return
+ *   0 on success, a negative EBUSY and rte_errno is set.
+ */
+int
+mlx5_geneve_tlv_options_check_busy(struct mlx5_priv *priv)
+{
+	struct mlx5_physical_device *phdev = mlx5_get_locked_physical_device(priv);
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
+
+	if (!phdev || phdev->sh != sh) {
+		mlx5_unlock_physical_device();
+		return 0;
+	}
+	if (!sh->phdev->tlv_options || sh->phdev->tlv_options->refcnt == 1) {
+		/* Mark port as being closed one */
+		sh->phdev->sh = NULL;
+		mlx5_unlock_physical_device();
+		return 0;
+	}
+	mlx5_unlock_physical_device();
+	rte_errno = EBUSY;
+	return -EBUSY;
+}
+
+/**
+ * Validate GENEVE TLV option user request structure.
+ *
+ * @param attr
+ *   Pointer to HCA attribute structure.
+ * @param option
+ *   Pointer to user configuration.
+ *
+ * @return
+ *   0 on success, a negative errno otherwise and rte_errno is set.
+ */
+static int
+mlx5_geneve_tlv_option_validate(struct mlx5_hca_attr *attr,
+				const struct rte_pmd_mlx5_geneve_tlv *option)
+{
+	if (option->option_len > attr->max_geneve_tlv_option_data_len) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option length (%u) exceeds the limit (%u).",
+			option->option_len,
+			attr->max_geneve_tlv_option_data_len);
+		rte_errno = ENOTSUP;
+		return -rte_errno;
+	}
+	if (option->option_len < option->offset + option->sample_len) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option length is smaller than (offset + sample_len).");
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	if (option->match_on_class_mode > 2) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option match_on_class_mode is invalid.");
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	return 0;
+}
+
+/**
+ * Get the number of requested DWs in given GENEVE TLV option.
+ *
+ * @param option
+ *   Pointer to user configuration.
+ *
+ * @return
+ *   Number of requested DWs for given GENEVE TLV option.
+ */
+static uint8_t
+mlx5_geneve_tlv_option_get_nb_dws(const struct rte_pmd_mlx5_geneve_tlv *option)
+{
+	uint8_t nb_dws = 0;
+	uint8_t i;
+
+	if (option->match_on_class_mode == 2)
+		nb_dws++;
+	for (i = 0; i < option->sample_len; ++i) {
+		if (option->match_data_mask[i] == 0xffffffff)
+			nb_dws++;
+	}
+	return nb_dws;
+}
+
+/**
+ * Compare GENEVE TLV option user request structure.
+ *
+ * @param option1
+ *   Pointer to first user configuration.
+ * @param option2
+ *   Pointer to second user configuration.
+ *
+ * @return
+ *   True if the options are equal, false otherwise.
+ */
+static bool
+mlx5_geneve_tlv_option_compare(const struct rte_pmd_mlx5_geneve_tlv *option1,
+			       const struct rte_pmd_mlx5_geneve_tlv *option2)
+{
+	uint8_t i;
+
+	if (option1->option_type != option2->option_type ||
+	    option1->option_class != option2->option_class ||
+	    option1->option_len != option2->option_len ||
+	    option1->offset != option2->offset ||
+	    option1->match_on_class_mode != option2->match_on_class_mode ||
+	    option1->sample_len != option2->sample_len)
+		return false;
+	for (i = 0; i < option1->sample_len; ++i) {
+		if (option1->match_data_mask[i] != option2->match_data_mask[i])
+			return false;
+	}
+	return true;
+}
+
+/**
+ * Check whether the given GENEVE TLV option list is equal to internal list.
+ * The lists are equal when they have same size and same options in the same
+ * order inside the list.
+ *
+ * @param options
+ *   Pointer to GENEVE TLV options structure.
+ * @param tlv_list
+ *   A list of GENEVE TLV options to compare.
+ * @param nb_options
+ *   The number of options in TLV list.
+ *
+ * @return
+ *   True if the lists are equal, false otherwise.
+ */
+static bool
+mlx5_is_same_geneve_tlv_options(const struct mlx5_geneve_tlv_options *options,
+				const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+				uint8_t nb_options)
+{
+	const struct rte_pmd_mlx5_geneve_tlv *spec = options->spec;
+	uint8_t i;
+
+	if (options->nb_options != nb_options)
+		return false;
+	for (i = 0; i < nb_options; ++i) {
+		if (!mlx5_geneve_tlv_option_compare(&spec[i], &tlv_list[i]))
+			return false;
+	}
+	return true;
+}
+
+void *
+mlx5_geneve_tlv_parser_create(uint16_t port_id,
+			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+			      uint8_t nb_options)
+{
+	struct mlx5_geneve_tlv_options *options = NULL;
+	struct mlx5_physical_device *phdev;
+	struct rte_eth_dev *dev;
+	struct mlx5_priv *priv;
+	struct mlx5_hca_attr *attr;
+	uint8_t total_dws = 0;
+	uint8_t i;
+
+	/*
+	 * Validate the input before taking a lock and before any memory
+	 * allocation.
+	 */
+	if (rte_eth_dev_is_valid_port(port_id) < 0) {
+		DRV_LOG(ERR, "There is no Ethernet device for port %u.",
+			port_id);
+		rte_errno = ENODEV;
+		return NULL;
+	}
+	dev = &rte_eth_devices[port_id];
+	priv = dev->data->dev_private;
+	if (priv->tlv_options) {
+		DRV_LOG(ERR, "Port %u already has GENEVE TLV parser.", port_id);
+		rte_errno = EEXIST;
+		return NULL;
+	}
+	if (priv->sh->config.dv_flow_en < 2) {
+		DRV_LOG(ERR,
+			"GENEVE TLV parser is only supported for HW steering.");
+		rte_errno = ENOTSUP;
+		return NULL;
+	}
+	attr = &priv->sh->cdev->config.hca_attr;
+	MLX5_ASSERT(MAX_GENEVE_OPTIONS_RESOURCES <=
+		    attr->max_geneve_tlv_options);
+	if (!attr->geneve_tlv_option_offset || !attr->geneve_tlv_sample ||
+	    !attr->query_match_sample_info || !attr->geneve_tlv_opt) {
+		DRV_LOG(ERR, "Not enough capabilities to support GENEVE TLV parser, maybe old FW version");
+		rte_errno = ENOTSUP;
+		return NULL;
+	}
+	if (nb_options > MAX_GENEVE_OPTIONS_RESOURCES) {
+		DRV_LOG(ERR,
+			"GENEVE TLV option number (%u) exceeds the limit (%u).",
+			nb_options, MAX_GENEVE_OPTIONS_RESOURCES);
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	for (i = 0; i < nb_options; ++i) {
+		if (mlx5_geneve_tlv_option_validate(attr, &tlv_list[i]) < 0) {
+			DRV_LOG(ERR, "GENEVE TLV option %u is invalid.", i);
+			return NULL;
+		}
+		total_dws += mlx5_geneve_tlv_option_get_nb_dws(&tlv_list[i]);
+	}
+	if (total_dws > MAX_GENEVE_OPTIONS_RESOURCES) {
+		DRV_LOG(ERR,
+			"Total requested DWs (%u) exceeds the limit (%u).",
+			total_dws, MAX_GENEVE_OPTIONS_RESOURCES);
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	/* Take lock for this physical device and manage the options. */
+	phdev = mlx5_get_locked_physical_device(priv);
+	options = priv->sh->phdev->tlv_options;
+	if (options) {
+		if (!mlx5_is_same_geneve_tlv_options(options, tlv_list,
+						     nb_options)) {
+			mlx5_unlock_physical_device();
+			DRV_LOG(ERR, "Another port has already prepared different GENEVE TLV parser.");
+			rte_errno = EEXIST;
+			return NULL;
+		}
+		if (phdev->sh == NULL) {
+			mlx5_unlock_physical_device();
+			DRV_LOG(ERR, "GENEVE TLV options are hosted on port being closed.");
+			rte_errno = EBUSY;
+			return NULL;
+		}
+		/* Use existing options. */
+		options->refcnt++;
+		goto exit;
+	}
+	/* Create GENEVE TLV options for this physical device. */
+	options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options);
+	if (!options) {
+		mlx5_unlock_physical_device();
+		return NULL;
+	}
+	phdev->tlv_options = options;
+exit:
+	mlx5_unlock_physical_device();
+	priv->tlv_options = options;
+	return priv;
+}
+
+int
+mlx5_geneve_tlv_parser_destroy(void *handle)
+{
+	struct mlx5_priv *priv = (struct mlx5_priv *)handle;
+	struct mlx5_physical_device *phdev;
+	int ret;
+
+	if (priv == NULL) {
+		DRV_LOG(ERR, "Handle input is invalid (NULL).");
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	if (priv->tlv_options == NULL) {
+		DRV_LOG(ERR, "This parser has been already released.");
+		rte_errno = ENOENT;
+		return -rte_errno;
+	}
+	/* Take lock for this physical device and manage the options. */
+	phdev = mlx5_get_locked_physical_device(priv);
+	/* Destroy the options */
+	ret = mlx5_geneve_tlv_options_destroy(phdev->tlv_options, phdev);
+	if (ret < 0) {
+		mlx5_unlock_physical_device();
+		return ret;
+	}
+	priv->tlv_options = NULL;
+	mlx5_unlock_physical_device();
+	return 0;
+}
+
+#endif /* defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) */
diff --git a/drivers/net/mlx5/rte_pmd_mlx5.h b/drivers/net/mlx5/rte_pmd_mlx5.h
index 654dd3cff3..004be0eea1 100644
--- a/drivers/net/mlx5/rte_pmd_mlx5.h
+++ b/drivers/net/mlx5/rte_pmd_mlx5.h
@@ -229,6 +229,108 @@ enum rte_pmd_mlx5_flow_engine_mode {
 __rte_experimental
 int rte_pmd_mlx5_flow_engine_set_mode(enum rte_pmd_mlx5_flow_engine_mode mode, uint32_t flags);
 
+/**
+ * User configuration structure using to create parser for single GENEVE TLV option.
+ */
+struct rte_pmd_mlx5_geneve_tlv {
+	/**
+	 * The class of the GENEVE TLV option.
+	 * Relevant only when 'match_on_class_mode' is 1.
+	 */
+	rte_be16_t option_class;
+	/**
+	 * The type of the GENEVE TLV option.
+	 * This field is the identifier of the option.
+	 */
+	uint8_t option_type;
+	/**
+	 * The length of the GENEVE TLV option data excluding the option header
+	 * in DW granularity.
+	 */
+	uint8_t option_len;
+	/**
+	 * Indicator about class field role in this option:
+	 *  0 - class is ignored.
+	 *  1 - class is fixed (the class defines the option along with the type).
+	 *  2 - class matching per flow.
+	 */
+	uint8_t match_on_class_mode;
+	/**
+	 * The offset of the first sample in DW granularity.
+	 * This offset is relative to first of option data.
+	 * The 'match_data_mask' corresponds to option data since this offset.
+	 */
+	uint8_t offset;
+	/**
+	 * The number of DW to sample.
+	 * This field describes the length of 'match_data_mask' in DW
+	 * granularity.
+	 */
+	uint8_t sample_len;
+	/**
+	 * Array of DWs which each bit marks if this bit should be sampled.
+	 * Each nonzero DW consumes one DW from maximum 7 DW in total.
+	 */
+	rte_be32_t *match_data_mask;
+};
+
+/**
+ * Creates GENEVE TLV parser for the selected port.
+ * This function must be called before first use of GENEVE option.
+ *
+ * This API is port oriented, but the configuration is done once for all ports
+ * under the same physical device. Each port should call this API before using
+ * GENEVE OPT item, but it must use the same options in the same order inside
+ * the list.
+ *
+ * Each physical device has 7 DWs for GENEVE TLV options. Each nonzero element
+ * in 'match_data_mask' array consumes one DW, and choosing matchable mode for
+ * class consumes additional one.
+ * Calling this API for second port under same physical device doesn't consume
+ * more DW, it uses same configuration.
+ *
+ * @param[in] port_id
+ *   The port identifier of the Ethernet device.
+ * @param[in] tlv_list
+ *   A list of GENEVE TLV options to create parser for them.
+ * @param[in] nb_options
+ *   The number of options in TLV list.
+ *
+ * @return
+ *   A pointer to TLV handle on success, NULL otherwise and rte_errno is set.
+ *   Possible values for rte_errno:
+ *   - ENOMEM - not enough memory to create GENEVE TLV parser.
+ *   - EEXIST - this port already has GENEVE TLV parser or another port under
+ *              same physical device has already prepared a different parser.
+ *   - EINVAL - invalid GENEVE TLV requested.
+ *   - ENODEV - there is no Ethernet device for this port id.
+ *   - ENOTSUP - the port doesn't support GENEVE TLV parsing.
+ */
+__rte_experimental
+void *
+rte_pmd_mlx5_create_geneve_tlv_parser(uint16_t port_id,
+				      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
+				      uint8_t nb_options);
+
+/**
+ * Destroy GENEVE TLV parser for the selected port.
+ * This function must be called after last use of GENEVE option and before port
+ * closing.
+ *
+ * @param[in] handle
+ *   Handle for the GENEVE TLV parser object to be destroyed.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ *   Possible values for rte_errno:
+ *   - EINVAL - invalid handle.
+ *   - ENOENT - there is no valid GENEVE TLV parser in this handle.
+ *   - EBUSY - one of options is in used by template table.
+ */
+__rte_experimental
+int
+rte_pmd_mlx5_destroy_geneve_tlv_parser(void *handle);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/drivers/net/mlx5/version.map b/drivers/net/mlx5/version.map
index 99f5ab754a..8fb0e07303 100644
--- a/drivers/net/mlx5/version.map
+++ b/drivers/net/mlx5/version.map
@@ -17,4 +17,7 @@ EXPERIMENTAL {
 	rte_pmd_mlx5_external_sq_enable;
 	# added in 23.03
 	rte_pmd_mlx5_flow_engine_set_mode;
+	# added in 24.03
+	rte_pmd_mlx5_create_geneve_tlv_parser;
+	rte_pmd_mlx5_destroy_geneve_tlv_parser;
 };
-- 
2.25.1

[PATCH v2 14/23] net/mlx5: add API to expose GENEVE option FW information

[Michael Baum]

Add a new API to expose GENEVE option FW information to DR layer.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.h        | 28 +++++++++
 drivers/net/mlx5/mlx5_flow_geneve.c | 94 +++++++++++++++++++++++++++++
 2 files changed, 122 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 4bf9ed7e4d..14806fa78e 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1772,6 +1772,34 @@ flow_hw_get_reg_id_from_ctx(void *dr_ctx,
 	return REG_NON;
 }
 
+/**
+ * Get GENEVE TLV option FW information according type and class.
+ *
+ * @param[in] dr_ctx
+ *   Pointer to HW steering DR context.
+ * @param[in] type
+ *   GENEVE TLV option type.
+ * @param[in] class
+ *   GENEVE TLV option class.
+ * @param[out] hl_ok_bit
+ *   Pointer to header layout structure describing OK bit FW information.
+ * @param[out] num_of_dws
+ *   Pointer to fill inside the size of 'hl_dws' array.
+ * @param[out] hl_dws
+ *   Pointer to header layout array describing data DWs FW information.
+ * @param[out] ok_bit_on_class
+ *   Pointer to an indicator whether OK bit includes class along with type.
+ *
+ * @return
+ *   0 on success, negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
+			struct mlx5_hl_data ** const hl_ok_bit,
+			uint8_t *num_of_dws,
+			struct mlx5_hl_data ** const hl_dws,
+			bool *ok_bit_on_class);
+
 void *
 mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index f23fb31aa0..2d593b70ba 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -58,6 +58,100 @@ struct mlx5_geneve_tlv_options {
 	RTE_ATOMIC(uint32_t) refcnt;
 };
 
+/**
+ * Check if type and class is matching to given GENEVE TLV option.
+ *
+ * @param type
+ *   GENEVE option type.
+ * @param class
+ *   GENEVE option class.
+ * @param option
+ *   Pointer to GENEVE TLV option structure.
+ *
+ * @return
+ *   True if this type and class match to this option, false otherwise.
+ */
+static inline bool
+option_match_type_and_class(uint8_t type, uint16_t class,
+			    struct mlx5_geneve_tlv_option *option)
+{
+	if (type != option->type)
+		return false;
+	if (option->class_mode == 1 && option->class != class)
+		return false;
+	return true;
+}
+
+/**
+ * Get GENEVE TLV option matching to given type and class.
+ *
+ * @param priv
+ *   Pointer to port's private data.
+ * @param type
+ *   GENEVE option type.
+ * @param class
+ *   GENEVE option class.
+ *
+ * @return
+ *   Pointer to option structure if exist, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_geneve_tlv_option *
+mlx5_geneve_tlv_option_get(const struct mlx5_priv *priv, uint8_t type,
+			   uint16_t class)
+{
+	struct mlx5_geneve_tlv_options *options;
+	uint8_t i;
+
+	if (priv->tlv_options == NULL) {
+		DRV_LOG(ERR,
+			"Port %u doesn't have configured GENEVE TLV options.",
+			priv->dev_data->port_id);
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	options = priv->tlv_options;
+	MLX5_ASSERT(options != NULL);
+	for (i = 0; i < options->nb_options; ++i) {
+		struct mlx5_geneve_tlv_option *option = &options->options[i];
+
+		if (option_match_type_and_class(type, class, option))
+			return option;
+	}
+	DRV_LOG(ERR, "TLV option type %u class %u doesn't exist.", type, class);
+	rte_errno = ENOENT;
+	return NULL;
+}
+
+int
+mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
+			struct mlx5_hl_data ** const hl_ok_bit,
+			uint8_t *num_of_dws,
+			struct mlx5_hl_data ** const hl_dws,
+			bool *ok_bit_on_class)
+{
+	uint16_t port_id;
+
+	MLX5_ETH_FOREACH_DEV(port_id, NULL) {
+		struct mlx5_priv *priv;
+		struct mlx5_geneve_tlv_option *option;
+
+		priv = rte_eth_devices[port_id].data->dev_private;
+		if (priv->dr_ctx != dr_ctx)
+			continue;
+		/* Find specific option inside list. */
+		option = mlx5_geneve_tlv_option_get(priv, type, class);
+		if (option == NULL)
+			return -rte_errno;
+		*hl_ok_bit = &option->hl_ok_bit;
+		*hl_dws = option->match_data;
+		*num_of_dws = option->match_data_size;
+		*ok_bit_on_class = !!(option->class_mode == 1);
+		return 0;
+	}
+	DRV_LOG(ERR, "DR CTX %p doesn't belong to any DPDK port.", dr_ctx);
+	return -EINVAL;
+}
+
 /**
  * Create single GENEVE TLV option sample.
  *
-- 
2.25.1

[PATCH v2 15/23] net/mlx5: add testpmd support for GENEVE TLV parser

[Michael Baum]

Add GENEVE TLV parser support for mlx5 testpmd using following commands:

1. Add single option to the global option list:

   testpmd> mlx5 set tlv_option class (class) type (type) len (length) \
            offset (sample_offset) sample_len (sample_len) \
            class_mode (ignore|fixed|matchable) \
            data (0xffffffff|0x0 [0xffffffff|0x0]*)

2. Remove several options from the global option list:

   testpmd> mlx5 flush tlv_options max (nb_option)

3. Print all options which are set in the global option list so far:

   testpmd> mlx5 list tlv_options

4. Create GENEVE TLV parser for specific port using option list which
   are set so far:

   testpmd> mlx5 port (port_id) apply tlv_options

5. Destroy GENEVE TLV parser for specific port:

   testpmd> mlx5 port (port_id) destroy tlv_options

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 doc/guides/nics/mlx5.rst        |  97 ++++++
 drivers/net/mlx5/mlx5_testpmd.c | 556 +++++++++++++++++++++++++++++++-
 2 files changed, 652 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index e82f7034aa..2e5274edb8 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -2515,3 +2515,100 @@ This command is used for testing live migration,
 and works for software steering only.
 Default FDB jump should be disabled if switchdev is enabled.
 The mode will propagate to all the probed ports.
+
+GENEVE TLV options parser
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+GENEVE TLV options parser management.
+See :ref:`options parser API <geneve_parser_api>` for more information.
+
+Setting Option
+^^^^^^^^^^^^^^
+
+Add single option to the global option list::
+
+   testpmd> mlx5 set tlv_option class (class) type (type) len (length) \
+            offset (sample_offset) sample_len (sample_len) \
+            class_mode (ignore|fixed|matchable) data (0xffffffff|0x0 [0xffffffff|0x0]*)
+
+where:
+
+* ``class``: option class.
+* ``type``: option type.
+* ``length``: option data length in 4 bytes granularity.
+* ``sample_offset``: offset to data list related to option data start.
+  The offset is in 4 bytes granularity.
+* ``sample_len``: length data list in 4 bytes granularity.
+* ``ignore``: ignore ``class`` field.
+* ``fixed``: option class is fixed and defines the option along with the type.
+* ``matchable``: ``class`` field is matchable.
+* ``data``: list of masks indicating which DW should be configure.
+  The size of list should be equal to ``sample_len``.
+* ``0xffffffff``: this DW should be configure.
+* ``0x0``: this DW shouldn't be configure.
+
+
+Flushing Options
+^^^^^^^^^^^^^^^^
+
+Remove several options from the global option list::
+
+   testpmd> mlx5 flush tlv_options max (nb_option)
+
+where:
+
+* ``nb_option``: maximum number of option to remove from list. The order is LIFO.
+
+
+Listing Options
+^^^^^^^^^^^^^^^
+
+Print all options which are set in the global option list so far::
+
+   testpmd> mlx5 list tlv_options
+
+Output contains the values of each option, one per line.
+There is no output at all when no options are configured on the global list::
+
+   ID      Type    Class   Class_mode   Len     Offset  Sample_len   Data
+   [...]   [...]   [...]   [...]        [...]   [...]   [...]        [...]
+
+Setting several options and listing them::
+
+   testpmd> mlx5 set tlv_option class 1 type 1 len 4 offset 1 sample_len 3
+            class_mode fixed data 0xffffffff 0x0 0xffffffff
+   testpmd: set new option in global list, now it has 1 options
+   testpmd> mlx5 set tlv_option class 1 type 2 len 2 offset 0 sample_len 2
+            class_mode fixed data 0xffffffff 0xffffffff
+   testpmd: set new option in global list, now it has 2 options
+   testpmd> mlx5 set tlv_option class 1 type 3 len 5 offset 4 sample_len 1
+            class_mode fixed data 0xffffffff
+   testpmd: set new option in global list, now it has 3 options
+   testpmd> mlx5 list tlv_options
+   ID      Type    Class   Class_mode   Len    Offset  Sample_len  Data
+   0       1       1       fixed        4      1       3           0xffffffff 0x0 0xffffffff
+   1       2       1       fixed        2      0       2           0xffffffff 0xffffffff
+   2       3       1       fixed        5      4       1           0xffffffff
+   testpmd>
+
+
+Applying Options
+^^^^^^^^^^^^^^^^
+
+Create GENEVE TLV parser for specific port using option list which are set so
+far::
+
+   testpmd> mlx5 port (port_id) apply tlv_options
+
+The same global option list can used by several ports.
+
+
+Destroying Options
+^^^^^^^^^^^^^^^^^^
+
+Destroy GENEVE TLV parser for specific port::
+
+   testpmd> mlx5 port (port_id) destroy tlv_options
+
+This command doesn't destroy the global list,
+For releasing options, ``flush`` command should be used.
diff --git a/drivers/net/mlx5/mlx5_testpmd.c b/drivers/net/mlx5/mlx5_testpmd.c
index 403f3a8f83..5bc4dd0551 100644
--- a/drivers/net/mlx5/mlx5_testpmd.c
+++ b/drivers/net/mlx5/mlx5_testpmd.c
@@ -23,9 +23,25 @@
 #include "mlx5_testpmd.h"
 #include "testpmd.h"
 
-static uint8_t host_shaper_avail_thresh_triggered[RTE_MAX_ETHPORTS];
 #define SHAPER_DISABLE_DELAY_US 100000 /* 100ms */
+#define MAX_GENEVE_OPTIONS_RESOURCES 7
 #define PARSE_DELIMITER " \f\n\r\t\v"
+#define SPACE_DELIMITER (" ")
+
+static uint8_t host_shaper_avail_thresh_triggered[RTE_MAX_ETHPORTS];
+
+struct mlx5_port {
+	void *geneve_tlv_parser_handle;
+};
+
+static struct mlx5_port private_port[RTE_MAX_ETHPORTS] = {{0}};
+
+struct tlv_list_manager {
+	uint8_t nb_options;
+	struct rte_pmd_mlx5_geneve_tlv tlv_list[MAX_GENEVE_OPTIONS_RESOURCES];
+};
+
+static struct tlv_list_manager tlv_mng = {.nb_options = 0};
 
 static int
 parse_uint(uint64_t *value, const char *str)
@@ -304,6 +320,88 @@ mlx5_test_attach_port_extend_devargs(char *identifier)
 }
 #endif
 
+static inline const char *
+mode2string(uint8_t mode)
+{
+	switch (mode) {
+	case 0:
+		return "ignored\t";
+	case 1:
+		return "fixed\t";
+	case 2:
+		return "matchable";
+	default:
+		break;
+	}
+	return "unknown";
+}
+
+static inline uint8_t
+string2mode(const char *mode)
+{
+	if (strcmp(mode, "ignored") == 0)
+		return 0;
+	if (strcmp(mode, "fixed") == 0)
+		return 1;
+	if (strcmp(mode, "matchable") == 0)
+		return 2;
+	return UINT8_MAX;
+}
+
+static int
+mlx5_test_parse_geneve_option_data(const char *buff, uint8_t data_len,
+				   rte_be32_t **match_data_mask)
+{
+	rte_be32_t *data;
+	char *buff2;
+	char *token;
+	uint8_t i = 0;
+
+	if (data_len == 0) {
+		*match_data_mask = NULL;
+		return 0;
+	}
+
+	data = calloc(data_len, sizeof(rte_be32_t));
+	if (data == NULL) {
+		TESTPMD_LOG(ERR, "Fail to allocate memory for GENEVE TLV option data\n");
+		return -ENOMEM;
+	}
+
+	buff2 = strdup(buff);
+	if (buff2 == NULL) {
+		TESTPMD_LOG(ERR,
+			    "Fail to duplicate GENEVE TLV option data string (%s)\n",
+			    buff);
+		free(data);
+		return -ENOMEM;
+	}
+
+	token = strtok(buff2, SPACE_DELIMITER);
+	while (token != NULL) {
+		if (i == data_len) {
+			TESTPMD_LOG(ERR,
+				    "GENEVE TLV option has more data then given data length %u\n",
+				    data_len);
+			free(buff2);
+			free(data);
+			return -EINVAL;
+		}
+
+		if (strcmp(token, "0xffffffff") == 0)
+			data[i] = 0xffffffff;
+		else
+			data[i] = 0x0;
+
+		token = strtok(NULL, SPACE_DELIMITER);
+		i++;
+	}
+
+	free(buff2);
+	*match_data_mask = data;
+	return 0;
+}
+
 /* *** SET HOST_SHAPER FOR A PORT *** */
 struct cmd_port_host_shaper_result {
 	cmdline_fixed_string_t mlx5;
@@ -680,6 +778,429 @@ cmdline_parse_inst_t mlx5_cmd_set_flow_engine_mode = {
 	}
 };
 
+/* Prepare single GENEVE TLV option and add it into global option list. */
+struct mlx5_cmd_set_tlv_option {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t set;
+	cmdline_fixed_string_t tlv_option;
+	cmdline_fixed_string_t class;
+	uint16_t class_id;
+	cmdline_fixed_string_t type;
+	uint8_t type_id;
+	cmdline_fixed_string_t len;
+	uint8_t option_len;
+	cmdline_fixed_string_t offset;
+	uint8_t off;
+	cmdline_fixed_string_t sample_len;
+	uint8_t length;
+	cmdline_fixed_string_t class_mode;
+	cmdline_fixed_string_t cmode;
+	cmdline_fixed_string_t data;
+	cmdline_fixed_string_t data_mask;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, mlx5, "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_set =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, set, "set");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_tlv_option =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, tlv_option,
+				 "tlv_option");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_class =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, class,
+				 "class");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_class_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, class_id,
+			      RTE_UINT16);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_type =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, type, "type");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_type_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, type_id,
+			      RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_len =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, len, "len");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_option_len =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, option_len,
+			      RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_offset =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, offset,
+				 "offset");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_off =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, off, RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_sample_len =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, sample_len,
+				 "sample_len");
+cmdline_parse_token_num_t mlx5_cmd_set_tlv_option_length =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_set_tlv_option, length,
+			      RTE_UINT8);
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_class_mode =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, class_mode,
+				 "class_mode");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_cmode =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, cmode,
+				 "ignored#fixed#matchable");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_data =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, data, "data");
+cmdline_parse_token_string_t mlx5_cmd_set_tlv_option_data_mask =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_set_tlv_option, data_mask, "");
+
+static void
+mlx5_cmd_set_tlv_option_parsed(void *parsed_result,
+			       __rte_unused struct cmdline *cl,
+			       __rte_unused void *data)
+{
+	struct mlx5_cmd_set_tlv_option *res = parsed_result;
+	struct rte_pmd_mlx5_geneve_tlv *option;
+	uint8_t class_mode;
+	int ret;
+
+	if (tlv_mng.nb_options == MAX_GENEVE_OPTIONS_RESOURCES) {
+		fprintf(stderr, "GENEVE TLV option list is full\n");
+		return;
+	}
+
+	if (res->option_len < res->length + res->off) {
+		fprintf(stderr,
+			"GENEVE TLV option length (%u) cannot be less than offset (%u) + sample_len (%u)\n",
+			res->option_len, res->length, res->off);
+		return;
+	}
+
+	if (res->option_len > 32) {
+		fprintf(stderr,
+			"GENEVE TLV option length (%u) must be less than 32\n",
+			res->option_len);
+		return;
+	}
+
+	class_mode = string2mode(res->cmode);
+	if (class_mode == UINT8_MAX) {
+		fprintf(stderr, "Invalid class mode \"%s\"\n", res->cmode);
+		return;
+	}
+
+	if (res->length > 0) {
+		if (strcmp(res->data, "data") || !strcmp(res->data_mask, "")) {
+			fprintf(stderr,
+				"sample_len is %u but any data isn't provided\n",
+				res->length);
+			return;
+		}
+	} else {
+		if (!strcmp(res->data, "data") && strcmp(res->data_mask, "")) {
+			fprintf(stderr,
+				"sample_len is 0 but data is provided (%s)\n",
+				res->data_mask);
+			return;
+		}
+	}
+
+	option = &tlv_mng.tlv_list[tlv_mng.nb_options];
+	ret = mlx5_test_parse_geneve_option_data(res->data_mask, res->length,
+						 &option->match_data_mask);
+	if (ret < 0)
+		return;
+
+	option->match_on_class_mode = class_mode;
+	option->option_class = rte_cpu_to_be_16(res->class_id);
+	option->option_type = res->type_id;
+	option->option_len = res->option_len;
+	option->offset = res->off;
+	option->sample_len = res->length;
+	tlv_mng.nb_options++;
+
+	TESTPMD_LOG(DEBUG,
+		    "set new option in global list, now it has %u options\n",
+		    tlv_mng.nb_options);
+}
+
+cmdline_parse_inst_t mlx5_cmd_set_tlv_option = {
+	.f = mlx5_cmd_set_tlv_option_parsed,
+	.data = NULL,
+	.help_str = "mlx5 set tlv_option class <class_id> type <type_id> len "
+		"<option_len> offset <sample_offset> sample_len "
+		"<sample_length> class_mode <ignored|fixed|matchable> data <mask1 [mask2 [...]>",
+	.tokens = {
+		(void *)&mlx5_cmd_set_tlv_option_mlx5,
+		(void *)&mlx5_cmd_set_tlv_option_set,
+		(void *)&mlx5_cmd_set_tlv_option_tlv_option,
+		(void *)&mlx5_cmd_set_tlv_option_class,
+		(void *)&mlx5_cmd_set_tlv_option_class_id,
+		(void *)&mlx5_cmd_set_tlv_option_type,
+		(void *)&mlx5_cmd_set_tlv_option_type_id,
+		(void *)&mlx5_cmd_set_tlv_option_len,
+		(void *)&mlx5_cmd_set_tlv_option_option_len,
+		(void *)&mlx5_cmd_set_tlv_option_offset,
+		(void *)&mlx5_cmd_set_tlv_option_off,
+		(void *)&mlx5_cmd_set_tlv_option_sample_len,
+		(void *)&mlx5_cmd_set_tlv_option_length,
+		(void *)&mlx5_cmd_set_tlv_option_class_mode,
+		(void *)&mlx5_cmd_set_tlv_option_cmode,
+		(void *)&mlx5_cmd_set_tlv_option_data,
+		(void *)&mlx5_cmd_set_tlv_option_data_mask,
+		NULL,
+	}
+};
+
+/* Print all GENEVE TLV options which are configured so far. */
+struct mlx5_cmd_list_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t list;
+	cmdline_fixed_string_t tlv_options;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_list_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_list_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_list_tlv_options_list =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_list_tlv_options, list,
+				 "list");
+cmdline_parse_token_string_t mlx5_cmd_list_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_list_tlv_options, tlv_options,
+				 "tlv_options");
+
+static void
+mlx5_cmd_list_tlv_options_parsed(__rte_unused void *parsed_result,
+				 __rte_unused struct cmdline *cl,
+				 __rte_unused void *data)
+{
+	struct rte_pmd_mlx5_geneve_tlv *option;
+	uint8_t i, j;
+
+	printf("ID\tType\tClass\tClass_mode\tLen\tOffset\tSample_len\tData\n");
+	for (i = 0; i < tlv_mng.nb_options; ++i) {
+		option = &tlv_mng.tlv_list[i];
+		printf("%u\t%u\t%u\t%s\t%u\t%u\t%u\t\t", i,
+		       option->option_type, rte_be_to_cpu_16(option->option_class),
+		       mode2string(option->match_on_class_mode),
+		       option->option_len,
+		       option->offset, option->sample_len);
+		for (j = 0; j < option->sample_len; ++j)
+			printf("0x%x ", option->match_data_mask[j]);
+		printf("\n");
+	}
+}
+
+cmdline_parse_inst_t mlx5_cmd_list_tlv_options = {
+	.f = mlx5_cmd_list_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 list tlv_options",
+	.tokens = {
+		(void *)&mlx5_cmd_list_tlv_options_mlx5,
+		(void *)&mlx5_cmd_list_tlv_options_list,
+		(void *)&mlx5_cmd_list_tlv_options_tlv_options,
+		NULL,
+	}
+};
+
+/* Clear all GENEVE TLV options which are configured so far. */
+struct mlx5_cmd_flush_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t flush;
+	cmdline_fixed_string_t tlv_options;
+	cmdline_fixed_string_t max;
+	uint8_t number;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_flush =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, flush,
+				 "flush");
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, tlv_options,
+				 "tlv_options");
+cmdline_parse_token_string_t mlx5_cmd_flush_tlv_options_max =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_flush_tlv_options, max, "max");
+cmdline_parse_token_num_t mlx5_cmd_flush_tlv_options_number =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_flush_tlv_options, number,
+			      RTE_UINT8);
+
+static void
+mlx5_cmd_flush_tlv_options_parsed(void *parsed_result,
+				  __rte_unused struct cmdline *cl,
+				  __rte_unused void *data)
+{
+	struct mlx5_cmd_flush_tlv_options *res = parsed_result;
+	struct rte_pmd_mlx5_geneve_tlv *option;
+	uint8_t nb_options_flush = tlv_mng.nb_options;
+	uint8_t nb_options_left = 0;
+
+	if (strcmp(res->max, "max") == 0 && res->number < tlv_mng.nb_options) {
+		nb_options_left = tlv_mng.nb_options - res->number;
+		nb_options_flush = RTE_MIN(res->number, nb_options_flush);
+	}
+
+	while (tlv_mng.nb_options > nb_options_left) {
+		tlv_mng.nb_options--;
+		option = &tlv_mng.tlv_list[tlv_mng.nb_options];
+		if (option->match_data_mask) {
+			free(option->match_data_mask);
+			option->match_data_mask = NULL;
+		}
+	}
+
+	TESTPMD_LOG(DEBUG, "Flush %u latest configured GENEVE TLV options, "
+		    "current number of options in the list is %u\n",
+		    nb_options_flush, nb_options_left);
+}
+
+cmdline_parse_inst_t mlx5_cmd_flush_tlv_options = {
+	.f = mlx5_cmd_flush_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 flush tlv_options max <nb_options>",
+	.tokens = {
+		(void *)&mlx5_cmd_flush_tlv_options_mlx5,
+		(void *)&mlx5_cmd_flush_tlv_options_flush,
+		(void *)&mlx5_cmd_flush_tlv_options_tlv_options,
+		(void *)&mlx5_cmd_flush_tlv_options_max,
+		(void *)&mlx5_cmd_flush_tlv_options_number,
+		NULL,
+	}
+};
+
+/* Create GENEVE TLV parser using option list which is configured before. */
+struct mlx5_cmd_apply_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t port;
+	portid_t port_id;
+	cmdline_fixed_string_t apply;
+	cmdline_fixed_string_t tlv_options;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_port =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, port,
+				 "port");
+cmdline_parse_token_num_t mlx5_cmd_apply_tlv_options_port_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_apply_tlv_options, port_id,
+			      RTE_UINT16);
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_apply =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, apply,
+				 "apply");
+cmdline_parse_token_string_t mlx5_cmd_apply_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_apply_tlv_options, tlv_options,
+				 "tlv_options");
+
+static void
+mlx5_cmd_apply_tlv_options_parsed(void *parsed_result,
+				  __rte_unused struct cmdline *cl,
+				  __rte_unused void *data)
+{
+	struct mlx5_cmd_apply_tlv_options *res = parsed_result;
+	struct mlx5_port *port;
+	void *handle;
+
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
+		return;
+
+	if (tlv_mng.nb_options == 0) {
+		fprintf(stderr, "The option list is empty, please set options\n");
+		return;
+	}
+
+	handle = rte_pmd_mlx5_create_geneve_tlv_parser(res->port_id,
+						       tlv_mng.tlv_list,
+						       tlv_mng.nb_options);
+	if (handle == NULL) {
+		fprintf(stderr,
+			"Fail to create GENEVE TLV parser, nb_option=%u: %s\n",
+			tlv_mng.nb_options, strerror(rte_errno));
+		return;
+	}
+
+	TESTPMD_LOG(DEBUG, "GENEVE TLV options parser is successfully created:"
+		    " nb_option=%u, handle=%p\n", tlv_mng.nb_options, handle);
+
+	port = &private_port[res->port_id];
+	port->geneve_tlv_parser_handle = handle;
+}
+
+cmdline_parse_inst_t mlx5_cmd_apply_tlv_options = {
+	.f = mlx5_cmd_apply_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 port <port_id> apply tlv_options",
+	.tokens = {
+		(void *)&mlx5_cmd_apply_tlv_options_mlx5,
+		(void *)&mlx5_cmd_apply_tlv_options_port,
+		(void *)&mlx5_cmd_apply_tlv_options_port_id,
+		(void *)&mlx5_cmd_apply_tlv_options_apply,
+		(void *)&mlx5_cmd_apply_tlv_options_tlv_options,
+		NULL,
+	}
+};
+
+/* Destroy GENEVE TLV parser created by apply command. */
+struct mlx5_cmd_destroy_tlv_options {
+	cmdline_fixed_string_t mlx5;
+	cmdline_fixed_string_t port;
+	portid_t port_id;
+	cmdline_fixed_string_t destroy;
+	cmdline_fixed_string_t tlv_options;
+};
+
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_mlx5 =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, mlx5,
+				 "mlx5");
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_port =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, port,
+				 "port");
+cmdline_parse_token_num_t mlx5_cmd_destroy_tlv_options_port_id =
+	TOKEN_NUM_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, port_id,
+			      RTE_UINT16);
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_destroy =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, destroy,
+				 "destroy");
+cmdline_parse_token_string_t mlx5_cmd_destroy_tlv_options_tlv_options =
+	TOKEN_STRING_INITIALIZER(struct mlx5_cmd_destroy_tlv_options, tlv_options,
+				 "tlv_options");
+
+static void
+mlx5_cmd_destroy_tlv_options_parsed(void *parsed_result,
+				    __rte_unused struct cmdline *cl,
+				    __rte_unused void *data)
+{
+	struct mlx5_cmd_destroy_tlv_options *res = parsed_result;
+	struct mlx5_port *port;
+	int ret;
+
+	if (port_id_is_invalid(res->port_id, ENABLED_WARN))
+		return;
+
+	port = &private_port[res->port_id];
+	if (!port->geneve_tlv_parser_handle)
+		return;
+
+	ret = rte_pmd_mlx5_destroy_geneve_tlv_parser(port->geneve_tlv_parser_handle);
+	if (ret < 0) {
+		fprintf(stderr, "Fail to destroy GENEVE TLV parser: %s\n",
+			strerror(-ret));
+		return;
+	}
+
+	TESTPMD_LOG(DEBUG, "GENEVE TLV options parser is successfully released:"
+		    " handle=%p\n", port->geneve_tlv_parser_handle);
+
+	port->geneve_tlv_parser_handle = NULL;
+}
+
+cmdline_parse_inst_t mlx5_cmd_destroy_tlv_options = {
+	.f = mlx5_cmd_destroy_tlv_options_parsed,
+	.data = NULL,
+	.help_str = "mlx5 port <port_id> destroy tlv_options",
+	.tokens = {
+		(void *)&mlx5_cmd_destroy_tlv_options_mlx5,
+		(void *)&mlx5_cmd_destroy_tlv_options_port,
+		(void *)&mlx5_cmd_destroy_tlv_options_port_id,
+		(void *)&mlx5_cmd_destroy_tlv_options_destroy,
+		(void *)&mlx5_cmd_destroy_tlv_options_tlv_options,
+		NULL,
+	}
+};
+
 static struct testpmd_driver_commands mlx5_driver_cmds = {
 	.commands = {
 		{
@@ -712,6 +1233,39 @@ static struct testpmd_driver_commands mlx5_driver_cmds = {
 			.help = "mlx5 set flow_engine (active|standby) [(flag)]\n"
 				"    Set flow_engine to the specific mode with flag.\n\n"
 		},
+		{
+			.ctx = &mlx5_cmd_set_tlv_option,
+			.help = "mlx5 set tlv_option class (class_id) type "
+				"(type_id) len (option_length) offset "
+				"(sample_offset) sample_len (sample_length) "
+				"class_mode (ignored|fixed|matchable) "
+				"data (mask1) [(mask2) [...]]\n"
+				"    Set single GENEVE TLV option inside global list "
+				"using later by apply command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_list_tlv_options,
+			.help = "mlx5 list tlv_options\n"
+				"    Print all GENEVE TLV options which are configured "
+				"so far by TLV option set command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_flush_tlv_options,
+			.help = "mlx5 flush tlv_options [max (number options)]\n"
+				"    Clear all GENEVE TLV options which are configured "
+				"so far by TLV option set command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_apply_tlv_options,
+			.help = "mlx5 port (port_id) apply tlv_options\n"
+				"    Create GENEVE TLV parser using option list which is "
+				"configured before by TLV option set command\n\n",
+		},
+		{
+			.ctx = &mlx5_cmd_destroy_tlv_options,
+			.help = "mlx5 port (port_id) destroy tlv_options\n"
+				"    Destroy GENEVE TLV parser\n\n",
+		},
 		{
 			.ctx = NULL,
 		},
-- 
2.25.1

[PATCH v2 16/23] net/mlx5/hws: increase hl size for future compatibility

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

In some cases we rely on header layout DW offset from FW caps,
this is done in case of future HW which may support current
flex fields natively, for this we must increase header layout to
255 DWs, which is the limit in current definer creation.

Signed-off-by: Alex Vesker <valex@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.h | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h b/drivers/net/mlx5/hws/mlx5dr_definer.h
index 7b7463fc91..f6a3a7ec28 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.h
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
@@ -534,10 +534,8 @@ struct mlx5_ifc_definer_hl_bits {
 	u8 unsupported_free_running_timestamp[0x40];
 	struct mlx5_ifc_definer_hl_flex_parser_bits flex_parser;
 	struct mlx5_ifc_definer_hl_registers_bits registers;
-	/* struct x ib_l3_extended; */
-	/* struct x rwh */
-	/* struct x dcceth */
-	/* struct x dceth */
+	/* Reserved in case header layout on future HW */
+	u8 unsupported_reserved[0xd40];
 };
 
 enum mlx5dr_definer_gtp {
-- 
2.25.1

[PATCH v2 17/23] net/mlx5/hws: support GENEVE matching

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

Add matching for GENEVE tunnel header.

Signed-off-by: Alex Vesker <valex@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 91 +++++++++++++++++++++++++++
 drivers/net/mlx5/hws/mlx5dr_definer.h | 19 ++++++
 2 files changed, 110 insertions(+)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 219bffd3b5..7c0ce805f1 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -12,6 +12,7 @@
 #define UDP_VXLAN_GPE_PORT	4790
 #define UDP_GTPU_PORT	2152
 #define UDP_PORT_MPLS	6635
+#define UDP_GENEVE_PORT 6081
 #define UDP_ROCEV2_PORT	4791
 #define DR_FLOW_LAYER_TUNNEL_NO_MPLS (MLX5_FLOW_LAYER_TUNNEL & ~MLX5_FLOW_LAYER_MPLS)
 
@@ -177,6 +178,9 @@ struct mlx5dr_definer_conv_data {
 	X(SET,		source_qp,		v->queue,		mlx5_rte_flow_item_sq) \
 	X(SET,		tag,			v->data,		rte_flow_item_tag) \
 	X(SET,		metadata,		v->data,		rte_flow_item_meta) \
+	X(SET_BE16,	geneve_protocol,	v->protocol,		rte_flow_item_geneve) \
+	X(SET,		geneve_udp_port,	UDP_GENEVE_PORT,	rte_flow_item_geneve) \
+	X(SET_BE16,	geneve_ctrl,		v->ver_opt_len_o_c_rsvd0,	rte_flow_item_geneve) \
 	X(SET_BE16,	gre_c_ver,		v->c_rsvd0_ver,		rte_flow_item_gre) \
 	X(SET_BE16,	gre_protocol_type,	v->protocol,		rte_flow_item_gre) \
 	X(SET,		ipv4_protocol_gre,	IPPROTO_GRE,		rte_flow_item_gre) \
@@ -688,6 +692,16 @@ mlx5dr_definer_mpls_label_set(struct mlx5dr_definer_fc *fc,
 	memcpy(tag + fc->byte_off + sizeof(v->label_tc_s), &v->ttl, sizeof(v->ttl));
 }
 
+static void
+mlx5dr_definer_geneve_vni_set(struct mlx5dr_definer_fc *fc,
+			      const void *item_spec,
+			      uint8_t *tag)
+{
+	const struct rte_flow_item_geneve *v = item_spec;
+
+	memcpy(tag + fc->byte_off, v->vni, sizeof(v->vni));
+}
+
 static void
 mlx5dr_definer_ib_l4_qp_set(struct mlx5dr_definer_fc *fc,
 			    const void *item_spec,
@@ -2227,6 +2241,79 @@ mlx5dr_definer_conv_item_random(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_geneve(struct mlx5dr_definer_conv_data *cd,
+				struct rte_flow_item *item,
+				int item_idx)
+{
+	const struct rte_flow_item_geneve *m = item->mask;
+	struct mlx5dr_definer_fc *fc;
+	bool inner = cd->tunnel;
+
+	if (inner) {
+		DR_LOG(ERR, "Inner GENEVE item not supported");
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
+	/* In order to match on Geneve we must match on ip_protocol and l4_dport */
+	if (!cd->relaxed) {
+		fc = &cd->fc[DR_CALC_FNAME(IP_PROTOCOL, inner)];
+		if (!fc->tag_set) {
+			fc->item_idx = item_idx;
+			fc->tag_mask_set = &mlx5dr_definer_ones_set;
+			fc->tag_set = &mlx5dr_definer_udp_protocol_set;
+			DR_CALC_SET(fc, eth_l2, l4_type_bwc, inner);
+		}
+
+		fc = &cd->fc[DR_CALC_FNAME(L4_DPORT, inner)];
+		if (!fc->tag_set) {
+			fc->item_idx = item_idx;
+			fc->tag_mask_set = &mlx5dr_definer_ones_set;
+			fc->tag_set = &mlx5dr_definer_geneve_udp_port_set;
+			DR_CALC_SET(fc, eth_l4, destination_port, inner);
+		}
+	}
+
+	if (!m)
+		return 0;
+
+	if (m->rsvd1) {
+		rte_errno = ENOTSUP;
+		return rte_errno;
+	}
+
+	if (m->ver_opt_len_o_c_rsvd0) {
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_CTRL];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_ctrl_set;
+		DR_CALC_SET_HDR(fc, tunnel_header, tunnel_header_0);
+		fc->bit_mask = __mlx5_mask(header_geneve, ver_opt_len_o_c_rsvd);
+		fc->bit_off = __mlx5_dw_bit_off(header_geneve, ver_opt_len_o_c_rsvd);
+	}
+
+	if (m->protocol) {
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_PROTO];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_protocol_set;
+		DR_CALC_SET_HDR(fc, tunnel_header, tunnel_header_0);
+		fc->byte_off += MLX5_BYTE_OFF(header_geneve, protocol_type);
+		fc->bit_mask = __mlx5_mask(header_geneve, protocol_type);
+		fc->bit_off = __mlx5_dw_bit_off(header_geneve, protocol_type);
+	}
+
+	if (!is_mem_zero(m->vni, 3)) {
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_VNI];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_vni_set;
+		DR_CALC_SET_HDR(fc, tunnel_header, tunnel_header_1);
+		fc->bit_mask = __mlx5_mask(header_geneve, vni);
+		fc->bit_off = __mlx5_dw_bit_off(header_geneve, vni);
+	}
+
+	return 0;
+}
+
 static int
 mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
 			 struct mlx5dr_definer_fc *fc,
@@ -2665,6 +2752,10 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			item_flags |= MLX5_FLOW_LAYER_MPLS;
 			cd.mpls_idx++;
 			break;
+		case RTE_FLOW_ITEM_TYPE_GENEVE:
+			ret = mlx5dr_definer_conv_item_geneve(&cd, items, i);
+			item_flags |= MLX5_FLOW_LAYER_GENEVE;
+			break;
 		case RTE_FLOW_ITEM_TYPE_IB_BTH:
 			ret = mlx5dr_definer_conv_item_ib_l4(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_IB_BTH;
diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h b/drivers/net/mlx5/hws/mlx5dr_definer.h
index f6a3a7ec28..eb70dcd39d 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.h
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
@@ -96,6 +96,9 @@ enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_VXLAN_GPE_PROTO,
 	MLX5DR_DEFINER_FNAME_VXLAN_GPE_VNI,
 	MLX5DR_DEFINER_FNAME_VXLAN_GPE_RSVD1,
+	MLX5DR_DEFINER_FNAME_GENEVE_CTRL,
+	MLX5DR_DEFINER_FNAME_GENEVE_PROTO,
+	MLX5DR_DEFINER_FNAME_GENEVE_VNI,
 	MLX5DR_DEFINER_FNAME_SOURCE_QP,
 	MLX5DR_DEFINER_FNAME_REG_0,
 	MLX5DR_DEFINER_FNAME_REG_1,
@@ -627,6 +630,22 @@ struct mlx5_ifc_header_gre_bits {
 	u8 reserved_at_30[0x10];
 };
 
+struct mlx5_ifc_header_geneve_bits {
+	union {
+		u8 ver_opt_len_o_c_rsvd[0x10];
+		struct {
+			u8 version[0x2];
+			u8 opt_len[0x6];
+			u8 o_flag[0x1];
+			u8 c_flag[0x1];
+			u8 reserved_at_a[0x6];
+		};
+	};
+	u8 protocol_type[0x10];
+	u8 vni[0x18];
+	u8 reserved_at_38[0x8];
+};
+
 struct mlx5_ifc_header_icmp_bits {
 	union {
 		u8 icmp_dw1[0x20];
-- 
2.25.1

[PATCH v2 18/23] net/mlx5/hws: support GENEVE options header

[Michael Baum]

From: Alex Vesker <valex@nvidia.com>

Add support for matching multiple GENEVE options. Options
header introduces new complexities since there can be more
than one GENEVE option. This requires us to track the total
DWs used for matching. Current code supports 8DWs for data
including type, class, length. There is also an optimization
to use a special OK bit to reduce the use of limited data DWs.

Signed-off-by: Alex Vesker <valex@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/hws/mlx5dr_definer.c | 147 ++++++++++++++++++++++++--
 drivers/net/mlx5/hws/mlx5dr_definer.h |  24 +++++
 2 files changed, 165 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c b/drivers/net/mlx5/hws/mlx5dr_definer.c
index 7c0ce805f1..79d98bbf78 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.c
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
@@ -118,6 +118,8 @@ struct mlx5dr_definer_conv_data {
 	uint8_t relaxed;
 	uint8_t tunnel;
 	uint8_t mpls_idx;
+	uint8_t geneve_opt_ok_idx;
+	uint8_t geneve_opt_data_idx;
 	enum rte_flow_item_type last_item;
 };
 
@@ -702,6 +704,29 @@ mlx5dr_definer_geneve_vni_set(struct mlx5dr_definer_fc *fc,
 	memcpy(tag + fc->byte_off, v->vni, sizeof(v->vni));
 }
 
+static void
+mlx5dr_definer_geneve_opt_ctrl_set(struct mlx5dr_definer_fc *fc,
+				   const void *item_spec,
+				   uint8_t *tag)
+{
+	const struct rte_flow_item_geneve_opt *v = item_spec;
+	uint32_t dw0 = 0;
+
+	dw0 |= v->option_type << __mlx5_dw_bit_off(header_geneve_opt, type);
+	dw0 |= rte_cpu_to_be_16(v->option_class) << __mlx5_dw_bit_off(header_geneve_opt, class);
+	DR_SET(tag, dw0, fc->byte_off, fc->bit_off, fc->bit_mask);
+}
+
+static void
+mlx5dr_definer_geneve_opt_data_set(struct mlx5dr_definer_fc *fc,
+				   const void *item_spec,
+				   uint8_t *tag)
+{
+	const struct rte_flow_item_geneve_opt *v = item_spec;
+
+	DR_SET_BE32(tag, v->data[fc->extra_data], fc->byte_off, fc->bit_off, fc->bit_mask);
+}
+
 static void
 mlx5dr_definer_ib_l4_qp_set(struct mlx5dr_definer_fc *fc,
 			    const void *item_spec,
@@ -1356,7 +1381,6 @@ mlx5dr_definer_conv_item_port(struct mlx5dr_definer_conv_data *cd,
 	struct mlx5dr_cmd_query_caps *caps = cd->ctx->caps;
 	const struct rte_flow_item_ethdev *m = item->mask;
 	struct mlx5dr_definer_fc *fc;
-	uint8_t bit_offset = 0;
 
 	if (m->port_id) {
 		if (!caps->wire_regc_mask) {
@@ -1365,16 +1389,13 @@ mlx5dr_definer_conv_item_port(struct mlx5dr_definer_conv_data *cd,
 			return rte_errno;
 		}
 
-		while (!(caps->wire_regc_mask & (1 << bit_offset)))
-			bit_offset++;
-
 		fc = &cd->fc[MLX5DR_DEFINER_FNAME_VPORT_REG_C_0];
 		fc->item_idx = item_idx;
 		fc->tag_set = &mlx5dr_definer_vport_set;
 		fc->tag_mask_set = &mlx5dr_definer_ones_set;
 		DR_CALC_SET_HDR(fc, registers, register_c_0);
-		fc->bit_off = bit_offset;
-		fc->bit_mask = caps->wire_regc_mask >> bit_offset;
+		fc->bit_off = __builtin_ctz(caps->wire_regc_mask);
+		fc->bit_mask = caps->wire_regc_mask >> fc->bit_off;
 	} else {
 		DR_LOG(ERR, "Pord ID item mask must specify ID mask");
 		rte_errno = EINVAL;
@@ -2314,6 +2335,116 @@ mlx5dr_definer_conv_item_geneve(struct mlx5dr_definer_conv_data *cd,
 	return 0;
 }
 
+static int
+mlx5dr_definer_conv_item_geneve_opt(struct mlx5dr_definer_conv_data *cd,
+				    struct rte_flow_item *item,
+				    int item_idx)
+{
+	const struct rte_flow_item_geneve_opt *m = item->mask;
+	const struct rte_flow_item_geneve_opt *v = item->spec;
+	struct mlx5_hl_data *hl_ok_bit, *hl_dws;
+	struct mlx5dr_definer_fc *fc;
+	uint8_t num_of_dws, i;
+	bool ok_bit_on_class;
+	int ret;
+
+	if (!m || !(m->option_class || m->option_type || m->data))
+		return 0;
+
+	if (!v || m->option_type != 0xff) {
+		DR_LOG(ERR, "Cannot match geneve opt without valid opt type");
+		goto out_not_supp;
+	}
+
+	if (m->option_class && m->option_class != RTE_BE16(UINT16_MAX)) {
+		DR_LOG(ERR, "Geneve option class has invalid mask");
+		goto out_not_supp;
+	}
+
+	ret = mlx5_get_geneve_hl_data(cd->ctx,
+				      v->option_type,
+				      v->option_class,
+				      &hl_ok_bit,
+				      &num_of_dws,
+				      &hl_dws,
+				      &ok_bit_on_class);
+	if (ret) {
+		DR_LOG(ERR, "Geneve opt type and class %d not supported", v->option_type);
+		goto out_not_supp;
+	}
+
+	if (!ok_bit_on_class && m->option_class) {
+		/* DW0 is used, we will match type, class */
+		if (!num_of_dws || hl_dws[0].dw_mask != UINT32_MAX) {
+			DR_LOG(ERR, "Geneve opt type %d DW0 not supported", v->option_type);
+			goto out_not_supp;
+		}
+
+		if (MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx >
+		    MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_7) {
+			DR_LOG(ERR, "Max match geneve opt DWs reached");
+			goto out_not_supp;
+		}
+
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx++];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_opt_ctrl_set;
+		fc->byte_off = hl_dws[0].dw_offset * DW_SIZE;
+		fc->bit_mask = UINT32_MAX;
+	} else {
+		/* DW0 is not used, we must verify geneve opt type exists in packet */
+		if (!hl_ok_bit->dw_mask) {
+			DR_LOG(ERR, "Geneve opt OK bits not supported");
+			goto out_not_supp;
+		}
+
+		if (MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_0 + cd->geneve_opt_ok_idx >
+		    MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_7) {
+			DR_LOG(ERR, "Max match geneve opt reached");
+			goto out_not_supp;
+		}
+
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_0 + cd->geneve_opt_ok_idx++];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_ones_set;
+		fc->byte_off = hl_ok_bit->dw_offset * DW_SIZE +
+				__builtin_clz(hl_ok_bit->dw_mask) / 8;
+		fc->bit_off = __builtin_ctz(hl_ok_bit->dw_mask);
+		fc->bit_mask = 0x1;
+	}
+
+	for (i = 1; i < num_of_dws; i++) {
+		/* Process each valid geneve option data DW1..N */
+		if (!m->data[i - 1])
+			continue;
+
+		if (hl_dws[i].dw_mask != UINT32_MAX) {
+			DR_LOG(ERR, "Matching Geneve opt data[%d] not supported", i - 1);
+			goto out_not_supp;
+		}
+
+		if (MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx >
+		    MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_7) {
+			DR_LOG(ERR, "Max match geneve options DWs reached");
+			goto out_not_supp;
+		}
+
+		fc = &cd->fc[MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0 + cd->geneve_opt_data_idx++];
+		fc->item_idx = item_idx;
+		fc->tag_set = &mlx5dr_definer_geneve_opt_data_set;
+		fc->byte_off = hl_dws[i].dw_offset * DW_SIZE;
+		fc->bit_mask = m->data[i - 1];
+		/* Use extra_data for data[] set offset */
+		fc->extra_data = i - 1;
+	}
+
+	return 0;
+
+out_not_supp:
+	rte_errno = ENOTSUP;
+	return rte_errno;
+}
+
 static int
 mlx5dr_definer_mt_set_fc(struct mlx5dr_match_template *mt,
 			 struct mlx5dr_definer_fc *fc,
@@ -2756,6 +2887,10 @@ mlx5dr_definer_conv_items_to_hl(struct mlx5dr_context *ctx,
 			ret = mlx5dr_definer_conv_item_geneve(&cd, items, i);
 			item_flags |= MLX5_FLOW_LAYER_GENEVE;
 			break;
+		case RTE_FLOW_ITEM_TYPE_GENEVE_OPT:
+			ret = mlx5dr_definer_conv_item_geneve_opt(&cd, items, i);
+			item_flags |= MLX5_FLOW_LAYER_GENEVE_OPT;
+			break;
 		case RTE_FLOW_ITEM_TYPE_IB_BTH:
 			ret = mlx5dr_definer_conv_item_ib_l4(&cd, items, i);
 			item_flags |= MLX5_FLOW_ITEM_IB_BTH;
diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h b/drivers/net/mlx5/hws/mlx5dr_definer.h
index eb70dcd39d..ced9d9da13 100644
--- a/drivers/net/mlx5/hws/mlx5dr_definer.h
+++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
@@ -146,6 +146,22 @@ enum mlx5dr_definer_fname {
 	MLX5DR_DEFINER_FNAME_OKS2_MPLS2_I,
 	MLX5DR_DEFINER_FNAME_OKS2_MPLS3_I,
 	MLX5DR_DEFINER_FNAME_OKS2_MPLS4_I,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_0,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_1,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_2,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_3,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_4,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_5,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_6,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_OK_7,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_0,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_1,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_2,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_3,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_4,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_5,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_6,
+	MLX5DR_DEFINER_FNAME_GENEVE_OPT_DW_7,
 	MLX5DR_DEFINER_FNAME_IB_L4_OPCODE,
 	MLX5DR_DEFINER_FNAME_IB_L4_QPN,
 	MLX5DR_DEFINER_FNAME_IB_L4_A,
@@ -171,6 +187,7 @@ enum mlx5dr_definer_type {
 struct mlx5dr_definer_fc {
 	uint8_t item_idx;
 	uint8_t is_range;
+	uint16_t extra_data;
 	uint32_t byte_off;
 	int bit_off;
 	uint32_t bit_mask;
@@ -646,6 +663,13 @@ struct mlx5_ifc_header_geneve_bits {
 	u8 reserved_at_38[0x8];
 };
 
+struct mlx5_ifc_header_geneve_opt_bits {
+	u8 class[0x10];
+	u8 type[0x8];
+	u8 reserved[0x3];
+	u8 len[0x5];
+};
+
 struct mlx5_ifc_header_icmp_bits {
 	union {
 		u8 icmp_dw1[0x20];
-- 
2.25.1

[PATCH v2 19/23] net/mlx5: add support for GENEVE and option item in HWS

[Michael Baum]

Add HW steering support for both "RTE_FLOW_ITEM_TYPE_GENEVE" and
"RTE_FLOW_ITEM_TYPE_GENEVE_OPT".

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 doc/guides/nics/mlx5.rst               |  15 ++-
 doc/guides/rel_notes/release_24_03.rst |   5 +
 drivers/net/mlx5/mlx5_flow.h           |  21 +++++
 drivers/net/mlx5/mlx5_flow_geneve.c    | 121 ++++++++++++++++++++++++-
 drivers/net/mlx5/mlx5_flow_hw.c        |  44 ++++++++-
 5 files changed, 199 insertions(+), 7 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 2e5274edb8..62fd27d859 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -337,12 +337,25 @@ Limitations
      - Length
      - Data
 
-  Only one Class/Type/Length Geneve TLV option is supported per shared device.
   Class/Type/Length fields must be specified as well as masks.
   Class/Type/Length specified masks must be full.
   Matching Geneve TLV option without specifying data is not supported.
   Matching Geneve TLV option with ``data & mask == 0`` is not supported.
 
+  In SW steering (``dv_flow_en`` = 1):
+
+     - Only one Class/Type/Length Geneve TLV option is supported per shared
+       device.
+     - Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 0.
+
+  In HW steering (``dv_flow_en`` = 2):
+
+     - Multiple Class/Type/Length Geneve TLV option are supported per physical
+       device. See :ref:`geneve_parser_api` for more information.
+     - Multiple of same Geneve TLV option isn't supported at the same pattern
+       template.
+     - Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+
 - VF: flow rules created on VF devices can only match traffic targeted at the
   configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``).
 
diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
index a1dfea263c..0c8491ce37 100644
--- a/doc/guides/rel_notes/release_24_03.rst
+++ b/doc/guides/rel_notes/release_24_03.rst
@@ -77,6 +77,11 @@ New Features
 
   * Added support for ``RTE_FLOW_ITEM_TYPE_RANDOM`` flow item.
 
+  * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE`` flow item.
+
+  * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE_OPT`` flow item.
+
+
 Removed Items
 -------------
 
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 14806fa78e..0459472fe4 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1338,6 +1338,15 @@ struct mlx5_action_construct_data {
 
 #define MAX_GENEVE_OPTIONS_RESOURCES 7
 
+/* GENEVE TLV options manager structure. */
+struct mlx5_geneve_tlv_options_mng {
+	uint8_t nb_options; /* Number of options inside the template. */
+	struct {
+		uint8_t opt_type;
+		uint16_t opt_class;
+	} options[MAX_GENEVE_OPTIONS_RESOURCES];
+};
+
 /* Flow item template struct. */
 struct rte_flow_pattern_template {
 	LIST_ENTRY(rte_flow_pattern_template) next;
@@ -1357,6 +1366,8 @@ struct rte_flow_pattern_template {
 	 * tag pattern item for representor matching.
 	 */
 	bool implicit_tag;
+	/* Manages all GENEVE TLV options used by this pattern template. */
+	struct mlx5_geneve_tlv_options_mng geneve_opt_mng;
 	uint8_t flex_item; /* flex item index. */
 };
 
@@ -1805,6 +1816,16 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
 			      uint8_t nb_options);
 int mlx5_geneve_tlv_parser_destroy(void *handle);
+int mlx5_flow_geneve_tlv_option_validate(struct mlx5_priv *priv,
+					 const struct rte_flow_item *geneve_opt,
+					 struct rte_flow_error *error);
+
+struct mlx5_geneve_tlv_options_mng;
+int mlx5_geneve_tlv_option_register(struct mlx5_priv *priv,
+				    const struct rte_flow_item_geneve_opt *spec,
+				    struct mlx5_geneve_tlv_options_mng *mng);
+void mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
+					struct mlx5_geneve_tlv_options_mng *mng);
 
 void flow_hw_set_port_info(struct rte_eth_dev *dev);
 void flow_hw_clear_port_info(struct rte_eth_dev *dev);
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index 2d593b70ba..2c8dc39e74 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -152,6 +152,106 @@ mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
 	return -EINVAL;
 }
 
+/**
+ * Calculate total data size.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] geneve_opt
+ *   Pointer to GENEVE option item structure.
+ * @param[out] error
+ *   Pointer to error structure.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_flow_geneve_tlv_option_validate(struct mlx5_priv *priv,
+				     const struct rte_flow_item *geneve_opt,
+				     struct rte_flow_error *error)
+{
+	const struct rte_flow_item_geneve_opt *spec = geneve_opt->spec;
+	const struct rte_flow_item_geneve_opt *mask = geneve_opt->mask;
+	struct mlx5_geneve_tlv_option *option;
+
+	option = mlx5_geneve_tlv_option_get(priv, spec->option_type, spec->option_class);
+	if (option == NULL)
+		return rte_flow_error_set(error, rte_errno,
+					  RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+					  "Unregistered GENEVE option");
+	if (mask->option_type != UINT8_MAX)
+		return rte_flow_error_set(error, EINVAL,
+					  RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+					  "GENEVE option type must be fully masked");
+	if (option->class_mode == 1 && mask->option_class != UINT16_MAX)
+		return rte_flow_error_set(error, EINVAL,
+					  RTE_FLOW_ERROR_TYPE_ITEM, NULL,
+					  "GENEVE option class must be fully masked");
+	return 0;
+}
+
+/**
+ * Register single GENEVE TLV option as used by pattern template.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] spec
+ *   Pointer to GENEVE option item structure.
+ * @param[out] mng
+ *   Pointer to GENEVE option manager.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_geneve_tlv_option_register(struct mlx5_priv *priv,
+				const struct rte_flow_item_geneve_opt *spec,
+				struct mlx5_geneve_tlv_options_mng *mng)
+{
+	struct mlx5_geneve_tlv_option *option;
+
+	option = mlx5_geneve_tlv_option_get(priv, spec->option_type, spec->option_class);
+	if (option == NULL)
+		return -rte_errno;
+	/* Increase the option reference counter. */
+	rte_atomic_fetch_add_explicit(&option->refcnt, 1,
+				      rte_memory_order_relaxed);
+	/* Update the manager with option information. */
+	mng->options[mng->nb_options].opt_type = spec->option_type;
+	mng->options[mng->nb_options].opt_class = spec->option_class;
+	mng->nb_options++;
+	return 0;
+}
+
+/**
+ * Unregister all GENEVE TLV options used by pattern template.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] mng
+ *   Pointer to GENEVE option manager.
+ */
+void
+mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
+				   struct mlx5_geneve_tlv_options_mng *mng)
+{
+	struct mlx5_geneve_tlv_option *option;
+	uint8_t i;
+
+	for (i = 0; i < mng->nb_options; ++i) {
+		option = mlx5_geneve_tlv_option_get(priv,
+						    mng->options[i].opt_type,
+						    mng->options[i].opt_class);
+		MLX5_ASSERT(option != NULL);
+		/* Decrease the option reference counter. */
+		rte_atomic_fetch_sub_explicit(&option->refcnt, 1,
+					      rte_memory_order_relaxed);
+		mng->options[i].opt_type = 0;
+		mng->options[i].opt_class = 0;
+	}
+	mng->nb_options = 0;
+}
+
 /**
  * Create single GENEVE TLV option sample.
  *
@@ -208,6 +308,24 @@ mlx5_geneve_tlv_option_destroy_sample(struct mlx5_geneve_tlv_resource *resource)
 	resource->obj = NULL;
 }
 
+/*
+ * Sample for DW0 are created when one of two conditions is met:
+ * 1. Header is matchable.
+ * 2. This option doesn't configure any data DW.
+ */
+static bool
+should_configure_sample_for_dw0(const struct rte_pmd_mlx5_geneve_tlv *spec)
+{
+	uint8_t i;
+
+	if (spec->match_on_class_mode == 2)
+		return true;
+	for (i = 0; i < spec->sample_len; ++i)
+		if (spec->match_data_mask[i] != 0)
+			return false;
+	return true;
+}
+
 /**
  * Create single GENEVE TLV option.
  *
@@ -237,8 +355,7 @@ mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *s
 	uint8_t i, resource_id = 0;
 	int ret;
 
-	if (spec->match_on_class_mode == 2) {
-		/* Header is matchable, create sample for DW0. */
+	if (should_configure_sample_for_dw0(spec)) {
 		attr.sample_offset = 0;
 		resource = &option->resources[resource_id];
 		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index f06d2ce273..00dc9bc890 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -6828,6 +6828,17 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 							  " attribute");
 			break;
 		}
+		case RTE_FLOW_ITEM_TYPE_GENEVE_OPT:
+		{
+			int ret;
+
+			ret = mlx5_flow_geneve_tlv_option_validate(priv,
+								   &items[i],
+								   error);
+			if (ret < 0)
+				return ret;
+			break;
+		}
 		case RTE_FLOW_ITEM_TYPE_VOID:
 		case RTE_FLOW_ITEM_TYPE_ETH:
 		case RTE_FLOW_ITEM_TYPE_VLAN:
@@ -6840,6 +6851,7 @@ flow_hw_pattern_validate(struct rte_eth_dev *dev,
 		case RTE_FLOW_ITEM_TYPE_VXLAN:
 		case RTE_FLOW_ITEM_TYPE_VXLAN_GPE:
 		case RTE_FLOW_ITEM_TYPE_MPLS:
+		case RTE_FLOW_ITEM_TYPE_GENEVE:
 		case MLX5_RTE_FLOW_ITEM_TYPE_SQ:
 		case RTE_FLOW_ITEM_TYPE_GRE:
 		case RTE_FLOW_ITEM_TYPE_GRE_KEY:
@@ -7008,24 +7020,45 @@ flow_hw_pattern_template_create(struct rte_eth_dev *dev,
 		}
 	}
 	for (i = 0; items[i].type != RTE_FLOW_ITEM_TYPE_END; ++i) {
-		if (items[i].type == RTE_FLOW_ITEM_TYPE_FLEX) {
+		switch (items[i].type) {
+		case RTE_FLOW_ITEM_TYPE_FLEX: {
 			const struct rte_flow_item_flex *spec =
 				(const struct rte_flow_item_flex *)items[i].spec;
 			struct rte_flow_item_flex_handle *handle = spec->handle;
 
 			if (flow_hw_flex_item_acquire(dev, handle, &it->flex_item)) {
-				claim_zero(mlx5dr_match_template_destroy(it->mt));
-				mlx5_free(it);
 				rte_flow_error_set(error, rte_errno,
 						   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
 						   "Failed to acquire flex item");
-				return NULL;
+				goto error;
 			}
+			break;
+		}
+		case RTE_FLOW_ITEM_TYPE_GENEVE_OPT: {
+			const struct rte_flow_item_geneve_opt *spec = items[i].spec;
+
+			if (mlx5_geneve_tlv_option_register(priv, spec,
+							    &it->geneve_opt_mng)) {
+				rte_flow_error_set(error, rte_errno,
+						   RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
+						   "Failed to register GENEVE TLV option");
+				goto error;
+			}
+			break;
+		}
+		default:
+			break;
 		}
 	}
 	__atomic_fetch_add(&it->refcnt, 1, __ATOMIC_RELAXED);
 	LIST_INSERT_HEAD(&priv->flow_hw_itt, it, next);
 	return it;
+error:
+	flow_hw_flex_item_release(dev, &it->flex_item);
+	mlx5_geneve_tlv_options_unregister(priv, &it->geneve_opt_mng);
+	claim_zero(mlx5dr_match_template_destroy(it->mt));
+	mlx5_free(it);
+	return NULL;
 }
 
 /**
@@ -7046,6 +7079,8 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 			      struct rte_flow_pattern_template *template,
 			      struct rte_flow_error *error __rte_unused)
 {
+	struct mlx5_priv *priv = dev->data->dev_private;
+
 	if (__atomic_load_n(&template->refcnt, __ATOMIC_RELAXED) > 1) {
 		DRV_LOG(WARNING, "Item template %p is still in use.",
 			(void *)template);
@@ -7059,6 +7094,7 @@ flow_hw_pattern_template_destroy(struct rte_eth_dev *dev,
 		mlx5_free_srh_flex_parser(dev);
 	LIST_REMOVE(template, next);
 	flow_hw_flex_item_release(dev, &template->flex_item);
+	mlx5_geneve_tlv_options_unregister(priv, &template->geneve_opt_mng);
 	claim_zero(mlx5dr_match_template_destroy(template->mt));
 	mlx5_free(template);
 	return 0;
-- 
2.25.1

[PATCH v2 20/23] net/mlx5: add GENEVE option support for profile 0

[Michael Baum]

Add support for matching and modifying GENEVE option for
FLEX_PARSER_PROFILE_ENABLE=0.
Before this patch it is supported when FLEX_PARSER_PROFILE_ENABLE=8 in
HW steering and when FLEX_PARSER_PROFILE_ENABLE=0 in SW steering.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 doc/guides/nics/mlx5.rst            |   9 ++-
 doc/guides/platform/mlx5.rst        |   6 +-
 drivers/net/mlx5/mlx5_flow_geneve.c | 114 +++++++++++++++++++++-------
 3 files changed, 95 insertions(+), 34 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 62fd27d859..a6d00ecd2b 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -355,6 +355,7 @@ Limitations
      - Multiple of same Geneve TLV option isn't supported at the same pattern
        template.
      - Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+     - Supported also when ``FLEX_PARSER_PROFILE_ENABLE`` = 0 for single DW only.
 
 - VF: flow rules created on VF devices can only match traffic targeted at the
   configured MAC addresses (see ``rte_eth_dev_mac_addr_add()``).
@@ -2445,8 +2446,14 @@ Limitations
 ~~~~~~~~~~~
 
 * Supported only in HW steering (``dv_flow_en`` = 2).
-* Supported only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
 * Supported for FW version **xx.37.0142** and above.
+* Parser creation can be done only for E-Switch manager.
+* Supported for multiple DW only when ``FLEX_PARSER_PROFILE_ENABLE`` = 8.
+* Supported for single DW also when ``FLEX_PARSER_PROFILE_ENABLE`` = 0 with some limitations:
+
+   - ``sample_len`` must be equal to ``option_len`` and not bigger than 1.
+   - ``match_on_class_mode`` different than 1 is not supported.
+   - ``offset`` must be 0.
 
 
 Testpmd driver specific commands
diff --git a/doc/guides/platform/mlx5.rst b/doc/guides/platform/mlx5.rst
index d16508d0da..a66cf778d1 100644
--- a/doc/guides/platform/mlx5.rst
+++ b/doc/guides/platform/mlx5.rst
@@ -536,12 +536,10 @@ Below are some firmware configurations listed.
    or
    FLEX_PARSER_PROFILE_ENABLE=1
 
-- enable Geneve TLV option flow matching in SW steering::
+- enable Geneve TLV option flow matching::
 
    FLEX_PARSER_PROFILE_ENABLE=0
-
-- enable Geneve TLV option flow matching in HW steering::
-
+   or
    FLEX_PARSER_PROFILE_ENABLE=8
 
 - enable GTP flow matching::
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index 2c8dc39e74..f3ee414d02 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -18,6 +18,8 @@
 #define MAX_GENEVE_OPTION_TOTAL_DATA_SIZE \
 		(MAX_GENEVE_OPTION_DATA_SIZE * MAX_GENEVE_OPTIONS_RESOURCES)
 
+#define INVALID_SAMPLE_ID (UINT8_MAX)
+
 /**
  * Single DW inside GENEVE TLV option.
  */
@@ -265,6 +267,8 @@ mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
  *   Pointer to header layout structure to update.
  * @param resource
  *   Pointer to single sample context to fill.
+ * @param sample_id
+ *   The flex parser id for single DW or UINT8_MAX for multiple DWs.
  *
  * @return
  *   0 on success, a negative errno otherwise and rte_errno is set.
@@ -274,7 +278,7 @@ mlx5_geneve_tlv_option_create_sample(void *ctx,
 		      struct mlx5_devx_geneve_tlv_option_attr *attr,
 		      struct mlx5_devx_match_sample_info_query_attr *query_attr,
 		      struct mlx5_hl_data *match_data,
-		      struct mlx5_geneve_tlv_resource *resource)
+		      struct mlx5_geneve_tlv_resource *resource, uint8_t sample_id)
 {
 	struct mlx5_devx_obj *obj;
 	int ret;
@@ -282,7 +286,10 @@ mlx5_geneve_tlv_option_create_sample(void *ctx,
 	obj = mlx5_devx_cmd_create_geneve_tlv_option(ctx, attr);
 	if (obj == NULL)
 		return -rte_errno;
-	ret = mlx5_devx_cmd_query_geneve_tlv_option(ctx, obj, query_attr);
+	if (sample_id == INVALID_SAMPLE_ID)
+		ret = mlx5_devx_cmd_query_geneve_tlv_option(ctx, obj, query_attr);
+	else
+		ret = mlx5_devx_cmd_match_sample_info_query(ctx, sample_id, query_attr);
 	if (ret) {
 		claim_zero(mlx5_devx_cmd_destroy(obj));
 		return ret;
@@ -335,20 +342,22 @@ should_configure_sample_for_dw0(const struct rte_pmd_mlx5_geneve_tlv *spec)
  *   Pointer to user configuration.
  * @param option
  *   Pointer to single GENEVE TLV option to fill.
+ * @param sample_id
+ *   The flex parser id for single DW or UINT8_MAX for multiple DWs.
  *
  * @return
  *   0 on success, a negative errno otherwise and rte_errno is set.
  */
 static int
 mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *spec,
-			      struct mlx5_geneve_tlv_option *option)
+			      struct mlx5_geneve_tlv_option *option, uint8_t sample_id)
 {
 	struct mlx5_devx_geneve_tlv_option_attr attr = {
 		.option_class = spec->option_class,
 		.option_type = spec->option_type,
 		.option_data_len = spec->option_len,
 		.option_class_ignore = spec->match_on_class_mode == 1 ? 0 : 1,
-		.offset_valid = 1,
+		.offset_valid = sample_id == INVALID_SAMPLE_ID ? 1 : 0,
 	};
 	struct mlx5_devx_match_sample_info_query_attr query_attr = {0};
 	struct mlx5_geneve_tlv_resource *resource;
@@ -356,12 +365,14 @@ mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *s
 	int ret;
 
 	if (should_configure_sample_for_dw0(spec)) {
+		MLX5_ASSERT(sample_id == INVALID_SAMPLE_ID);
 		attr.sample_offset = 0;
 		resource = &option->resources[resource_id];
 		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
 							   &query_attr,
 							   &option->match_data[0],
-							   resource);
+							   resource,
+							   INVALID_SAMPLE_ID);
 		if (ret)
 			return ret;
 		resource_id++;
@@ -379,7 +390,8 @@ mlx5_geneve_tlv_option_create(void *ctx, const struct rte_pmd_mlx5_geneve_tlv *s
 		ret = mlx5_geneve_tlv_option_create_sample(ctx, &attr,
 							   &query_attr,
 							   &option->match_data[i],
-							   resource);
+							   resource,
+							   sample_id);
 		if (ret)
 			goto error;
 		resource_id++;
@@ -467,6 +479,8 @@ mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst,
  *   A list of GENEVE TLV options to create parser for them.
  * @param nb_options
  *   The number of options in TLV list.
+ * @param sample_id
+ *   The flex parser id for single DW or UINT8_MAX for multiple DWs.
  *
  * @return
  *   A pointer to GENEVE TLV options parser structure on success,
@@ -475,7 +489,7 @@ mlx5_geneve_tlv_option_copy(struct rte_pmd_mlx5_geneve_tlv *dst,
 static struct mlx5_geneve_tlv_options *
 mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh,
 			       const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
-			       uint8_t nb_options)
+			       uint8_t nb_options, uint8_t sample_id)
 {
 	struct mlx5_geneve_tlv_options *options;
 	const struct rte_pmd_mlx5_geneve_tlv *spec;
@@ -495,7 +509,7 @@ mlx5_geneve_tlv_options_create(struct mlx5_dev_ctx_shared *sh,
 	for (i = 0; i < nb_options; ++i) {
 		spec = &tlv_list[i];
 		ret = mlx5_geneve_tlv_option_create(sh->cdev->ctx, spec,
-						    &options->options[i]);
+						    &options->options[i], sample_id);
 		if (ret < 0)
 			goto error;
 		/* Copy the user list for comparing future configuration. */
@@ -705,6 +719,12 @@ mlx5_is_same_geneve_tlv_options(const struct mlx5_geneve_tlv_options *options,
 	return true;
 }
 
+static inline bool
+multiple_dws_supported(struct mlx5_hca_attr *attr)
+{
+	return attr->geneve_tlv_option_offset && attr->geneve_tlv_sample;
+}
+
 void *
 mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
@@ -715,8 +735,7 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 	struct rte_eth_dev *dev;
 	struct mlx5_priv *priv;
 	struct mlx5_hca_attr *attr;
-	uint8_t total_dws = 0;
-	uint8_t i;
+	uint8_t sample_id;
 
 	/*
 	 * Validate the input before taking a lock and before any memory
@@ -742,34 +761,71 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 		return NULL;
 	}
 	attr = &priv->sh->cdev->config.hca_attr;
-	MLX5_ASSERT(MAX_GENEVE_OPTIONS_RESOURCES <=
-		    attr->max_geneve_tlv_options);
-	if (!attr->geneve_tlv_option_offset || !attr->geneve_tlv_sample ||
-	    !attr->query_match_sample_info || !attr->geneve_tlv_opt) {
-		DRV_LOG(ERR, "Not enough capabilities to support GENEVE TLV parser, maybe old FW version");
+	if (!attr->query_match_sample_info || !attr->geneve_tlv_opt) {
+		DRV_LOG(ERR, "Not enough capabilities to support GENEVE TLV parser, is this device eswitch manager?");
 		rte_errno = ENOTSUP;
 		return NULL;
 	}
-	if (nb_options > MAX_GENEVE_OPTIONS_RESOURCES) {
+	DRV_LOG(DEBUG, "Max DWs supported for GENEVE TLV option is %u",
+		attr->max_geneve_tlv_options);
+	if (nb_options > attr->max_geneve_tlv_options) {
 		DRV_LOG(ERR,
 			"GENEVE TLV option number (%u) exceeds the limit (%u).",
-			nb_options, MAX_GENEVE_OPTIONS_RESOURCES);
+			nb_options, attr->max_geneve_tlv_options);
 		rte_errno = EINVAL;
 		return NULL;
 	}
-	for (i = 0; i < nb_options; ++i) {
-		if (mlx5_geneve_tlv_option_validate(attr, &tlv_list[i]) < 0) {
-			DRV_LOG(ERR, "GENEVE TLV option %u is invalid.", i);
+	if (multiple_dws_supported(attr)) {
+		uint8_t total_dws = 0;
+		uint8_t i;
+
+		MLX5_ASSERT(attr->max_geneve_tlv_options >= MAX_GENEVE_OPTIONS_RESOURCES);
+		for (i = 0; i < nb_options; ++i) {
+			if (mlx5_geneve_tlv_option_validate(attr, &tlv_list[i]) < 0) {
+				DRV_LOG(ERR, "GENEVE TLV option %u is invalid.", i);
+				return NULL;
+			}
+			total_dws += mlx5_geneve_tlv_option_get_nb_dws(&tlv_list[i]);
+		}
+		if (total_dws > MAX_GENEVE_OPTIONS_RESOURCES) {
+			DRV_LOG(ERR,
+				"Total requested DWs (%u) exceeds the limit (%u).",
+				total_dws, MAX_GENEVE_OPTIONS_RESOURCES);
+			rte_errno = EINVAL;
 			return NULL;
 		}
-		total_dws += mlx5_geneve_tlv_option_get_nb_dws(&tlv_list[i]);
-	}
-	if (total_dws > MAX_GENEVE_OPTIONS_RESOURCES) {
-		DRV_LOG(ERR,
-			"Total requested DWs (%u) exceeds the limit (%u).",
-			total_dws, MAX_GENEVE_OPTIONS_RESOURCES);
-		rte_errno = EINVAL;
-		return NULL;
+		/* Multiple DWs is supported, each of the has sample ID given later. */
+		sample_id = INVALID_SAMPLE_ID;
+		DRV_LOG(DEBUG, "GENEVE TLV parser supports multiple DWs, FLEX_PARSER_PROFILE_ENABLE == 8");
+	} else {
+		const struct rte_pmd_mlx5_geneve_tlv *option = &tlv_list[0];
+
+		if (option->offset != 0) {
+			DRV_LOG(ERR,
+				"GENEVE TLV option offset %u is required but not supported.",
+				option->offset);
+			rte_errno = ENOTSUP;
+			return NULL;
+		}
+		if (option->sample_len != option->option_len) {
+			DRV_LOG(ERR,
+				"GENEVE TLV option length (%u) should be equal to sample length (%u).",
+				option->option_len, option->sample_len);
+			rte_errno = ENOTSUP;
+			return NULL;
+		}
+		if (option->match_on_class_mode != 1) {
+			DRV_LOG(ERR,
+				"GENEVE TLV option match_on_class_mode %u is invalid for flex parser profile 0.",
+				option->match_on_class_mode);
+			rte_errno = EINVAL;
+			return NULL;
+		}
+		if (mlx5_geneve_tlv_option_validate(attr, option) < 0)
+			return NULL;
+		/* Single DW is supported, its sample ID is given. */
+		sample_id = attr->geneve_tlv_option_sample_id;
+		DRV_LOG(DEBUG, "GENEVE TLV parser supports only single DW, FLEX_PARSER_PROFILE_ENABLE == 0");
 	}
 	/* Take lock for this physical device and manage the options. */
 	phdev = mlx5_get_locked_physical_device(priv);
@@ -793,7 +849,7 @@ mlx5_geneve_tlv_parser_create(uint16_t port_id,
 		goto exit;
 	}
 	/* Create GENEVE TLV options for this physical device. */
-	options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options);
+	options = mlx5_geneve_tlv_options_create(priv->sh, tlv_list, nb_options, sample_id);
 	if (!options) {
 		mlx5_unlock_physical_device();
 		return NULL;
-- 
2.25.1

[PATCH v2 21/23] net/mlx5: add GENEVE option support for group 0

[Michael Baum]

Add support for HWS GENEVE options for flex parser profile 0 and group
0.

This patch avoids parser creation during matcher/flow preparation for HW
steering (MLX5_SET_MATCHER_HS) and removes some logic done in
"flow_dev_geneve_tlv_option_resource_*()" functions when dv_flow_en=2.

After this change, those functions became static and they were removed
from header file.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         |  8 +-------
 drivers/net/mlx5/mlx5_flow.h    |  4 ----
 drivers/net/mlx5/mlx5_flow_dv.c | 24 +++++++++++-------------
 3 files changed, 12 insertions(+), 24 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 5f8af31aea..881c42a97a 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2049,13 +2049,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 	} while (++i <= sh->bond.n_port);
 	if (sh->td)
 		claim_zero(mlx5_devx_cmd_destroy(sh->td));
-#ifdef HAVE_MLX5_HWS_SUPPORT
-	/* HWS manages geneve_tlv_option resource as global. */
-	if (sh->config.dv_flow_en == 2)
-		flow_dev_geneve_tlv_option_resource_release(sh);
-	else
-#endif
-		MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL);
+	MLX5_ASSERT(sh->geneve_tlv_option_resource == NULL);
 	pthread_mutex_destroy(&sh->txpp.mutex);
 	mlx5_lwm_unset(sh);
 	mlx5_physical_device_destroy(sh->phdev);
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 0459472fe4..655e4d3d86 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -2832,10 +2832,6 @@ void flow_hw_grp_clone_free_cb(void *tool_ctx, struct mlx5_list_entry *entry);
 
 struct mlx5_aso_age_action *flow_aso_age_get_by_idx(struct rte_eth_dev *dev,
 						    uint32_t age_idx);
-int flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
-					     const struct rte_flow_item *item,
-					     struct rte_flow_error *error);
-void flow_dev_geneve_tlv_option_resource_release(struct mlx5_dev_ctx_shared *sh);
 
 void flow_release_workspace(void *data);
 int mlx5_flow_os_init_workspace_once(void);
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index ae10981165..857813368b 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -9998,7 +9998,7 @@ flow_dv_translate_item_geneve(void *key, const struct rte_flow_item *item,
 /**
  * Create Geneve TLV option resource.
  *
- * @param dev[in, out]
+ * @param[in, out] dev
  *   Pointer to rte_eth_dev structure.
  * @param[in] item
  *   Flow pattern to translate.
@@ -10008,8 +10008,7 @@ flow_dv_translate_item_geneve(void *key, const struct rte_flow_item *item,
  * @return
  *   0 on success otherwise -errno and errno is set.
  */
-
-int
+static int
 flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 					     const struct rte_flow_item *item,
 					     struct rte_flow_error *error)
@@ -10022,6 +10021,7 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 	const struct rte_flow_item_geneve_opt *geneve_opt_v = item->spec;
 	int ret = 0;
 
+	MLX5_ASSERT(sh->config.dv_flow_en == 1);
 	if (!geneve_opt_v)
 		return -1;
 	rte_spinlock_lock(&sh->geneve_tlv_opt_sl);
@@ -10032,13 +10032,8 @@ flow_dev_geneve_tlv_option_resource_register(struct rte_eth_dev *dev,
 			geneve_opt_v->option_type &&
 			geneve_opt_resource->length ==
 			geneve_opt_v->option_len) {
-			/*
-			 * We already have GENEVE TLV option obj allocated.
-			 * Increasing refcnt only in SWS. HWS uses it as global.
-			 */
-			if (priv->sh->config.dv_flow_en == 1)
-				__atomic_fetch_add(&geneve_opt_resource->refcnt, 1,
-						   __ATOMIC_RELAXED);
+			__atomic_fetch_add(&geneve_opt_resource->refcnt, 1,
+					   __ATOMIC_RELAXED);
 		} else {
 			ret = rte_flow_error_set(error, ENOMEM,
 				RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
@@ -10117,8 +10112,11 @@ flow_dv_translate_item_geneve_opt(struct rte_eth_dev *dev, void *key,
 		return -1;
 	MLX5_ITEM_UPDATE(item, key_type, geneve_opt_v, geneve_opt_m,
 			 &rte_flow_item_geneve_opt_mask);
-	/* Register resource requires item spec. */
-	if (key_type & MLX5_SET_MATCHER_V) {
+	/*
+	 * Register resource requires item spec for SW steering,
+	 * for HW steering resources is registered explicitly by user.
+	 */
+	if (key_type & MLX5_SET_MATCHER_SW_V) {
 		ret = flow_dev_geneve_tlv_option_resource_register(dev, item,
 								   error);
 		if (ret) {
@@ -15900,7 +15898,7 @@ flow_dv_dest_array_resource_release(struct rte_eth_dev *dev,
 				    &resource->entry);
 }
 
-void
+static void
 flow_dev_geneve_tlv_option_resource_release(struct mlx5_dev_ctx_shared *sh)
 {
 	struct mlx5_geneve_tlv_option_resource *geneve_opt_resource =
-- 
2.25.1

[PATCH v2 22/23] net/mlx5: add support for GENEVE VNI modify field

[Michael Baum]

Add support for GENEVE VNI field modification.
The support is only using HW steering.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 doc/guides/nics/mlx5.rst               |  6 +++++-
 doc/guides/rel_notes/release_24_03.rst |  1 +
 drivers/net/mlx5/mlx5_flow_dv.c        |  4 +---
 drivers/net/mlx5/mlx5_flow_hw.c        | 12 ++++++++++--
 4 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index a6d00ecd2b..0e3d0bc099 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -585,8 +585,12 @@ Limitations
   - Modification of an arbitrary place in a packet via the special ``RTE_FLOW_FIELD_START`` Field ID is not supported.
   - Modification of the MPLS header is supported only in HWS and only to copy from,
     the encapsulation level is always 0.
-  - Modification of the 802.1Q Tag, VXLAN Network or GENEVE Network ID's is not supported.
   - Modify field action using ``RTE_FLOW_FIELD_RANDOM`` is not supported.
+  - Modification of the 802.1Q Tag is not supported.
+  - Modification of VXLAN Network or GENEVE Network ID's is supported only for HW steering.
+  - Modification of GENEVE Network ID's is not supported when configured
+    ``FLEX_PARSER_PROFILE_ENABLE`` supports Geneve TLV options.
+    See :ref:`mlx5_firmware_config` for more flex parser information.
   - Encapsulation levels are not supported, can modify outermost header fields only.
   - Offsets cannot skip past the boundary of a field.
   - If the field type is ``RTE_FLOW_FIELD_MAC_TYPE``
diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
index 0c8491ce37..8b14ab8986 100644
--- a/doc/guides/rel_notes/release_24_03.rst
+++ b/doc/guides/rel_notes/release_24_03.rst
@@ -80,6 +80,7 @@ New Features
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE`` flow item.
 
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE_OPT`` flow item.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_VNI`` flow action.
 
 
 Removed Items
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 857813368b..5d5e2cadf6 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1957,6 +1957,7 @@ mlx5_flow_field_id_to_modify_info
 			info[idx].offset = off_be;
 		break;
 	case RTE_FLOW_FIELD_VXLAN_VNI:
+	case RTE_FLOW_FIELD_GENEVE_VNI:
 		MLX5_ASSERT(data->offset + width <= 24);
 		/* VNI is on bits 31-8 of TUNNEL_HDR_DW_1. */
 		off_be = 24 - (data->offset + width) + 8;
@@ -1967,9 +1968,6 @@ mlx5_flow_field_id_to_modify_info
 		else
 			info[idx].offset = off_be;
 		break;
-	case RTE_FLOW_FIELD_GENEVE_VNI:
-		/* not supported yet*/
-		break;
 	case RTE_FLOW_FIELD_GTP_TEID:
 		MLX5_ASSERT(data->offset + width <= 32);
 		off_be = 32 - (data->offset + width);
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 00dc9bc890..687d809b1b 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -4990,6 +4990,8 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 {
 	const struct rte_flow_action_modify_field *action_conf = action->conf;
 	const struct rte_flow_action_modify_field *mask_conf = mask->conf;
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_hca_attr *attr = &priv->sh->cdev->config.hca_attr;
 	int ret;
 
 	if (!mask_conf)
@@ -5089,10 +5091,16 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
 				"modifying random value is not supported");
-	if (flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_VNI))
+	/**
+	 * Geneve VNI modification is supported only when Geneve header is
+	 * parsed natively. When GENEVE options are supported, they both Geneve
+	 * and options headers are parsed as a flex parser.
+	 */
+	if (flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_VNI) &&
+	    attr->geneve_tlv_opt)
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"modifying Geneve VNI is not supported");
+				"modifying Geneve VNI is not supported when GENEVE opt is supported");
 	/* Due to HW bug, tunnel MPLS header is read only. */
 	if (action_conf->dst.field == RTE_FLOW_FIELD_MPLS)
 		return rte_flow_error_set(error, EINVAL,
-- 
2.25.1

[PATCH v2 23/23] net/mlx5: add support for modify GENEVE option header

[Michael Baum]

Add support for GENEVE option fields modification.
Only fields configured in parser creation can be modified.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Suanming Mou <suanmingm@nvidia.com>
---
 doc/guides/nics/mlx5.rst               |   4 +
 doc/guides/rel_notes/release_24_03.rst |   3 +
 drivers/net/mlx5/mlx5_flow.h           |  21 +++++
 drivers/net/mlx5/mlx5_flow_dv.c        |  78 ++++++++++++++++-
 drivers/net/mlx5/mlx5_flow_geneve.c    | 117 +++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_flow_hw.c        |  71 ++++++++++-----
 6 files changed, 268 insertions(+), 26 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index 0e3d0bc099..6e1e2df79a 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -591,6 +591,10 @@ Limitations
   - Modification of GENEVE Network ID's is not supported when configured
     ``FLEX_PARSER_PROFILE_ENABLE`` supports Geneve TLV options.
     See :ref:`mlx5_firmware_config` for more flex parser information.
+  - Modification of GENEVE TLV option fields is supported only for HW steering.
+    Only DWs configured in :ref:`parser creation <geneve_parser_api>` can be modified,
+    'type' and 'class' fields can be modified when ``match_on_class_mode=2``.
+  - Modification of GENEVE TLV option data supports one DW per action.
   - Encapsulation levels are not supported, can modify outermost header fields only.
   - Offsets cannot skip past the boundary of a field.
   - If the field type is ``RTE_FLOW_FIELD_MAC_TYPE``
diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_notes/release_24_03.rst
index 8b14ab8986..73515aad1e 100644
--- a/doc/guides/rel_notes/release_24_03.rst
+++ b/doc/guides/rel_notes/release_24_03.rst
@@ -81,6 +81,9 @@ New Features
 
   * Added HW steering support for ``RTE_FLOW_ITEM_TYPE_GENEVE_OPT`` flow item.
   * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_VNI`` flow action.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_OPT_TYPE`` flow action.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_OPT_CLASS`` flow action.
+  * Added HW steering support for modify field ``RTE_FLOW_FIELD_GENEVE_OPT_DATA`` flow action.
 
 
 Removed Items
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 655e4d3d86..c9cc942d80 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -1811,6 +1811,25 @@ mlx5_get_geneve_hl_data(const void *dr_ctx, uint8_t type, uint16_t class,
 			struct mlx5_hl_data ** const hl_dws,
 			bool *ok_bit_on_class);
 
+/**
+ * Get modify field ID for single DW inside configured GENEVE TLV option.
+ *
+ * @param[in] dr_ctx
+ *   Pointer to HW steering DR context.
+ * @param[in] type
+ *   GENEVE TLV option type.
+ * @param[in] class
+ *   GENEVE TLV option class.
+ * @param[in] dw_offset
+ *   Offset of DW inside the option.
+ *
+ * @return
+ *   Modify field ID on success, negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_get_geneve_option_modify_field_id(const void *dr_ctx, uint8_t type,
+				       uint16_t class, uint8_t dw_offset);
+
 void *
 mlx5_geneve_tlv_parser_create(uint16_t port_id,
 			      const struct rte_pmd_mlx5_geneve_tlv tlv_list[],
@@ -1819,6 +1838,8 @@ int mlx5_geneve_tlv_parser_destroy(void *handle);
 int mlx5_flow_geneve_tlv_option_validate(struct mlx5_priv *priv,
 					 const struct rte_flow_item *geneve_opt,
 					 struct rte_flow_error *error);
+int mlx5_geneve_opt_modi_field_get(struct mlx5_priv *priv,
+				   const struct rte_flow_action_modify_data *data);
 
 struct mlx5_geneve_tlv_options_mng;
 int mlx5_geneve_tlv_option_register(struct mlx5_priv *priv,
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 5d5e2cadf6..6998be107f 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -1465,6 +1465,21 @@ mlx5_mpls_modi_field_get(const struct rte_flow_action_modify_data *data)
 	return MLX5_MODI_IN_MPLS_LABEL_0 + data->tag_index;
 }
 
+static __rte_always_inline int
+flow_geneve_opt_modi_field_get(struct mlx5_priv *priv,
+			       const struct rte_flow_action_modify_data *data)
+{
+#ifdef HAVE_MLX5_HWS_SUPPORT
+	return mlx5_geneve_opt_modi_field_get(priv, data);
+#else
+	(void)priv;
+	(void)data;
+	DRV_LOG(ERR, "GENEVE option modification is not supported.");
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+#endif
+}
+
 static void
 mlx5_modify_flex_item(const struct rte_eth_dev *dev,
 		      const struct mlx5_flex_item *flex,
@@ -1604,9 +1619,11 @@ mlx5_flow_field_id_to_modify_info
 		 const struct rte_flow_attr *attr, struct rte_flow_error *error)
 {
 	struct mlx5_priv *priv = dev->data->dev_private;
+	enum mlx5_modification_field modi_id;
 	uint32_t idx = 0;
 	uint32_t off_be = 0;
 	uint32_t length = 0;
+
 	switch ((int)data->field) {
 	case RTE_FLOW_FIELD_START:
 		/* not supported yet */
@@ -1968,6 +1985,48 @@ mlx5_flow_field_id_to_modify_info
 		else
 			info[idx].offset = off_be;
 		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_TYPE:
+		MLX5_ASSERT(data->offset + width <= 8);
+		modi_id = flow_geneve_opt_modi_field_get(priv, data);
+		if (modi_id < 0)
+			return;
+		/* Type is on bits 16-8 of GENEVE option header (DW0). */
+		off_be = 32 - (16 + data->offset + width);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
+		if (mask)
+			mask[idx] = flow_modify_info_mask_32(width, off_be);
+		else
+			info[idx].offset = off_be;
+		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_CLASS:
+		MLX5_ASSERT(data->offset + width <= 16);
+		modi_id = flow_geneve_opt_modi_field_get(priv, data);
+		if (modi_id < 0)
+			return;
+		/* Class is on bits 31-16 of GENEVE option header (DW0). */
+		off_be = 32 - (data->offset + width);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
+		if (mask)
+			mask[idx] = flow_modify_info_mask_32(width, off_be);
+		else
+			info[idx].offset = off_be;
+		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_DATA:
+		if ((data->offset % 32) + width > 32) {
+			DRV_LOG(ERR, "Geneve TLV option data is per DW.");
+			return;
+		}
+		modi_id = flow_geneve_opt_modi_field_get(priv, data);
+		if (modi_id < 0)
+			return;
+		/* Use offset inside DW. */
+		off_be = 32 - ((data->offset % 32) + width);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
+		if (mask)
+			mask[idx] = flow_modify_info_mask_32(width, off_be);
+		else
+			info[idx].offset = off_be;
+		break;
 	case RTE_FLOW_FIELD_GTP_TEID:
 		MLX5_ASSERT(data->offset + width <= 32);
 		off_be = 32 - (data->offset + width);
@@ -1981,8 +2040,8 @@ mlx5_flow_field_id_to_modify_info
 	case RTE_FLOW_FIELD_MPLS:
 		MLX5_ASSERT(data->offset + width <= 32);
 		off_be = 32 - (data->offset + width);
-		info[idx] = (struct field_modify_info){4, 0,
-					mlx5_mpls_modi_field_get(data)};
+		modi_id = mlx5_mpls_modi_field_get(data);
+		info[idx] = (struct field_modify_info){4, 0, modi_id};
 		if (mask)
 			mask[idx] = flow_modify_info_mask_32(width, off_be);
 		else
@@ -5488,6 +5547,21 @@ flow_dv_validate_action_modify_field(struct rte_eth_dev *dev,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
 				"modifications of the GENEVE Network"
 				" Identifier is not supported");
+	if (dst_data->field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE ||
+	    src_data->field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifications of the GENEVE option type is not supported");
+	if (dst_data->field == RTE_FLOW_FIELD_GENEVE_OPT_CLASS ||
+	    src_data->field == RTE_FLOW_FIELD_GENEVE_OPT_CLASS)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifications of the GENEVE option class is not supported");
+	if (dst_data->field == RTE_FLOW_FIELD_GENEVE_OPT_DATA ||
+	    src_data->field == RTE_FLOW_FIELD_GENEVE_OPT_DATA)
+		return rte_flow_error_set(error, ENOTSUP,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifications of the GENEVE option data is not supported");
 	if (dst_data->field == RTE_FLOW_FIELD_MPLS ||
 	    src_data->field == RTE_FLOW_FIELD_MPLS)
 		return rte_flow_error_set(error, ENOTSUP,
diff --git a/drivers/net/mlx5/mlx5_flow_geneve.c b/drivers/net/mlx5/mlx5_flow_geneve.c
index f3ee414d02..d5847a60e9 100644
--- a/drivers/net/mlx5/mlx5_flow_geneve.c
+++ b/drivers/net/mlx5/mlx5_flow_geneve.c
@@ -254,6 +254,123 @@ mlx5_geneve_tlv_options_unregister(struct mlx5_priv *priv,
 	mng->nb_options = 0;
 }
 
+/**
+ * Get single DW resource from given option.
+ *
+ * @param option
+ *   Pointer to single GENEVE TLV option.
+ * @param offset
+ *   Offset of DW related to option start.
+ *
+ * @return
+ *   DW resource on success, NULL otherwise and rte_errno is set.
+ */
+static struct mlx5_geneve_tlv_resource *
+mlx5_geneve_tlv_option_get_resource_by_offset(struct mlx5_geneve_tlv_option *option,
+					      uint8_t offset)
+{
+	uint8_t i;
+
+	for (i = 0; option->resources[i].obj != NULL; ++i) {
+		if (option->resources[i].offset < offset)
+			continue;
+		if (option->resources[i].offset == offset)
+			return &option->resources[i];
+		break;
+	}
+	DRV_LOG(ERR, "The DW in offset %u wasn't configured.", offset);
+	rte_errno = EINVAL;
+	return NULL;
+}
+
+int
+mlx5_get_geneve_option_modify_field_id(const void *dr_ctx, uint8_t type,
+				       uint16_t class, uint8_t dw_offset)
+{
+	uint16_t port_id;
+
+	MLX5_ETH_FOREACH_DEV(port_id, NULL) {
+		struct mlx5_priv *priv;
+		struct mlx5_geneve_tlv_option *option;
+		struct mlx5_geneve_tlv_resource *resource;
+
+		priv = rte_eth_devices[port_id].data->dev_private;
+		if (priv->dr_ctx != dr_ctx)
+			continue;
+		/* Find specific option inside list. */
+		option = mlx5_geneve_tlv_option_get(priv, type, class);
+		if (option == NULL)
+			return -rte_errno;
+		/* Find specific FW object inside option resources. */
+		resource = mlx5_geneve_tlv_option_get_resource_by_offset(option,
+									 dw_offset);
+		if (resource == NULL)
+			return -rte_errno;
+		return resource->modify_field;
+	}
+	DRV_LOG(ERR, "DR CTX %p doesn't belong to any DPDK port.", dr_ctx);
+	rte_errno = EINVAL;
+	return -rte_errno;
+}
+
+/**
+ * Get modify field ID for single DW inside configured GENEVE TLV option.
+ *
+ * @param[in] priv
+ *   Pointer to port's private data.
+ * @param[in] data
+ *   Pointer to modify field data structure.
+ *
+ * @return
+ *   Modify field ID on success, negative errno otherwise and rte_errno is set.
+ */
+int
+mlx5_geneve_opt_modi_field_get(struct mlx5_priv *priv,
+			       const struct rte_flow_action_modify_data *data)
+{
+	uint16_t class = data->class_id;
+	uint8_t type = data->type;
+	struct mlx5_geneve_tlv_option *option;
+	struct mlx5_geneve_tlv_resource *resource;
+	uint8_t offset;
+
+	option = mlx5_geneve_tlv_option_get(priv, type, class);
+	if (option == NULL)
+		return -rte_errno;
+	switch (data->field) {
+	case RTE_FLOW_FIELD_GENEVE_OPT_TYPE:
+	case RTE_FLOW_FIELD_GENEVE_OPT_CLASS:
+		if (!option->match_data[0].dw_mask) {
+			DRV_LOG(ERR, "DW0 isn't configured");
+			rte_errno = EINVAL;
+			return -rte_errno;
+		}
+		resource = &option->resources[0];
+		MLX5_ASSERT(resource->offset == 0);
+		break;
+	case RTE_FLOW_FIELD_GENEVE_OPT_DATA:
+		/*
+		 * Convert offset twice:
+		 *  - First conversion from bit offset to DW offset.
+		 *  - Second conversion is to be related to data start instead
+		 *    of option start.
+		 */
+		offset = (data->offset >> 5) + 1;
+		resource = mlx5_geneve_tlv_option_get_resource_by_offset(option,
+									 offset);
+		break;
+	default:
+		DRV_LOG(ERR,
+			"Field ID %u doesn't describe GENEVE option header.",
+			data->field);
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	if (resource == NULL)
+		return -rte_errno;
+	return resource->modify_field;
+}
+
 /**
  * Create single GENEVE TLV option sample.
  *
diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c
index 687d809b1b..7510715189 100644
--- a/drivers/net/mlx5/mlx5_flow_hw.c
+++ b/drivers/net/mlx5/mlx5_flow_hw.c
@@ -1257,10 +1257,12 @@ flow_hw_modify_field_compile(struct rte_eth_dev *dev,
 			else
 				value = rte_cpu_to_be_32(value);
 			item.spec = &value;
-		} else if (conf->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI) {
+		} else if (conf->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI ||
+			   conf->dst.field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE) {
 			/*
-			 * QFI is passed as an uint8_t integer, but it is accessed through
-			 * a 2nd least significant byte of a 32-bit field in modify header command.
+			 * Both QFI and Geneve option type are passed as an uint8_t integer,
+			 * but it is accessed through a 2nd least significant byte of a 32-bit
+			 * field in modify header command.
 			 */
 			value = *(const uint8_t *)item.spec;
 			value = rte_cpu_to_be_32(value << 8);
@@ -2829,12 +2831,14 @@ flow_hw_modify_field_construct(struct mlx5_hw_q_job *job,
 			*value_p = rte_cpu_to_be_32(*value_p << 16);
 		else
 			*value_p = rte_cpu_to_be_32(*value_p);
-	} else if (mhdr_action->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI) {
+	} else if (mhdr_action->dst.field == RTE_FLOW_FIELD_GTP_PSC_QFI ||
+		   mhdr_action->dst.field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE) {
 		uint32_t tmp;
 
 		/*
-		 * QFI is passed as an uint8_t integer, but it is accessed through
-		 * a 2nd least significant byte of a 32-bit field in modify header command.
+		 * Both QFI and Geneve option type are passed as an uint8_t integer,
+		 * but it is accessed through a 2nd least significant byte of a 32-bit
+		 * field in modify header command.
 		 */
 		tmp = values[0];
 		value_p = (unaligned_uint32_t *)values;
@@ -4955,6 +4959,14 @@ flow_hw_modify_field_is_used(const struct rte_flow_action_modify_field *action,
 	return action->src.field == field || action->dst.field == field;
 }
 
+static bool
+flow_hw_modify_field_is_geneve_opt(enum rte_flow_field_id field)
+{
+	return field == RTE_FLOW_FIELD_GENEVE_OPT_TYPE ||
+	       field == RTE_FLOW_FIELD_GENEVE_OPT_CLASS ||
+	       field == RTE_FLOW_FIELD_GENEVE_OPT_DATA;
+}
+
 static bool
 flow_hw_modify_field_is_add_dst_valid(const struct rte_flow_action_modify_field *conf)
 {
@@ -5015,15 +5027,17 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 	ret = flow_validate_modify_field_level(&action_conf->dst, error);
 	if (ret)
 		return ret;
-	if (action_conf->dst.tag_index &&
-	    !flow_modify_field_support_tag_array(action_conf->dst.field))
-		return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"destination tag index is not supported");
-	if (action_conf->dst.class_id)
-		return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"destination class id is not supported");
+	if (!flow_hw_modify_field_is_geneve_opt(action_conf->dst.field)) {
+		if (action_conf->dst.tag_index &&
+		    !flow_modify_field_support_tag_array(action_conf->dst.field))
+			return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"destination tag index is not supported");
+		if (action_conf->dst.class_id)
+			return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"destination class id is not supported");
+	}
 	if (mask_conf->dst.level != UINT8_MAX)
 		return rte_flow_error_set(error, EINVAL,
 			RTE_FLOW_ERROR_TYPE_ACTION, action,
@@ -5038,15 +5052,17 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 				"destination field mask and template are not equal");
 	if (action_conf->src.field != RTE_FLOW_FIELD_POINTER &&
 	    action_conf->src.field != RTE_FLOW_FIELD_VALUE) {
-		if (action_conf->src.tag_index &&
-		    !flow_modify_field_support_tag_array(action_conf->src.field))
-			return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"source tag index is not supported");
-		if (action_conf->src.class_id)
-			return rte_flow_error_set(error, EINVAL,
-				RTE_FLOW_ERROR_TYPE_ACTION, action,
-				"source class id is not supported");
+		if (!flow_hw_modify_field_is_geneve_opt(action_conf->src.field)) {
+			if (action_conf->src.tag_index &&
+			    !flow_modify_field_support_tag_array(action_conf->src.field))
+				return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"source tag index is not supported");
+			if (action_conf->src.class_id)
+				return rte_flow_error_set(error, EINVAL,
+					RTE_FLOW_ERROR_TYPE_ACTION, action,
+					"source class id is not supported");
+		}
 		if (mask_conf->src.level != UINT8_MAX)
 			return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
@@ -5101,6 +5117,13 @@ flow_hw_validate_action_modify_field(struct rte_eth_dev *dev,
 		return rte_flow_error_set(error, EINVAL,
 				RTE_FLOW_ERROR_TYPE_ACTION, action,
 				"modifying Geneve VNI is not supported when GENEVE opt is supported");
+	if (priv->tlv_options == NULL &&
+	    (flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_OPT_TYPE) ||
+	     flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_OPT_CLASS) ||
+	     flow_hw_modify_field_is_used(action_conf, RTE_FLOW_FIELD_GENEVE_OPT_DATA)))
+		return rte_flow_error_set(error, EINVAL,
+				RTE_FLOW_ERROR_TYPE_ACTION, action,
+				"modifying Geneve TLV option is supported only after parser configuration");
 	/* Due to HW bug, tunnel MPLS header is read only. */
 	if (action_conf->dst.field == RTE_FLOW_FIELD_MPLS)
 		return rte_flow_error_set(error, EINVAL,
-- 
2.25.1

RE: [PATCH v2 00/23] net/mlx5: support Geneve and options for HWS

[Raslan Darawsheh]

Hi,


> -----Original Message-----
> From: Michael Baum <michaelba@nvidia.com>
> Sent: Thursday, January 25, 2024 3:30 PM
> To: dev@dpdk.org
> Cc: Matan Azrad <matan@nvidia.com>; Raslan Darawsheh
> <rasland@nvidia.com>; Dariusz Sosnowski <dsosnowski@nvidia.com>; Slava
> Ovsiienko <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>;
> Suanming Mou <suanmingm@nvidia.com>
> Subject: [PATCH v2 00/23] net/mlx5: support Geneve and options for HWS
> 
> Add HWS support for both GENEVE and GENEVE TLV option headers.
> This patchset supports:
> 
>  - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE" flow item.
>  - Add HW support for "RTE_FLOW_ITEM_TYPE_GENEVE_OPT" flow item.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_VNI" for modify field flow
>    action.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_TYPE" for modify
> field
>    flow action.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_CLASS" for modify
> field
>    flow action.
>  - Add HW support for "RTE_FLOW_FIELD_GENEVE_OPT_DATA" for modify
> field
>    flow action.
> 
> The GENEVE TLV options support using flex parser.
> The profile should be specified to either 8 for multiple option or 0 for single
> option.
> A new API is added to create the GENEVE option parser before using it in
> templates API.
> 
> v2:
>  - Rebase.
>  - Add "Acked-by" from v1.
> 
> Alex Vesker (4):
>   net/mlx5/hws: fix tunnel protocol checks
>   net/mlx5/hws: increase hl size for future compatibility
>   net/mlx5/hws: support GENEVE matching
>   net/mlx5/hws: support GENEVE options header
> 
> Michael Baum (19):
>   common/mlx5: fix duplicate read of general capabilities
>   common/mlx5: fix query sample info capability
>   net/mlx5: remove GENEVE options length limitation
>   net/mlx5: fix GENEVE option item translation
>   common/mlx5: add system image GUID attribute
>   common/mlx5: add GENEVE TLV option attribute structure
>   common/mlx5: add PRM attribute for TLV sample
>   common/mlx5: add sample info query syndrome into error log
>   common/mlx5: query GENEVE option sample ID from HCA attr
>   common/mlx5: add function to query GENEVE TLV option
>   net/mlx5: add physical device handle
>   net/mlx5: add GENEVE TLV options parser API
>   net/mlx5: add API to expose GENEVE option FW information
>   net/mlx5: add testpmd support for GENEVE TLV parser
>   net/mlx5: add support for GENEVE and option item in HWS
>   net/mlx5: add GENEVE option support for profile 0
>   net/mlx5: add GENEVE option support for group 0
>   net/mlx5: add support for GENEVE VNI modify field
>   net/mlx5: add support for modify GENEVE option header
> 
>  doc/guides/nics/mlx5.rst               |  251 +++++-
>  doc/guides/platform/mlx5.rst           |    2 +
>  doc/guides/rel_notes/release_24_03.rst |    9 +
>  drivers/common/mlx5/mlx5_devx_cmds.c   |  139 +++-
>  drivers/common/mlx5/mlx5_devx_cmds.h   |   29 +-
>  drivers/common/mlx5/mlx5_prm.h         |   20 +-
>  drivers/common/mlx5/version.map        |    1 +
>  drivers/net/mlx5/hws/mlx5dr_definer.c  |  281 ++++++-
>  drivers/net/mlx5/hws/mlx5dr_definer.h  |   49 +-
>  drivers/net/mlx5/meson.build           |    1 +
>  drivers/net/mlx5/mlx5.c                |  115 ++-
>  drivers/net/mlx5/mlx5.h                |   21 +
>  drivers/net/mlx5/mlx5_flow.c           |   30 +
>  drivers/net/mlx5/mlx5_flow.h           |   92 ++-
>  drivers/net/mlx5/mlx5_flow_dv.c        |  158 ++--
>  drivers/net/mlx5/mlx5_flow_geneve.c    | 1011
> ++++++++++++++++++++++++
>  drivers/net/mlx5/mlx5_flow_hw.c        |  127 ++-
>  drivers/net/mlx5/mlx5_testpmd.c        |  556 ++++++++++++-
>  drivers/net/mlx5/rte_pmd_mlx5.h        |  102 +++
>  drivers/net/mlx5/version.map           |    3 +
>  20 files changed, 2811 insertions(+), 186 deletions(-)  create mode 100644
> drivers/net/mlx5/mlx5_flow_geneve.c
> 
> --
> 2.25.1
Series applied to next-net-mlx,
Kindest regards
Raslan Darawsheh