[dpdk-dev] [PATCH v1 00/72] mlx5 Windows support - part #5

[dpdk-dev] [PATCH v1 00/72] mlx5 Windows support - part #5

[Ophir Munk]

This patch series is part of preparing mlx5 PMD to compile and run under
Windows OS. Part #5

v1: initial version

Ophir Munk (46):
  net/mlx5: fix flow sample definitions
  net/mlx5/linux: extend device attributes getter
  net/mlx5: remove Linux files from Windows compilation
  net/mlx5: fix freeing packet pacing
  net/mlx5: replace Linux sleep with rte sleep
  net/mlx5: define mprq functions as static inline
  net/mlx5: do not define static_assert in Windows
  net/mlx5: move static_assert calls to global scope
  net/mlx5: wrap glue alloc/dealloc PD with OS calls
  net/mlx5: fix adding destroy flow action wrapper
  common/mlx5: add definition HAVE_INFINIBAND_VERBS_H
  common/mlx5/linux: handle memory allocations with alignment
  common/mlx5/windows: handle memory allocations with alignment
  mlx5/windows: add initialization routine for external lib
  mlx5/windows: generate file mlx5_autoconf.h
  common/mlx5/windows: add getter functions
  net/mlx5: update MR prototypes for DevX
  common/mlx5/windows: add OS reg/dereg MR
  drivers/common: enable Windows common mlx5 compilation
  net/mlx5/windows: implement device attribute getter
  net/mlx5/windows: add mlx5_os.c stubs
  net/mlx5: refactor eth dev ops for Windows
  common/mlx5/windows: add missing DV and IBV definitions
  mlx5/windows: add mlx5_os header file under net
  net/mlx5/windows: add pthread initializer definition
  net/mlx5/windows: define epoll API to do nothing
  net/mlx5/windows: define errno ETOOMANYREFS
  common/mlx5: add rte compatibility header file
  common/mlx5/windows: add DevX UAR getters
  net/mlx5/windows: add memory region callbacks
  net/mlx5/windows: add stubs for MP requests
  net/mlx5/windows: add ethdev stub operations
  net/mlx5/windws: add VLAN stubs
  net/mlx5: exclude rte_intr_callback_register call
  net/mlx5/windows: support open device
  net/mlx5/windws: spawn eth devices
  net/mlx5/linux: wrap adjust flow priority with OS calls
  net/mlx5/linux: add OS default miss flow action
  net/mlx5/linux: fix add OS dest_devx_tir action
  drivers/net: enable Windows net/mlx5 compilation
  net/mlx5/windows: introduce flow support
  net/mlx5/windows: create flow matcher object
  net/mlx5/windows: create flow action dest TIR object
  net/mlx5/windows: create flow rule
  net/mlx5: use HAVE_INFINIBAND_VERBS_H in shared code
  net/mlx5: fix separating eth_dev_ops per OS

Tal Shnaiderman (26):
  mlx5: fix relaxed ordering DevX flow
  net/mlx5: fix folding constant array error
  net/mlx5: wrap glue reg/dereg UMEM with OS calls
  common/mlx5/linux: wrap event channel APIs with OS calls
  common/mlx5: add Windows exports file
  common/mlx5: extend DevX query hca attributes command
  common/mlx5: add DevX alloc PD command
  common/mlx5/windows: add glue functions APIs
  mlx5/windows: add mlx5 meson file
  common/mlx5/windows: extend PRM match_param_bits struct
  common/mlx5/windows: add OS alloc/dealloc pd
  common/mlx5/windows: add OS umem reg/dereg API
  net/mlx5/windows: implement mlx5 mac addr add
  common/mlx5/windows: wrap event channel APIs with OS calls
  net/mlx5/windows: support get mac
  net/mlx5/windows: support link update
  net/mlx5/windows: support read clock
  net/mlx5/windows: support get mtu
  net/mlx5/windows: support get interface name
  net/mlx5/windows: support is removed
  net/mlx5/windows: support get pdn
  net/mlx5/windows: initial probing implementation
  net/mlx5/windows: support VF PCI address
  common/mlx5: fix Windows warnings on missing enum
  net/mlx5: fix Windows warnings on get_if_name
  mlx5: build pmd only with the clang compiler

 drivers/common/meson.build                      |    8 +-
 drivers/common/mlx5/linux/meson.build           |    2 +
 drivers/common/mlx5/linux/mlx5_common_os.h      |   83 ++
 drivers/common/mlx5/meson.build                 |    4 +-
 drivers/common/mlx5/mlx5_common.c               |    4 +-
 drivers/common/mlx5/mlx5_common_mp.h            |    2 +-
 drivers/common/mlx5/mlx5_common_mr.h            |    1 +
 drivers/common/mlx5/mlx5_devx_cmds.c            |   54 +-
 drivers/common/mlx5/mlx5_devx_cmds.h            |   17 +-
 drivers/common/mlx5/mlx5_malloc.c               |   14 +-
 drivers/common/mlx5/mlx5_prm.h                  |   94 ++
 drivers/common/mlx5/rte_common_mlx5_exports.def |   67 ++
 drivers/common/mlx5/version.map                 |    1 +
 drivers/common/mlx5/windows/meson.build         |   46 +
 drivers/common/mlx5/windows/mlx5_common_os.c    |  209 ++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  257 +++++
 drivers/common/mlx5/windows/mlx5_glue.c         |  354 +++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |   60 ++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  250 +++++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |   46 +
 drivers/net/meson.build                         |    9 +-
 drivers/net/mlx5/linux/mlx5_flow_os.h           |   62 ++
 drivers/net/mlx5/linux/mlx5_os.c                |  175 +---
 drivers/net/mlx5/linux/mlx5_os.h                |    4 -
 drivers/net/mlx5/linux/mlx5_verbs.c             |    2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h             |    8 +-
 drivers/net/mlx5/meson.build                    |   20 +-
 drivers/net/mlx5/mlx5.c                         |  168 +++-
 drivers/net/mlx5/mlx5.h                         |   25 +-
 drivers/net/mlx5/mlx5_defs.h                    |    7 +-
 drivers/net/mlx5/mlx5_devx.c                    |   39 +-
 drivers/net/mlx5/mlx5_flow.c                    |   26 +-
 drivers/net/mlx5/mlx5_flow.h                    |   16 +-
 drivers/net/mlx5/mlx5_flow_dv.c                 |   53 +-
 drivers/net/mlx5/mlx5_flow_verbs.c              |    6 +
 drivers/net/mlx5/mlx5_rxq.c                     |   71 --
 drivers/net/mlx5/mlx5_rxtx.c                    |   98 +-
 drivers/net/mlx5/mlx5_rxtx.h                    |   73 +-
 drivers/net/mlx5/mlx5_trigger.c                 |    3 +-
 drivers/net/mlx5/mlx5_txpp.c                    |   36 +-
 drivers/net/mlx5/windows/meson.build            |   12 +
 drivers/net/mlx5/windows/mlx5_ethdev_os.c       |  386 ++++++++
 drivers/net/mlx5/windows/mlx5_flow_os.c         |  237 +++++
 drivers/net/mlx5/windows/mlx5_flow_os.h         |  364 +++++++
 drivers/net/mlx5/windows/mlx5_mp_os.c           |   71 ++
 drivers/net/mlx5/windows/mlx5_os.c              | 1180 +++++++++++++++++++++++
 drivers/net/mlx5/windows/mlx5_os.h              |   27 +
 drivers/net/mlx5/windows/mlx5_vlan_os.c         |   83 ++
 drivers/vdpa/mlx5/mlx5_vdpa_event.c             |    7 +-
 drivers/vdpa/mlx5/mlx5_vdpa_lm.c                |    3 +-
 drivers/vdpa/mlx5/mlx5_vdpa_mem.c               |    3 +-
 51 files changed, 4433 insertions(+), 414 deletions(-)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def
 create mode 100644 drivers/common/mlx5/windows/meson.build
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_ethdev_os.c
 create mode 100644 drivers/net/mlx5/windows/mlx5_flow_os.c
 create mode 100644 drivers/net/mlx5/windows/mlx5_flow_os.h
 create mode 100644 drivers/net/mlx5/windows/mlx5_mp_os.c
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.h
 create mode 100644 drivers/net/mlx5/windows/mlx5_vlan_os.c

-- 
2.8.4

[dpdk-dev] [PATCH v1 01/72] mlx5: fix relaxed ordering DevX flow

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

The current DevX implementation of the relaxed ordering feature is
enabling relaxed ordering usage only if both relaxed ordering read AND
write are supported.  In that case both relaxed ordering read and write
are activated.

This commit will optimize the usage of relaxed ordering by enabling it
when the read OR write features are supported.  Each relaxed ordering
type will be activated according to it's own capability bit.

This will align the DevX flow with the verbs implementation of
ibv_reg_mr when using the flag IBV_ACCESS_RELAXED_ORDERING

Fixes: 53ac93f71ad1 ("net/mlx5: create relaxed ordering memory regions")
Fixes: d4322061770b ("net/mlx5: synchronize flow counter pool creation")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c |  8 ++++----
 drivers/common/mlx5/mlx5_devx_cmds.h |  3 ++-
 drivers/net/mlx5/linux/mlx5_os.c     | 12 ++++++++----
 drivers/net/mlx5/mlx5.h              |  3 ++-
 drivers/net/mlx5/mlx5_flow.c         |  3 ++-
 drivers/vdpa/mlx5/mlx5_vdpa_lm.c     |  3 ++-
 drivers/vdpa/mlx5/mlx5_vdpa_mem.c    |  3 ++-
 7 files changed, 22 insertions(+), 13 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 8aee12d..27eff5f 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -267,10 +267,10 @@ mlx5_devx_cmd_mkey_create(void *ctx,
 	MLX5_SET(mkc, mkc, pd, attr->pd);
 	MLX5_SET(mkc, mkc, mkey_7_0, attr->umem_id & 0xFF);
 	MLX5_SET(mkc, mkc, translations_octword_size, translation_size);
-	if (attr->relaxed_ordering == 1) {
-		MLX5_SET(mkc, mkc, relaxed_ordering_write, 0x1);
-		MLX5_SET(mkc, mkc, relaxed_ordering_read, 0x1);
-	}
+	MLX5_SET(mkc, mkc, relaxed_ordering_write,
+		attr->relaxed_ordering_write);
+	MLX5_SET(mkc, mkc, relaxed_ordering_read,
+		attr->relaxed_ordering_read);
 	MLX5_SET64(mkc, mkc, start_addr, attr->addr);
 	MLX5_SET64(mkc, mkc, len, attr->size);
 	mkey->obj = mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4, out,
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index abbea67..25cf12e 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -20,7 +20,8 @@ struct mlx5_devx_mkey_attr {
 	uint32_t pd;
 	uint32_t log_entity_size;
 	uint32_t pg_access:1;
-	uint32_t relaxed_ordering:1;
+	uint32_t relaxed_ordering_write:1;
+	uint32_t relaxed_ordering_read:1;
 	struct mlx5_klm *klm_array;
 	int klm_num;
 };
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index c890998..25897d9 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1078,10 +1078,14 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 			goto error;
 		}
 		/* Check relax ordering support. */
-		if (config->hca_attr.relaxed_ordering_write &&
-		    config->hca_attr.relaxed_ordering_read  &&
-		    !haswell_broadwell_cpu)
-			sh->cmng.relaxed_ordering = 1;
+		sh->cmng.relaxed_ordering_read = 0;
+		sh->cmng.relaxed_ordering_write = 0;
+		if (!haswell_broadwell_cpu) {
+			sh->cmng.relaxed_ordering_write =
+				config->hca_attr.relaxed_ordering_write;
+			sh->cmng.relaxed_ordering_read =
+				config->hca_attr.relaxed_ordering_read;
+		}
 		/* Check for LRO support. */
 		if (config->dest_tir && config->hca_attr.lro_cap &&
 		    config->dv_flow_en) {
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 88bbd31..c148e8e 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -455,7 +455,8 @@ struct mlx5_flow_counter_mng {
 	uint8_t pending_queries;
 	uint16_t pool_index;
 	uint8_t query_thread_on;
-	bool relaxed_ordering;
+	bool relaxed_ordering_read;
+	bool relaxed_ordering_write;
 	bool counter_fallback; /* Use counter fallback management. */
 	LIST_HEAD(mem_mngs, mlx5_counter_stats_mem_mng) mem_mngs;
 	LIST_HEAD(stat_raws, mlx5_counter_stats_raw) free_stat_raws;
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 082d886..89d6d70 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -7221,7 +7221,8 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	mkey_attr.pg_access = 0;
 	mkey_attr.klm_array = NULL;
 	mkey_attr.klm_num = 0;
-	mkey_attr.relaxed_ordering = sh->cmng.relaxed_ordering;
+	mkey_attr.relaxed_ordering_write = sh->cmng.relaxed_ordering_write;
+	mkey_attr.relaxed_ordering_read = sh->cmng.relaxed_ordering_read;
 	mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr);
 	if (!mem_mng->dm) {
 		mlx5_glue->devx_umem_dereg(mem_mng->umem);
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_lm.c b/drivers/vdpa/mlx5/mlx5_vdpa_lm.c
index 273c46f..6c4284f 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_lm.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_lm.c
@@ -43,7 +43,8 @@ mlx5_vdpa_dirty_bitmap_set(struct mlx5_vdpa_priv *priv, uint64_t log_base,
 			.pg_access = 1,
 			.klm_array = NULL,
 			.klm_num = 0,
-			.relaxed_ordering = 0,
+			.relaxed_ordering_read = 0,
+			.relaxed_ordering_write = 0,
 	};
 	struct mlx5_devx_virtq_attr attr = {
 		.type = MLX5_VIRTQ_MODIFY_TYPE_DIRTY_BITMAP_PARAMS,
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_mem.c b/drivers/vdpa/mlx5/mlx5_vdpa_mem.c
index b6c7cb8..f8861d5 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_mem.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_mem.c
@@ -223,7 +223,8 @@ mlx5_vdpa_mem_register(struct mlx5_vdpa_priv *priv)
 		mkey_attr.pg_access = 1;
 		mkey_attr.klm_array = NULL;
 		mkey_attr.klm_num = 0;
-		mkey_attr.relaxed_ordering = 0;
+		mkey_attr.relaxed_ordering_read = 0;
+		mkey_attr.relaxed_ordering_write = 0;
 		entry->mkey = mlx5_devx_cmd_mkey_create(priv->ctx, &mkey_attr);
 		if (!entry->mkey) {
 			DRV_LOG(ERR, "Failed to create direct Mkey.");
-- 
2.8.4

[dpdk-dev] [PATCH v1 02/72] net/mlx5: fix flow sample definitions

[Ophir Munk]

Flow sampling is dependent on rdma-core support. The definitions which
enable sampling code are HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE and
HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY. This commit expands these
definitions to more functions which use sampling logic and structs:
flow_dv_sample_resource_register, flow_dv_dest_array_resource_register,
flow_dv_sample_resource_release, flow_dv_dest_array_resource_release.
Hence any system without the required rdma-core support will not compile
or execute redundant sampling code.

Fixes: eb7368b0109a ("net/mlx5: update translate function for sample action")
Fixes: e8a1d23ae9a8 ("net/mlx5: update translate function for mirror")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow_dv.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index dafe07f..2560559 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -8629,6 +8629,7 @@ flow_dv_sample_resource_register(struct rte_eth_dev *dev,
 			 void **sample_dv_actions,
 			 struct rte_flow_error *error)
 {
+#ifdef HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE
 	struct mlx5_flow_dv_sample_resource *cache_resource;
 	struct mlx5dv_dr_flow_sampler_attr sampler_attr;
 	struct mlx5_priv *priv = dev->data->dev_private;
@@ -8746,6 +8747,17 @@ flow_dv_sample_resource_register(struct rte_eth_dev *dev,
 				dev_flow->handle->dvh.rix_sample);
 	dev_flow->handle->dvh.rix_sample = 0;
 	return -rte_errno;
+#else
+	RTE_SET_USED(dev);
+	RTE_SET_USED(attr);
+	RTE_SET_USED(resource);
+	RTE_SET_USED(dev_flow);
+	RTE_SET_USED(sample_dv_actions);
+	RTE_SET_USED(error);
+	DRV_LOG(ERR, "Sample resource registration is not supported.");
+	rte_errno = ENOTSUP;
+	return -ENOTSUP;
+#endif /* HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE */
 }
 
 /**
@@ -8772,6 +8784,7 @@ flow_dv_dest_array_resource_register(struct rte_eth_dev *dev,
 			 struct mlx5_flow *dev_flow,
 			 struct rte_flow_error *error)
 {
+#ifdef HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY
 	struct mlx5_flow_dv_dest_array_resource *cache_resource;
 	struct mlx5dv_dr_action_dest_attr *dest_attr[MLX5_MAX_DEST_NUM] = { 0 };
 	struct mlx5dv_dr_action_dest_reformat dest_reformat[MLX5_MAX_DEST_NUM];
@@ -8894,6 +8907,16 @@ flow_dv_dest_array_resource_register(struct rte_eth_dev *dev,
 				dev_flow->handle->dvh.rix_dest_array);
 	dev_flow->handle->dvh.rix_dest_array = 0;
 	return -rte_errno;
+#else
+	RTE_SET_USED(dev);
+	RTE_SET_USED(attr);
+	RTE_SET_USED(resource);
+	RTE_SET_USED(dev_flow);
+	RTE_SET_USED(error);
+	DRV_LOG(ERR, "Dest array resource registration is not supported.");
+	rte_errno = ENOTSUP;
+	return -ENOTSUP;
+#endif /* HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY */
 }
 
 /**
@@ -10758,6 +10781,7 @@ static int
 flow_dv_sample_resource_release(struct rte_eth_dev *dev,
 				     struct mlx5_flow_handle *handle)
 {
+#ifdef HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE
 	struct mlx5_priv *priv = dev->data->dev_private;
 	uint32_t idx = handle->dvh.rix_sample;
 	struct mlx5_flow_dv_sample_resource *cache_resource;
@@ -10807,6 +10831,12 @@ flow_dv_sample_resource_release(struct rte_eth_dev *dev,
 		return 0;
 	}
 	return 1;
+#else
+	RTE_SET_USED(dev);
+	RTE_SET_USED(handle);
+	DRV_LOG(ERR, "Sample resource release is not supported.");
+	return 0;
+#endif /* HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE */
 }
 
 /**
@@ -10824,6 +10854,7 @@ static int
 flow_dv_dest_array_resource_release(struct rte_eth_dev *dev,
 				     struct mlx5_flow_handle *handle)
 {
+#ifdef HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY
 	struct mlx5_priv *priv = dev->data->dev_private;
 	struct mlx5_flow_dv_dest_array_resource *cache_resource;
 	struct mlx5_flow_sub_actions_idx *mdest_act_res;
@@ -10875,6 +10906,12 @@ flow_dv_dest_array_resource_release(struct rte_eth_dev *dev,
 		return 0;
 	}
 	return 1;
+#else
+	RTE_SET_USED(dev);
+	RTE_SET_USED(handle);
+	DRV_LOG(ERR, "Dest array resource release is not supported.");
+	return 0;
+#endif /* ifdef HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY */
 }
 
 /**
-- 
2.8.4

[dpdk-dev] [PATCH v1 03/72] net/mlx5: fix folding constant array error

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Before this commit the PMD used:
   const int elt_n = 8
   const int *stack[elt_n];

In Windows clang compiler complains:
net/mlx5/mlx5_flow.c:215:19: error: variable length array folded
to constant array as an extension [-Werror,-Wgnu-folding-constant]

Fix it by using a constant macro definition instead of a variable:
   #define MLX5_RSS_EXP_ELT_N 8
   const int *stack[MLX5_RSS_EXP_ELT_N];

Fixes: c7870bfe09dc ("ethdev: move RSS expansion code to mlx5 driver")

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 89d6d70..43d3acd 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -190,6 +190,8 @@ mlx5_flow_expand_rss_item_complete(const struct rte_flow_item *item)
 	return ret;
 }
 
+#define MLX5_RSS_EXP_ELT_N 8
+
 /**
  * Expand RSS flows into several possible flows according to the RSS hash
  * fields requested and the driver capabilities.
@@ -220,13 +222,12 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		     const struct mlx5_flow_expand_node graph[],
 		     int graph_root_index)
 {
-	const int elt_n = 8;
 	const struct rte_flow_item *item;
 	const struct mlx5_flow_expand_node *node = &graph[graph_root_index];
 	const int *next_node;
-	const int *stack[elt_n];
+	const int *stack[MLX5_RSS_EXP_ELT_N];
 	int stack_pos = 0;
-	struct rte_flow_item flow_items[elt_n];
+	struct rte_flow_item flow_items[MLX5_RSS_EXP_ELT_N];
 	unsigned int i;
 	size_t lsize;
 	size_t user_pattern_size = 0;
@@ -239,10 +240,10 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 
 	memset(&missed_item, 0, sizeof(missed_item));
 	lsize = offsetof(struct mlx5_flow_expand_rss, entry) +
-		elt_n * sizeof(buf->entry[0]);
+		MLX5_RSS_EXP_ELT_N * sizeof(buf->entry[0]);
 	if (lsize <= size) {
 		buf->entry[0].priority = 0;
-		buf->entry[0].pattern = (void *)&buf->entry[elt_n];
+		buf->entry[0].pattern = (void *)&buf->entry[MLX5_RSS_EXP_ELT_N];
 		buf->entries = 0;
 		addr = buf->entry[0].pattern;
 	}
@@ -345,7 +346,7 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		/* Go deeper. */
 		if (node->next) {
 			next_node = node->next;
-			if (stack_pos++ == elt_n) {
+			if (stack_pos++ == MLX5_RSS_EXP_ELT_N) {
 				rte_errno = E2BIG;
 				return -rte_errno;
 			}
-- 
2.8.4

[dpdk-dev] [PATCH v1 04/72] net/mlx5/linux: extend device attributes getter

[Ophir Munk]

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 5 +++++
 drivers/net/mlx5/mlx5.h          | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 25897d9..f0162d5 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -116,7 +116,12 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	device_attr->max_qp_wr = attr_ex.orig_attr.max_qp_wr;
 	device_attr->max_sge = attr_ex.orig_attr.max_sge;
 	device_attr->max_cq = attr_ex.orig_attr.max_cq;
+	device_attr->max_cqe = attr_ex.orig_attr.max_cqe;
+	device_attr->max_mr = attr_ex.orig_attr.max_mr;
+	device_attr->max_pd = attr_ex.orig_attr.max_pd;
 	device_attr->max_qp = attr_ex.orig_attr.max_qp;
+	device_attr->max_srq = attr_ex.orig_attr.max_srq;
+	device_attr->max_srq_wr = attr_ex.orig_attr.max_srq_wr;
 	device_attr->raw_packet_caps = attr_ex.raw_packet_caps;
 	device_attr->max_rwq_indirection_table_size =
 		attr_ex.rss_caps.max_rwq_indirection_table_size;
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index c148e8e..2004fa2 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -69,6 +69,11 @@ struct mlx5_dev_attr {
 	int		max_sge;
 	int		max_cq;
 	int		max_qp;
+	int		max_cqe;
+	uint32_t	max_pd;
+	uint32_t	max_mr;
+	uint32_t	max_srq;
+	uint32_t	max_srq_wr;
 	uint32_t	raw_packet_caps;
 	uint32_t	max_rwq_indirection_table_size;
 	uint32_t	max_tso;
-- 
2.8.4

[dpdk-dev] [PATCH v1 05/72] net/mlx5: remove Linux files from Windows compilation

[Ophir Munk]

This commit removes Linux files flow_verbs.c and mlx5_rxtx_vec.c
from Windows compilation.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/meson.build | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index 9a97bb9..f75a169 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -15,7 +15,6 @@ sources = files(
 	'mlx5_flow.c',
 	'mlx5_flow_meter.c',
 	'mlx5_flow_dv.c',
-	'mlx5_flow_verbs.c',
 	'mlx5_mac.c',
 	'mlx5_mr.c',
 	'mlx5_rss.c',
@@ -30,11 +29,18 @@ sources = files(
 	'mlx5_utils.c',
 	'mlx5_devx.c',
 )
-if (dpdk_conf.has('RTE_ARCH_X86_64')
-	or dpdk_conf.has('RTE_ARCH_ARM64')
-	or dpdk_conf.has('RTE_ARCH_PPC_64'))
-	sources += files('mlx5_rxtx_vec.c')
+
+if is_linux
+	sources += files(
+		'mlx5_flow_verbs.c',
+	)
+	if (dpdk_conf.has('RTE_ARCH_X86_64')
+		or dpdk_conf.has('RTE_ARCH_ARM64')
+		or dpdk_conf.has('RTE_ARCH_PPC_64'))
+		sources += files('mlx5_rxtx_vec.c')
+	endif
 endif
+
 cflags_options = [
 	'-std=c11',
 	'-Wno-strict-prototypes',
-- 
2.8.4

[dpdk-dev] [PATCH v1 06/72] net/mlx5: fix freeing packet pacing

[Ophir Munk]

Packet pacing is allocated under condition #ifdef HAVE_MLX5DV_PP_ALLOC.
In a similar way - free packet pacing index under the same condition.
This update is required to successfully compile under operating systems
which do not support packet pacing.

Fixes: aef1e20ebeb2 ("net/mlx5: allocate packet pacing context")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_txpp.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 37355fa..3a16f87 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -57,11 +57,16 @@ mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 static void
 mlx5_txpp_free_pp_index(struct mlx5_dev_ctx_shared *sh)
 {
+#ifdef HAVE_MLX5DV_PP_ALLOC
 	if (sh->txpp.pp) {
 		mlx5_glue->dv_free_pp(sh->txpp.pp);
 		sh->txpp.pp = NULL;
 		sh->txpp.pp_id = 0;
 	}
+#else
+	RTE_SET_USED(sh);
+	DRV_LOG(ERR, "Freeing pacing index is not supported.");
+#endif
 }
 
 /* Allocate Packet Pacing index from kernel via mlx5dv call. */
-- 
2.8.4

[dpdk-dev] [PATCH v1 07/72] net/mlx5: replace Linux sleep with rte sleep

[Ophir Munk]

Replace Linux API usleep() and nanosleep() with rte_delay_us_sleep().
The replacement occurs in shared files compiled under different
operating systems.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_trigger.c | 3 ++-
 drivers/net/mlx5/mlx5_txpp.c    | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 91aaee3..812a066 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -21,6 +21,7 @@
 #include <rte_spinlock.h>
 #include <rte_string_fns.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
@@ -1415,7 +1416,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_flex_parser_ecpri_release(dev);
 	if (priv->rxqs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
@@ -1423,7 +1424,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->txqs_n); ++i)
 			mlx5_txq_release(dev, i);
 		priv->txqs_n = 0;
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index 52691b6..014ff49 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -9,6 +9,7 @@
 #include <rte_ethdev_driver.h>
 #include <rte_interrupts.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_malloc.h>
 
@@ -1170,7 +1171,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	rte_wmb();
 	/* Disable datapath on secondary process. */
 	mlx5_mp_os_req_stop_rxtx(dev);
-	usleep(1000 * priv->rxqs_n);
+	rte_delay_us_sleep(1000 * priv->rxqs_n);
 	DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id);
 	mlx5_flow_stop_default(dev);
 	/* Control flows for default traffic can be removed firstly. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 3a16f87..9e3eae4 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -740,15 +740,11 @@ mlx5_txpp_init_timestamp(struct mlx5_dev_ctx_shared *sh)
 	sh->txpp.ts_p = 0;
 	sh->txpp.ts_n = 0;
 	for (wait = 0; wait < MLX5_TXPP_WAIT_INIT_TS; wait++) {
-		struct timespec onems;
-
 		mlx5_txpp_update_timestamp(sh);
 		if (wq->sq_ci)
 			return;
 		/* Wait one millisecond and try again. */
-		onems.tv_sec = 0;
-		onems.tv_nsec = NS_PER_S / MS_PER_S;
-		nanosleep(&onems, 0);
+		rte_delay_us_sleep(US_PER_S / MS_PER_S);
 	}
 	DRV_LOG(ERR, "Unable to initialize timestamp.");
 	sh->txpp.sync_lost = 1;
-- 
2.8.4

[dpdk-dev] [PATCH v1 08/72] net/mlx5: define mprq functions as static inline

[Ophir Munk]

Functions mlx5_check_mprq_support(), mlx5_rxq_mprq_enabled(),
mlx5_mprq_enabled() are moved from source file mlx5_rxq.c to header file
mlx5_rxtx.h and their type is updated to 'static __rte_always_inline'.
Previously the functions were declared as 'inline' in the source file
which was reported as 'unresolved external symbol' error by some Windows
linkers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxq.c  | 71 ------------------------------------------
 drivers/net/mlx5/mlx5_rxtx.h | 73 ++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 70 insertions(+), 74 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index ddd5df7..e2d02b5 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -49,77 +49,6 @@ static_assert(MLX5_RSS_HASH_KEY_LEN ==
 	      "wrong RSS default key size.");
 
 /**
- * Check whether Multi-Packet RQ can be enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   1 if supported, negative errno value if not.
- */
-inline int
-mlx5_check_mprq_support(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	if (priv->config.mprq.enabled &&
-	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
-		return 1;
-	return -ENOTSUP;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the Rx queue.
- *
- *  @param rxq
- *     Pointer to receive queue structure.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
-{
-	return rxq->strd_num_n > 0;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_mprq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	uint32_t i;
-	uint16_t n = 0;
-	uint16_t n_ibv = 0;
-
-	if (mlx5_check_mprq_support(dev) < 0)
-		return 0;
-	/* All the configured queues should be enabled. */
-	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
-
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
-			continue;
-		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
-			++n;
-	}
-	/* Multi-Packet RQ can't be partially configured. */
-	MLX5_ASSERT(n == 0 || n == n_ibv);
-	return n == n_ibv;
-}
-
-/**
  * Calculate the number of CQEs in CQ for the Rx queue.
  *
  *  @param rxq_data
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 1b5fba4..d8014d3 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -308,9 +308,6 @@ struct mlx5_txq_ctrl {
 
 extern uint8_t rss_hash_default_key[];
 
-int mlx5_check_mprq_support(struct rte_eth_dev *dev);
-int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq);
-int mlx5_mprq_enabled(struct rte_eth_dev *dev);
 unsigned int mlx5_rxq_cqe_num(struct mlx5_rxq_data *rxq_data);
 int mlx5_mprq_free_mp(struct rte_eth_dev *dev);
 int mlx5_mprq_alloc_mp(struct rte_eth_dev *dev);
@@ -889,4 +886,74 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Multi-Packet RQ can be enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   1 if supported, negative errno value if not.
+ */
+static __rte_always_inline int
+mlx5_check_mprq_support(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	if (priv->config.mprq.enabled &&
+	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
+		return 1;
+	return -ENOTSUP;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the Rx queue.
+ *
+ *  @param rxq
+ *     Pointer to receive queue structure.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
+{
+	return rxq->strd_num_n > 0;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_mprq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint32_t i;
+	uint16_t n = 0;
+	uint16_t n_ibv = 0;
+
+	if (mlx5_check_mprq_support(dev) < 0)
+		return 0;
+	/* All the configured queues should be enabled. */
+	for (i = 0; i < priv->rxqs_n; ++i) {
+		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
+			(rxq, struct mlx5_rxq_ctrl, rxq);
+
+		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+			continue;
+		n_ibv++;
+		if (mlx5_rxq_mprq_enabled(rxq))
+			++n;
+	}
+	/* Multi-Packet RQ can't be partially configured. */
+	MLX5_ASSERT(n == 0 || n == n_ibv);
+	return n == n_ibv;
+}
 #endif /* RTE_PMD_MLX5_RXTX_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 09/72] net/mlx5: do not define static_assert in Windows

[Ophir Munk]

In Linux 'static_assert' is defined in file mlx5_defs.h:

 #ifndef HAVE_STATIC_ASSERT
    #define static_assert _Static_assert
 #endif

The same definition can originate from Linux file /usr/include/assert.h.

In Windows static_assert is used while _Static_assert is unknown.
Therefore update the definition condition to be:

 #if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
    #define static_assert _Static_assert
 #endif

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_defs.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index 2657081..27e25b1 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -199,8 +199,11 @@
 /* Maximum number of shared actions supported by rte_flow */
 #define MLX5_MAX_SHARED_ACTIONS 1
 
-/* Definition of static_assert found in /usr/include/assert.h */
-#ifndef HAVE_STATIC_ASSERT
+/*
+ * Linux definition of static_assert is found in /usr/include/assert.h.
+ * Windows does not require a redefinition.
+ */
+#if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
 #define static_assert _Static_assert
 #endif
 
-- 
2.8.4

[dpdk-dev] [PATCH v1 10/72] net/mlx5: move static_assert calls to global scope

[Ophir Munk]

Some Windows compilers consider static_assert() as calls to another
function rather than a compiler directive which allows checking type
information at compile time.  This only occurs if the static_assert call
appears inside another function scope. To solve it move the
static_assert calls to global scope in the files where they are used.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxtx.c | 98 ++++++++++++++++++++++----------------------
 drivers/net/mlx5/mlx5_txpp.c |  5 ++-
 2 files changed, 53 insertions(+), 50 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index 2ffacf8..05d3688 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -79,6 +79,56 @@ static uint16_t mlx5_tx_burst_##func(void *txq, \
 
 #define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx},
 
+/* static asserts */
+static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
+static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
+		(2 * RTE_ETHER_ADDR_LEN),
+		"invalid Data Segment data size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert((sizeof(struct rte_vlan_hdr) +
+			sizeof(struct rte_ether_hdr)) ==
+		MLX5_ESEG_MIN_INLINE_SIZE,
+		"invalid min inline data size");
+static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
+		MLX5_DSEG_MAX, "invalid WQE max size");
+static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Control Segment size");
+static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Ethernet Segment size");
+static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Data Segment size");
+static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
+		"invalid WQE size");
+
 static __rte_always_inline uint32_t
 rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe);
 
@@ -2008,8 +2058,6 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq,
 	bool ring_doorbell = false;
 	int ret;
 
-	static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
-	static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
 	do {
 		volatile struct mlx5_cqe *cqe;
 
@@ -2317,15 +2365,6 @@ mlx5_tx_eseg_dmin(struct mlx5_txq_data *__rte_restrict txq __rte_unused,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = RTE_BE16(MLX5_ESEG_MIN_INLINE_SIZE);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2410,15 +2449,6 @@ mlx5_tx_eseg_data(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = rte_cpu_to_be_16(inlen);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2633,15 +2663,6 @@ mlx5_tx_eseg_mdat(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	MLX5_ASSERT(inlen >= MLX5_ESEG_MIN_INLINE_SIZE);
 	pdst = (uint8_t *)&es->inline_data;
 	if (MLX5_TXOFF_CONFIG(VLAN) && vlan) {
@@ -2888,9 +2909,6 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *__rte_restrict txq,
 	uint8_t *pdst;
 
 	MLX5_ASSERT(len > MLX5_ESEG_MIN_INLINE_SIZE);
-	static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
-				 (2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Data Segment data size");
 	if (!MLX5_TXOFF_CONFIG(MPW)) {
 		/* Store the descriptor byte counter for eMPW sessions. */
 		dseg->bcount = rte_cpu_to_be_32
@@ -4006,7 +4024,6 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4183,7 +4200,6 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4497,10 +4513,6 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq,
 			    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
 				vlan = sizeof(struct rte_vlan_hdr);
 				inlen += vlan;
-				static_assert((sizeof(struct rte_vlan_hdr) +
-					       sizeof(struct rte_ether_hdr)) ==
-					       MLX5_ESEG_MIN_INLINE_SIZE,
-					       "invalid min inline data size");
 			}
 			/*
 			 * If inlining is enabled at configuration time
@@ -5503,16 +5515,6 @@ mlx5_select_tx_function(struct rte_eth_dev *dev)
 	uint64_t tx_offloads = dev->data->dev_conf.txmode.offloads;
 	unsigned int diff = 0, olx = 0, i, m;
 
-	static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
-		      MLX5_DSEG_MAX, "invalid WQE max size");
-	static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Control Segment size");
-	static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Ethernet Segment size");
-	static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Data Segment size");
-	static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
-		      "invalid WQE size");
 	MLX5_ASSERT(priv);
 	if (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) {
 		/* We should support Multi-Segment Packets. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 9e3eae4..85bbea9 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -18,6 +18,9 @@
 #include "mlx5_rxtx.h"
 #include "mlx5_common_os.h"
 
+static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
+		"Wrong timestamp CQE part size");
+
 static const char * const mlx5_txpp_stat_names[] = {
 	"txpp_err_miss_int", /* Missed service interrupt. */
 	"txpp_err_rearm_queue",	/* Rearm Queue errors. */
@@ -713,8 +716,6 @@ mlx5_txpp_update_timestamp(struct mlx5_dev_ctx_shared *sh)
 	uint64_t ts;
 	uint16_t ci;
 
-	static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
-		      "Wrong timestamp CQE part size");
 	mlx5_atomic_read_cqe((rte_int128_t *)&cqe->timestamp, &to.u128);
 	if (to.cts.op_own >> 4) {
 		DRV_LOG(DEBUG, "Clock Queue error sync lost.");
-- 
2.8.4

[dpdk-dev] [PATCH v1 11/72] net/mlx5: wrap glue alloc/dealloc PD with OS calls

[Ophir Munk]

Wrap glue calls alloc_pd() and dealloc_pd() with generic OS calls.  In
Linux - protection domain allocations are implemented by Verbs glue API
while in Windows it is by DevX API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 11 +++++++++++
 drivers/net/mlx5/mlx5.c                    |  6 +++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index 3420937..c9f8d7c 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -201,4 +201,15 @@ mlx5_os_get_devx_uar_page_id(void *uar)
 #endif
 }
 
+static inline void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	return mlx5_glue->alloc_pd(ctx);
+}
+
+static inline int
+mlx5_os_dealloc_pd(void *pd)
+{
+	return mlx5_glue->dealloc_pd(pd);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 812a066..1d89615 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -938,7 +938,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sh->port[i].ih_port_id = RTE_MAX_ETHPORTS;
 		sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS;
 	}
-	sh->pd = mlx5_glue->alloc_pd(sh->ctx);
+	sh->pd = mlx5_os_alloc_pd(sh->ctx);
 	if (sh->pd == NULL) {
 		DRV_LOG(ERR, "PD allocation failure");
 		err = ENOMEM;
@@ -1045,7 +1045,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	if (sh->tx_uar)
 		mlx5_glue->devx_free_uar(sh->tx_uar);
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->ctx)
 		claim_zero(mlx5_glue->close_device(sh->ctx));
 	if (sh->flow_id_pool)
@@ -1111,7 +1111,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		sh->tx_uar = NULL;
 	}
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->tis)
 		claim_zero(mlx5_devx_cmd_destroy(sh->tis));
 	if (sh->td)
-- 
2.8.4

[dpdk-dev] [PATCH v1 12/72] net/mlx5: wrap glue reg/dereg UMEM with OS calls

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Wrap glue calls for UMEM registration and deregistration with generic OS
calls since each OS (Linux or Windows) has a different glue API
parameters.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 12 ++++++++++++
 drivers/common/mlx5/mlx5_common.c          |  4 ++--
 drivers/net/mlx5/mlx5.c                    |  2 +-
 drivers/net/mlx5/mlx5_devx.c               | 16 ++++++++--------
 drivers/net/mlx5/mlx5_flow.c               |  4 ++--
 drivers/net/mlx5/mlx5_txpp.c               | 12 ++++++------
 6 files changed, 31 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index c9f8d7c..f8b215c 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -212,4 +212,16 @@ mlx5_os_dealloc_pd(void *pd)
 {
 	return mlx5_glue->dealloc_pd(pd);
 }
+
+static inline void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	return mlx5_glue->devx_umem_reg(ctx, addr, size, access);
+}
+
+static inline int
+mlx5_os_umem_dereg(void *pumem)
+{
+	return mlx5_glue->devx_umem_dereg(pumem);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c
index 06f0a64..df2ba34 100644
--- a/drivers/common/mlx5/mlx5_common.c
+++ b/drivers/common/mlx5/mlx5_common.c
@@ -148,7 +148,7 @@ mlx5_alloc_dbr_page(void *ctx)
 		return NULL;
 	}
 	/* Register allocated memory. */
-	page->umem = mlx5_glue->devx_umem_reg(ctx, page->dbrs,
+	page->umem = mlx5_os_umem_reg(ctx, page->dbrs,
 					      MLX5_DBR_PAGE_SIZE, 0);
 	if (!page->umem) {
 		DRV_LOG(ERR, "cannot umem reg dbr page");
@@ -232,7 +232,7 @@ mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id,
 		/* Page not used, free it and remove from list. */
 		LIST_REMOVE(page, next);
 		if (page->umem)
-			ret = -mlx5_glue->devx_umem_dereg(page->umem);
+			ret = -mlx5_os_umem_dereg(page->umem);
 		mlx5_free(page);
 	} else {
 		/* Mark in bitmap that this door-bell is not in use. */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 1d89615..cb131da 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -508,7 +508,7 @@ mlx5_flow_destroy_counter_stat_mem_mng(struct mlx5_counter_stats_mem_mng *mng)
 
 	LIST_REMOVE(mng, next);
 	claim_zero(mlx5_devx_cmd_destroy(mng->dm));
-	claim_zero(mlx5_glue->devx_umem_dereg(mng->umem));
+	claim_zero(mlx5_os_umem_dereg(mng->umem));
 	mlx5_free(mem);
 }
 
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 5fce4cd..563e56d 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -159,7 +159,7 @@ mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 		rxq_ctrl->rxq.wqes = NULL;
 	}
 	if (rxq_ctrl->wq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->wq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->wq_umem);
 		rxq_ctrl->wq_umem = NULL;
 	}
 	if (dbr_page) {
@@ -186,7 +186,7 @@ mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 		rxq_ctrl->rxq.cqes = NULL;
 	}
 	if (rxq_ctrl->cq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->cq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->cq_umem);
 		rxq_ctrl->cq_umem = NULL;
 	}
 	if (dbr_page) {
@@ -375,7 +375,7 @@ mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	if (!buf)
 		return NULL;
 	rxq_data->wqes = buf;
-	rxq_ctrl->wq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	rxq_ctrl->wq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						     buf, wq_size, 0);
 	if (!rxq_ctrl->wq_umem)
 		goto error;
@@ -475,7 +475,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	rxq_data->cqes = (volatile struct mlx5_cqe (*)[])(uintptr_t)buf;
-	rxq_ctrl->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx, buf,
+	rxq_ctrl->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, buf,
 						     cq_size,
 						     IBV_ACCESS_LOCAL_WRITE);
 	if (!rxq_ctrl->cq_umem) {
@@ -1038,7 +1038,7 @@ mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->sq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->sq_devx));
 	if (txq_obj->sq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->sq_umem));
 	if (txq_obj->sq_buf)
 		mlx5_free(txq_obj->sq_buf);
 	if (txq_obj->sq_dbrec_page)
@@ -1060,7 +1060,7 @@ mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
 	if (txq_obj->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->cq_umem));
 	if (txq_obj->cq_buf)
 		mlx5_free(txq_obj->cq_buf);
 	if (txq_obj->cq_dbrec_page)
@@ -1148,7 +1148,7 @@ mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		return 0;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	txq_obj->cq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						(void *)txq_obj->cq_buf,
 						cqe_n * sizeof(struct mlx5_cqe),
 						IBV_ACCESS_LOCAL_WRITE);
@@ -1247,7 +1247,7 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->sq_umem = mlx5_glue->devx_umem_reg
+	txq_obj->sq_umem = mlx5_os_umem_reg
 					(priv->sh->ctx,
 					 (void *)txq_obj->sq_buf,
 					 wqe_n * sizeof(struct mlx5_wqe),
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 43d3acd..93ed113 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -7207,7 +7207,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	}
 	mem_mng = (struct mlx5_counter_stats_mem_mng *)(mem + size) - 1;
 	size = sizeof(*raw_data) * MLX5_COUNTERS_PER_POOL * raws_n;
-	mem_mng->umem = mlx5_glue->devx_umem_reg(sh->ctx, mem, size,
+	mem_mng->umem = mlx5_os_umem_reg(sh->ctx, mem, size,
 						 IBV_ACCESS_LOCAL_WRITE);
 	if (!mem_mng->umem) {
 		rte_errno = errno;
@@ -7226,7 +7226,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	mkey_attr.relaxed_ordering_read = sh->cmng.relaxed_ordering_read;
 	mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr);
 	if (!mem_mng->dm) {
-		mlx5_glue->devx_umem_dereg(mem_mng->umem);
+		mlx5_os_umem_dereg(mem_mng->umem);
 		rte_errno = errno;
 		mlx5_free(mem);
 		return -rte_errno;
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 85bbea9..9d6b921 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -131,13 +131,13 @@ mlx5_txpp_destroy_send_queue(struct mlx5_txpp_wq *wq)
 	if (wq->sq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->sq));
 	if (wq->sq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->sq_umem));
 	if (wq->sq_buf)
 		mlx5_free((void *)(uintptr_t)wq->sq_buf);
 	if (wq->cq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->cq));
 	if (wq->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->cq_umem));
 	if (wq->cq_buf)
 		mlx5_free((void *)(uintptr_t)wq->cq_buf);
 	memset(wq, 0, sizeof(*wq));
@@ -268,7 +268,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -318,7 +318,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -506,7 +506,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -562,7 +562,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
-- 
2.8.4

[dpdk-dev] [PATCH v1 13/72] net/mlx5: fix adding destroy flow action wrapper

[Ophir Munk]

Glue function destroy_flow_action() was wrapped by OS specific operation
mlx5_flow_os_destroy_flow_action(). It was skipped in file mlx5.c.

Fixes: b293fbf9672b ("net/mlx5: add OS specific flow actions operations")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index cb131da..f4b466f 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -38,6 +38,7 @@
 #include "mlx5_autoconf.h"
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
+#include "mlx5_flow_os.h"
 #include "rte_pmd_mlx5.h"
 
 /* Device parameter to enable RX completion queue compression. */
@@ -549,7 +550,7 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh)
 
 				if (cnt->action)
 					claim_zero
-					 (mlx5_glue->destroy_flow_action
+					 (mlx5_flow_os_destroy_flow_action
 					  (cnt->action));
 				if (fallback && MLX5_POOL_GET_CNT
 				    (pool, j)->dcs_when_free)
-- 
2.8.4

[dpdk-dev] [PATCH v1 14/72] common/mlx5: add definition HAVE_INFINIBAND_VERBS_H

[Ophir Munk]

Add a Verbs file presence indication. Under Linux it is required that
file infiniband/verbs.h is installed for building DPDK.  Other
operating systems (e.g. Windows) ignore Verbs completely.  This commit
adds definition HAVE_INFINIBAND_VERBS_H (file mlx5_autoconf.h) to
indicate whether DPDK compiles with Verbs or not.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/meson.build | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build
index 9ef8e18..4da5bde 100644
--- a/drivers/common/mlx5/linux/meson.build
+++ b/drivers/common/mlx5/linux/meson.build
@@ -179,6 +179,8 @@ has_sym_args = [
 	[ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h',
 	'mlx5dv_dr_action_create_dest_array'],
 	[ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ],
+	[ 'HAVE_INFINIBAND_VERBS_H', 'infiniband/verbs.h',
+	'INFINIBAND_VERBS_H' ],
 ]
 config = configuration_data()
 foreach arg:has_sym_args
-- 
2.8.4

[dpdk-dev] [PATCH v1 15/72] common/mlx5/linux: handle memory allocations with alignment

[Ophir Munk]

mlx5_malloc() API has an alignment parameter for system memory
allocations. malloc() is called for non-aligned allocations and
posix_memalign() is called for aligned allocations. When calling
mlx5_free() there is no distinction whether the memory was originally
allocated with or without alignment. Freeing a memory may be handled
differently by operating systems. Therefore this commit wraps these APIs
with OS specific calls: mlx5_os_malloc(), mlx5_os_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 38 ++++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_malloc.c          | 14 +++++------
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index f8b215c..bd44ecb 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -6,6 +6,7 @@
 #define RTE_PMD_MLX5_COMMON_OS_H_
 
 #include <stdio.h>
+#include <malloc.h>
 
 #include <rte_pci.h>
 #include <rte_debug.h>
@@ -16,6 +17,7 @@
 
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
+#include "mlx5_malloc.h"
 
 /**
  * Get device name. Given an ibv_device pointer - return a
@@ -224,4 +226,40 @@ mlx5_os_umem_dereg(void *pumem)
 {
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
+
+/**
+ * Memory allocation optionally with alignment.
+ *
+ * @param[in] align
+ *    Alignment size (may be zero)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	void *buf;
+
+	if (posix_memalign(&buf, align, size))
+		return NULL;
+	return buf;
+}
+
+/**
+ * This API de-allocates a memory that originally could have been
+ * allocated aligned or non-aligned. In Linux it is a wrapper
+ * around free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	free(addr);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_malloc.c b/drivers/common/mlx5/mlx5_malloc.c
index 4489971..dd8b793 100644
--- a/drivers/common/mlx5/mlx5_malloc.c
+++ b/drivers/common/mlx5/mlx5_malloc.c
@@ -11,6 +11,7 @@
 #include <rte_atomic.h>
 
 #include "mlx5_common_utils.h"
+#include "mlx5_common_os.h"
 #include "mlx5_malloc.h"
 
 struct mlx5_sys_mem {
@@ -151,14 +152,11 @@ static void *
 mlx5_alloc_align(size_t size, unsigned int align, unsigned int zero)
 {
 	void *buf;
-	int ret;
-
-	ret = posix_memalign(&buf, align, size);
-	if (ret) {
-		DRV_LOG(ERR,
-			"Couldn't allocate buf size=%zu align=%u. Err=%d\n",
-			size, align, ret);
 
+	buf = mlx5_os_malloc(align, size);
+	if (!buf) {
+		DRV_LOG(ERR, "Couldn't allocate buf size=%zu align=%u.",
+			size, align);
 		return NULL;
 	}
 	if (zero)
@@ -262,7 +260,7 @@ mlx5_free(void *addr)
 #ifdef RTE_LIBRTE_MLX5_DEBUG
 		rte_atomic64_inc(&mlx5_sys_mem.free_sys);
 #endif
-		free(addr);
+		mlx5_os_free(addr);
 	} else {
 #ifdef RTE_LIBRTE_MLX5_DEBUG
 		rte_atomic64_inc(&mlx5_sys_mem.free_rte);
-- 
2.8.4

[dpdk-dev] [PATCH v1 16/72] common/mlx5/windows: handle memory allocations with alignment

[Ophir Munk]

This commit is the Windows equivalent of the Linux implementation.  The
APIs included in this commit: mlx5_os_malloc(), mlx5_os_free(). For
memory allocations (with or without alignment) we always call
_aligned_malloc(). Even if zero alignment was requested in the first
place - we always select a minimal alignment value. In this way when the
memory is free - it is always safe to call _aligned_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 51 ++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
new file mode 100644
index 0000000..cce7c88
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_COMMON_OS_H_
+#define RTE_PMD_MLX5_COMMON_OS_H_
+
+#include <stdio.h>
+
+#include "mlx5_autoconf.h"
+#include "mlx5_glue.h"
+#include "mlx5_malloc.h"
+
+/**
+ * This API allocates aligned or non-aligned memory.  The free can be on either
+ * aligned or nonaligned memory.  To be protected - even though there may be no
+ * alignment - in Windows this API will unconditioanlly call _aligned_malloc()
+ * with at least a minimal alignment size.
+ *
+ * @param[in] align
+ *    The alignment value, which must be an integer power of 2 (or 0 for
+ *    non-alignment)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	if (align < MLX5_MALLOC_ALIGNMENT)
+		align = MLX5_MALLOC_ALIGNMENT;
+	return _aligned_malloc(size, align);
+}
+
+/**
+ * This API de-allocates a memory that originally could have been allocated
+ * aligned or non-aligned. In Windows since the allocation was with
+ * _aligned_malloc() - it is safe to always call _aligned_free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	_aligned_free(addr);
+}
+#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 17/72] common/mlx5/linux: wrap event channel APIs with OS calls

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Wrap the API to create/destroy event channel and to subscribe an event
with OS calls. In Linux those calls are implemented by glue functions
while in Windows they are not supported.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 22 ++++++++++++++++++++++
 drivers/net/mlx5/mlx5_devx.c               |  8 ++++----
 drivers/net/mlx5/mlx5_txpp.c               |  6 +++---
 drivers/vdpa/mlx5/mlx5_vdpa_event.c        |  7 ++++---
 4 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index bd44ecb..63f070d 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -227,6 +227,28 @@ mlx5_os_umem_dereg(void *pumem)
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
 
+static inline void *
+mlx5_os_devx_create_event_channel(void *ctx, int flags)
+{
+	return mlx5_glue->devx_create_event_channel(ctx, flags);
+}
+
+static inline void
+mlx5_os_devx_destroy_event_channel(void *eventc)
+{
+	mlx5_glue->devx_destroy_event_channel(eventc);
+}
+
+static inline int
+mlx5_os_devx_subscribe_devx_event(void *eventc,
+				  void *obj,
+				  uint16_t events_sz, uint16_t events_num[],
+				  uint64_t cookie)
+{
+	return mlx5_glue->devx_subscribe_devx_event(eventc, obj, events_sz,
+						    events_num, cookie);
+}
+
 /**
  * Memory allocation optionally with alignment.
  *
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 563e56d..23d4190 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -216,7 +216,7 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_obj *rxq_obj)
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->devx_cq));
 		if (rxq_obj->devx_channel)
-			mlx5_glue->devx_destroy_event_channel
+			mlx5_os_devx_destroy_event_channel
 							(rxq_obj->devx_channel);
 		mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
 		mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
@@ -511,7 +511,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cqe_n = log_cqe_n;
 	rxq_data->cqn = cq_obj->id;
 	if (rxq_ctrl->obj->devx_channel) {
-		ret = mlx5_glue->devx_subscribe_devx_event
+		ret = mlx5_os_devx_subscribe_devx_event
 						(rxq_ctrl->obj->devx_channel,
 						 cq_obj->obj,
 						 sizeof(event_nums),
@@ -622,7 +622,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		int devx_ev_flag =
 			  MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA;
 
-		tmpl->devx_channel = mlx5_glue->devx_create_event_channel
+		tmpl->devx_channel = mlx5_os_devx_create_event_channel
 								(priv->sh->ctx,
 								 devx_ev_flag);
 		if (!tmpl->devx_channel) {
@@ -664,7 +664,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	if (tmpl->devx_cq)
 		claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq));
 	if (tmpl->devx_channel)
-		mlx5_glue->devx_destroy_event_channel(tmpl->devx_channel);
+		mlx5_os_devx_destroy_event_channel(tmpl->devx_channel);
 	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
 	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	rte_errno = ret; /* Restore rte_errno. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 9d6b921..c22ddc9 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -37,7 +37,7 @@ static void
 mlx5_txpp_destroy_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	if (sh->txpp.echan) {
-		mlx5_glue->devx_destroy_event_channel(sh->txpp.echan);
+		mlx5_os_devx_destroy_event_channel(sh->txpp.echan);
 		sh->txpp.echan = NULL;
 	}
 }
@@ -47,7 +47,7 @@ static int
 mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	MLX5_ASSERT(!sh->txpp.echan);
-	sh->txpp.echan = mlx5_glue->devx_create_event_channel(sh->ctx,
+	sh->txpp.echan = mlx5_os_devx_create_event_channel(sh->ctx,
 			MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!sh->txpp.echan) {
 		rte_errno = errno;
@@ -903,7 +903,7 @@ mlx5_txpp_start_service(struct mlx5_dev_ctx_shared *sh)
 		return -rte_errno;
 	}
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(sh->txpp.echan,
+	ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan,
 						   sh->txpp.rearm_queue.cq->obj,
 						   sizeof(event_nums),
 						   event_nums, 0);
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
index 8a01e42..a0e988a 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
@@ -15,6 +15,7 @@
 #include <rte_alarm.h>
 
 #include <mlx5_common.h>
+#include <mlx5_common_os.h>
 
 #include "mlx5_vdpa_utils.h"
 #include "mlx5_vdpa.h"
@@ -40,7 +41,7 @@ mlx5_vdpa_event_qp_global_release(struct mlx5_vdpa_priv *priv)
 		       sizeof(out.buf)) >=
 		       (ssize_t)sizeof(out.event_resp.cookie))
 			;
-		mlx5_glue->devx_destroy_event_channel(priv->eventc);
+		mlx5_os_devx_destroy_event_channel(priv->eventc);
 		priv->eventc = NULL;
 	}
 #endif
@@ -60,7 +61,7 @@ mlx5_vdpa_event_qp_global_prepare(struct mlx5_vdpa_priv *priv)
 		DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno);
 		return -1;
 	}
-	priv->eventc = mlx5_glue->devx_create_event_channel(priv->ctx,
+	priv->eventc = mlx5_os_devx_create_event_channel(priv->ctx,
 			   MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!priv->eventc) {
 		rte_errno = errno;
@@ -168,7 +169,7 @@ mlx5_vdpa_cq_create(struct mlx5_vdpa_priv *priv, uint16_t log_desc_n,
 	cq->cq_ci = 0;
 	rte_spinlock_init(&cq->sl);
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
+	ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
 						   sizeof(event_nums),
 						   event_nums,
 						   (uint64_t)(uintptr_t)cq);
-- 
2.8.4

[dpdk-dev] [PATCH v1 18/72] common/mlx5: add Windows exports file

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

File drivers/common/mlx5/rte_common_mlx5_exports.def contains mlx5
Windows exported symbols under common/mlx5 directory (DLL file
name librte_common_mlx5*.dll). It is the equivalent of Linux map
file rte_common_mlx5_version.map but the list of symbols may be
different between the two operating systems.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def | 60 +++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
new file mode 100644
index 0000000..8d1f373
--- /dev/null
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -0,0 +1,60 @@
+EXPORTS
+	haswell_broadwell_cpu
+
+	mlx5_common_init
+
+	mlx5_create_mr_ext
+
+	mlx5_devx_cmd_create_cq
+	mlx5_devx_cmd_create_flex_parser
+	mlx5_devx_cmd_create_qp
+	mlx5_devx_cmd_create_rq
+	mlx5_devx_cmd_create_rqt
+	mlx5_devx_cmd_create_sq
+	mlx5_devx_cmd_create_tir
+	mlx5_devx_cmd_create_td
+	mlx5_devx_cmd_create_tis
+	mlx5_devx_cmd_create_virtq
+	mlx5_devx_cmd_destroy
+	mlx5_devx_cmd_flow_counter_alloc
+	mlx5_devx_cmd_flow_counter_query
+	mlx5_devx_cmd_flow_dump
+	mlx5_devx_cmd_mkey_create
+	mlx5_devx_cmd_modify_qp_state
+	mlx5_devx_cmd_modify_rq
+	mlx5_devx_cmd_modify_rqt
+	mlx5_devx_cmd_modify_sq
+	mlx5_devx_cmd_modify_tir
+	mlx5_devx_cmd_modify_virtq
+	mlx5_devx_cmd_qp_query_tis_td
+	mlx5_devx_cmd_query_hca_attr
+	mlx5_devx_cmd_query_parse_samples
+	mlx5_devx_cmd_query_virtq
+	mlx5_devx_cmd_register_read
+	mlx5_devx_get_out_command_status
+
+	mlx5_get_dbr
+
+	mlx5_malloc_mem_select
+	mlx5_mr_btree_init
+	mlx5_mr_btree_free
+	mlx5_mr_btree_dump
+	mlx5_mr_addr2mr_bh
+	mlx5_mr_release_cache
+	mlx5_mr_dump_cache
+	mlx5_mr_rebuild_cache
+	mlx5_mr_insert_cache
+	mlx5_mr_lookup_cache
+	mlx5_mr_lookup_list
+	mlx5_mr_create_primary
+	mlx5_mr_flush_local_cache
+	mlx5_mp_req_queue_state_modify
+
+	mlx5_pci_driver_register
+
+	mlx5_release_dbr
+
+	mlx5_malloc
+	mlx5_realloc
+	mlx5_free
+
-- 
2.8.4

[dpdk-dev] [PATCH v1 19/72] common/mlx5: extend DevX query hca attributes command

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Extend DevX API mlx5_devx_cmd_query_hca_attr() to report on max number
of available objects including: CQ, QP, PD, SRQ.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 11 +++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++++++++-
 drivers/common/mlx5/mlx5_prm.h       |  1 +
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 27eff5f..ee1baf8 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -718,6 +718,14 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->regex = MLX5_GET(cmd_hca_cap, hcattr, regexp);
 	attr->regexp_num_of_engines = MLX5_GET(cmd_hca_cap, hcattr,
 					       regexp_num_of_engines);
+	attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
+	attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
+	attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
+	attr->log_max_qp_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp_sz);
+	attr->log_max_mrw_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_mrw_sz);
+	attr->log_max_pd = MLX5_GET(cmd_hca_cap, hcattr, log_max_pd);
+	attr->log_max_srq = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq);
+	attr->log_max_srq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq_sz);
 	if (attr->qos.sup) {
 		MLX5_SET(query_hca_cap_in, in, op_mod,
 			 MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
@@ -832,6 +840,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->tunnel_stateless_gtp = MLX5_GET
 					(per_protocol_networking_offload_caps,
 					 hcattr, tunnel_stateless_gtp);
+	attr->rss_ind_tbl_cap = MLX5_GET
+					(per_protocol_networking_offload_caps,
+					 hcattr, rss_ind_tbl_cap);
 	if (attr->wqe_inline_mode != MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
 		return 0;
 	if (attr->eth_virt) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 25cf12e..b654e42 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -107,6 +107,15 @@ struct mlx5_hca_attr {
 	uint32_t log_max_ft_sampler_num:8;
 	struct mlx5_hca_qos_attr qos;
 	struct mlx5_hca_vdpa_attr vdpa;
+	int log_max_qp_sz;
+	int log_max_cq_sz;
+	int log_max_qp;
+	int log_max_cq;
+	uint32_t log_max_pd;
+	uint32_t log_max_mrw_sz;
+	uint32_t log_max_srq;
+	uint32_t log_max_srq_sz;
+	uint32_t rss_ind_tbl_cap;
 };
 
 struct mlx5_devx_wq_attr {
@@ -483,5 +492,4 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_virtio_q_counters(void *ctx);
 __rte_internal
 int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 				  struct mlx5_devx_virtio_q_couners_attr *attr);
-
 #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index d342263..8b9b694 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -649,6 +649,7 @@ typedef uint8_t u8;
 #define MLX5_GET64(typ, p, fld) rte_be_to_cpu_64(*((rte_be64_t *)(p) + \
 						   __mlx5_64_off(typ, fld)))
 #define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
+#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
 
 struct mlx5_ifc_fte_match_set_misc_bits {
 	u8 gre_c_present[0x1];
-- 
2.8.4

[dpdk-dev] [PATCH v1 20/72] common/mlx5: add DevX alloc PD command

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Add a new DevX API mlx5_devx_cmd_alloc_pd() that creates a new protection
domain (PD).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c            | 35 ++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  2 ++
 drivers/common/mlx5/mlx5_prm.h                  | 36 +++++++++++++++++++++++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  1 +
 drivers/common/mlx5/version.map                 |  1 +
 5 files changed, 75 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index ee1baf8..b9d11c3 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2002,3 +2002,38 @@ mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 					invalid_buffer);
 	return ret;
 }
+
+/*
+ * Create PD using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_alloc_pd(void *ctx)
+{
+	struct mlx5_devx_obj *ppd =
+		mlx5_malloc(MLX5_MEM_ZERO, sizeof(*ppd), 0, SOCKET_ID_ANY);
+	u32 in[MLX5_ST_SZ_DW(alloc_pd_in)] = {0};
+	u32 out[MLX5_ST_SZ_DW(alloc_pd_out)] = {0};
+
+	if (!ppd) {
+		DRV_LOG(ERR, "Failed to allocate PD data.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(alloc_pd_in, in, opcode, MLX5_CMD_OP_ALLOC_PD);
+	ppd->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+				out, sizeof(out));
+	if (!ppd->obj) {
+		mlx5_free(ppd);
+		DRV_LOG(ERR, "Failed to allocate PD Obj using DevX.");
+		rte_errno = errno;
+		return NULL;
+	}
+	ppd->id = MLX5_GET(alloc_pd_out, out, pd);
+	return ppd;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index b654e42..f3b2a7e 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -492,4 +492,6 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_virtio_q_counters(void *ctx);
 __rte_internal
 int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 				  struct mlx5_devx_virtio_q_couners_attr *attr);
+__rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_alloc_pd(void *ctx);
 #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 8b9b694..818f8c4 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -828,6 +828,8 @@ enum {
 	MLX5_CMD_OP_SUSPEND_QP = 0x50F,
 	MLX5_CMD_OP_RESUME_QP = 0x510,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
+	MLX5_CMD_OP_ALLOC_PD = 0x800,
+	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ACCESS_REGISTER = 0x805,
 	MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -2662,6 +2664,40 @@ struct mlx5_ifc_init2init_qp_in_bits {
 	u8 reserved_at_800[0x80];
 };
 
+struct mlx5_ifc_dealloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x8];
+	u8 pd[0x18];
+	u8 reserved_3[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x8];
+	u8 pd[0x18];
+	u8 reserved_2[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x40];
+};
+
 #ifdef PEDANTIC
 #pragma GCC diagnostic ignored "-Wpedantic"
 #endif
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 8d1f373..650f656 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -5,6 +5,7 @@ EXPORTS
 
 	mlx5_create_mr_ext
 
+	mlx5_devx_cmd_alloc_pd
 	mlx5_devx_cmd_create_cq
 	mlx5_devx_cmd_create_flex_parser
 	mlx5_devx_cmd_create_qp
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 884001c..bd1e965 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -10,6 +10,7 @@ INTERNAL {
 
 	mlx5_dev_to_pci_addr;
 
+	mlx5_devx_cmd_alloc_pd;
 	mlx5_devx_cmd_create_cq;
 	mlx5_devx_cmd_create_flex_parser;
 	mlx5_devx_cmd_create_qp;
-- 
2.8.4

[dpdk-dev] [PATCH v1 21/72] common/mlx5/windows: add glue functions APIs

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Windows glue functions are added to file mlx5/windows/mlx5_glue.c.
The following APIs are supported:
get_device_list, free_device_list, open_device, close_device,
query_device, query_hca_iseg, devx_obj_create, devx_obj_destroy,
devx_obj_query, devx_obj_modify, devx_general_cmd, devx_umem_reg,
devx_umem_dereg, devx_alloc_uar, devx_free_uar, devx_fs_rule_add,
devx_fs_rule_del, devx_query_eqn
New added files:
mlx5_win_defs.h - this file imports missing definitions from Linux
rdma-core library and Linux OS.
mlx5_win_ext.h - this file contains structs that enable a unified
Linux/Windows API. Each struct has an equivalent (but different) Linux
struct. By calling with 'void *' pointers - the Linux/Windows API is
identical.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |   1 +
 drivers/common/mlx5/windows/mlx5_glue.c         | 304 ++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 +++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  34 +++
 5 files changed, 422 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 650f656..2a68cd6 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -35,6 +35,7 @@ EXPORTS
 	mlx5_devx_get_out_command_status
 
 	mlx5_get_dbr
+	mlx5_glue
 
 	mlx5_malloc_mem_select
 	mlx5_mr_btree_init
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
new file mode 100644
index 0000000..7f8a00a
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -0,0 +1,304 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+
+#include "mlx5_glue.h"
+#include "mlx5_common_utils.h"
+#include "mlx5_win_ext.h"
+
+/*
+ * The returned value of this API is an array of pointers to mlx5
+ * devices under Windows. The interesting parameters of a device:
+ * Device PCI parameters: domain, bus, device id, function.
+ * Device name.
+ */
+static void *
+mlx5_glue_devx_get_device_list(int *num_devices)
+{
+	struct devx_device_bdf *devx_bdf_devs = NULL;
+	size_t n_devx_devx = 0;
+	int32_t ret = devx_get_device_list(&n_devx_devx, &devx_bdf_devs);
+
+	if (ret) {
+		errno = ret;
+		*num_devices = 0;
+		return NULL;
+	}
+	*num_devices = (int)n_devx_devx;
+	return devx_bdf_devs;
+}
+
+static void
+mlx5_glue_devx_free_device_list(void *list)
+{
+	if (!list) {
+		errno = EINVAL;
+		return;
+	}
+	devx_free_device_list(list);
+}
+
+static int
+mlx5_glue_devx_close_device(void *ctx)
+{
+	mlx5_context_st *mlx5_ctx;
+	int rc;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (mlx5_context_st *)ctx;
+	rc = devx_close_device(mlx5_ctx->devx_ctx);
+	free(mlx5_ctx);
+	return rc;
+}
+
+static void *
+mlx5_glue_devx_open_device(void *device)
+{
+	struct mlx5_context *mlx5_ctx;
+
+	if (!device) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = malloc((sizeof(struct mlx5_context)));
+	if (!mlx5_ctx) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(mlx5_ctx, 0, sizeof(*mlx5_ctx));
+	mlx5_ctx->devx_ctx = devx_open_device(device);
+	if (DEVX_IS_ERR(mlx5_ctx->devx_ctx)) {
+		errno = -DEVX_PTR_ERR(mlx5_ctx->devx_ctx);
+		free(mlx5_ctx);
+		return NULL;
+	}
+	return mlx5_ctx;
+}
+
+static int
+mlx5_glue_devx_query_device(void *device_bdf, void *dev_inf)
+{
+	struct devx_device_bdf *dev_bdf;
+	struct devx_device *mlx5_dev;
+
+	if (!device_bdf)
+		return -EINVAL;
+	dev_bdf = (struct devx_device_bdf *)device_bdf;
+	mlx5_dev = (struct devx_device *)dev_inf;
+	int err = devx_query_device(dev_bdf, mlx5_dev);
+	if (err)
+		return -E_FAIL;
+	return 0;
+}
+
+static void *
+mlx5_glue_devx_query_hca_iseg_mapping(void *ctx, uint32_t *cb_iseg)
+{
+	struct mlx5_context *mlx5_ctx;
+	void *pv_iseg;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	err = devx_query_hca_iseg_mapping(mlx5_ctx->devx_ctx,
+						cb_iseg, &pv_iseg);
+	if (err) {
+		errno = err;
+		return NULL;
+	}
+	return pv_iseg;
+}
+
+static void *
+mlx5_glue_devx_obj_create(void *ctx,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	devx_obj = malloc((sizeof(*devx_obj)));
+	if (!devx_obj) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(devx_obj, 0, sizeof(*devx_obj));
+	devx_obj->devx_ctx = GET_DEVX_CTX(ctx);
+	devx_obj->obj = devx_obj_create(devx_obj->devx_ctx,
+					in, inlen, out, outlen);
+	if (DEVX_IS_ERR(devx_obj->obj)) {
+		errno = -DEVX_PTR_ERR(devx_obj->obj);
+		free(devx_obj);
+		return NULL;
+	}
+	return devx_obj;
+}
+
+static int
+mlx5_glue_devx_obj_destroy(void *obj)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!obj)
+		return -EINVAL;
+	devx_obj = obj;
+	int rc = devx_obj_destroy(devx_obj->obj);
+	free(devx_obj);
+	return rc;
+}
+
+static int
+mlx5_glue_devx_general_cmd(void *ctx,
+			   void *in, size_t inlen,
+			   void *out, size_t outlen)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_cmd(GET_DEVX_CTX(ctx), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_query(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_modify(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_umem_dereg(void *pumem)
+{
+	struct devx_obj_handle *umem;
+
+	if (!pumem)
+		return -EINVAL;
+	umem = pumem;
+	return devx_umem_unreg(umem);
+}
+
+static void *
+mlx5_glue_devx_umem_reg(void *ctx, void *addr, size_t size,
+				  uint32_t access, uint32_t *id)
+{
+	struct devx_obj_handle *umem_hdl;
+	int w_access = DEVX_UMEM_ACCESS_READ;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	if (access)
+		w_access |= DEVX_UMEM_ACCESS_WRITE;
+
+	umem_hdl = devx_umem_reg(GET_DEVX_CTX(ctx), addr,
+					size, w_access, id);
+	if (DEVX_IS_ERR(umem_hdl)) {
+		errno = -DEVX_PTR_ERR(umem_hdl);
+		return NULL;
+	}
+	return umem_hdl;
+}
+
+static void *
+mlx5_glue_devx_alloc_uar(void *ctx,
+		uint32_t flags)
+{
+	devx_uar_handle *uar;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	uar = devx_alloc_uar(GET_DEVX_CTX(ctx), flags);
+	if (DEVX_IS_ERR(uar)) {
+		errno = -DEVX_PTR_ERR(uar);
+		return NULL;
+	}
+	return uar;
+}
+
+static int
+mlx5_glue_devx_query_eqn(void *ctx,
+		uint32_t cpus, uint32_t *eqn)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_query_eqn(GET_DEVX_CTX(ctx), cpus, eqn);
+}
+
+static void
+mlx5_glue_devx_free_uar(void *uar)
+{
+	devx_free_uar((devx_uar_handle *)uar);
+}
+
+static void*
+mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
+
+{
+	struct devx_obj_handle *rule_hdl = NULL;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	rule_hdl = devx_fs_rule_add(GET_DEVX_CTX(ctx), in, inlen);
+	if (DEVX_IS_ERR(rule_hdl)) {
+		errno = -DEVX_PTR_ERR(rule_hdl);
+		return NULL;
+	}
+	return rule_hdl;
+}
+
+static int
+mlx5_glue_devx_fs_rule_del(void *flow)
+{
+	return devx_fs_rule_del(flow);
+}
+
+alignas(RTE_CACHE_LINE_SIZE)
+const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+	.version = MLX5_GLUE_VERSION,
+	.get_device_list = mlx5_glue_devx_get_device_list,
+	.free_device_list = mlx5_glue_devx_free_device_list,
+	.open_device = mlx5_glue_devx_open_device,
+	.close_device = mlx5_glue_devx_close_device,
+	.query_device = mlx5_glue_devx_query_device,
+	.query_hca_iseg = mlx5_glue_devx_query_hca_iseg_mapping,
+	.devx_obj_create = mlx5_glue_devx_obj_create,
+	.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
+	.devx_obj_query = mlx5_glue_devx_obj_query,
+	.devx_obj_modify = mlx5_glue_devx_obj_modify,
+	.devx_general_cmd = mlx5_glue_devx_general_cmd,
+	.devx_umem_reg = mlx5_glue_devx_umem_reg,
+	.devx_umem_dereg = mlx5_glue_devx_umem_dereg,
+	.devx_alloc_uar = mlx5_glue_devx_alloc_uar,
+	.devx_free_uar = mlx5_glue_devx_free_uar,
+	.devx_fs_rule_add = mlx5_glue_devx_fs_rule_add,
+	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
+	.devx_query_eqn = mlx5_glue_devx_query_eqn,
+};
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
new file mode 100644
index 0000000..f2261ec
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_GLUE_H_
+#define MLX5_GLUE_H_
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include <rte_byteorder.h>
+#include <mlx5_win_defs.h>
+
+#ifndef MLX5_GLUE_VERSION
+#define MLX5_GLUE_VERSION ""
+#endif
+
+/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
+struct mlx5_glue {
+	const char *version;
+	void *(*devx_obj_create)(void *ctx,
+				 void *in, size_t inlen,
+				 void *out, size_t outlen);
+	int (*devx_obj_destroy)(void *obj);
+	int (*devx_obj_query)(void *obj,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen);
+	int (*devx_obj_modify)(void *obj,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_general_cmd)(void *ctx,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_umem_dereg)(void *umem);
+	void *(*devx_umem_reg)(void *ctx,
+			void *addr, size_t size,
+			uint32_t access, uint32_t *id);
+	void *(*devx_alloc_uar)(void *ctx,
+			uint32_t flags);
+	void (*devx_free_uar)(void *uar);
+	void *(*get_device_list)(int *num_devices);
+	void (*free_device_list)(void *list);
+	void *(*open_device)(void *device);
+	int (*close_device)(void *ctx);
+	int (*query_device)(void *device_bdf, void *dev_inf);
+	void* (*query_hca_iseg)(void *ctx, uint32_t *cb_iseg);
+	int (*devx_obj_query_async)(void *obj,
+				    const void *in, size_t inlen,
+				    size_t outlen, uint64_t wr_id,
+				    void *cmd_comp);
+	void *(*devx_fs_rule_add)(void *ctx, void *in, uint32_t inlen);
+	int (*devx_fs_rule_del)(void *flow);
+	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
+};
+
+extern const struct mlx5_glue *mlx5_glue;
+
+#endif /* MLX5_GLUE_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
new file mode 100644
index 0000000..72a3131
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_DEFS_H__
+#define __MLX5_WIN_DEFS_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum {
+	MLX5_CQE_OWNER_MASK	= 1,
+	MLX5_CQE_REQ		= 0,
+	MLX5_CQE_RESP_WR_IMM	= 1,
+	MLX5_CQE_RESP_SEND	= 2,
+	MLX5_CQE_RESP_SEND_IMM	= 3,
+	MLX5_CQE_RESP_SEND_INV	= 4,
+	MLX5_CQE_RESIZE_CQ	= 5,
+	MLX5_CQE_NO_PACKET	= 6,
+	MLX5_CQE_REQ_ERR	= 13,
+	MLX5_CQE_RESP_ERR	= 14,
+	MLX5_CQE_INVALID	= 15,
+};
+#endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
new file mode 100644
index 0000000..0e74910
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_ETX_H__
+#define __MLX5_WIN_ETX_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "mlx5devx.h"
+
+typedef struct mlx5_context {
+	devx_device_ctx        *devx_ctx;
+	struct devx_device mlx5_dev;
+
+} mlx5_context_st;
+
+typedef struct {
+	devx_device_ctx *devx_ctx;
+	struct devx_obj_handle *obj;
+} mlx5_devx_obj_st;
+
+struct mlx5_devx_umem {
+	void                   *addr;
+	struct devx_obj_handle *umem_hdl;
+	uint32_t                umem_id;
+};
+
+#define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
+#define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
+
+#endif /* __MLX5_WIN_ETX_H__ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 22/72] mlx5/windows: add mlx5 meson file

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

File drivers/common/mlx5/windows/meson.build is added to enable mlx5
source files compilation under common windows directory.
A Devx SDK tool must be installed to export two external H files:
mlx5devx.h and mlx5_ifc_devx.h. The installation is based on environment
variable DEVX_INC_PATH. In addition a DLL lib file is installed based on
environment variable DEVX_LIB_PATH. The meson file is using the
environment variables for compilation and linkage.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/meson.build         |  4 ++--
 drivers/common/mlx5/windows/meson.build | 42 +++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 2 deletions(-)
 create mode 100644 drivers/common/mlx5/windows/meson.build

diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build
index 3dacc6f..f8515d2 100644
--- a/drivers/common/mlx5/meson.build
+++ b/drivers/common/mlx5/meson.build
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2019 Mellanox Technologies, Ltd
 
-if not is_linux
+if not (is_linux or is_windows)
 	build = false
-	reason = 'only supported on Linux'
+	reason = 'only supported on Linux and Windows'
 	subdir_done()
 endif
 
diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
new file mode 100644
index 0000000..2b2aa3b
--- /dev/null
+++ b/drivers/common/mlx5/windows/meson.build
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2019 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+
+allow_experimental_apis = true
+sources += files(
+	'mlx5_glue.c',
+)
+
+res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
+res_inc = run_command(python3, '-c', 'import os; print(os.environ["DEVX_INC_PATH"])')
+
+if (res_lib.returncode() != 0 or res_inc.returncode() != 0)
+	build = false
+	reason = 'DevX environment variables are not set, DEVX_LIB_PATH and DEVX_INC_PATH vars must be exported'
+	subdir_done()
+endif
+
+devx_lib_dir = res_lib.stdout().strip()
+devx_inc_dir = res_inc.stdout().strip()
+
+ext_deps += cc.find_library('mlx5devx', dirs: devx_lib_dir, required: true)
+includes += include_directories(devx_inc_dir)
+cflags_options = [
+	'-std=c11',
+	'-Wno-strict-prototypes',
+	'-D_BSD_SOURCE',
+	'-D_DEFAULT_SOURCE',
+	'-D_XOPEN_SOURCE=600'
+]
+foreach option:cflags_options
+	if cc.has_argument(option)
+		cflags += option
+	endif
+endforeach
+if get_option('buildtype').contains('debug')
+	cflags += [ '-pedantic', '-DPEDANTIC' ]
+else
+	cflags += [ '-UPEDANTIC' ]
+endif
+
-- 
2.8.4

[dpdk-dev] [PATCH v1 23/72] mlx5/windows: add initialization routine for external lib

[Ophir Munk]

Add function mlx5_glue_constructor() as an initialization routine for
run-time dependency on external lib.  Currently the routine has an empty
body. It is used for compatibility with Linux.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build      |  1 +
 drivers/common/mlx5/windows/mlx5_common_os.c | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 2b2aa3b..9e74ebd 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -6,6 +6,7 @@ includes += include_directories('.')
 allow_experimental_apis = true
 sources += files(
 	'mlx5_glue.c',
+	'mlx5_common_os.c',
 )
 
 res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
new file mode 100644
index 0000000..5707fb6
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <unistd.h>
+#include <string.h>
+#include <stdio.h>
+
+#include <rte_errno.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "mlx5_devx_cmds.h"
+#include "mlx5_common_utils.h"
+#include <mlx5_common.h>
+
+/**
+ * Initialization routine for run-time dependency on external lib
+ */
+void
+mlx5_glue_constructor(void)
+{
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 24/72] mlx5/windows: generate file mlx5_autoconf.h

[Ophir Munk]

File mlx5_autoconf.h is generated under Windows to maintain
compatibility with the Linux build system.  This file is included in
Linux/Windows shared source files therefore it is required. Currently
the file is created empty.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 9e74ebd..5ac5724 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -41,3 +41,6 @@ else
 	cflags += [ '-UPEDANTIC' ]
 endif
 
+# Generate an empty mlx5_autoconf.h file for compatibility with Linux
+config = configuration_data()
+configure_file(output : 'mlx5_autoconf.h', configuration : config)
-- 
2.8.4

[dpdk-dev] [PATCH v1 25/72] common/mlx5/windows: extend PRM match_param_bits struct

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Add reserved size to PRM 'struct mlx5_ifc_fte_match_param_bits' for non
Linux OS.  Windows drivers require this extension since their expected
size should match the actual struct size. Linux drivers do not require
this extension and already use calculations based on the shorter size.
Use a static assert to verify that the PRM fte_match_param struct size
remains correct.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h          | 7 +++++++
 drivers/common/mlx5/windows/mlx5_glue.c | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 818f8c4..bdf8799 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -799,6 +799,13 @@ struct mlx5_ifc_fte_match_param_bits {
 	struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
 	struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
 	struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
+/*
+ * Add reserved bit to match the struct size with the size defined in PRM.
+ * This extension is not required in Linux.
+ */
+#ifndef HAVE_INFINIBAND_VERBS_H
+	u8 reserved_0[0x400];
+#endif
 };
 
 enum {
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 7f8a00a..3896cf0 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -256,6 +256,9 @@ mlx5_glue_devx_free_uar(void *uar)
 	devx_free_uar((devx_uar_handle *)uar);
 }
 
+static_assert(MLX5_ST_SZ_BYTES(fte_match_param) == 0x200,
+	"PRM size of fte_match_param is broken! cannot compile Windows!");
+
 static void*
 mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
 
-- 
2.8.4

[dpdk-dev] [PATCH v1 26/72] common/mlx5/windows: add getter functions

[Ophir Munk]

Add file mlx5/windows/mlx5_common_os.h the equivalent of Linux file
mlx5/linux/mlx5_common_os.h. It contains getters functions
mlx5_os_get_dev_device_name, mlx5_os_get_ctx_device_name,
mlx5_os_get_ctx_device_path, mlx5_os_get_umem_id,
mlx5_os_get_devx_channel_fd.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 91 ++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index cce7c88..2abdb2c 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -48,4 +48,95 @@ mlx5_os_free(void *addr)
 {
 	_aligned_free(addr);
 }
+
+/**
+ * Get fd. Given a pointer to DevX channel object of type
+ * 'struct mlx5dv_devx_event_channel*' - return its fd.
+ * Under Windows it is a stub.
+ *
+ * @param[in] channel
+ *    Pointer to channel object.
+ *
+ * @return
+ *    0
+ */
+static inline int
+mlx5_os_get_devx_channel_fd(void *channel)
+{
+	if (!channel)
+		return 0;
+	return 0;
+}
+
+/**
+ * Get device name. Given a device pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] dev
+ *   Pointer to device.
+ *
+ * @return
+ *   Pointer to device name if dev is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_dev_device_name(void *dev)
+{
+	if (!dev)
+		return NULL;
+	return ((struct devx_device *)dev)->name;
+}
+
+/**
+ * Get device name. Given a context pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device name if ctx is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_ctx_device_name(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.name;
+}
+
+/**
+ * Get a device path name. Given acontext pointer - return a
+ * pointer to the corresponding device path name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device path name if ctx is valid, NULL otherwise.
+ */
+
+static inline const char *
+mlx5_os_get_ctx_device_path(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.dev_pnp_id;
+}
+
+/**
+ * Get umem id. Given a pointer to umem object of type return its id.
+ *
+ * @param[in] umem
+ *    Pointer to umem object.
+ *
+ * @return
+ *    The umem id if umem is valid, 0 otherwise.
+ */
+static inline uint32_t
+mlx5_os_get_umem_id(void *umem)
+{
+	if (!umem)
+		return 0;
+	return ((struct mlx5_devx_umem *)umem)->umem_id;
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 27/72] common/mlx5/windows: add OS alloc/dealloc pd

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd(). They
are equivalent to the Linux implementation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
 drivers/common/mlx5/windows/mlx5_common_os.c    | 47 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
 4 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 2a68cd6..15dfa2e 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -59,4 +59,5 @@ EXPORTS
 	mlx5_malloc
 	mlx5_realloc
 	mlx5_free
-
+	mlx5_os_alloc_pd
+	mlx5_os_dealloc_pd
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 5707fb6..f77dfca 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -22,3 +22,50 @@ void
 mlx5_glue_constructor(void)
 {
 }
+
+/**
+ * Allocate PD. Given a devx context object
+ * return an mlx5-pd object.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *    The mlx5_pd if pd is valid, NULL and errno otherwise.
+ */
+void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
+		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
+	if (!ppd)
+		return NULL;
+
+	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
+	if (!obj)
+		return NULL;
+
+	ppd->obj = obj;
+	ppd->pdn = obj->id;
+	ppd->devx_ctx = ctx;
+	return ppd;
+}
+
+/**
+ * Release PD. Releases a given mlx5_pd object
+ *
+ * @param[in] pd
+ *   Pointer to mlx5_pd.
+ *
+ * @return
+ *    Zero if pd is released successfully, negative number otherwise.
+ */
+int
+mlx5_os_dealloc_pd(void *pd)
+{
+	if (!pd)
+		return -EINVAL;
+	mlx5_devx_cmd_destroy(((struct mlx5_pd *)pd)->obj);
+	mlx5_free(pd);
+	return 0;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index 2abdb2c..f47351e 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -139,4 +139,7 @@ mlx5_os_get_umem_id(void *umem)
 		return 0;
 	return ((struct mlx5_devx_umem *)umem)->umem_id;
 }
+
+void *mlx5_os_alloc_pd(void *ctx);
+int mlx5_os_dealloc_pd(void *pd);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index 0e74910..8e697b3 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -28,6 +28,12 @@ struct mlx5_devx_umem {
 	uint32_t                umem_id;
 };
 
+struct mlx5_pd {
+	void                   *obj;
+	uint32_t                pdn;
+	devx_device_ctx        *devx_ctx;
+};
+
 #define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
 #define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
 
-- 
2.8.4

[dpdk-dev] [PATCH v1 28/72] common/mlx5/windows: add OS umem reg/dereg API

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Implement Windows API mlx5_os_umem_reg() and mlx5_os_umem_dereg(). They
are equivalent to the Linux implementation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 60 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  2 +
 3 files changed, 64 insertions(+)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 15dfa2e..1d12e71 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -61,3 +61,5 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_umem_reg
+	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index f77dfca..c8343ad 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -69,3 +69,63 @@ mlx5_os_dealloc_pd(void *pd)
 	mlx5_free(pd);
 	return 0;
 }
+
+/**
+ * Register umem.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ * @param[in] addr
+ *   Pointer to memory start address.
+ * @param[in] size
+ *   Size of the memory to register.
+ * @param[out] access
+ *   UMEM access type
+ *
+ * @return
+ *   umem on successful registration, NULL and errno otherwise
+ */
+void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	struct mlx5_devx_umem *umem;
+
+	umem = mlx5_malloc(MLX5_MEM_ZERO,
+		(sizeof(*umem)), 0, SOCKET_ID_ANY);
+	if (!umem) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	umem->umem_hdl = mlx5_glue->devx_umem_reg(ctx, addr, size, access,
+		&umem->umem_id);
+	if (!umem->umem_hdl) {
+		mlx5_free(umem);
+		return NULL;
+	}
+	umem->addr = addr;
+	return umem;
+}
+
+/**
+ * Deregister umem.
+ *
+ * @param[in] pumem
+ *   Pointer to umem.
+ *
+ * @return
+ *   0 on successful release, negative number otherwise
+ */
+int
+mlx5_os_umem_dereg(void *pumem)
+{
+	struct mlx5_devx_umem *umem;
+	int err = 0;
+
+	if (!pumem)
+		return err;
+	umem = pumem;
+	if (umem->umem_hdl)
+		err = mlx5_glue->devx_umem_dereg(umem->umem_hdl);
+	mlx5_free(umem);
+	return err;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index f47351e..decb5ac 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -142,4 +142,6 @@ mlx5_os_get_umem_id(void *umem)
 
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
+void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
+int mlx5_os_umem_dereg(void *pumem);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 29/72] net/mlx5: update MR prototypes for DevX

[Ophir Munk]

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well.  Rename 'struct mlx5_verbs_ops' as 'struct mlx5_mr_ops' and
move it to shared file mlx5.h.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c    | 4 ++--
 drivers/net/mlx5/linux/mlx5_verbs.c | 2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h | 8 +-------
 drivers/net/mlx5/mlx5.h             | 6 ++++++
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index f0162d5..e55c16c 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -2387,8 +2387,8 @@ void
 mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
 		      mlx5_dereg_mr_t *dereg_mr_cb)
 {
-	*reg_mr_cb = mlx5_verbs_ops.reg_mr;
-	*dereg_mr_cb = mlx5_verbs_ops.dereg_mr;
+	*reg_mr_cb = mlx5_mr_verbs_ops.reg_mr;
+	*dereg_mr_cb = mlx5_mr_verbs_ops.dereg_mr;
 }
 
 /**
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c
index 494ddba..9d5ba23 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.c
+++ b/drivers/net/mlx5/linux/mlx5_verbs.c
@@ -62,7 +62,7 @@ mlx5_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
 }
 
 /* verbs operations. */
-const struct mlx5_verbs_ops mlx5_verbs_ops = {
+const struct mlx5_mr_ops mlx5_mr_verbs_ops = {
 	.reg_mr = mlx5_reg_mr,
 	.dereg_mr = mlx5_dereg_mr,
 };
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.h b/drivers/net/mlx5/linux/mlx5_verbs.h
index 0670f6c..76a79bf 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.h
+++ b/drivers/net/mlx5/linux/mlx5_verbs.h
@@ -7,16 +7,10 @@
 
 #include "mlx5.h"
 
-struct mlx5_verbs_ops {
-	mlx5_reg_mr_t reg_mr;
-	mlx5_dereg_mr_t dereg_mr;
-};
-
 int mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
 
 /* Verbs ops struct */
-extern const struct mlx5_verbs_ops mlx5_verbs_ops;
+extern const struct mlx5_mr_ops mlx5_mr_verbs_ops;
 extern struct mlx5_obj_ops ibv_obj_ops;
-
 #endif /* RTE_PMD_MLX5_VERBS_H_ */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 2004fa2..c8192df 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -823,6 +823,12 @@ struct mlx5_obj_ops {
 	void (*txq_obj_release)(struct mlx5_txq_obj *txq_obj);
 };
 
+/* MR operations structure. */
+struct mlx5_mr_ops {
+	mlx5_reg_mr_t reg_mr;
+	mlx5_dereg_mr_t dereg_mr;
+};
+
 struct mlx5_priv {
 	struct rte_eth_dev_data *dev_data;  /* Pointer to device data. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared device context. */
-- 
2.8.4

[dpdk-dev] [PATCH v1 30/72] common/mlx5/windows: add OS reg/dereg MR

[Ophir Munk]

This commits implements Windows API for MR registration and
deregistration. It is based on DevX.  Is support the relaxed ordering
flow in Windows by checking the capabilities and machine type.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.h            |  1 +
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 78 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  7 +++
 4 files changed, 88 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index a2c426d..c4f87a5 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -28,6 +28,7 @@ struct mlx5_pmd_mr {
 	void		     *addr;
 	size_t		     len;
 	void		     *obj;  /* verbs mr object or devx umem object. */
+	struct mlx5_devx_obj *mkey; /* devx mkey object. */
 };
 
 /**
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 1d12e71..23eee59 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -61,5 +61,7 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_dereg_mr
+	mlx5_os_reg_mr
 	mlx5_os_umem_reg
 	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index c8343ad..52d609a 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -14,6 +14,8 @@
 #include "mlx5_devx_cmds.h"
 #include "mlx5_common_utils.h"
 #include <mlx5_common.h>
+#include "mlx5_common_os.h"
+#include "mlx5_malloc.h"
 
 /**
  * Initialization routine for run-time dependency on external lib
@@ -129,3 +131,79 @@ mlx5_os_umem_dereg(void *pumem)
 	mlx5_free(umem);
 	return err;
 }
+
+/**
+ * Register mr. Given protection doamin pointer, pointer to addr and length
+ * register the memory region.
+ *
+ * @param[in] pd
+ *   Pointer to protection domain context (type mlx5_pd).
+ * @param[in] addr
+ *   Pointer to memory start address (type devx_device_ctx).
+ * @param[in] length
+ *   Lengtoh of the memory to register.
+ * @param[out] pmd_mr
+ *   pmd_mr struct set with lkey, address, length, pointer to mr object, mkey
+ *
+ * @return
+ *   0 on successful registration, -1 otherwise
+ */
+int
+mlx5_os_reg_mr(void *pd,
+	       void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr)
+{
+	struct mlx5_devx_mkey_attr mkey_attr;
+	struct mlx5_pd *mlx5_pd = (struct mlx5_pd *)pd;
+	struct mlx5_hca_attr attr;
+
+	if (!pd || !addr) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+	if (mlx5_devx_cmd_query_hca_attr(mlx5_pd->devx_ctx, &attr))
+		return -1;
+	pmd_mr->addr = addr;
+	pmd_mr->len = length;
+	pmd_mr->obj = mlx5_os_umem_reg(mlx5_pd->devx_ctx, pmd_mr->addr,
+				       pmd_mr->len, IBV_ACCESS_LOCAL_WRITE);
+	if (!pmd_mr->obj)
+		return -1;
+	mkey_attr.addr = (uintptr_t)addr;
+	mkey_attr.size = length;
+	mkey_attr.umem_id = ((struct mlx5_devx_umem *)(pmd_mr->obj))->umem_id;
+	mkey_attr.pd = mlx5_pd->pdn;
+	mkey_attr.log_entity_size = 0;
+	mkey_attr.pg_access = 0;
+	mkey_attr.klm_array = NULL;
+	mkey_attr.klm_num = 0;
+	mkey_attr.relaxed_ordering_read = 0;
+	mkey_attr.relaxed_ordering_write = 0;
+	if (!haswell_broadwell_cpu) {
+		mkey_attr.relaxed_ordering_write = attr.relaxed_ordering_write;
+		mkey_attr.relaxed_ordering_read = attr.relaxed_ordering_read;
+	}
+	pmd_mr->mkey = mlx5_devx_cmd_mkey_create(mlx5_pd->devx_ctx, &mkey_attr);
+	if (!pmd_mr->mkey) {
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+		return -1;
+	}
+	pmd_mr->lkey = pmd_mr->mkey->id;
+	return 0;
+}
+
+/**
+ * De-register mr.
+ *
+ * @param[in] pmd_mr
+ *  Pointer to PMD mr object
+ */
+void
+mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
+{
+	if (pmd_mr && pmd_mr->mkey)
+		claim_zero(mlx5_glue->devx_obj_destroy(pmd_mr->mkey->obj));
+	if (pmd_mr && pmd_mr->obj)
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index decb5ac..ba16641 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -7,9 +7,13 @@
 
 #include <stdio.h>
 
+#include <rte_errno.h>
+
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
 #include "mlx5_malloc.h"
+#include "mlx5_common_mr.h"
+#include "mlx5_win_ext.h"
 
 /**
  * This API allocates aligned or non-aligned memory.  The free can be on either
@@ -144,4 +148,7 @@ void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
 int mlx5_os_umem_dereg(void *pumem);
+int mlx5_os_reg_mr(void *pd,
+		   void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr);
+void mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 31/72] drivers/common: enable Windows common mlx5 compilation

[Ophir Munk]

Prior to this commit meson did not progress beyond directory
drivers/common for Windows compilations.  This commit enables drivers
compilation under this directory.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/meson.build | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index d78882f..be97424 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -1,10 +1,8 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Cavium, Inc
 
-if is_windows
-	subdir_done()
+if not is_windows
+	std_deps = ['eal']
+	drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
 endif
-
-std_deps = ['eal']
-drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
 config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON'
-- 
2.8.4

[dpdk-dev] [PATCH v1 32/72] net/mlx5/windows: implement device attribute getter

[Ophir Munk]

This commit is the Windows implementation of mlx5_os_get_dev_attr() API.
It follows the commit in (1). A new file named mlx5_os.c is added under
windows directory as its Linux counterpart file: linux/mlx5_os.c.

(1).
commit e85f623e13ea ("net/mlx5: remove attributes dependency on Verbs")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h       | 43 +++++++++++++++++++++
 drivers/net/mlx5/windows/meson.build |  8 ++++
 drivers/net/mlx5/windows/mlx5_os.c   | 75 ++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index bdf8799..33353a3 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -2193,6 +2193,49 @@ struct mlx5_ifc_cqc_bits {
 	u8 dbr_addr[0x40];
 };
 
+struct mlx5_ifc_health_buffer_bits {
+	u8         reserved_0[0x100];
+	u8         assert_existptr[0x20];
+	u8         assert_callra[0x20];
+	u8         reserved_1[0x40];
+	u8         fw_version[0x20];
+	u8         hw_id[0x20];
+	u8         reserved_2[0x20];
+	u8         irisc_index[0x8];
+	u8         synd[0x8];
+	u8         ext_synd[0x10];
+};
+
+struct mlx5_ifc_initial_seg_bits {
+	u8         fw_rev_minor[0x10];
+	u8         fw_rev_major[0x10];
+	u8         cmd_interface_rev[0x10];
+	u8         fw_rev_subminor[0x10];
+	u8         reserved_0[0x40];
+	u8         cmdq_phy_addr_63_32[0x20];
+	u8         cmdq_phy_addr_31_12[0x14];
+	u8         reserved_1[0x2];
+	u8         nic_interface[0x2];
+	u8         log_cmdq_size[0x4];
+	u8         log_cmdq_stride[0x4];
+	u8         command_doorbell_vector[0x20];
+	u8         reserved_2[0xf00];
+	u8         initializing[0x1];
+	u8         nic_interface_supported[0x7];
+	u8         reserved_4[0x18];
+	struct mlx5_ifc_health_buffer_bits health_buffer;
+	u8         no_dram_nic_offset[0x20];
+	u8         reserved_5[0x6de0];
+	u8         internal_timer_h[0x20];
+	u8         internal_timer_l[0x20];
+	u8         reserved_6[0x20];
+	u8         reserved_7[0x1f];
+	u8         clear_int[0x1];
+	u8         health_syndrome[0x8];
+	u8         health_counter[0x18];
+	u8         reserved_8[0x17fc0];
+};
+
 struct mlx5_ifc_create_cq_out_bits {
 	u8 status[0x8];
 	u8 reserved_at_8[0x18];
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
new file mode 100644
index 0000000..2ea0792
--- /dev/null
+++ b/drivers/net/mlx5/windows/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2020 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+sources += files(
+	'mlx5_os.c',
+)
+
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
new file mode 100644
index 0000000..e064667
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <rte_windows.h>
+
+#include <mlx5_glue.h>
+#include <mlx5_devx_cmds.h>
+#include <mlx5_common.h>
+
+#include "mlx5_defs.h"
+#include "mlx5.h"
+#include "mlx5_autoconf.h"
+
+/**
+ * Get mlx5 device attributes.
+ *
+ * @param ctx
+ *   Pointer to device context.
+ *
+ * @param device_attr
+ *   Pointer to mlx5 device attributes.
+ *
+ * @return
+ *   0 on success, non zero error number otherwise
+ */
+int
+mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
+{
+	struct mlx5_context *mlx5_ctx;
+	struct mlx5_hca_attr hca_attr;
+	void *pv_iseg = NULL;
+	u32 cb_iseg = 0;
+	int err = 0;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	memset(device_attr, 0, sizeof(*device_attr));
+	err = mlx5_devx_cmd_query_hca_attr(mlx5_ctx, &hca_attr);
+	if (err) {
+		DRV_LOG(ERR, "Failed to get device hca_cap");
+		return err;
+	}
+	device_attr->max_cq = 1 << hca_attr.log_max_cq;
+	device_attr->max_qp = 1 << hca_attr.log_max_qp;
+	device_attr->max_qp_wr = 1 << hca_attr.log_max_qp_sz;
+	device_attr->max_cqe = 1 << hca_attr.log_max_cq_sz;
+	device_attr->max_mr = 1 << hca_attr.log_max_mrw_sz;
+	device_attr->max_pd = 1 << hca_attr.log_max_pd;
+	device_attr->max_srq = 1 << hca_attr.log_max_srq;
+	device_attr->max_srq_wr = 1 << hca_attr.log_max_srq_sz;
+	if (hca_attr.rss_ind_tbl_cap) {
+		device_attr->max_rwq_indirection_table_size =
+			1 << hca_attr.rss_ind_tbl_cap;
+	}
+	pv_iseg = mlx5_glue->query_hca_iseg(mlx5_ctx, &cb_iseg);
+	if (pv_iseg == NULL) {
+		DRV_LOG(ERR, "Failed to get device hca_iseg");
+		return errno;
+	}
+	if (!err) {
+		snprintf(device_attr->fw_ver, 64, "%x.%x.%04x",
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_major),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_minor),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_subminor));
+	}
+	return err;
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 33/72] net/mlx5/windows: add mlx5_os.c stubs

[Ophir Munk]

mlx5_os_set_nonblock_channel_fd
mlx5_os_dev_shared_handler_install
mlx5_os_dev_shared_handler_uninstall
mlx5_os_read_dev_stat
mlx5_os_mac_addr_flush
mlx5_os_mac_addr_remove
mlx5_os_vf_mac_addr_modify
mlx5_os_set_promisc
mlx5_os_set_allmulti

Set struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops with NULL
pointers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 178 +++++++++++++++++++++++++++++++++++++
 1 file changed, 178 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index e064667..b0cc9f3 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -13,10 +13,18 @@
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
 #include <mlx5_common.h>
+#include <mlx5_common_mp.h>
+#include <mlx5_common_mr.h>
+#include <mlx5_malloc.h>
 
 #include "mlx5_defs.h"
 #include "mlx5.h"
+#include "mlx5_common_os.h"
+#include "mlx5_utils.h"
+#include "mlx5_rxtx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_mr.h"
+#include "mlx5_flow.h"
 
 /**
  * Get mlx5 device attributes.
@@ -73,3 +81,173 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	}
 	return err;
 }
+
+/**
+ * Set the completion channel file descriptor interrupt as non-blocking.
+ * Currently it has no support under Windows.
+ *
+ * @param[in] rxq_obj
+ *   Pointer to RQ channel object, which includes the channel fd
+ *
+ * @param[out] fd
+ *   The file descriptor (representing the intetrrupt) used in this channel.
+ *
+ * @return
+ *   0 on successfully setting the fd to non-blocking, non-zero otherwise.
+ */
+int
+mlx5_os_set_nonblock_channel_fd(int fd)
+{
+	(void)fd;
+	return -ENOTSUP;
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param sh
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+}
+
+/**
+ * Read statistics by a named counter.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ * @param[in] ctr_name
+ *   Pointer to the name of the statistic counter to read
+ * @param[out] stat
+ *   Pointer to read statistic value.
+ * @return
+ *   0 on success and stat is valud, 1 if failed to read the value
+ *   rte_errno is set.
+ *
+ */
+int
+mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name,
+		      uint64_t *stat)
+{
+	RTE_SET_USED(priv);
+	RTE_SET_USED(ctr_name);
+	RTE_SET_USED(stat);
+	return -ENOTSUP;
+}
+
+/**
+ * Flush device MAC addresses
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ */
+void
+mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
+{
+	(void)dev;
+}
+
+/**
+ * Remove a MAC address from device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param index
+ *   MAC address index.
+ */
+void
+mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+	(void)dev;
+	(void)(index);
+}
+
+/**
+ * Modify a VF MAC address
+ * Currently it has no support under Windows.
+ *
+ * @param priv
+ *   Pointer to device private data.
+ * @param mac_addr
+ *   MAC address to modify into.
+ * @param iface_idx
+ *   Net device interface index
+ * @param vf_index
+ *   VF index
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_vf_mac_addr_modify(struct mlx5_priv *priv,
+			   unsigned int iface_idx,
+			   struct rte_ether_addr *mac_addr,
+			   int vf_index)
+{
+	(void)priv;
+	(void)iface_idx;
+	(void)mac_addr;
+	(void)vf_index;
+	return -ENOTSUP;
+}
+
+/**
+ * Set device promiscuous mode
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - promiscuous is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_promisc(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	return -ENOTSUP;
+}
+
+/**
+ * Set device allmulti mode
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - all multicase is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	return -ENOTSUP;
+}
+
+const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
-- 
2.8.4

[dpdk-dev] [PATCH v1 34/72] net/mlx5/windows: implement mlx5 mac addr add

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Get the list of MAC addresses and verify if the input mac parameter
already exists. If not - return -ENOTSUP (as Windows does not support
adding new MAC addresses). If the MAC address exists (EEXIST) return 0
(the equivalent of Linux implementation of this API).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index b0cc9f3..6e27474 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -183,6 +183,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
 }
 
 /**
+ * Adds a MAC address to the device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param mac_addr
+ *   MAC address to register.
+ * @param index
+ *   MAC address index.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
+		     uint32_t index)
+{
+	(void)index;
+	struct rte_ether_addr lmac;
+
+	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
+		DRV_LOG(ERR,
+			"port %u cannot get MAC address, is mlx5_en"
+			" loaded? (errno: %s)",
+			dev->data->port_id, strerror(rte_errno));
+		return rte_errno;
+	}
+	if (memcmp(&lmac, mac, sizeof(struct rte_ether_addr))) {
+		DRV_LOG(ERR,
+			"adding new mac address to device is unsupported");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
+/**
  * Modify a VF MAC address
  * Currently it has no support under Windows.
  *
-- 
2.8.4

[dpdk-dev] [PATCH v1 35/72] net/mlx5: refactor eth dev ops for Windows

[Ophir Munk]

There are two types of eth_dev_ops used under Windows: primary and
isolate mode. Their function calls initialization is added to the OS
specific file mlx5_os.c. Secondary process eth_dev_ops is nullified.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 108 +++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 6e27474..572f586 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -287,3 +287,111 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
+
+const struct eth_dev_ops mlx5_os_dev_ops = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.read_clock = mlx5_read_clock,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.reta_update = mlx5_dev_rss_reta_update,
+	.reta_query = mlx5_dev_rss_reta_query,
+	.rss_hash_update = mlx5_rss_hash_update,
+	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
+
+/* Available operations from secondary process. */
+const struct eth_dev_ops mlx5_os_dev_sec_ops = {0};
+
+/* Available operations in flow isolated mode. */
+const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
-- 
2.8.4

[dpdk-dev] [PATCH v1 36/72] common/mlx5/windows: add missing DV and IBV definitions

[Ophir Munk]

Add missing DV and IBV definition to file mlx5_win_defs.h. The
definitions originated from rdma-core library which is not part of
Windows. They are referenced in shared files that must compile under
Windows such as mlx5_flow_dv.c and mlx5_rxtx.c.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_win_defs.h | 141 ++++++++++++++++++++++++++++
 1 file changed, 141 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
index 72a3131..e9569e4 100644
--- a/drivers/common/mlx5/windows/mlx5_win_defs.h
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -22,4 +22,145 @@ enum {
 	MLX5_CQE_RESP_ERR	= 14,
 	MLX5_CQE_INVALID	= 15,
 };
+
+enum {
+	MLX5_OPCODE_NOP			= 0x00,
+	MLX5_OPCODE_SEND_INVAL		= 0x01,
+	MLX5_OPCODE_RDMA_WRITE		= 0x08,
+	MLX5_OPCODE_RDMA_WRITE_IMM	= 0x09,
+	MLX5_OPCODE_SEND		= 0x0a,
+	MLX5_OPCODE_SEND_IMM		= 0x0b,
+	MLX5_OPCODE_TSO			= 0x0e,
+	MLX5_OPCODE_RDMA_READ		= 0x10,
+	MLX5_OPCODE_ATOMIC_CS		= 0x11,
+	MLX5_OPCODE_ATOMIC_FA		= 0x12,
+	MLX5_OPCODE_ATOMIC_MASKED_CS	= 0x14,
+	MLX5_OPCODE_ATOMIC_MASKED_FA	= 0x15,
+	MLX5_OPCODE_FMR			= 0x19,
+	MLX5_OPCODE_LOCAL_INVAL		= 0x1b,
+	MLX5_OPCODE_CONFIG_CMD		= 0x1f,
+	MLX5_OPCODE_UMR			= 0x25,
+	MLX5_OPCODE_TAG_MATCHING	= 0x28
+};
+
+enum mlx5dv_cq_init_attr_mask {
+	MLX5DV_CQ_INIT_ATTR_MASK_COMPRESSED_CQE	= 1 << 0,
+	MLX5DV_CQ_INIT_ATTR_MASK_FLAGS		= 1 << 1,
+	MLX5DV_CQ_INIT_ATTR_MASK_CQE_SIZE = 1 << 2,
+};
+
+enum mlx5dv_cqe_comp_res_format {
+	MLX5DV_CQE_RES_FORMAT_HASH		= 1 << 0,
+	MLX5DV_CQE_RES_FORMAT_CSUM		= 1 << 1,
+	MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX	= 1 << 2,
+};
+
+enum ibv_access_flags {
+	IBV_ACCESS_LOCAL_WRITE		= 1,
+	IBV_ACCESS_REMOTE_WRITE		= 1 << 1,
+	IBV_ACCESS_REMOTE_READ		= 1 << 2,
+	IBV_ACCESS_REMOTE_ATOMIC	= 1 << 3,
+	IBV_ACCESS_MW_BIND		= 1 << 4,
+	IBV_ACCESS_ZERO_BASED		= 1 << 5,
+	IBV_ACCESS_ON_DEMAND		= 1 << 6,
+};
+
+enum mlx5_ib_uapi_devx_create_event_channel_flags {
+	MLX5_IB_UAPI_DEVX_CR_EV_CH_FLAGS_OMIT_DATA = 1 << 0,
+};
+
+#define MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA \
+	MLX5_IB_UAPI_DEVX_CR_EV_CH_FLAGS_OMIT_DATA
+
+enum {
+	MLX5_CQE_SYNDROME_LOCAL_LENGTH_ERR		= 0x01,
+	MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR		= 0x02,
+	MLX5_CQE_SYNDROME_LOCAL_PROT_ERR		= 0x04,
+	MLX5_CQE_SYNDROME_WR_FLUSH_ERR			= 0x05,
+	MLX5_CQE_SYNDROME_MW_BIND_ERR			= 0x06,
+	MLX5_CQE_SYNDROME_BAD_RESP_ERR			= 0x10,
+	MLX5_CQE_SYNDROME_LOCAL_ACCESS_ERR		= 0x11,
+	MLX5_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR		= 0x12,
+	MLX5_CQE_SYNDROME_REMOTE_ACCESS_ERR		= 0x13,
+	MLX5_CQE_SYNDROME_REMOTE_OP_ERR			= 0x14,
+	MLX5_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR	= 0x15,
+	MLX5_CQE_SYNDROME_RNR_RETRY_EXC_ERR		= 0x16,
+	MLX5_CQE_SYNDROME_REMOTE_ABORTED_ERR		= 0x22,
+};
+
+enum {
+	MLX5_ETH_WQE_L3_CSUM = (1 << 6),
+	MLX5_ETH_WQE_L4_CSUM = (1 << 7),
+};
+
+/*
+ * RX Hash fields enable to set which incoming packet's field should
+ * participates in RX Hash. Each flag represent certain packet's field,
+ * when the flag is set the field that is represented by the flag will
+ * participate in RX Hash calculation.
+ * Note: IPV4 and IPV6 flags can't be enabled together on the same QP,
+ * TCP and UDP flags can't be enabled together on the same QP.
+ */
+enum ibv_rx_hash_fields {
+	IBV_RX_HASH_SRC_IPV4	= 1 << 0,
+	IBV_RX_HASH_DST_IPV4	= 1 << 1,
+	IBV_RX_HASH_SRC_IPV6	= 1 << 2,
+	IBV_RX_HASH_DST_IPV6	= 1 << 3,
+	IBV_RX_HASH_SRC_PORT_TCP	= 1 << 4,
+	IBV_RX_HASH_DST_PORT_TCP	= 1 << 5,
+	IBV_RX_HASH_SRC_PORT_UDP	= 1 << 6,
+	IBV_RX_HASH_DST_PORT_UDP	= 1 << 7,
+	IBV_RX_HASH_IPSEC_SPI		= 1 << 8,
+	IBV_RX_HASH_INNER		= (1 << 31),
+};
+
+enum {
+	MLX5_RCV_DBR	= 0,
+	MLX5_SND_DBR	= 1,
+};
+
+#ifndef MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2
+#define MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TUNNEL_TO_L2	0x0
+#endif
+#ifndef MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL
+#define MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L2_TUNNEL	0x1
+#endif
+#ifndef MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2
+#define MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L3_TUNNEL_TO_L2	0x2
+#endif
+#ifndef MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL
+#define MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL	0x3
+#endif
+
+struct mlx5_err_cqe {
+	uint8_t		rsvd0[32];
+	uint32_t	srqn;
+	uint8_t		rsvd1[18];
+	uint8_t		vendor_err_synd;
+	uint8_t		syndrome;
+	uint32_t	s_wqe_opcode_qpn;
+	uint16_t	wqe_counter;
+	uint8_t		signature;
+	uint8_t		op_own;
+};
+
+struct mlx5_wqe_srq_next_seg {
+	uint8_t			rsvd0[2];
+	rte_be16_t		next_wqe_index;
+	uint8_t			signature;
+	uint8_t			rsvd1[11];
+};
+
+enum ibv_wq_state {
+	IBV_WQS_RESET,
+	IBV_WQS_RDY,
+	IBV_WQS_ERR,
+	IBV_WQS_UNKNOWN
+};
+
+struct mlx5_wqe_data_seg {
+	rte_be32_t		byte_count;
+	rte_be32_t		lkey;
+	rte_be64_t		addr;
+};
 #endif /* __MLX5_WIN_DEFS_H__ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 37/72] mlx5/windows: add mlx5_os header file under net

[Ophir Munk]

File drivers/net/mlx5/windows/mlx5_os.h is added. It includes specific
Windows definitions such as PCI driver flags link state changes
interrupts, link removal interrupts.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.h

diff --git a/drivers/net/mlx5/windows/mlx5_os.h b/drivers/net/mlx5/windows/mlx5_os.h
new file mode 100644
index 0000000..bb1aa27
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_os.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_OS_H_
+#define RTE_PMD_MLX5_OS_H_
+/* sysfs equivalent under Windows */
+enum {
+	SYSFS_NAME_MAX = 64,
+	SYSFS_PATH_MAX = 256
+};
+enum {
+	DEV_SYSFS_NAME_MAX = SYSFS_NAME_MAX + 1,
+	DEV_SYSFS_PATH_MAX = SYSFS_PATH_MAX + 1
+};
+
+#define PCI_DRV_FLAGS 0
+
+#endif /* RTE_PMD_MLX5_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 38/72] net/mlx5/windows: add pthread initializer definition

[Ophir Munk]

PTHREAD_MUTEX_INITIALIZER is a Linux macro defined in
/usr/include/pthread.h. It is used by mlx5 PMD but it is not included
in DPDK Windows pthread implementation. Therefore define it privately
in windows/mlx5_os.h file.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.h b/drivers/net/mlx5/windows/mlx5_os.h
index bb1aa27..5f4abc3 100644
--- a/drivers/net/mlx5/windows/mlx5_os.h
+++ b/drivers/net/mlx5/windows/mlx5_os.h
@@ -16,4 +16,5 @@ enum {
 
 #define PCI_DRV_FLAGS 0
 
+#define PTHREAD_MUTEX_INITIALIZER {(void *)-1, -1, 0, 0, 0, 0}
 #endif /* RTE_PMD_MLX5_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 39/72] net/mlx5/windows: define epoll API to do nothing

[Ophir Munk]

This is a hack commit. Linux epoll mechanism is not implemented in
Windows.  Therefore define rte_intr_free_epoll_fd API to do nothing.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.h b/drivers/net/mlx5/windows/mlx5_os.h
index 5f4abc3..dd3515c 100644
--- a/drivers/net/mlx5/windows/mlx5_os.h
+++ b/drivers/net/mlx5/windows/mlx5_os.h
@@ -17,4 +17,5 @@ enum {
 #define PCI_DRV_FLAGS 0
 
 #define PTHREAD_MUTEX_INITIALIZER {(void *)-1, -1, 0, 0, 0, 0}
+#define rte_intr_free_epoll_fd
 #endif /* RTE_PMD_MLX5_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 40/72] net/mlx5/windows: define errno ETOOMANYREFS

[Ophir Munk]

Errno ETOOMANYREFS may not be supported under Windows. In such cases
add its definition.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.h b/drivers/net/mlx5/windows/mlx5_os.h
index dd3515c..563cfa4 100644
--- a/drivers/net/mlx5/windows/mlx5_os.h
+++ b/drivers/net/mlx5/windows/mlx5_os.h
@@ -18,4 +18,7 @@ enum {
 
 #define PTHREAD_MUTEX_INITIALIZER {(void *)-1, -1, 0, 0, 0, 0}
 #define rte_intr_free_epoll_fd
+#ifndef ETOOMANYREFS
+#define ETOOMANYREFS 109     /* Too many references: cannot splice */
+#endif
 #endif /* RTE_PMD_MLX5_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 41/72] common/mlx5: add rte compatibility header file

[Ophir Munk]

Add #include <rte_compat.h> to file mlx5_devx_cmds.h. It is required for
Windows to identify the  __rte_internal definition.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index f3b2a7e..c972391 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -7,7 +7,7 @@
 
 #include "mlx5_glue.h"
 #include "mlx5_prm.h"
-
+#include <rte_compat.h>
 
 /* This is limitation of libibverbs: in length variable type is u16. */
 #define MLX5_DEVX_MAX_KLM_ENTRIES ((UINT16_MAX - \
-- 
2.8.4

[dpdk-dev] [PATCH v1 42/72] common/mlx5/windows: add DevX UAR getters

[Ophir Munk]

The following getters are added: mlx5_os_get_devx_uar_mmap_offset,
mlx5_os_get_devx_uar_base_addr, mlx5_os_get_devx_uar_reg_addr,
mlx5_os_get_devx_uar_page_id.  This commit is the Windows equivalent of
the Linux implementation in (1).

(1)
commit 8638e19a10aa ("net/mlx5: remove more DV dependencies")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 74 ++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index ba16641..9ff6bef 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -15,6 +15,8 @@
 #include "mlx5_common_mr.h"
 #include "mlx5_win_ext.h"
 
+#define MLX5_BF_OFFSET 0x800
+
 /**
  * This API allocates aligned or non-aligned memory.  The free can be on either
  * aligned or nonaligned memory.  To be protected - even though there may be no
@@ -144,6 +146,78 @@ mlx5_os_get_umem_id(void *umem)
 	return ((struct mlx5_devx_umem *)umem)->umem_id;
 }
 
+/**
+ * Get mmap offset. Given a pointer to an DevX UAR object of type
+ * 'struct mlx5dv_devx_uar *' - return its mmap offset.
+ *
+ * @param[in] uar
+ *    Pointer to UAR object.
+ *
+ * @return
+ *    The mmap offset if uar is valid, 0 otherwise.
+ */
+static inline off_t
+mlx5_os_get_devx_uar_mmap_offset(void *uar)
+{
+	if (!uar)
+		return 0;
+	return 0;
+}
+
+/**
+ * Get base addr pointer. Given a pointer to an UAR object of type
+ * 'struct mlx5dv_devx_uar *' - return its base address.
+ *
+ * @param[in] uar
+ *    Pointer to an UAR object.
+ *
+ * @return
+ *    The base address if UAR is valid, NULL otherwise.
+ */
+static inline void *
+mlx5_os_get_devx_uar_base_addr(void *uar)
+{
+	if (!uar)
+		return NULL;
+	return ((devx_uar_handle *)uar)->uar_page;
+}
+
+/**
+ * Get reg addr pointer. Given a pointer to an UAR object of type
+ * 'struct mlx5dv_devx_uar *' - return its reg address.
+ *
+ * @param[in] uar
+ *    Pointer to an UAR object.
+ *
+ * @return
+ *    The reg address if UAR is valid, NULL otherwise.
+ */
+static inline void *
+mlx5_os_get_devx_uar_reg_addr(void *uar)
+{
+	if (!uar)
+		return NULL;
+	return ((char *)((devx_uar_handle *)uar)->uar_page) + MLX5_BF_OFFSET;
+}
+
+/**
+ * Get page id. Given a pointer to an UAR object of type
+ * 'struct mlx5dv_devx_uar *' - return its page id.
+ *
+ * @param[in] uar
+ *    Pointer to an UAR object.
+ *
+ * @return
+ *    The page id if UAR is valid, 0 otherwise.
+ */
+static inline uint32_t
+mlx5_os_get_devx_uar_page_id(void *uar)
+{
+	if (!uar)
+		return 0;
+	return ((devx_uar_handle *)uar)->uar_index;
+}
+
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
-- 
2.8.4

[dpdk-dev] [PATCH v1 43/72] common/mlx5/windows: wrap event channel APIs with OS calls

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit is the Windows equivalent of the Linux implementation.
Windows returns an error ENOTSUP for the APIs to create/destroy event
channel or to subscribe an event.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 29 ++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index 9ff6bef..855a93f 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -218,6 +218,35 @@ mlx5_os_get_devx_uar_page_id(void *uar)
 	return ((devx_uar_handle *)uar)->uar_index;
 }
 
+static inline void *
+mlx5_os_devx_create_event_channel(void *ctx, int flags)
+{
+	(void)ctx;
+	(void)flags;
+	errno = ENOTSUP;
+	return NULL;
+}
+
+static inline void
+mlx5_os_devx_destroy_event_channel(void *eventc)
+{
+	(void)eventc;
+}
+
+static inline int
+mlx5_os_devx_subscribe_devx_event(void *eventc,
+				    void *obj,
+				    uint16_t events_sz, uint16_t events_num[],
+				    uint64_t cookie)
+{
+	(void)eventc;
+	(void)obj;
+	(void)events_sz;
+	(void)events_num;
+	(void)cookie;
+	return -ENOTSUP;
+}
+
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
-- 
2.8.4

[dpdk-dev] [PATCH v1 44/72] net/mlx5/windows: add memory region callbacks

[Ophir Munk]

This commit is the Windows part implementation of (1).

(1)
commit d5ed8aa9449d ("net/mlx5: add memory region callbacks
			 in per-device cache")'

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 572f586..e334110 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -286,6 +286,23 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 	return -ENOTSUP;
 }
 
+/**
+ * Set the reg_mr and dereg_mr call backs
+ *
+ * @param reg_mr_cb[out]
+ *   Pointer to reg_mr func
+ * @param dereg_mr_cb[out]
+ *   Pointer to dereg_mr func
+ *
+ */
+void
+mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
+		      mlx5_dereg_mr_t *dereg_mr_cb)
+{
+	*reg_mr_cb = mlx5_os_reg_mr;
+	*dereg_mr_cb = mlx5_os_dereg_mr;
+}
+
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
 
 const struct eth_dev_ops mlx5_os_dev_ops = {
-- 
2.8.4

[dpdk-dev] [PATCH v1 45/72] net/mlx5/windows: add stubs for MP requests

[Ophir Munk]

Windows supports the primary process with no secondary process control.
This commit adds stubs for requests to start/stop the data-path to the
secondary process and for requests to start/stop a queue of the primary
process.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/meson.build  |  1 +
 drivers/net/mlx5/windows/mlx5_mp_os.c | 71 +++++++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/mlx5_mp_os.c

diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
index 2ea0792..f87dcb1 100644
--- a/drivers/net/mlx5/windows/meson.build
+++ b/drivers/net/mlx5/windows/meson.build
@@ -4,5 +4,6 @@
 includes += include_directories('.')
 sources += files(
 	'mlx5_os.c',
+	'mlx5_mp_os.c',
 )
 
diff --git a/drivers/net/mlx5/windows/mlx5_mp_os.c b/drivers/net/mlx5/windows/mlx5_mp_os.c
new file mode 100644
index 0000000..db793b1
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_mp_os.c
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <stdio.h>
+#include <time.h>
+
+#include <rte_eal.h>
+#include <rte_ethdev_driver.h>
+#include <rte_string_fns.h>
+
+#include <mlx5_common_mp.h>
+#include <mlx5_common_mr.h>
+#include <mlx5_malloc.h>
+
+#include "mlx5.h"
+#include "mlx5_rxtx.h"
+#include "mlx5_utils.h"
+
+/**
+ * Broadcast request of starting data-path to secondary processes. The request
+ * is synchronous.
+ *
+ * @param[in] dev
+ *   Pointer to Ethernet structure.
+ */
+void
+mlx5_mp_os_req_start_rxtx(struct rte_eth_dev *dev)
+{
+	RTE_SET_USED(dev);
+}
+
+/**
+ * Broadcast request of stopping data-path to secondary processes. The request
+ * is synchronous.
+ *
+ * @param[in] dev
+ *   Pointer to Ethernet structure.
+ */
+void
+mlx5_mp_os_req_stop_rxtx(struct rte_eth_dev *dev)
+{
+	RTE_SET_USED(dev);
+}
+
+/**
+ * Request Verbs Rx/Tx queue stop or start to the primary process.
+ *
+ * @param[in] dev
+ *   Pointer to Ethernet structure.
+ * @param queue_id
+ *   Queue ID to control.
+ * @param req_type
+ *   request type
+ *     MLX5_MP_REQ_QUEUE_RX_START - start Rx queue
+ *     MLX5_MP_REQ_QUEUE_TX_START - stop Tx queue
+ *     MLX5_MP_REQ_QUEUE_RX_STOP - stop Rx queue
+ *     MLX5_MP_REQ_QUEUE_TX_STOP - stop Tx queue
+ * @return
+ *   0 on success, a negative errno value otherwise and
+ *     rte_errno is set.
+ */
+int
+mlx5_mp_os_req_queue_control(struct rte_eth_dev *dev, uint16_t queue_id,
+			  enum mlx5_mp_req_type req_type)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(queue_id);
+	RTE_SET_USED(req_type);
+	return -ENOTSUP;
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 46/72] net/mlx5/windows: support get mac

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commits implements API mlx5_get_mac().  It returns the MAC address
saved in the device context since its creation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/meson.build      |  1 +
 drivers/net/mlx5/windows/mlx5_ethdev_os.c | 43 +++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/mlx5_ethdev_os.c

diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
index f87dcb1..b5d9f0e 100644
--- a/drivers/net/mlx5/windows/meson.build
+++ b/drivers/net/mlx5/windows/meson.build
@@ -5,5 +5,6 @@ includes += include_directories('.')
 sources += files(
 	'mlx5_os.c',
 	'mlx5_mp_os.c',
+	'mlx5_ethdev_os.c',
 )
 
diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
new file mode 100644
index 0000000..0662c09
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+#include <stdio.h>
+
+#include <rte_errno.h>
+#include <rte_ether.h>
+#include <rte_ethdev_driver.h>
+#include <rte_interrupts.h>
+
+#include <mlx5_glue.h>
+#include <mlx5_devx_cmds.h>
+#include <mlx5_common.h>
+#include <mlx5_win_ext.h>
+#include <mlx5_malloc.h>
+#include <mlx5.h>
+
+/**
+ * Get MAC address by querying netdevice.
+ *
+ * @param[in] dev
+ *   Pointer to Ethernet device.
+ * @param[out] mac
+ *   MAC address output buffer.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN])
+{
+	struct mlx5_priv *priv;
+	mlx5_context_st *context_obj;
+
+	if (!dev) {
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	priv = dev->data->dev_private;
+	context_obj = (mlx5_context_st *)priv->sh->ctx;
+	memcpy(mac, context_obj->mlx5_dev.eth_mac, RTE_ETHER_ADDR_LEN);
+	return 0;
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 47/72] net/mlx5/windows: add ethdev stub operations

[Ophir Munk]

This commits adds ethdev stubs. These APIs are called from shared code
that must compile under Linux and Windows. The following stubs are added:
mlx5_set_mtu
mlx5_os_read_dev_counters
mlx5_intr_callback_unregister
mlx5_os_get_stats_n
mlx5_os_stats_init
mlx5_set_link_down
mlx5_set_link_up
mlx5_dev_get_flow_ctrl
mlx5_dev_set_flow_ctrl
mlx5_get_module_info
mlx5_get_module_eeprom

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_ethdev_os.c | 199 ++++++++++++++++++++++++++++++
 1 file changed, 199 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index 0662c09..4925fd8 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -41,3 +41,202 @@ mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN])
 	memcpy(mac, context_obj->mlx5_dev.eth_mac, RTE_ETHER_ADDR_LEN);
 	return 0;
 }
+
+/**
+ * Set device MTU.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param mtu
+ *   MTU value to set.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_set_mtu(struct rte_eth_dev *dev, uint16_t mtu)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(mtu);
+	return -ENOTSUP;
+}
+
+/*
+ * Unregister callback handler safely. The handler may be active
+ * while we are trying to unregister it, in this case code -EAGAIN
+ * is returned by rte_intr_callback_unregister(). This routine checks
+ * the return code and tries to unregister handler again.
+ *
+ * @param handle
+ *   interrupt handle
+ * @param cb_fn
+ *   pointer to callback routine
+ * @cb_arg
+ *   opaque callback parameter
+ */
+void
+mlx5_intr_callback_unregister(const struct rte_intr_handle *handle,
+			      rte_intr_callback_fn cb_fn, void *cb_arg)
+{
+	RTE_SET_USED(handle);
+	RTE_SET_USED(cb_fn);
+	RTE_SET_USED(cb_arg);
+}
+
+/**
+ * DPDK callback to get flow control status.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param[out] fc_conf
+ *   Flow control output buffer.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_dev_get_flow_ctrl(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(fc_conf);
+	return -ENOTSUP;
+}
+
+/**
+ * DPDK callback to modify flow control parameters.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param[in] fc_conf
+ *   Flow control parameters.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_dev_set_flow_ctrl(struct rte_eth_dev *dev, struct rte_eth_fc_conf *fc_conf)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(fc_conf);
+	return -ENOTSUP;
+}
+
+/**
+ * Query the number of statistics provided by ETHTOOL.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   Number of statistics on success, negative errno value otherwise and
+ *   rte_errno is set.
+ */
+int
+mlx5_os_get_stats_n(struct rte_eth_dev *dev)
+{
+	RTE_SET_USED(dev);
+	return -ENOTSUP;
+}
+
+/**
+ * Init the structures to read device counters.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ */
+void
+mlx5_os_stats_init(struct rte_eth_dev *dev)
+{
+	RTE_SET_USED(dev);
+}
+
+/**
+ * Read device counters table.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param[out] stats
+ *   Counters table output buffer.
+ *
+ * @return
+ *   0 on success and stats is filled, negative errno value otherwise and
+ *   rte_errno is set.
+ */
+int
+mlx5_os_read_dev_counters(struct rte_eth_dev *dev, uint64_t *stats)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(stats);
+	return -ENOTSUP;
+}
+
+/**
+ * DPDK callback to bring the link DOWN.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_set_link_down(struct rte_eth_dev *dev)
+{
+	RTE_SET_USED(dev);
+	return -ENOTSUP;
+}
+
+/**
+ * DPDK callback to bring the link UP.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_set_link_up(struct rte_eth_dev *dev)
+{
+	RTE_SET_USED(dev);
+	return -ENOTSUP;
+}
+
+/**
+ * DPDK callback to retrieve plug-in module EEPROM information (type and size).
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param[out] modinfo
+ *   Storage for plug-in module EEPROM information.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_get_module_info(struct rte_eth_dev *dev,
+		     struct rte_eth_dev_module_info *modinfo)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(modinfo);
+	return -ENOTSUP;
+}
+
+/**
+ * DPDK callback to retrieve plug-in module EEPROM data.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param[out] info
+ *   Storage for plug-in module EEPROM data.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int mlx5_get_module_eeprom(struct rte_eth_dev *dev,
+			   struct rte_dev_eeprom_info *info)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(info);
+	return -ENOTSUP;
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 48/72] net/mlx5/windows: support link update

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Add support for mlx5_link_update() to get link speed and link state.
Other parameters are currently hard-coded.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_ethdev_os.c | 39 +++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index 4925fd8..0c45101 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -171,6 +171,45 @@ mlx5_os_read_dev_counters(struct rte_eth_dev *dev, uint64_t *stats)
 }
 
 /**
+ * DPDK callback to retrieve physical link information.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param wait_to_complete
+ *   Wait for request completion.
+ *
+ * @return
+ *   0 if link status was not updated, positive if it was, a negative errno
+ *   value otherwise and rte_errno is set.
+ */
+int
+mlx5_link_update(struct rte_eth_dev *dev, int wait_to_complete)
+{
+	RTE_SET_USED(wait_to_complete);
+	struct mlx5_priv *priv;
+	mlx5_context_st *context_obj;
+	struct rte_eth_link dev_link;
+	int ret;
+
+	ret = 0;
+	if (!dev) {
+		rte_errno = EINVAL;
+		return rte_errno;
+	}
+	priv = dev->data->dev_private;
+	context_obj = (mlx5_context_st *)priv->sh->ctx;
+	dev_link.link_speed = context_obj->mlx5_dev.link_speed / (1024 * 1024);
+	dev_link.link_status =
+	      (context_obj->mlx5_dev.link_state == 1 && !mlx5_is_removed(dev))
+	      ? 1 : 0;
+	dev_link.link_duplex = 1;
+	ret = !!memcmp(&dev->data->dev_link, &dev_link,
+		       sizeof(struct rte_eth_link));
+	dev->data->dev_link = dev_link;
+	return ret;
+}
+
+/**
  * DPDK callback to bring the link DOWN.
  *
  * @param dev
-- 
2.8.4

[dpdk-dev] [PATCH v1 49/72] net/mlx5/windows: support read clock

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit adds a new glue function query_rt_values to support the new
API mlx5_read_clock().

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h             |  1 +
 drivers/common/mlx5/windows/mlx5_glue.c    | 26 ++++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h    |  1 +
 drivers/common/mlx5/windows/mlx5_win_ext.h |  6 ++++++
 drivers/net/mlx5/windows/mlx5_ethdev_os.c  | 30 ++++++++++++++++++++++++++++++
 5 files changed, 64 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 33353a3..1dfd6bd 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -578,6 +578,7 @@ struct mlx5_modification_cmd {
 	};
 };
 
+typedef uint64_t u64;
 typedef uint32_t u32;
 typedef uint16_t u16;
 typedef uint8_t u8;
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 3896cf0..1675318 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -283,6 +283,31 @@ mlx5_glue_devx_fs_rule_del(void *flow)
 	return devx_fs_rule_del(flow);
 }
 
+static int
+mlx5_glue_query_rt_values(void *ctx, void *devx_clock)
+{
+	struct mlx5_context *mlx5_ctx;
+	struct mlx5_devx_clock *clock;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return errno;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	clock = (struct mlx5_devx_clock *)devx_clock;
+	err = devx_hca_clock_query(
+			mlx5_ctx->devx_ctx,
+			&clock->p_iseg_internal_timer,
+			&clock->clock_frequency_hz,
+			&clock->is_stable_clock_frequency);
+	if (err) {
+		errno = err;
+		return errno;
+	}
+	return 0;
+}
+
 alignas(RTE_CACHE_LINE_SIZE)
 const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
 	.version = MLX5_GLUE_VERSION,
@@ -304,4 +329,5 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
 	.devx_fs_rule_add = mlx5_glue_devx_fs_rule_add,
 	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
 	.devx_query_eqn = mlx5_glue_devx_query_eqn,
+	.query_rt_values = mlx5_glue_query_rt_values,
 };
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
index f2261ec..bfaaa2a 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.h
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -51,6 +51,7 @@ struct mlx5_glue {
 	void *(*devx_fs_rule_add)(void *ctx, void *in, uint32_t inlen);
 	int (*devx_fs_rule_del)(void *flow);
 	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
+	int (*query_rt_values)(void *ctx, void *devx_clock);
 };
 
 extern const struct mlx5_glue *mlx5_glue;
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index 8e697b3..a14a6cc 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -34,6 +34,12 @@ struct mlx5_pd {
 	devx_device_ctx        *devx_ctx;
 };
 
+struct mlx5_devx_clock {
+	void *p_iseg_internal_timer;
+	u64 clock_frequency_hz;
+	int is_stable_clock_frequency;
+};
+
 #define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
 #define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
 
diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index 0c45101..e7f7462 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -14,6 +14,7 @@
 #include <mlx5_win_ext.h>
 #include <mlx5_malloc.h>
 #include <mlx5.h>
+#include <mlx5_utils.h>
 
 /**
  * Get MAC address by querying netdevice.
@@ -279,3 +280,32 @@ int mlx5_get_module_eeprom(struct rte_eth_dev *dev,
 	RTE_SET_USED(info);
 	return -ENOTSUP;
 }
+
+/**
+ * Get device current raw clock counter
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param[out] time
+ *   Current raw clock counter of the device.
+ *
+ * @return
+ *   0 if the clock has correctly been read
+ *   The value of errno in case of error
+ */
+int
+mlx5_read_clock(struct rte_eth_dev *dev, uint64_t *clock)
+{
+	int err;
+	struct mlx5_devx_clock mlx5_clock;
+	struct mlx5_priv *priv = dev->data->dev_private;
+	mlx5_context_st *context_obj = (mlx5_context_st *)priv->sh->ctx;
+
+	err = mlx5_glue->query_rt_values(context_obj, &mlx5_clock);
+	if (err != 0) {
+		DRV_LOG(WARNING, "Could not query the clock !");
+		return err;
+	}
+	*clock = *(u64 volatile *)mlx5_clock.p_iseg_internal_timer;
+	return 0;
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 50/72] net/mlx5/windows: support get mtu

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit implements API mlx5_get_mtu(). It returns the MTU size as
saved in the device context since its creation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_ethdev_os.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index e7f7462..933acf4 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -44,6 +44,33 @@ mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN])
 }
 
 /**
+ * Get device MTU.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ * @param[out] mtu
+ *   MTU value output buffer.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu)
+{
+	struct mlx5_priv *priv;
+	mlx5_context_st *context_obj;
+
+	if (!dev) {
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	priv = dev->data->dev_private;
+	context_obj = (mlx5_context_st *)priv->sh->ctx;
+	*mtu = context_obj->mlx5_dev.mtu_bytes;
+	return 0;
+}
+
+/**
  * Set device MTU.
  *
  * @param dev
-- 
2.8.4

[dpdk-dev] [PATCH v1 51/72] net/mlx5/windows: support get interface name

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit copies the interface name as saved in the device context
since its creation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_ethdev_os.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index 933acf4..05f25cc 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -44,6 +44,34 @@ mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN])
 }
 
 /**
+ * Get interface name from private structure.
+ *
+ *
+ * @param[in] dev
+ *   Pointer to Ethernet device.
+ * @param[out] ifname
+ *   Interface name output buffer.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE])
+{
+	struct mlx5_priv *priv;
+	mlx5_context_st *context_obj;
+
+	if (!dev) {
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	priv = dev->data->dev_private;
+	context_obj = (mlx5_context_st *)priv->sh->ctx;
+	strncpy(*ifname, context_obj->mlx5_dev.name, IF_NAMESIZE);
+	return 0;
+}
+
+/**
  * Get device MTU.
  *
  * @param dev
-- 
2.8.4

[dpdk-dev] [PATCH v1 52/72] net/mlx5/windows: support is removed

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit implements mlx5_is_removed() API. A new glue call
'init_shutdown_event' is added to support the new API.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_glue.c    | 21 +++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h    |  1 +
 drivers/common/mlx5/windows/mlx5_win_ext.h |  2 +-
 drivers/net/mlx5/windows/mlx5_ethdev_os.c  | 20 ++++++++++++++++++++
 4 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 1675318..aef6d3b 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -308,6 +308,26 @@ mlx5_glue_query_rt_values(void *ctx, void *devx_clock)
 	return 0;
 }
 
+static int
+mlx5_glue_devx_init_showdown_event(void *ctx)
+{
+	struct mlx5_context *mlx5_ctx;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return errno;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	err = devx_query_shutdown_event(mlx5_ctx->devx_ctx,
+			&mlx5_ctx->shutdown_event_obj);
+	if (err) {
+		errno = err;
+		return errno;
+	}
+	return 0;
+}
+
 alignas(RTE_CACHE_LINE_SIZE)
 const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
 	.version = MLX5_GLUE_VERSION,
@@ -330,4 +350,5 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
 	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
 	.devx_query_eqn = mlx5_glue_devx_query_eqn,
 	.query_rt_values = mlx5_glue_query_rt_values,
+	.devx_init_showdown_event = mlx5_glue_devx_init_showdown_event,
 };
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
index bfaaa2a..420bfb2 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.h
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -52,6 +52,7 @@ struct mlx5_glue {
 	int (*devx_fs_rule_del)(void *flow);
 	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
 	int (*query_rt_values)(void *ctx, void *devx_clock);
+	int (*devx_init_showdown_event)(void *ctx);
 };
 
 extern const struct mlx5_glue *mlx5_glue;
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index a14a6cc..111af2e 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -14,7 +14,7 @@ extern "C" {
 typedef struct mlx5_context {
 	devx_device_ctx        *devx_ctx;
 	struct devx_device mlx5_dev;
-
+	struct devx_shutdown_event shutdown_event_obj;
 } mlx5_context_st;
 
 typedef struct {
diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
index 05f25cc..c843605 100644
--- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
@@ -364,3 +364,23 @@ mlx5_read_clock(struct rte_eth_dev *dev, uint64_t *clock)
 	*clock = *(u64 volatile *)mlx5_clock.p_iseg_internal_timer;
 	return 0;
 }
+
+/**
+ * Check if mlx5 device was removed.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ * @return
+ *   1 when device is removed, otherwise 0.
+ */
+int
+mlx5_is_removed(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	mlx5_context_st *context_obj = (mlx5_context_st *)priv->sh->ctx;
+
+	if (*context_obj->shutdown_event_obj.p_flag)
+		return 1;
+	return 0;
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 53/72] net/mlx5/windws: add VLAN stubs

[Ophir Munk]

This commit adds stubs to VLAN VM operations.  It is the Windows
equivalent implementation of (1).  The Linux implementation was based on
Netlink APIs which are not supported in Windows.

(1)
commit 7af10d29a4a0 ("net/mlx5/linux: refactor VLAN")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/meson.build    |  1 +
 drivers/net/mlx5/windows/mlx5_vlan_os.c | 83 +++++++++++++++++++++++++++++++++
 2 files changed, 84 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/mlx5_vlan_os.c

diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
index b5d9f0e..87e34d3 100644
--- a/drivers/net/mlx5/windows/meson.build
+++ b/drivers/net/mlx5/windows/meson.build
@@ -6,5 +6,6 @@ sources += files(
 	'mlx5_os.c',
 	'mlx5_mp_os.c',
 	'mlx5_ethdev_os.c',
+	'mlx5_vlan_os.c',
 )
 
diff --git a/drivers/net/mlx5/windows/mlx5_vlan_os.c b/drivers/net/mlx5/windows/mlx5_vlan_os.c
new file mode 100644
index 0000000..d3b11e0
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_vlan_os.c
@@ -0,0 +1,83 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <stddef.h>
+#include <errno.h>
+#include <stdint.h>
+#include <unistd.h>
+
+/*
+ * Not needed by this file; included to work around the lack of off_t
+ * definition for mlx5dv.h with unpatched rdma-core versions.
+ */
+#include <sys/types.h>
+
+#include <rte_ethdev_driver.h>
+#include <rte_common.h>
+
+#include <mlx5.h>
+#include <mlx5_malloc.h>
+
+/*
+ * Release VLAN network device, created for VM workaround.
+ *
+ * @param[in] dev
+ *   Ethernet device object, Netlink context provider.
+ * @param[in] vlan
+ *   Object representing the network device to release.
+ */
+void
+mlx5_vlan_vmwa_release(struct rte_eth_dev *dev,
+			    struct mlx5_vf_vlan *vlan)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(vlan);
+}
+
+/**
+ * Acquire VLAN interface with specified tag for VM workaround.
+ *
+ * @param[in] dev
+ *   Ethernet device object, Netlink context provider.
+ * @param[in] vlan
+ *   Object representing the network device to acquire.
+ */
+void
+mlx5_vlan_vmwa_acquire(struct rte_eth_dev *dev,
+			    struct mlx5_vf_vlan *vlan)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(vlan);
+}
+
+/*
+ * Create per ethernet device VLAN VM workaround context
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param ifindex
+ *   Interface index.
+ *
+ * @Return
+ *   Pointer to mlx5_nl_vlan_vmwa_context
+ */
+void *
+mlx5_vlan_vmwa_init(struct rte_eth_dev *dev, uint32_t ifindex)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(ifindex);
+	return NULL;
+}
+
+/*
+ * Destroy per ethernet device VLAN VM workaround context
+ *
+ * @param dev
+ *   Pointer to VM context
+ */
+void
+mlx5_vlan_vmwa_exit(void *vmctx)
+{
+	RTE_SET_USED(vmctx);
+}
-- 
2.8.4

[dpdk-dev] [PATCH v1 54/72] net/mlx5: exclude rte_intr_callback_register call

[Ophir Munk]

Exclude call to rte_intr_callback_register under Windows using
definition RTE_EXEC_ENV_WINDOWS.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/mlx5_txpp.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index c22ddc9..4405054 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -896,12 +896,14 @@ mlx5_txpp_start_service(struct mlx5_dev_ctx_shared *sh)
 	fd = mlx5_os_get_devx_channel_fd(sh->txpp.echan);
 	sh->txpp.intr_handle.fd = fd;
 	sh->txpp.intr_handle.type = RTE_INTR_HANDLE_EXT;
+#ifndef RTE_EXEC_ENV_WINDOWS
 	if (rte_intr_callback_register(&sh->txpp.intr_handle,
 				       mlx5_txpp_interrupt_handler, sh)) {
 		sh->txpp.intr_handle.fd = 0;
 		DRV_LOG(ERR, "Failed to register CQE interrupt %d.", rte_errno);
 		return -rte_errno;
 	}
+#endif
 	/* Subscribe CQ event to the event channel controlled by the driver. */
 	ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan,
 						   sh->txpp.rearm_queue.cq->obj,
-- 
2.8.4

[dpdk-dev] [PATCH v1 55/72] net/mlx5/windows: support get pdn

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Implement OS function call to get pdn.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index e334110..56bb969 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -303,6 +303,27 @@ mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
 	*dereg_mr_cb = mlx5_os_dereg_mr;
 }
 
+/**
+ * Extract pdn of PD object using DevX
+ *
+ * @param[in] pd
+ *   Pointer to the DevX PD object.
+ * @param[out] pdn
+ *   Pointer to the PD object number variable.
+ *
+ * @return
+ *   0 on success, error value otherwise.
+ */
+int
+mlx5_os_get_pdn(void *pd, uint32_t *pdn)
+{
+	if (!pd)
+		return -EINVAL;
+
+	*pdn = ((struct mlx5_pd *)pd)->pdn;
+	return 0;
+}
+
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
 
 const struct eth_dev_ops mlx5_os_dev_ops = {
-- 
2.8.4

[dpdk-dev] [PATCH v1 56/72] net/mlx5/windows: support open device

[Ophir Munk]

This commit implements mlx5_os_open_device() API. It calls glue API
open_device() then glue API query_device() to fill in 'struct
mlx5_context' with data for later usage.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 43 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 56bb969..fbfc044 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -103,6 +103,49 @@ mlx5_os_set_nonblock_channel_fd(int fd)
 }
 
 /**
+ * Function API open device under Windows
+ *
+ * This function calls the Windows glue APIs to open a device.
+ *
+ * @param[in] spawn
+ *   Pointer to the device attributes (name, port, etc).
+ * @param[out] config
+ *   Pointer to device configuration structure.
+ * @param[out] sh
+ *   Pointer to shared context structure.
+ *
+ * @return
+ *   0 on success, a positive error value otherwise.
+ */
+int
+mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn,
+		 const struct mlx5_dev_config *config,
+		 struct mlx5_dev_ctx_shared *sh)
+{
+	(void)config;
+	int err = 0;
+	/* Set numa node from pci probe */
+	sh->numa_node = spawn->pci_dev->device.numa_node;
+
+	/* Try to open device with DevX */
+	rte_errno = 0;
+	sh->ctx = mlx5_glue->open_device(spawn->phys_dev);
+	if (!sh->ctx) {
+		DRV_LOG(ERR, "open_device failed");
+		err = errno;
+		return err;
+	}
+	sh->devx = 1;
+
+	struct mlx5_context *mlx5_ctx = (struct mlx5_context *)sh->ctx;
+	err = mlx5_glue->query_device(spawn->phys_dev, &mlx5_ctx->mlx5_dev);
+	if (err)
+		DRV_LOG(ERR, "query_device failed");
+
+	return err;
+}
+
+/**
  * This function should share events between multiple ports of single IB
  * device.  Currently it has no support under Windows.
  *
-- 
2.8.4

[dpdk-dev] [PATCH v1 57/72] net/mlx5/windows: initial probing implementation

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit implements mlx5_os_pci_probe API under Windows. It does all
required initializations then it gets the PCI device list using glue API
get_device_list().  Next, all non MLX5 matched devices are filtered out.
The supported NIC types are: CONNECTX4VF, CONNECTX4LXVF, CONNECTX5VF,
CONNECTX5EXVF, CONNECTX5BFVF, CONNECTX6VF, MELLANOX_CONNECTX6DXVF.  Each
device in the list is assigned with default configuration parameters,
most of them are 0. The default dv_flow_en parameter value is 1 (which
means Windows match and action flows are based on DV code). Next for
each PCI device call mlx5_dev_spawn() to create an eth device (struct
rte_ethdev). The implementation of device spawn is in the follow up
commit.  Finally, the device list is free.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 278 +++++++++++++++++++++++++++++++++++++
 1 file changed, 278 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index fbfc044..a3e9a6a 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -9,6 +9,7 @@
 #include <stdlib.h>
 
 #include <rte_windows.h>
+#include <rte_ethdev_pci.h>
 
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
@@ -26,6 +27,80 @@
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
 
+static const char *MZ_MLX5_PMD_SHARED_DATA = "mlx5_pmd_shared_data";
+
+/* Spinlock for mlx5_shared_data allocation. */
+static rte_spinlock_t mlx5_shared_data_lock = RTE_SPINLOCK_INITIALIZER;
+/* Process local data for secondary processes. */
+static struct mlx5_local_data mlx5_local_data;
+
+/**
+ * Initialize shared data between primary and secondary process.
+ *
+ * A memzone is reserved by primary process and secondary processes attach to
+ * the memzone.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_init_shared_data(void)
+{
+	const struct rte_memzone *mz;
+	int ret = 0;
+
+	rte_spinlock_lock(&mlx5_shared_data_lock);
+	if (mlx5_shared_data == NULL) {
+		if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+			/* Allocate shared memory. */
+			mz = rte_memzone_reserve(MZ_MLX5_PMD_SHARED_DATA,
+						 sizeof(*mlx5_shared_data),
+						 SOCKET_ID_ANY, 0);
+			if (mz == NULL) {
+				DRV_LOG(ERR,
+					"Cannot allocate mlx5 shared data");
+				ret = -rte_errno;
+				goto error;
+			}
+			mlx5_shared_data = mz->addr;
+			memset(mlx5_shared_data, 0, sizeof(*mlx5_shared_data));
+			rte_spinlock_init(&mlx5_shared_data->lock);
+		} else {
+			/* Lookup allocated shared memory. */
+			mz = rte_memzone_lookup(MZ_MLX5_PMD_SHARED_DATA);
+			if (mz == NULL) {
+				DRV_LOG(ERR,
+					"Cannot attach mlx5 shared data");
+				ret = -rte_errno;
+				goto error;
+			}
+			mlx5_shared_data = mz->addr;
+			memset(&mlx5_local_data, 0, sizeof(mlx5_local_data));
+		}
+	}
+error:
+	rte_spinlock_unlock(&mlx5_shared_data_lock);
+	return ret;
+}
+
+/**
+ * PMD global initialization.
+ *
+ * Independent from individual device, this function initializes global
+ * per-PMD data structures distinguishing primary and secondary processes.
+ * Hence, each initialization is called once per a process.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+static int
+mlx5_init_once(void)
+{
+	if (mlx5_init_shared_data())
+		return -rte_errno;
+	return 0;
+}
+
 /**
  * Get mlx5 device attributes.
  *
@@ -146,6 +221,31 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn,
 }
 
 /**
+ * Spawn an Ethernet device from Verbs information.
+ *
+ * @param dpdk_dev
+ *   Backing DPDK device.
+ * @param spawn
+ *   Verbs device parameters (name, port, switch_info) to spawn.
+ * @param config
+ *   Device configuration parameters.
+ *
+ * @return
+ *   NULL pointer. Operation is not supported and rte_errno is set to ENOTSUP.
+ */
+static struct rte_eth_dev *
+mlx5_dev_spawn(struct rte_device *dpdk_dev,
+	       struct mlx5_dev_spawn_data *spawn,
+	       struct mlx5_dev_config *config)
+{
+	(void)dpdk_dev;
+	(void)spawn;
+	(void)config;
+	rte_errno = -ENOTSUP;
+	return NULL;
+}
+
+/**
  * This function should share events between multiple ports of single IB
  * device.  Currently it has no support under Windows.
  *
@@ -330,6 +430,184 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 /**
+ * DPDK callback to register a PCI device.
+ *
+ * This function spawns Ethernet devices out of a given PCI device.
+ *
+ * @param[in] pci_drv
+ *   PCI driver structure (mlx5_driver).
+ * @param[in] pci_dev
+ *   PCI device information.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
+		  struct rte_pci_device *pci_dev)
+{
+	struct devx_device_bdf *devx_bdf_devs, *orig_devx_bdf_devs;
+	/*
+	 * Number of found IB Devices matching with requested PCI BDF.
+	 * nd != 1 means there are multiple IB devices over the same
+	 * PCI device and we have representors and master.
+	 */
+	unsigned int nd = 0;
+	/*
+	 * Number of found IB device Ports. nd = 1 and np = 1..n means
+	 * we have the single multiport IB device, and there may be
+	 * representors attached to some of found ports.
+	 * Currently not supported.
+	 * unsigned int np = 0;
+	 */
+
+	/*
+	 * Number of DPDK ethernet devices to Spawn - either over
+	 * multiple IB devices or multiple ports of single IB device.
+	 * Actually this is the number of iterations to spawn.
+	 */
+	unsigned int ns = 0;
+	/*
+	 * Bonding device
+	 *   < 0 - no bonding device (single one)
+	 *  >= 0 - bonding device (value is slave PF index)
+	 */
+	int bd = -1;
+	struct mlx5_dev_spawn_data *list = NULL;
+	struct mlx5_dev_config dev_config;
+	unsigned int dev_config_vf;
+	int ret;
+	uint32_t restore;
+
+	ret = mlx5_init_once();
+	if (ret) {
+		DRV_LOG(ERR, "unable to init PMD global data: %s",
+			strerror(rte_errno));
+		return -rte_errno;
+	}
+	errno = 0;
+	devx_bdf_devs = mlx5_glue->get_device_list(&ret);
+	orig_devx_bdf_devs = devx_bdf_devs;
+	if (!devx_bdf_devs) {
+		rte_errno = errno ? errno : ENOSYS;
+		DRV_LOG(ERR, "cannot list devices, is ib_uverbs loaded?");
+		return -rte_errno;
+	}
+	/*
+	 * First scan the list of all Infiniband devices to find
+	 * matching ones, gathering into the list.
+	 */
+	struct devx_device_bdf *devx_bdf_match[ret + 1];
+
+	while (ret-- > 0) {
+		if (pci_dev->addr.bus != devx_bdf_devs->bus_id ||
+		    pci_dev->addr.devid != devx_bdf_devs->dev_id ||
+		    pci_dev->addr.function != devx_bdf_devs->fnc_id) {
+			devx_bdf_devs++;
+			continue;
+		}
+
+		devx_bdf_match[nd++] = devx_bdf_devs;
+	}
+	devx_bdf_match[nd] = NULL;
+	if (!nd) {
+		/* No device matches, just complain and bail out. */
+		DRV_LOG(WARNING,
+			"no DevX device matches PCI device " PCI_PRI_FMT ","
+			" is DevX Configured?",
+			pci_dev->addr.domain, pci_dev->addr.bus,
+			pci_dev->addr.devid, pci_dev->addr.function);
+		rte_errno = ENOENT;
+		ret = -rte_errno;
+		goto exit;
+	}
+	/*
+	 * Now we can determine the maximal
+	 * amount of devices to be spawned.
+	 */
+	list = mlx5_malloc(MLX5_MEM_ZERO,
+			   sizeof(struct mlx5_dev_spawn_data),
+			   RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (!list) {
+		DRV_LOG(ERR, "spawn data array allocation failure");
+		rte_errno = ENOMEM;
+		ret = -rte_errno;
+		goto exit;
+	}
+	memset(&list[ns].info, 0, sizeof(list[ns].info));
+	list[ns].max_port = 1;
+	list[ns].phys_port = 1;
+	list[ns].phys_dev = devx_bdf_match[ns];
+	list[ns].eth_dev = NULL;
+	list[ns].pci_dev = pci_dev;
+	list[ns].pf_bond = bd;
+	list[ns].ifindex = -1; /* Spawn will assign */
+	list[ns].info =
+		(struct mlx5_switch_info){
+			.master = 0,
+			.representor = 0,
+			.name_type = MLX5_PHYS_PORT_NAME_TYPE_UPLINK,
+			.port_name = 0,
+			.switch_id = 0,
+		};
+	/* Device specific configuration. */
+	switch (pci_dev->id.device_id) {
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF:
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF:
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX5VF:
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX5EXVF:
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX5BFVF:
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX6VF:
+	case PCI_DEVICE_ID_MELLANOX_CONNECTX6DXVF:
+		dev_config_vf = 1;
+		break;
+	default:
+		dev_config_vf = 0;
+		break;
+	}
+	/* Default configuration. */
+	memset(&dev_config, 0, sizeof(struct mlx5_dev_config));
+	dev_config.vf = dev_config_vf;
+	dev_config.mps = 0;
+	dev_config.dbnc = MLX5_ARG_UNSET;
+	dev_config.rx_vec_en = 1;
+	dev_config.txq_inline_max = MLX5_ARG_UNSET;
+	dev_config.txq_inline_min = MLX5_ARG_UNSET;
+	dev_config.txq_inline_mpw = MLX5_ARG_UNSET;
+	dev_config.txqs_inline = MLX5_ARG_UNSET;
+	dev_config.vf_nl_en = 0;
+	dev_config.mr_ext_memseg_en = 1;
+	dev_config.mprq.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN;
+	dev_config.mprq.min_rxqs_num = MLX5_MPRQ_MIN_RXQS;
+	dev_config.dv_esw_en = 0;
+	dev_config.dv_flow_en = 1;
+	dev_config.decap_en = 0;
+	dev_config.log_hp_size = MLX5_ARG_UNSET;
+	list[ns].eth_dev = mlx5_dev_spawn(&pci_dev->device,
+					  &list[ns],
+					  &dev_config);
+	if (!list[ns].eth_dev)
+		goto exit;
+	restore = list[ns].eth_dev->data->dev_flags;
+	rte_eth_copy_pci_info(list[ns].eth_dev, pci_dev);
+	/* Restore non-PCI flags cleared by the above call. */
+	list[ns].eth_dev->data->dev_flags |= restore;
+	rte_eth_dev_probing_finish(list[ns].eth_dev);
+	ret = 0;
+exit:
+	/*
+	 * Do the routine cleanup:
+	 * - free allocated spawn data array
+	 * - free the device list
+	 */
+	if (list)
+		mlx5_free(list);
+	MLX5_ASSERT(orig_devx_bdf_devs);
+	mlx5_glue->free_device_list(orig_devx_bdf_devs);
+	return ret;
+}
+
+/**
  * Set the reg_mr and dereg_mr call backs
  *
  * @param reg_mr_cb[out]
-- 
2.8.4

[dpdk-dev] [PATCH v1 58/72] net/mlx5/windws: spawn eth devices

[Ophir Munk]

This commit implements mlx5_dev_spawn() API which allocates an eth
device (struct rte_eth_dev) for each PCI device. When working with
representors virtual functions (as in Linux), one PCI device may spawn
several eth devices: the master device for the main physical function
(PF) and several representors for the virtual functions (VFs).  However,
currently Windows does not work in switch dev mode, therefore, no VFs
are created and no representors are spawned. In this case one eth device
is created per one PCI main port.  In addition to device creation - the
device configuration must be correctly set. The device arguments
(devargs - set by the user) are parsed but they may be overridden by
Windows limitations or hardware configurations. Some associated network
parameters are stored in eth device (e.g. ifindex, MAC address, MTU) and
some callback (e.g. burst functions) are set.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_win_defs.h |   6 +
 drivers/net/mlx5/windows/mlx5_os.c          | 481 +++++++++++++++++++++++++++-
 2 files changed, 482 insertions(+), 5 deletions(-)

diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
index e9569e4..d8f2099 100644
--- a/drivers/common/mlx5/windows/mlx5_win_defs.h
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -163,4 +163,10 @@ struct mlx5_wqe_data_seg {
 	rte_be32_t		lkey;
 	rte_be64_t		addr;
 };
+
+#define MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP	(1 << 4)
+#define IBV_DEVICE_RAW_IP_CSUM			(1 << 26)
+#define IBV_RAW_PACKET_CAP_CVLAN_STRIPPING	(1 << 0)
+#define IBV_RAW_PACKET_CAP_SCATTER_FCS		(1 << 1)
+#define IBV_QPT_RAW_PACKET			8
 #endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index a3e9a6a..f9b469f 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -26,6 +26,9 @@
 #include "mlx5_autoconf.h"
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
+#include "mlx5_devx.h"
+
+#define MLX5_TAGS_HLIST_ARRAY_SIZE 8192
 
 static const char *MZ_MLX5_PMD_SHARED_DATA = "mlx5_pmd_shared_data";
 
@@ -158,6 +161,43 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 }
 
 /**
+ * Initialize DR related data within private structure.
+ * Routine checks the reference counter and does actual
+ * resources creation/initialization only if counter is zero.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ *
+ * @return
+ *   Zero on success, positive error code otherwise.
+ */
+static int
+mlx5_alloc_shared_dr(struct mlx5_priv *priv)
+{
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
+	char s[MLX5_HLIST_NAMESIZE];
+	int err = 0;
+
+	if (!sh->flow_tbls)
+		err = mlx5_alloc_table_hash_list(priv);
+	else
+		DRV_LOG(DEBUG, "sh->flow_tbls[%p] already created, reuse\n",
+			(void *)sh->flow_tbls);
+	return err;
+}
+/**
+ * Destroy DR related data within private structure.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ */
+void
+mlx5_os_free_shared_dr(struct mlx5_priv *priv)
+{
+	mlx5_free_table_hash_list(priv);
+}
+
+/**
  * Set the completion channel file descriptor interrupt as non-blocking.
  * Currently it has no support under Windows.
  *
@@ -221,6 +261,45 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn,
 }
 
 /**
+ * DV flow counter mode detect and config.
+ *
+ * @param dev
+ *   Pointer to rte_eth_dev structure.
+ *
+ */
+static void
+mlx5_flow_counter_mode_config(struct rte_eth_dev *dev __rte_unused)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+	struct mlx5_priv *priv = dev->data->dev_private;
+	struct mlx5_dev_ctx_shared *sh = priv->sh;
+	bool fallback;
+
+#ifndef HAVE_IBV_DEVX_ASYNC
+	fallback = true;
+#else
+	fallback = false;
+	if (!priv->config.devx || !priv->config.dv_flow_en ||
+	    !priv->config.hca_attr.flow_counters_dump ||
+	    !(priv->config.hca_attr.flow_counter_bulk_alloc_bitmap & 0x4) ||
+	    (mlx5_flow_dv_discover_counter_offset_support(dev) == -ENOTSUP))
+		fallback = true;
+#endif
+	if (fallback)
+		DRV_LOG(INFO, "Use fall-back DV counter management. Flow "
+			"counter dump:%d, bulk_alloc_bitmap:0x%hhx.",
+			priv->config.hca_attr.flow_counters_dump,
+			priv->config.hca_attr.flow_counter_bulk_alloc_bitmap);
+	/* Initialize fallback mode only on the port initializes sh. */
+	if (sh->refcnt == 1)
+		sh->cmng.counter_fallback = fallback;
+	else if (fallback != sh->cmng.counter_fallback)
+		DRV_LOG(WARNING, "Port %d in sh has different fallback mode "
+			"with others:%d.", PORT_ID(priv), fallback);
+#endif
+}
+
+/**
  * Spawn an Ethernet device from Verbs information.
  *
  * @param dpdk_dev
@@ -231,17 +310,409 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn,
  *   Device configuration parameters.
  *
  * @return
- *   NULL pointer. Operation is not supported and rte_errno is set to ENOTSUP.
+ *   A valid Ethernet device object on success, NULL otherwise and rte_errno
+ *   is set. The following errors are defined:
+ *
+ *   EEXIST: device is already spawned
  */
 static struct rte_eth_dev *
 mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	       struct mlx5_dev_spawn_data *spawn,
 	       struct mlx5_dev_config *config)
 {
-	(void)dpdk_dev;
-	(void)spawn;
-	(void)config;
-	rte_errno = -ENOTSUP;
+	const struct mlx5_switch_info *switch_info = &spawn->info;
+	struct mlx5_dev_ctx_shared *sh = NULL;
+	struct mlx5_dev_attr device_attr;
+	struct rte_eth_dev *eth_dev = NULL;
+	struct mlx5_priv *priv = NULL;
+	int err = 0;
+	unsigned int cqe_comp;
+	unsigned int cqe_pad = 0;
+	struct rte_ether_addr mac;
+	char name[RTE_ETH_NAME_MAX_LEN];
+	int own_domain_id = 0;
+	uint16_t port_id;
+
+	/* Build device name. */
+	strlcpy(name, dpdk_dev->name, sizeof(name));
+	/* check if the device is already spawned */
+	if (rte_eth_dev_get_port_by_name(name, &port_id) == 0) {
+		rte_errno = EEXIST;
+		return NULL;
+	}
+	DRV_LOG(DEBUG, "naming Ethernet device \"%s\"", name);
+	/*
+	 * Some parameters are needed in advance to create device context. We
+	 * process the devargs here to get ones, and later process devargs
+	 * again to override some hardware settings.
+	 */
+	err = mlx5_args(config, dpdk_dev->devargs);
+	if (err) {
+		err = rte_errno;
+		DRV_LOG(ERR, "failed to process device arguments: %s",
+			strerror(rte_errno));
+		goto error;
+	}
+	mlx5_malloc_mem_select(config->sys_mem_en);
+	sh = mlx5_alloc_shared_dev_ctx(spawn, config);
+	if (!sh)
+		return NULL;
+	config->devx = sh->devx;
+	/* Initialize the shutdown event in mlx5_dev_spawn to
+	 * support mlx5_is_removed for Windows.
+	 */
+	err = mlx5_glue->devx_init_showdown_event(sh->ctx);
+	if (err) {
+		DRV_LOG(ERR, "failed to init showdown event: %s",
+			strerror(errno));
+		goto error;
+	}
+	DRV_LOG(DEBUG, "MPW isn't supported");
+	mlx5_os_get_dev_attr(sh->ctx, &device_attr);
+	config->swp = 0;
+	config->ind_table_max_size =
+		sh->device_attr.max_rwq_indirection_table_size;
+	if (RTE_CACHE_LINE_SIZE == 128 &&
+	    !(device_attr.flags & MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP))
+		cqe_comp = 0;
+	else
+		cqe_comp = 1;
+	config->cqe_comp = cqe_comp;
+	DRV_LOG(DEBUG, "tunnel offloading is not supported");
+	config->tunnel_en = 0;
+	DRV_LOG(DEBUG, "MPLS over GRE/UDP tunnel offloading is no supported");
+	config->mpls_en = 0;
+	/* Allocate private eth device data. */
+	priv = mlx5_malloc(MLX5_MEM_ZERO | MLX5_MEM_RTE,
+			   sizeof(*priv),
+			   RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
+	if (priv == NULL) {
+		DRV_LOG(ERR, "priv allocation failure");
+		err = ENOMEM;
+		goto error;
+	}
+	priv->sh = sh;
+	priv->dev_port = spawn->phys_port;
+	priv->pci_dev = spawn->pci_dev;
+	priv->mtu = RTE_ETHER_MTU;
+	priv->mp_id.port_id = port_id;
+	strlcpy(priv->mp_id.name, MLX5_MP_NAME, RTE_MP_MAX_NAME_LEN);
+	priv->representor = !!switch_info->representor;
+	priv->master = !!switch_info->master;
+	priv->domain_id = RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID;
+	priv->vport_meta_tag = 0;
+	priv->vport_meta_mask = 0;
+	priv->pf_bond = spawn->pf_bond;
+	priv->vport_id = -1;
+	/* representor_id field keeps the unmodified VF index. */
+	priv->representor_id = -1;
+	/*
+	 * Look for sibling devices in order to reuse their switch domain
+	 * if any, otherwise allocate one.
+	 */
+	MLX5_ETH_FOREACH_DEV(port_id, priv->pci_dev) {
+		const struct mlx5_priv *opriv =
+			rte_eth_devices[port_id].data->dev_private;
+
+		if (!opriv ||
+		    opriv->sh != priv->sh ||
+			opriv->domain_id ==
+			RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID)
+			continue;
+		priv->domain_id = opriv->domain_id;
+		break;
+	}
+	if (priv->domain_id == RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID) {
+		err = rte_eth_switch_domain_alloc(&priv->domain_id);
+		if (err) {
+			err = rte_errno;
+			DRV_LOG(ERR, "unable to allocate switch domain: %s",
+				strerror(rte_errno));
+			goto error;
+		}
+		own_domain_id = 1;
+	}
+	/* Override some values set by hardware configuration. */
+	mlx5_args(config, dpdk_dev->devargs);
+	err = mlx5_dev_check_sibling_config(priv, config);
+	if (err)
+		goto error;
+	config->hw_csum = !!(sh->device_attr.device_cap_flags_ex &
+			    IBV_DEVICE_RAW_IP_CSUM);
+	DRV_LOG(DEBUG, "checksum offloading is %ssupported",
+		(config->hw_csum ? "" : "not "));
+	DRV_LOG(DEBUG, "counters are not supported");
+	config->ind_table_max_size =
+		sh->device_attr.max_rwq_indirection_table_size;
+	/*
+	 * Remove this check once DPDK supports larger/variable
+	 * indirection tables.
+	 */
+	if (config->ind_table_max_size > (unsigned int)ETH_RSS_RETA_SIZE_512)
+		config->ind_table_max_size = ETH_RSS_RETA_SIZE_512;
+	DRV_LOG(DEBUG, "maximum Rx indirection table size is %u",
+		config->ind_table_max_size);
+	config->hw_vlan_strip = !!(sh->device_attr.raw_packet_caps &
+				  IBV_RAW_PACKET_CAP_CVLAN_STRIPPING);
+	DRV_LOG(DEBUG, "VLAN stripping is %ssupported",
+		(config->hw_vlan_strip ? "" : "not "));
+	config->hw_fcs_strip = !!(sh->device_attr.raw_packet_caps &
+				 IBV_RAW_PACKET_CAP_SCATTER_FCS);
+	if (config->hw_padding) {
+		DRV_LOG(DEBUG, "Rx end alignment padding isn't supported");
+		config->hw_padding = 0;
+	}
+	config->tso = (sh->device_attr.max_tso > 0 &&
+		      (sh->device_attr.tso_supported_qpts &
+		       (1 << IBV_QPT_RAW_PACKET)));
+	if (config->tso)
+		config->tso_max_payload_sz = sh->device_attr.max_tso;
+	DRV_LOG(DEBUG, "%sMPS is %s",
+		config->mps == MLX5_MPW_ENHANCED ? "enhanced " :
+		config->mps == MLX5_MPW ? "legacy " : "",
+		config->mps != MLX5_MPW_DISABLED ? "enabled" : "disabled");
+	if (config->cqe_comp && !cqe_comp) {
+		DRV_LOG(WARNING, "Rx CQE compression isn't supported");
+		config->cqe_comp = 0;
+	}
+	if (config->cqe_pad && !cqe_pad) {
+		DRV_LOG(WARNING, "Rx CQE padding isn't supported");
+		config->cqe_pad = 0;
+	} else if (config->cqe_pad) {
+		DRV_LOG(INFO, "Rx CQE padding is enabled");
+	}
+	if (config->devx) {
+		err = mlx5_devx_cmd_query_hca_attr(sh->ctx, &config->hca_attr);
+		if (err) {
+			err = -err;
+			goto error;
+		}
+		/* Check relax ordering support. */
+		sh->cmng.relaxed_ordering_read = 0;
+		sh->cmng.relaxed_ordering_write = 0;
+		if (!haswell_broadwell_cpu) {
+			sh->cmng.relaxed_ordering_write =
+				config->hca_attr.relaxed_ordering_write;
+			sh->cmng.relaxed_ordering_read =
+				config->hca_attr.relaxed_ordering_read;
+		}
+	}
+	if (config->devx) {
+		uint32_t reg[MLX5_ST_SZ_DW(register_mtutc)];
+
+		err = config->hca_attr.access_register_user ?
+			mlx5_devx_cmd_register_read
+				(sh->ctx, MLX5_REGISTER_ID_MTUTC, 0,
+				reg, MLX5_ST_SZ_DW(register_mtutc)) : ENOTSUP;
+		if (!err) {
+			uint32_t ts_mode;
+
+			/* MTUTC register is read successfully. */
+			ts_mode = MLX5_GET(register_mtutc, reg,
+					   time_stamp_mode);
+			if (ts_mode == MLX5_MTUTC_TIMESTAMP_MODE_REAL_TIME)
+				config->rt_timestamp = 1;
+		} else {
+			/* Kernel does not support register reading. */
+			if (config->hca_attr.dev_freq_khz ==
+						 (NS_PER_S / MS_PER_S))
+				config->rt_timestamp = 1;
+		}
+	}
+	if (config->mprq.enabled) {
+		DRV_LOG(WARNING, "Multi-Packet RQ isn't supported");
+		config->mprq.enabled = 0;
+	}
+	if (config->max_dump_files_num == 0)
+		config->max_dump_files_num = 128;
+	eth_dev = rte_eth_dev_allocate(name);
+	if (eth_dev == NULL) {
+		DRV_LOG(ERR, "can not allocate rte ethdev");
+		err = ENOMEM;
+		goto error;
+	}
+	if (priv->representor) {
+		eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
+		eth_dev->data->representor_id = priv->representor_id;
+	}
+	/*
+	 * Store associated network device interface index. This index
+	 * is permanent throughout the lifetime of device. So, we may store
+	 * the ifindex here and use the cached value further.
+	 */
+	MLX5_ASSERT(spawn->ifindex);
+	priv->if_index = spawn->ifindex;
+	eth_dev->data->dev_private = priv;
+	priv->dev_data = eth_dev->data;
+	eth_dev->data->mac_addrs = priv->mac;
+	eth_dev->device = dpdk_dev;
+	eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
+	/* Configure the first MAC address by default. */
+	if (mlx5_get_mac(eth_dev, &mac.addr_bytes)) {
+		DRV_LOG(ERR,
+			"port %u cannot get MAC address, is mlx5_en"
+			" loaded? (errno: %s)",
+			eth_dev->data->port_id, strerror(rte_errno));
+		err = ENODEV;
+		goto error;
+	}
+	DRV_LOG(INFO,
+		"port %u MAC address is %02x:%02x:%02x:%02x:%02x:%02x",
+		eth_dev->data->port_id,
+		mac.addr_bytes[0], mac.addr_bytes[1],
+		mac.addr_bytes[2], mac.addr_bytes[3],
+		mac.addr_bytes[4], mac.addr_bytes[5]);
+#ifdef RTE_LIBRTE_MLX5_DEBUG
+	{
+		char ifname[IF_NAMESIZE];
+
+		if (mlx5_get_ifname(eth_dev, &ifname) == 0)
+			DRV_LOG(DEBUG, "port %u ifname is \"%s\"",
+				eth_dev->data->port_id, ifname);
+		else
+			DRV_LOG(DEBUG, "port %u ifname is unknown",
+				eth_dev->data->port_id);
+	}
+#endif
+	/* Get actual MTU if possible. */
+	err = mlx5_get_mtu(eth_dev, &priv->mtu);
+	if (err) {
+		err = rte_errno;
+		goto error;
+	}
+	DRV_LOG(DEBUG, "port %u MTU is %u", eth_dev->data->port_id,
+		priv->mtu);
+	/* Initialize burst functions to prevent crashes before link-up. */
+	eth_dev->rx_pkt_burst = removed_rx_burst;
+	eth_dev->tx_pkt_burst = removed_tx_burst;
+	eth_dev->dev_ops = &mlx5_os_dev_ops;
+	eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
+	eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
+	eth_dev->rx_queue_count = mlx5_rx_queue_count;
+	/* Register MAC address. */
+	claim_zero(mlx5_mac_addr_add(eth_dev, &mac, 0, 0));
+	priv->flows = 0;
+	priv->ctrl_flows = 0;
+	TAILQ_INIT(&priv->flow_meters);
+	TAILQ_INIT(&priv->flow_meter_profiles);
+	/* Bring Ethernet device up. */
+	DRV_LOG(DEBUG, "port %u forcing Ethernet interface up",
+		eth_dev->data->port_id);
+	mlx5_set_link_up(eth_dev);
+	/*
+	 * Even though the interrupt handler is not installed yet,
+	 * interrupts will still trigger on the async_fd from
+	 * Verbs context returned by ibv_open_device().
+	 */
+	mlx5_link_update(eth_dev, 0);
+	config->dv_esw_en = 0;
+	/* Detect minimal data bytes to inline. */
+	mlx5_set_min_inline(spawn, config);
+	/* Store device configuration on private structure. */
+	priv->config = *config;
+	/* Create context for virtual machine VLAN workaround. */
+	priv->vmwa_context = NULL;
+	if (config->dv_flow_en) {
+		err = mlx5_alloc_shared_dr(priv);
+		if (err)
+			goto error;
+		/*
+		 * RSS id is shared with meter flow id. Meter flow id can only
+		 * use the 24 MSB of the register.
+		 */
+		priv->qrss_id_pool = mlx5_flow_id_pool_alloc(UINT32_MAX >>
+				     MLX5_MTR_COLOR_BITS);
+		if (!priv->qrss_id_pool) {
+			DRV_LOG(ERR, "can't create flow id pool");
+			err = ENOMEM;
+			goto error;
+		}
+	}
+	/* No supported flow priority number detection. */
+	priv->config.flow_prio = -1;
+	if (!priv->config.dv_esw_en &&
+	    priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) {
+		DRV_LOG(WARNING, "metadata mode %u is not supported "
+				 "(no E-Switch)", priv->config.dv_xmeta_en);
+		priv->config.dv_xmeta_en = MLX5_XMETA_MODE_LEGACY;
+	}
+	mlx5_set_metadata_mask(eth_dev);
+	if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY &&
+	    !priv->sh->dv_regc0_mask) {
+		DRV_LOG(ERR, "metadata mode %u is not supported "
+			     "(no metadata reg_c[0] is available)",
+			     priv->config.dv_xmeta_en);
+			err = ENOTSUP;
+			goto error;
+	}
+	/*
+	 * Allocate the buffer for flow creating, just once.
+	 * The allocation must be done before any flow creating.
+	 */
+	mlx5_flow_alloc_intermediate(eth_dev);
+	/* Query availability of metadata reg_c's. */
+	err = mlx5_flow_discover_mreg_c(eth_dev);
+	if (err < 0) {
+		err = -err;
+		goto error;
+	}
+	if (!mlx5_flow_ext_mreg_supported(eth_dev)) {
+		DRV_LOG(DEBUG,
+			"port %u extensive metadata register is not supported",
+			eth_dev->data->port_id);
+		if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) {
+			DRV_LOG(ERR, "metadata mode %u is not supported "
+				     "(no metadata registers available)",
+				     priv->config.dv_xmeta_en);
+			err = ENOTSUP;
+			goto error;
+		}
+	}
+	if (priv->config.dv_flow_en &&
+	    priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY &&
+	    mlx5_flow_ext_mreg_supported(eth_dev) &&
+	    priv->sh->dv_regc0_mask) {
+		priv->mreg_cp_tbl = mlx5_hlist_create(MLX5_FLOW_MREG_HNAME,
+						      MLX5_FLOW_MREG_HTABLE_SZ);
+		if (!priv->mreg_cp_tbl) {
+			err = ENOMEM;
+			goto error;
+		}
+	}
+	if (config->devx && config->dv_flow_en) {
+		priv->obj_ops = devx_obj_ops;
+	} else {
+		DRV_LOG(ERR, "Flow mode %u is not supported "
+				"(Windows flow must be DevX with DV flow enabled)",
+				priv->config.dv_flow_en);
+		err = ENOTSUP;
+		goto error;
+	}
+	mlx5_flow_counter_mode_config(eth_dev);
+	return eth_dev;
+error:
+	if (priv) {
+		if (priv->mreg_cp_tbl)
+			mlx5_hlist_destroy(priv->mreg_cp_tbl, NULL, NULL);
+		if (priv->qrss_id_pool)
+			mlx5_flow_id_pool_release(priv->qrss_id_pool);
+		if (own_domain_id)
+			claim_zero(rte_eth_switch_domain_free(priv->domain_id));
+		mlx5_free(priv);
+		if (eth_dev != NULL)
+			eth_dev->data->dev_private = NULL;
+	}
+	if (eth_dev != NULL) {
+		/* mac_addrs must not be freed alone because part of
+		 * dev_private
+		 **/
+		eth_dev->data->mac_addrs = NULL;
+		rte_eth_dev_release_port(eth_dev);
+	}
+	if (sh)
+		mlx5_free_shared_dev_ctx(sh);
+	MLX5_ASSERT(err > 0);
+	rte_errno = err;
 	return NULL;
 }
 
-- 
2.8.4

[dpdk-dev] [PATCH v1 59/72] net/mlx5/windows: support VF PCI address

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Support VF BDF scanning by checking both the BDF and raw BDF provided by
DevX. In Linux a PCI address is formatted as: domain, bus, device,
function (DBDF).  This is right for both a PF and a VF. In Windows a PF
also has a DBDF format, but the domain is always 0, while a VF has a
special "domain" called "Virtual PCI Bus, Serial" (for example: "Virtual
PCI Bus Slot 2 Serial 2") or segment.  The full VF format under Windows
is called raw DBF.  Windows special domain must be considered and DevX
must be called to support it.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 67 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 64 insertions(+), 3 deletions(-)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index f9b469f..4374b05 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 /**
+ * Detect if a devx_device_bdf object has identical DBDF values to the
+ * rte_pci_addr found in bus/pci probing
+ *
+ * @param[in] devx_bdf
+ *   Pointer to the devx_device_bdf structure.
+ * @param[in] addr
+ *   Pointer to the rte_pci_addr structure.
+ *
+ * @return
+ *   1 on Device match, 0 on mismatch.
+ */
+static int
+mlx5_match_devx_bdf_to_addr(struct devx_device_bdf *devx_bdf,
+			    struct rte_pci_addr *addr)
+{
+	if (addr->domain != (devx_bdf->bus_id >> 8) ||
+	    addr->bus != (devx_bdf->bus_id & 0xff) ||
+	    addr->devid != devx_bdf->dev_id ||
+	    addr->function != devx_bdf->fnc_id) {
+		return 0;
+	}
+	return 1;
+}
+
+/**
+ * Detect if a devx_device_bdf object matches the rte_pci_addr
+ * found in bus/pci probing
+ * Compare both the Native/PF BDF and the raw_bdf representing a VF BDF.
+ *
+ * @param[in] devx_bdf
+ *   Pointer to the devx_device_bdf structure.
+ * @param[in] addr
+ *   Pointer to the rte_pci_addr structure.
+ *
+ * @return
+ *   1 on Device match, 0 on mismatch, rte_errno code on failure.
+ */
+static int
+mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
+				struct rte_pci_addr *addr)
+{
+	int err;
+	struct devx_device mlx5_dev;
+	if (mlx5_match_devx_bdf_to_addr(devx_bdf, addr))
+		return 1;
+
+	/**
+	 * Didn't match on Native/PF BDF, could still
+	 * Match a VF BDF, check it next
+	 */
+	err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
+	if (err) {
+		DRV_LOG(ERR, "query_device failed");
+		rte_errno = err;
+		return 0;
+	}
+	if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
+		return 1;
+	return 0;
+}
+
+/**
  * DPDK callback to register a PCI device.
  *
  * This function spawns Ethernet devices out of a given PCI device.
@@ -971,9 +1033,8 @@ mlx5_os_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
 	struct devx_device_bdf *devx_bdf_match[ret + 1];
 
 	while (ret-- > 0) {
-		if (pci_dev->addr.bus != devx_bdf_devs->bus_id ||
-		    pci_dev->addr.devid != devx_bdf_devs->dev_id ||
-		    pci_dev->addr.function != devx_bdf_devs->fnc_id) {
+		if (!mlx5_match_devx_devices_to_addr(devx_bdf_devs,
+		    &pci_dev->addr)) {
 			devx_bdf_devs++;
 			continue;
 		}
-- 
2.8.4

[dpdk-dev] [PATCH v1 60/72] net/mlx5/linux: wrap adjust flow priority with OS calls

[Ophir Munk]

mlx5_flow_adjust_priority() is used to adjust priorities according to
priorities levels. It is Verbs based and it is called from shared code
(mlx5_flow_dv.c). Therefore, wrap it in an OS API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_flow_os.h | 20 ++++++++++++++++++++
 drivers/net/mlx5/mlx5_flow_dv.c       |  5 +++--
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_flow_os.h b/drivers/net/mlx5/linux/mlx5_flow_os.h
index a6bd2c0..c7003a1 100644
--- a/drivers/net/mlx5/linux/mlx5_flow_os.h
+++ b/drivers/net/mlx5/linux/mlx5_flow_os.h
@@ -365,4 +365,24 @@ mlx5_flow_os_destroy_flow_action(void *action)
 	return mlx5_glue->destroy_flow_action(action);
 }
 
+/**
+ * OS wrapper over Verbs API.
+ * Adjust flow priority based on the highest layer and the request priority.
+ *
+ * @param[in] dev
+ *    Pointer to the Ethernet device structure.
+ * @param[in] priority
+ *    The rule base priority.
+ * @param[in] subpriority
+ *    The priority based on the items.
+ *
+ * @return
+ *    The new priority.
+ */
+static inline uint32_t
+mlx5_os_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
+			  uint32_t subpriority)
+{
+	return mlx5_flow_adjust_priority(dev, priority, subpriority);
+}
 #endif /* RTE_PMD_MLX5_FLOW_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 2560559..18ab409 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -10172,8 +10172,9 @@ __flow_dv_translate(struct rte_eth_dev *dev,
 	/* Register matcher. */
 	matcher.crc = rte_raw_cksum((const void *)matcher.mask.buf,
 				    matcher.mask.size);
-	matcher.priority = mlx5_flow_adjust_priority(dev, priority,
-						     matcher.priority);
+	matcher.priority = mlx5_os_flow_adjust_priority(dev,
+							priority,
+							matcher.priority);
 	/* reserved field no needs to be set to 0 here. */
 	tbl_key.domain = attr->transfer;
 	tbl_key.direction = attr->egress;
-- 
2.8.4

[dpdk-dev] [PATCH v1 61/72] net/mlx5/linux: add OS default miss flow action

[Ophir Munk]

Wrap glue call dr_create_flow_action_default_miss() with an OS API. This
commit is a follow up on (1).

(1)
commit d4d85aa6f13a ("common/mlx5: add default miss action")
commit b293fbf9672b ("net/mlx5: add OS specific flow actions operations")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_flow_os.h | 16 ++++++++++++++++
 drivers/net/mlx5/mlx5_flow_dv.c       |  9 +++++----
 2 files changed, 21 insertions(+), 4 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_flow_os.h b/drivers/net/mlx5/linux/mlx5_flow_os.h
index c7003a1..7706b3b 100644
--- a/drivers/net/mlx5/linux/mlx5_flow_os.h
+++ b/drivers/net/mlx5/linux/mlx5_flow_os.h
@@ -351,6 +351,22 @@ mlx5_flow_os_create_flow_action_drop(void **action)
 }
 
 /**
+ * Create flow action: default miss.
+ *
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or -1 on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_default_miss(void **action)
+{
+	*action = mlx5_glue->dr_create_flow_action_default_miss();
+	return (*action) ? 0 : -1;
+}
+
+/**
  * Destroy flow action.
  *
  * @param[in] action
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 18ab409..5c772e7 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -2987,12 +2987,13 @@ flow_dv_default_miss_resource_register(struct rte_eth_dev *dev,
 	struct mlx5_flow_default_miss_resource *cache_resource =
 			&sh->default_miss;
 	int cnt = rte_atomic32_read(&cache_resource->refcnt);
+	int ret;
 
 	if (!cnt) {
 		MLX5_ASSERT(cache_resource->action);
-		cache_resource->action =
-		mlx5_glue->dr_create_flow_action_default_miss();
-		if (!cache_resource->action)
+		ret = mlx5_flow_os_create_flow_action_default_miss
+						(&cache_resource->action);
+		if (ret)
 			return rte_flow_error_set(error, ENOMEM,
 					RTE_FLOW_ERROR_TYPE_UNSPECIFIED, NULL,
 					"cannot create default miss action");
@@ -10602,7 +10603,7 @@ flow_dv_default_miss_resource_release(struct rte_eth_dev *dev)
 			(void *)cache_resource->action,
 			rte_atomic32_read(&cache_resource->refcnt));
 	if (rte_atomic32_dec_and_test(&cache_resource->refcnt)) {
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 				(cache_resource->action));
 		DRV_LOG(DEBUG, "default miss resource %p: removed",
 				(void *)cache_resource->action);
-- 
2.8.4

[dpdk-dev] [PATCH v1 62/72] net/mlx5/linux: fix add OS dest_devx_tir action

[Ophir Munk]

Wrap glue call dv_create_flow_action_dest_devx_tir() with an OS API.

Fixes: b293fbf9672b ("net/mlx5: add OS specific flow actions operations")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_flow_os.h | 26 ++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5_devx.c          |  7 +++----
 2 files changed, 29 insertions(+), 4 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_flow_os.h b/drivers/net/mlx5/linux/mlx5_flow_os.h
index 7706b3b..6f3b732 100644
--- a/drivers/net/mlx5/linux/mlx5_flow_os.h
+++ b/drivers/net/mlx5/linux/mlx5_flow_os.h
@@ -367,6 +367,32 @@ mlx5_flow_os_create_flow_action_default_miss(void **action)
 }
 
 /**
+ * Create flow action: dest_devx_tir
+ *
+ * @param[in] tir
+ *   Pointer to DevX tir object
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or -1 on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_dest_devx_tir(struct mlx5_devx_obj *tir,
+					      void **action)
+{
+#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+	*action = mlx5_glue->dv_create_flow_action_dest_devx_tir(tir->obj);
+	return (*action) ? 0 : -1;
+#else
+	/* If no DV support - skip the operation and return success */
+	RTE_SET_USED(tir);
+	*action = 0;
+	return 0;
+#endif
+}
+
+/**
  * Destroy flow action.
  *
  * @param[in] action
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 23d4190..1aff17c 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -23,7 +23,7 @@
 #include "mlx5_utils.h"
 #include "mlx5_devx.h"
 #include "mlx5_flow.h"
-
+#include "mlx5_flow_os.h"
 
 /**
  * Modify RQ vlan stripping offload
@@ -855,9 +855,8 @@ mlx5_devx_hrxq_new(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq,
 		goto error;
 	}
 #ifdef HAVE_IBV_FLOW_DV_SUPPORT
-	hrxq->action = mlx5_glue->dv_create_flow_action_dest_devx_tir
-							       (hrxq->tir->obj);
-	if (!hrxq->action) {
+	if (mlx5_flow_os_create_flow_action_dest_devx_tir(hrxq->tir,
+							  &hrxq->action)) {
 		rte_errno = errno;
 		goto error;
 	}
-- 
2.8.4

[dpdk-dev] [PATCH v1 63/72] drivers/net: enable Windows net/mlx5 compilation

[Ophir Munk]

Add mlx5 as the first driver to be compiled under Windows (file
drivers/net/meson.build).

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/meson.build | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/net/meson.build b/drivers/net/meson.build
index 4e4c2c9..a341962 100644
--- a/drivers/net/meson.build
+++ b/drivers/net/meson.build
@@ -1,10 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-if is_windows
-	subdir_done()
-endif
-
 drivers = ['af_packet',
 	'af_xdp',
 	'ark',
@@ -55,6 +51,11 @@ drivers = ['af_packet',
 	'virtio',
 	'vmxnet3',
 ]
+
+if is_windows
+drivers = ['mlx5',]
+endif
+
 std_deps = ['ethdev', 'kvargs'] # 'ethdev' also pulls in mbuf, net, eal etc
 std_deps += ['bus_pci']         # very many PMDs depend on PCI, so make std
 std_deps += ['bus_vdev']        # same with vdev bus
-- 
2.8.4

[dpdk-dev] [PATCH v1 64/72] net/mlx5/windows: introduce flow support

[Ophir Munk]

This patch adds the initial flow framework under Windows OS. It supports
a subset of filters (ETH, IPV4, UDP) and a QUEUE action.  It is based on
DevX mechanism to send commands to the NIC through the kernel. It does
not support steering rules (i.e. writing directly to the NIC memory).
The Windows framework uses the existing DV framework where file
mlx5_flow_dv.c remains intact.

Steps involved in flow creation:
1. Create a domain (RX, TX, FDB). Since domains are created by steering
rules and not with DevX, Windows does not require a domain object (this
means switch dev mode which requires an FDB domain is not supported).
2. Create a table object. Windows only supports table 0. The call to
mlx5_flow_os_create_flow_tbl() verifies the table number.
3. Create a matcher object. A matcher struct is created by calling
mlx5_flow_os_create_flow_matcher().  The matcher validation and
translation are part of the DV implementation. The matcher bits that
were created by DV in standard PRM format are copied into the matcher
struct.
4. Create an action object. The call to
mlx5_flow_os_create_flow_action_dest_devx_tir() creates an action struct
with the TIR type and id.  This struct will be a parameter later in a
call to flow creation.  All other action calls (e.g. packet reformat,
header modification, jump to flow table, etc) return with a non
supported error.
5. Create the flow. The call to mlx5_flow_os_create_flow() receives the
matcher struct, action struct, and copy them into Windows specific
fs_rule struct, then it calls glue API devx_fs_rule_add().

Details on additional APIs:
* mlx5_flow_os_get_type() is called during flow type selection. In
Windows it always returns MLX5_FLOW_TYPE_DV.
* mlx5_flow_os_item_supported() is called before starting DV items
validation or translation. It filters out the OS non supported items in
advance.
* mlx5_flow_os_action_supported() is called before starting DV actions
validation or translation. It filters out the OS non supported actions
in advance.
* mlx5_flow_adjust_priority() is an OS stub for flow priority
adjustment. Windows only supports flow priority 0.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Dekel Peled <dekelp@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_win_defs.h |   7 +
 drivers/net/mlx5/windows/meson.build        |   1 +
 drivers/net/mlx5/windows/mlx5_flow_os.c     | 188 ++++++++++++++
 drivers/net/mlx5/windows/mlx5_flow_os.h     | 363 ++++++++++++++++++++++++++++
 4 files changed, 559 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/mlx5_flow_os.c
 create mode 100644 drivers/net/mlx5/windows/mlx5_flow_os.h

diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
index d8f2099..27dcf40 100644
--- a/drivers/common/mlx5/windows/mlx5_win_defs.h
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -169,4 +169,11 @@ struct mlx5_wqe_data_seg {
 #define IBV_RAW_PACKET_CAP_CVLAN_STRIPPING	(1 << 0)
 #define IBV_RAW_PACKET_CAP_SCATTER_FCS		(1 << 1)
 #define IBV_QPT_RAW_PACKET			8
+
+enum {
+	MLX5_FLOW_CONTEXT_DEST_TYPE_VPORT                    = 0x0,
+	MLX5_FLOW_CONTEXT_DEST_TYPE_FLOW_TABLE               = 0x1,
+	MLX5_FLOW_CONTEXT_DEST_TYPE_TIR                      = 0x2,
+	MLX5_FLOW_CONTEXT_DEST_TYPE_QP                       = 0x3,
+};
 #endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
index 87e34d3..8404dc1 100644
--- a/drivers/net/mlx5/windows/meson.build
+++ b/drivers/net/mlx5/windows/meson.build
@@ -7,5 +7,6 @@ sources += files(
 	'mlx5_mp_os.c',
 	'mlx5_ethdev_os.c',
 	'mlx5_vlan_os.c',
+	'mlx5_flow_os.c',
 )
 
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.c b/drivers/net/mlx5/windows/mlx5_flow_os.c
new file mode 100644
index 0000000..8c504fc
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.c
@@ -0,0 +1,188 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include "mlx5_flow_os.h"
+#include "mlx5_win_ext.h"
+
+/**
+ * Verify the @p attributes will be correctly understood by the NIC and store
+ * them in the @p flow if everything is correct.
+ *
+ * @param[in] dev
+ *   Pointer to dev struct.
+ * @param[in] attributes
+ *   Pointer to flow attributes
+ * @param[in] external
+ *   This flow rule is created by request external to PMD.
+ * @param[out] error
+ *   Pointer to error structure.
+ *
+ * @return
+ *   - 0 on success and non root table (not a valid option for Windows yet).
+ *   - 1 on success and root table.
+ *   - a negative errno value otherwise and rte_errno is set.
+ */
+int
+mlx5_flow_os_validate_flow_attributes(struct rte_eth_dev *dev,
+				      const struct rte_flow_attr *attributes,
+				      bool external,
+				      struct rte_flow_error *error)
+{
+	int ret = 1;
+
+	RTE_SET_USED(dev);
+	RTE_SET_USED(external);
+	if (attributes->group)
+		return rte_flow_error_set(error, ENOTSUP,
+					  RTE_FLOW_ERROR_TYPE_ATTR_GROUP,
+					  NULL,
+					  "groups are not supported");
+	if (attributes->priority)
+		return rte_flow_error_set(error, ENOTSUP,
+					  RTE_FLOW_ERROR_TYPE_ATTR_PRIORITY,
+					  NULL,
+					  "priorities are not supported");
+	if (attributes->transfer)
+		return rte_flow_error_set(error, ENOTSUP,
+					  RTE_FLOW_ERROR_TYPE_ATTR_TRANSFER,
+					  NULL,
+					  "transfer not supported");
+	if (!(attributes->ingress))
+		return rte_flow_error_set(error, ENOTSUP,
+					  RTE_FLOW_ERROR_TYPE_ATTR_INGRESS,
+					  NULL, "must specify ingress only");
+	return ret;
+}
+
+/**
+ * Create flow matcher in a flow table.
+ *
+ * @param[in] ctx
+ *   Pointer to relevant device context.
+ * @param[in] attr
+ *   Pointer to relevant attributes.
+ * @param[in] table
+ *   Pointer to table object.
+ * @param[out] matcher
+ *   Pointer to a valid flow matcher object on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or errno on failure.
+ */
+int
+mlx5_flow_os_create_flow_matcher(void *ctx,
+				 void *attr,
+				 void *table,
+				 void **matcher)
+{
+	RTE_SET_USED(ctx);
+	RTE_SET_USED(attr);
+	RTE_SET_USED(table);
+	*matcher = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Destroy flow matcher.
+ *
+ * @param[in] matcher
+ *   Pointer to matcher object to destroy.
+ *
+ * @return
+ *   0 on success, or the value of errno on failure.
+ */
+int
+mlx5_flow_os_destroy_flow_matcher(void *matcher)
+{
+	RTE_SET_USED(matcher);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: dest_devx_tir
+ *
+ * @param[in] tir
+ *   Pointer to DevX tir object
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or errno on failure.
+ */
+int
+mlx5_flow_os_create_flow_action_dest_devx_tir(struct mlx5_devx_obj *tir,
+					      void **action)
+{
+	RTE_SET_USED(tir);
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Destroy flow action.
+ *
+ * @param[in] action
+ *   Pointer to action object to destroy.
+ *
+ * @return
+ *   0 on success, or the value of errno on failure.
+ */
+int
+mlx5_flow_os_destroy_flow_action(void *action)
+{
+	RTE_SET_USED(action);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow rule.
+ *
+ * @param[in] matcher
+ *   Pointer to match mask structure.
+ * @param[in] match_value
+ *   Pointer to match value structure.
+ * @param[in] num_actions
+ *   Number of actions in flow rule.
+ * @param[in] actions
+ *   Pointer to array of flow rule actions.
+ * @param[out] flow
+ *   Pointer to a valid flow rule object on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or errno on failure.
+ */
+int
+mlx5_flow_os_create_flow(void *matcher, void *match_value,
+			 size_t num_actions,
+			 void *actions[], void **flow)
+{
+	RTE_SET_USED(matcher);
+	RTE_SET_USED(match_value);
+	RTE_SET_USED(num_actions);
+	RTE_SET_USED(actions);
+	*flow = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Destroy flow rule.
+ *
+ * @param[in] drv_flow_ptr
+ *   Pointer to flow rule object.
+ *
+ * @return
+ *   0 on success, errno on failure.
+ */
+int
+mlx5_flow_os_destroy_flow(void *drv_flow_ptr)
+{
+	RTE_SET_USED(dev_flow_ptr);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.h b/drivers/net/mlx5/windows/mlx5_flow_os.h
new file mode 100644
index 0000000..1e762c1
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.h
@@ -0,0 +1,363 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_FLOW_OS_H_
+#define RTE_PMD_MLX5_FLOW_OS_H_
+
+#include "mlx5_flow.h"
+
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
+extern const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops;
+#endif
+
+/**
+ * Get OS enforced flow type. MLX5_FLOW_TYPE_MAX means "non enforced type".
+ *
+ * @return
+ *   Flow type (MLX5_FLOW_TYPE_MAX)
+ */
+static inline enum mlx5_flow_drv_type
+mlx5_flow_os_get_type(void)
+{
+	return MLX5_FLOW_TYPE_DV;
+}
+
+/**
+ * Check if item type is supported.
+ *
+ * @param item
+ *   Item type to check.
+ *
+ * @return
+ *   True is this item type is supported, false if not supported.
+ */
+static inline bool
+mlx5_flow_os_item_supported(int item)
+{
+	switch (item) {
+	case RTE_FLOW_ITEM_TYPE_END:
+	case RTE_FLOW_ITEM_TYPE_VOID:
+	case RTE_FLOW_ITEM_TYPE_ETH:
+	case RTE_FLOW_ITEM_TYPE_IPV4:
+	case RTE_FLOW_ITEM_TYPE_UDP:
+		return true;
+	default:
+		return false;
+	}
+}
+
+/**
+ * Check if action type is supported.
+ *
+ * @param action
+ *   Action type to check.
+ *
+ * @return
+ *   True is this action type is supported, false if not supported.
+ */
+static inline bool
+mlx5_flow_os_action_supported(int action)
+{
+	switch (action) {
+	case RTE_FLOW_ACTION_TYPE_END:
+	case RTE_FLOW_ACTION_TYPE_VOID:
+	case RTE_FLOW_ACTION_TYPE_QUEUE:
+	case RTE_FLOW_ACTION_TYPE_RSS:
+		return true;
+	default:
+		return false;
+	}
+}
+
+/**
+ * Create flow table.
+ *
+ * @param[in] domain
+ *   Pointer to relevant domain.
+ * @param[in] table_id
+ *   Table ID.
+ * @param[out] table
+ *   NULL (no table object required)
+ *
+ * @return
+ *   0 if table_id is 0, negative value otherwise and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_tbl(void *domain, uint32_t table_id, void **table)
+{
+	RTE_SET_USED(domain);
+	*table = NULL;
+	if (table_id) {
+		rte_errno = ENOTSUP;
+		return -rte_errno;
+	}
+	return 0;
+}
+
+/**
+ * Destroy flow table.
+ *
+ * @param table
+ *   Pointer to table to destroy.
+ *
+ * @return
+ *   0 on success (silently ignored).
+ */
+static inline int
+mlx5_flow_os_destroy_flow_tbl(void *table)
+{
+	RTE_SET_USED(table);
+	/* Silently ignore */
+	return 0;
+}
+
+/**
+ * Create flow action: packet reformat.
+ *
+ * @param[in] ctx
+ *   Pointer to relevant device context.
+ * @param[in] domain
+ *   Pointer to domain handler.
+ * @param[in] resource
+ *   Pointer to action data resource.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_packet_reformat(void *ctx, void *domain,
+						void *resource, void **action)
+{
+	RTE_SET_USED(ctx);
+	RTE_SET_USED(domain);
+	RTE_SET_USED(resource);
+	RTE_SET_USED(action);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: modify header.
+ *
+ * @param[in] ctx
+ *   Pointer to relevant device context.
+ * @param[in] domain
+ *   Pointer to domain handler.
+ * @param[in] resource
+ *   Pointer to action data resource.
+ * @param[in] actions_len
+ *   Total length of actions data in resource.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ *
+ * @return
+ *   0 on success, or -1 on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_modify_header(void *ctx,
+					      void *domain,
+					      void *resource,
+					      uint32_t actions_len,
+					      void **action)
+{
+	RTE_SET_USED(ctx);
+	RTE_SET_USED(domain);
+	RTE_SET_USED(resource);
+	RTE_SET_USED(actions_len);
+	RTE_SET_USED(action);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: destination flow table.
+ *
+ * @param[in] tbl_obj
+ *   Pointer to destination table object.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_dest_flow_tbl(void *tbl_obj, void **action)
+{
+	RTE_SET_USED(tbl_obj);
+	RTE_SET_USED(action);
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: destination port.
+ *
+ * @param[in] domain
+ *   Pointer to domain handler.
+ * @param[in] port_id
+ *   Destination port ID.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_dest_port(void *domain, uint32_t port_id,
+					  void **action)
+{
+	RTE_SET_USED(domain);
+	RTE_SET_USED(port_id);
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: push vlan.
+ *
+ * @param[in] domain
+ *   Pointer to domain handler.
+ * @param[in] vlan_tag
+ *   VLAN tag value.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_push_vlan(void *domain, rte_be32_t vlan_tag,
+					  void **action)
+{
+	RTE_SET_USED(domain);
+	RTE_SET_USED(vlan_tag);
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: count.
+ *
+ * @param[in] cnt_obj
+ *   Pointer to DevX counter object.
+ * @param[in] offset
+ *   Offset of counter in array.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_count(void *cnt_obj, uint16_t offset,
+				      void **action)
+{
+	RTE_SET_USED(cnt_obj);
+	RTE_SET_USED(offset);
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: tag.
+ *
+ * @param[in] tag
+ *   Tag value.
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_tag(uint32_t tag, void **action)
+{
+	RTE_SET_USED(tag);
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: drop.
+ *
+ * @param[out] action
+ *   Pointer to a valid action on success, NULL otherwise.
+ *
+ * @return
+ *   0 on success, or negative value on failure and errno is set.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_drop(void **action)
+{
+	*action = NULL;
+	rte_errno = ENOTSUP;
+	return -rte_errno;
+}
+
+/**
+ * Create flow action: default miss.
+ *
+ * @param[out] action
+ *   NULL action pointer.
+ *
+ * @return
+ *   0 as success.
+ */
+static inline int
+mlx5_flow_os_create_flow_action_default_miss(void **action)
+{
+	*action = 0;
+	/* Silently ignore */
+	return 0;
+}
+
+/**
+ * OS stub for mlx5_flow_adjust_priority() API.
+ * Windows only supports flow priority 0 that cannot be adjusted.
+ *
+ * @param[in] dev
+ *    Pointer to the Ethernet device structure.
+ * @param[in] priority
+ *    The rule base priority.
+ * @param[in] subpriority
+ *    The priority based on the items.
+ *
+ * @return
+ *    0
+ */
+static inline uint32_t
+mlx5_os_flow_adjust_priority(struct rte_eth_dev *dev, int32_t priority,
+			  uint32_t subpriority)
+{
+	RTE_SET_USED(dev);
+	RTE_SET_USED(priority);
+	RTE_SET_USED(subpriority);
+	return 0;
+}
+
+int mlx5_flow_os_validate_flow_attributes(struct rte_eth_dev *dev,
+					const struct rte_flow_attr *attributes,
+					bool external,
+					struct rte_flow_error *error);
+int mlx5_flow_os_create_flow_matcher(void *ctx,
+				     void *attr,
+				     void *table,
+				     void **matcher);
+int mlx5_flow_os_destroy_flow_matcher(void *matcher);
+int mlx5_flow_os_create_flow_action_dest_devx_tir(struct mlx5_devx_obj *tir,
+						  void **action);
+int mlx5_flow_os_destroy_flow_action(void *action);
+int mlx5_flow_os_create_flow(void *matcher, void *match_value,
+			     size_t num_actions,
+			     void *actions[], void **flow);
+int mlx5_flow_os_destroy_flow(void *drv_flow_ptr);
+#endif /* RTE_PMD_MLX5_FLOW_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 65/72] net/mlx5/windows: create flow matcher object

[Ophir Munk]

This commit implements the mlx5_flow_os_create_flow_matcher() API. It is
the Linux rdma-core equivalent implementation. Missing rdma-core
parameters (e.g. struct mlx5dv_flow_match_parameters) are added to file
mlx5_win_defs.h. The API allocates space to hold the PRM bits in PRM
fte_match_param format and copy the DV translated PRM bits into the
matcher struct. This matcher struct will be used later by the flow
creation API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_win_defs.h | 55 +++++++++++++++++++++++++++++
 drivers/net/mlx5/windows/mlx5_flow_os.c     | 30 ++++++++++++----
 drivers/net/mlx5/windows/mlx5_flow_os.h     |  1 +
 3 files changed, 79 insertions(+), 7 deletions(-)

diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
index 27dcf40..f75329b 100644
--- a/drivers/common/mlx5/windows/mlx5_win_defs.h
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -132,6 +132,61 @@ enum {
 #define MLX5DV_FLOW_ACTION_PACKET_REFORMAT_TYPE_L2_TO_L3_TUNNEL	0x3
 #endif
 
+enum ibv_flow_flags {
+	IBV_FLOW_ATTR_FLAGS_ALLOW_LOOP_BACK = 1 << 0,
+	IBV_FLOW_ATTR_FLAGS_DONT_TRAP = 1 << 1,
+	IBV_FLOW_ATTR_FLAGS_EGRESS = 1 << 2,
+};
+
+enum ibv_flow_attr_type {
+	/* steering according to rule specifications */
+	IBV_FLOW_ATTR_NORMAL		= 0x0,
+	/* default unicast and multicast rule -
+	 * receive all Eth traffic which isn't steered to any QP
+	 */
+	IBV_FLOW_ATTR_ALL_DEFAULT	= 0x1,
+	/* default multicast rule -
+	 * receive all Eth multicast traffic which isn't steered to any QP
+	 */
+	IBV_FLOW_ATTR_MC_DEFAULT	= 0x2,
+	/* sniffer rule - receive all port traffic */
+	IBV_FLOW_ATTR_SNIFFER		= 0x3,
+};
+
+enum mlx5dv_flow_table_type {
+	MLX5_IB_UAPI_FLOW_TABLE_TYPE_NIC_RX     = 0x0,
+	MLX5_IB_UAPI_FLOW_TABLE_TYPE_NIC_TX	= 0x1,
+	MLX5_IB_UAPI_FLOW_TABLE_TYPE_FDB	= 0x2,
+	MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_RX	= 0x3,
+};
+
+#define MLX5DV_FLOW_TABLE_TYPE_NIC_RX	MLX5_IB_UAPI_FLOW_TABLE_TYPE_NIC_RX
+#define MLX5DV_FLOW_TABLE_TYPE_NIC_TX	MLX5_IB_UAPI_FLOW_TABLE_TYPE_NIC_TX
+#define MLX5DV_FLOW_TABLE_TYPE_FDB	MLX5_IB_UAPI_FLOW_TABLE_TYPE_FDB
+#define MLX5DV_FLOW_TABLE_TYPE_RDMA_RX	MLX5_IB_UAPI_FLOW_TABLE_TYPE_RDMA_RX
+
+struct mlx5dv_flow_match_parameters {
+	size_t match_sz;
+	uint64_t match_buf[]; /* Device spec format */
+};
+
+struct mlx5dv_flow_matcher_attr {
+	enum ibv_flow_attr_type type;
+	uint32_t flags; /* From enum ibv_flow_flags */
+	uint16_t priority;
+	uint8_t match_criteria_enable; /* Device spec format */
+	struct mlx5dv_flow_match_parameters *match_mask;
+	uint64_t comp_mask; /* use mlx5dv_flow_matcher_attr_mask */
+	enum mlx5dv_flow_table_type ft_type;
+};
+
+/* Windows specific mlx5_matcher */
+struct mlx5_matcher {
+	void *ctx;
+	struct mlx5dv_flow_matcher_attr attr;
+	uint64_t match_buf[];
+};
+
 struct mlx5_err_cqe {
 	uint8_t		rsvd0[32];
 	uint32_t	srqn;
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.c b/drivers/net/mlx5/windows/mlx5_flow_os.c
index 8c504fc..d572821 100644
--- a/drivers/net/mlx5/windows/mlx5_flow_os.c
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.c
@@ -76,12 +76,29 @@ mlx5_flow_os_create_flow_matcher(void *ctx,
 				 void *table,
 				 void **matcher)
 {
-	RTE_SET_USED(ctx);
-	RTE_SET_USED(attr);
 	RTE_SET_USED(table);
 	*matcher = NULL;
-	rte_errno = ENOTSUP;
-	return -rte_errno;
+	struct mlx5dv_flow_matcher_attr *mattr = attr;
+	if (mattr->type != IBV_FLOW_ATTR_NORMAL) {
+		rte_errno = ENOTSUP;
+		return -rte_errno;
+	}
+	struct mlx5_matcher *mlx5_matcher =
+		mlx5_malloc(MLX5_MEM_ZERO,
+		       sizeof(struct mlx5_matcher) +
+		       MLX5_ST_SZ_BYTES(fte_match_param),
+		       0, SOCKET_ID_ANY);
+	if (!mlx5_matcher) {
+		rte_errno = ENOMEM;
+		return -rte_errno;
+	}
+	mlx5_matcher->ctx = ctx;
+	memcpy(&mlx5_matcher->attr, attr, sizeof(mlx5_matcher->attr));
+	memcpy(&mlx5_matcher->match_buf,
+	       mattr->match_mask->match_buf,
+	       MLX5_ST_SZ_BYTES(fte_match_param));
+	*matcher = mlx5_matcher;
+	return 0;
 }
 
 /**
@@ -96,9 +113,8 @@ mlx5_flow_os_create_flow_matcher(void *ctx,
 int
 mlx5_flow_os_destroy_flow_matcher(void *matcher)
 {
-	RTE_SET_USED(matcher);
-	rte_errno = ENOTSUP;
-	return -rte_errno;
+	mlx5_free(matcher);
+	return 0;
 }
 
 /**
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.h b/drivers/net/mlx5/windows/mlx5_flow_os.h
index 1e762c1..5018566 100644
--- a/drivers/net/mlx5/windows/mlx5_flow_os.h
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.h
@@ -6,6 +6,7 @@
 #define RTE_PMD_MLX5_FLOW_OS_H_
 
 #include "mlx5_flow.h"
+#include "mlx5_malloc.h"
 
 #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 extern const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops;
-- 
2.8.4

[dpdk-dev] [PATCH v1 66/72] net/mlx5/windows: create flow action dest TIR object

[Ophir Munk]

This commit implements mlx5_flow_os_create_flow_action_dest_devx_tir()
API as the Linux rdma-core equivalent. Missing rdma-core parameters are
added to file mlx5_win_defs.h. The action TIR id and type
(MLX5_FLOW_CONTEXT_DEST_TYPE_TIR) are saved in the action struct.  The
action struct will be added to array of actions and will be used later
by the flow creation API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_win_defs.h | 16 ++++++++++++++++
 drivers/net/mlx5/windows/mlx5_flow_os.c     | 21 ++++++++++++++-------
 2 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
index f75329b..191f4b0 100644
--- a/drivers/common/mlx5/windows/mlx5_win_defs.h
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -187,6 +187,16 @@ struct mlx5_matcher {
 	uint64_t match_buf[];
 };
 
+/* Windows mlx5_action. This struct is the
+ * equivalent of rdma-core struct mlx5dv_dr_action
+ */
+struct mlx5_action {
+	int type;
+	struct {
+		uint32_t id;
+	} dest_tir;
+};
+
 struct mlx5_err_cqe {
 	uint8_t		rsvd0[32];
 	uint32_t	srqn;
@@ -231,4 +241,10 @@ enum {
 	MLX5_FLOW_CONTEXT_DEST_TYPE_TIR                      = 0x2,
 	MLX5_FLOW_CONTEXT_DEST_TYPE_QP                       = 0x3,
 };
+
+enum {
+	MLX5_MATCH_OUTER_HEADERS        = 1 << 0,
+	MLX5_MATCH_MISC_PARAMETERS      = 1 << 1,
+	MLX5_MATCH_INNER_HEADERS        = 1 << 2,
+};
 #endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.c b/drivers/net/mlx5/windows/mlx5_flow_os.c
index d572821..05010f4 100644
--- a/drivers/net/mlx5/windows/mlx5_flow_os.c
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.c
@@ -132,10 +132,18 @@ int
 mlx5_flow_os_create_flow_action_dest_devx_tir(struct mlx5_devx_obj *tir,
 					      void **action)
 {
-	RTE_SET_USED(tir);
-	*action = NULL;
-	rte_errno = ENOTSUP;
-	return -rte_errno;
+	struct mlx5_action *mlx5_action =
+		mlx5_malloc(MLX5_MEM_ZERO,
+		       sizeof(struct mlx5_action),
+		       0, SOCKET_ID_ANY);
+	if (!mlx5_action) {
+		rte_errno = ENOMEM;
+		return -rte_errno;
+	}
+	mlx5_action->type = MLX5_FLOW_CONTEXT_DEST_TYPE_TIR;
+	mlx5_action->dest_tir.id = tir->id;
+	*action = mlx5_action;
+	return 0;
 }
 
 /**
@@ -150,9 +158,8 @@ mlx5_flow_os_create_flow_action_dest_devx_tir(struct mlx5_devx_obj *tir,
 int
 mlx5_flow_os_destroy_flow_action(void *action)
 {
-	RTE_SET_USED(action);
-	rte_errno = ENOTSUP;
-	return -rte_errno;
+	mlx5_free(action);
+	return 0;
 }
 
 /**
-- 
2.8.4

[dpdk-dev] [PATCH v1 67/72] net/mlx5/windows: create flow rule

[Ophir Munk]

This commit implements mlx5_flow_os_create_flow() API. It is equivalent
to Linux rdma-core implementation. The API receives the matcher mask,
matcher value and an array of actions. They are copied into a PRM-like
struct devx_fs_rule_add_in. Then glue API devx_fs_rule_add() is called.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h          |  6 +++++
 drivers/net/mlx5/windows/mlx5_flow_os.c | 46 ++++++++++++++++++++++++++-------
 2 files changed, 42 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 1dfd6bd..bbd15bf 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -809,6 +809,12 @@ struct mlx5_ifc_fte_match_param_bits {
 #endif
 };
 
+struct mlx5_ifc_dest_format_struct_bits {
+	u8 destination_type[0x8];
+	u8 destination_id[0x18];
+	u8 reserved_0[0x20];
+};
+
 enum {
 	MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT,
 	MLX5_MATCH_CRITERIA_ENABLE_MISC_BIT,
diff --git a/drivers/net/mlx5/windows/mlx5_flow_os.c b/drivers/net/mlx5/windows/mlx5_flow_os.c
index 05010f4..bc5fd39 100644
--- a/drivers/net/mlx5/windows/mlx5_flow_os.c
+++ b/drivers/net/mlx5/windows/mlx5_flow_os.c
@@ -184,13 +184,41 @@ mlx5_flow_os_create_flow(void *matcher, void *match_value,
 			 size_t num_actions,
 			 void *actions[], void **flow)
 {
-	RTE_SET_USED(matcher);
-	RTE_SET_USED(match_value);
-	RTE_SET_USED(num_actions);
-	RTE_SET_USED(actions);
-	*flow = NULL;
-	rte_errno = ENOTSUP;
-	return -rte_errno;
+	struct mlx5_action *action;
+	int i;
+	struct mlx5_matcher *mlx5_matcher = matcher;
+	struct mlx5_flow_dv_match_params *mlx5_match_value = match_value;
+	uint32_t in[MLX5_ST_SZ_DW(devx_fs_rule_add_in)] = {0};
+	void *matcher_c = MLX5_ADDR_OF(devx_fs_rule_add_in, in,
+				       match_criteria);
+	void *matcher_v = MLX5_ADDR_OF(devx_fs_rule_add_in, in,
+				       match_value);
+
+	MLX5_ASSERT(mlx5_matcher->ctx);
+	memcpy(matcher_c, mlx5_matcher->match_buf,
+	       mlx5_match_value->size);
+	/* Use mlx5_match_value->size for match criteria */
+	memcpy(matcher_v, mlx5_match_value->buf,
+	       mlx5_match_value->size);
+	for (i = 0; i < num_actions; i++) {
+		action = actions[i];
+		switch (action->type) {
+		case MLX5_FLOW_CONTEXT_DEST_TYPE_TIR:
+			MLX5_SET(devx_fs_rule_add_in, in,
+				 dest.destination_type,
+				 MLX5_FLOW_CONTEXT_DEST_TYPE_TIR);
+			MLX5_SET(devx_fs_rule_add_in, in,
+				 dest.destination_id,
+				 action->dest_tir.id);
+			break;
+		default:
+			break;
+		}
+		MLX5_SET(devx_fs_rule_add_in, in, match_criteria_enable,
+			 MLX5_MATCH_OUTER_HEADERS);
+	}
+	*flow = mlx5_glue->devx_fs_rule_add(mlx5_matcher->ctx, in, sizeof(in));
+	return (*flow) ? 0 : -1;
 }
 
 /**
@@ -205,7 +233,5 @@ mlx5_flow_os_create_flow(void *matcher, void *match_value,
 int
 mlx5_flow_os_destroy_flow(void *drv_flow_ptr)
 {
-	RTE_SET_USED(dev_flow_ptr);
-	rte_errno = ENOTSUP;
-	return -rte_errno;
+	return mlx5_glue->devx_fs_rule_del(drv_flow_ptr);
 }
-- 
2.8.4

[dpdk-dev] [PATCH v1 68/72] net/mlx5: use HAVE_INFINIBAND_VERBS_H in shared code

[Ophir Munk]

Use macro HAVE_INFINIBAND_VERBS_H to successfully compile files both
under Linux and Windows (or any non Linux in general). Under Windows
this macro:
1. Hides Verbs references.
2. Exposes required DV structs that are under ifdefs related to rdma
core.

Linux code under definitions such as #ifdef HAVE_IBV_FLOW_DV_SUPPORT is
required unconditionally under Windows however those definitions are
never effective without rdma-core presence. Therefore update the #ifdef
condition to consider HAVE_INFINIBAND_VERBS_H as well (undefined macro
when running without an rdma-core library).

For example:
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/mlx5.c            |  2 +-
 drivers/net/mlx5/mlx5.h            |  4 ++--
 drivers/net/mlx5/mlx5_devx.c       |  8 ++++----
 drivers/net/mlx5/mlx5_flow.c       |  2 +-
 drivers/net/mlx5/mlx5_flow.h       | 16 +++++++---------
 drivers/net/mlx5/mlx5_flow_dv.c    |  2 +-
 drivers/net/mlx5/mlx5_flow_verbs.c |  6 ++++++
 7 files changed, 22 insertions(+), 18 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index f4b466f..19d8497 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -187,7 +187,7 @@ static LIST_HEAD(, mlx5_dev_ctx_shared) mlx5_dev_ctx_list =
 static pthread_mutex_t mlx5_dev_ctx_list_mutex = PTHREAD_MUTEX_INITIALIZER;
 
 static const struct mlx5_indexed_pool_config mlx5_ipool_cfg[] = {
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	{
 		.size = sizeof(struct mlx5_flow_dv_encap_decap_resource),
 		.trunk_size = 64,
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index c8192df..3f0211a 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -33,7 +33,7 @@
 #include "mlx5_autoconf.h"
 
 enum mlx5_ipool_index {
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	MLX5_IPOOL_DECAP_ENCAP = 0, /* Pool for encap/decap resource. */
 	MLX5_IPOOL_PUSH_VLAN, /* Pool for push vlan resource. */
 	MLX5_IPOOL_TAG, /* Pool for tag resource. */
@@ -746,7 +746,7 @@ struct mlx5_hrxq {
 		void *qp; /* Verbs queue pair. */
 		struct mlx5_devx_obj *tir; /* DevX TIR object. */
 	};
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	void *action; /* DV QP action pointer. */
 #endif
 	uint64_t hash_fields; /* Verbs Hash fields. */
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 1aff17c..e03832f 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -854,7 +854,7 @@ mlx5_devx_hrxq_new(struct rte_eth_dev *dev, struct mlx5_hrxq *hrxq,
 		rte_errno = errno;
 		goto error;
 	}
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	if (mlx5_flow_os_create_flow_action_dest_devx_tir(hrxq->tir,
 							  &hrxq->action)) {
 		rte_errno = errno;
@@ -1024,7 +1024,7 @@ mlx5_txq_obj_hairpin_new(struct rte_eth_dev *dev, uint16_t idx)
 	return 0;
 }
 
-#ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
+#if defined(HAVE_MLX5DV_DEVX_UAR_OFFSET) || !defined(HAVE_INFINIBAND_VERBS_H)
 /**
  * Release DevX SQ resources.
  *
@@ -1326,7 +1326,7 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 
 	if (txq_ctrl->type == MLX5_TXQ_TYPE_HAIRPIN)
 		return mlx5_txq_obj_hairpin_new(dev, idx);
-#ifndef HAVE_MLX5DV_DEVX_UAR_OFFSET
+#if !defined(HAVE_MLX5DV_DEVX_UAR_OFFSET) && defined(HAVE_INFINIBAND_VERBS_H)
 	DRV_LOG(ERR, "Port %u Tx queue %u cannot create with DevX, no UAR.",
 		     dev->data->port_id, idx);
 	rte_errno = ENOMEM;
@@ -1426,7 +1426,7 @@ mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->txq_ctrl->type == MLX5_TXQ_TYPE_HAIRPIN) {
 		if (txq_obj->tis)
 			claim_zero(mlx5_devx_cmd_destroy(txq_obj->tis));
-#ifdef HAVE_MLX5DV_DEVX_UAR_OFFSET
+#if defined(HAVE_MLX5DV_DEVX_UAR_OFFSET) || !defined(HAVE_INFINIBAND_VERBS_H)
 	} else {
 		mlx5_txq_release_devx_resources(txq_obj);
 #endif
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 93ed113..f009178 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -51,7 +51,7 @@ const struct mlx5_flow_driver_ops mlx5_flow_null_drv_ops;
 
 const struct mlx5_flow_driver_ops *flow_drv_ops[] = {
 	[MLX5_FLOW_TYPE_MIN] = &mlx5_flow_null_drv_ops,
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	[MLX5_FLOW_TYPE_DV] = &mlx5_flow_dv_drv_ops,
 #endif
 	[MLX5_FLOW_TYPE_VERBS] = &mlx5_flow_verbs_drv_ops,
diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h
index 8b5a93f..4819930 100644
--- a/drivers/net/mlx5/mlx5_flow.h
+++ b/drivers/net/mlx5/mlx5_flow.h
@@ -592,12 +592,6 @@ struct mlx5_flow_dv_dest_array_resource {
 	/**< Action resources. */
 };
 
-/* Verbs specification header. */
-struct ibv_spec_header {
-	enum ibv_flow_spec_type type;
-	uint16_t size;
-};
-
 /* RSS description. */
 struct mlx5_flow_rss_desc {
 	uint32_t level;
@@ -651,7 +645,7 @@ struct mlx5_flow_handle {
 		uint32_t rix_default_fate;
 		/**< Indicates default miss fate action. */
 	};
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 	struct mlx5_flow_handle_dv dvh;
 #endif
 } __rte_packed;
@@ -661,7 +655,7 @@ struct mlx5_flow_handle {
  * structure in Verbs. No DV flows attributes will be accessed.
  * Macro offsetof() could also be used here.
  */
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 #define MLX5_FLOW_HANDLE_VERBS_SIZE \
 	(sizeof(struct mlx5_flow_handle) - sizeof(struct mlx5_flow_handle_dv))
 #else
@@ -699,6 +693,7 @@ struct mlx5_flow_dv_workspace {
 	/**< Pointer to the destination array resource. */
 };
 
+#ifdef HAVE_INFINIBAND_VERBS_H
 /*
  * Maximal Verbs flow specifications & actions size.
  * Some elements are mutually exclusive, but enough space should be allocated.
@@ -755,6 +750,7 @@ struct mlx5_flow_verbs_workspace {
 	uint8_t specs[MLX5_VERBS_MAX_SPEC_ACT_SIZE];
 	/**< Specifications & actions buffer of verbs flow. */
 };
+#endif /* HAVE_INFINIBAND_VERBS_H */
 
 /** Maximal number of device sub-flows supported. */
 #define MLX5_NUM_MAX_DEV_FLOWS 32
@@ -769,10 +765,12 @@ struct mlx5_flow {
 	bool external; /**< true if the flow is created external to PMD. */
 	uint8_t ingress; /**< 1 if the flow is ingress. */
 	union {
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 		struct mlx5_flow_dv_workspace dv;
 #endif
+#ifdef HAVE_INFINIBAND_VERBS_H
 		struct mlx5_flow_verbs_workspace verbs;
+#endif
 	};
 	struct mlx5_flow_handle *handle;
 	uint32_t handle_idx; /* Index of the mlx5 flow handle memory. */
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index 5c772e7..f1de2d3 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -34,7 +34,7 @@
 #include "mlx5_flow_os.h"
 #include "mlx5_rxtx.h"
 
-#ifdef HAVE_IBV_FLOW_DV_SUPPORT
+#if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H)
 
 #ifndef HAVE_IBV_FLOW_DEVX_COUNTERS
 #define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0
diff --git a/drivers/net/mlx5/mlx5_flow_verbs.c b/drivers/net/mlx5/mlx5_flow_verbs.c
index 6bcc009..d6d3834 100644
--- a/drivers/net/mlx5/mlx5_flow_verbs.c
+++ b/drivers/net/mlx5/mlx5_flow_verbs.c
@@ -39,6 +39,12 @@ static const uint32_t priority_map_5[][MLX5_PRIORITY_MAP_MAX] = {
 	{ 9, 10, 11 }, { 12, 13, 14 },
 };
 
+/* Verbs specification header. */
+struct ibv_spec_header {
+	enum ibv_flow_spec_type type;
+	uint16_t size;
+};
+
 /**
  * Discover the maximum number of priority available.
  *
-- 
2.8.4

[dpdk-dev] [PATCH v1 69/72] net/mlx5: fix separating eth_dev_ops per OS

[Ophir Munk]

There are three types of eth_dev_ops: primary, secondary and isolate
represented in three callback tables per OS.  In this commit the OS
specific eth dev tables are unified into shared tables in file mlx5.c.
Starting from this commit all operating systems must implement the same
eth dev APIs. In case an OS does not support an API - it can return in
its implementation an error ENOTSUP.

Fixes: 042f5c94fd3a ("net/mlx5: refactor device operations for Linux")
Fixes: b541ecb63618 ("net/mlx5: refactor eth dev ops for Windows")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c   | 154 +------------------------------------
 drivers/net/mlx5/mlx5.c            | 150 ++++++++++++++++++++++++++++++++++++
 drivers/net/mlx5/mlx5.h            |   6 +-
 drivers/net/mlx5/mlx5_flow.c       |   4 +-
 drivers/net/mlx5/windows/mlx5_os.c | 110 +-------------------------
 5 files changed, 158 insertions(+), 266 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index e55c16c..6b36be9 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -711,7 +711,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 			return NULL;
 		}
 		eth_dev->device = dpdk_dev;
-		eth_dev->dev_ops = &mlx5_os_dev_sec_ops;
+		eth_dev->dev_ops = &mlx5_dev_sec_ops;
 		eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
 		eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
 		err = mlx5_proc_priv_init(eth_dev);
@@ -1354,7 +1354,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	/* Initialize burst functions to prevent crashes before link-up. */
 	eth_dev->rx_pkt_burst = removed_rx_burst;
 	eth_dev->tx_pkt_burst = removed_tx_burst;
-	eth_dev->dev_ops = &mlx5_os_dev_ops;
+	eth_dev->dev_ops = &mlx5_dev_ops;
 	eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
 	eth_dev->rx_queue_count = mlx5_rx_queue_count;
@@ -2520,153 +2520,3 @@ mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
 			       dev->data->mac_addrs,
 			       MLX5_MAX_MAC_ADDRESSES, priv->mac_own);
 }
-
-const struct eth_dev_ops mlx5_os_dev_ops = {
-	.dev_configure = mlx5_dev_configure,
-	.dev_start = mlx5_dev_start,
-	.dev_stop = mlx5_dev_stop,
-	.dev_set_link_down = mlx5_set_link_down,
-	.dev_set_link_up = mlx5_set_link_up,
-	.dev_close = mlx5_dev_close,
-	.promiscuous_enable = mlx5_promiscuous_enable,
-	.promiscuous_disable = mlx5_promiscuous_disable,
-	.allmulticast_enable = mlx5_allmulticast_enable,
-	.allmulticast_disable = mlx5_allmulticast_disable,
-	.link_update = mlx5_link_update,
-	.stats_get = mlx5_stats_get,
-	.stats_reset = mlx5_stats_reset,
-	.xstats_get = mlx5_xstats_get,
-	.xstats_reset = mlx5_xstats_reset,
-	.xstats_get_names = mlx5_xstats_get_names,
-	.fw_version_get = mlx5_fw_version_get,
-	.dev_infos_get = mlx5_dev_infos_get,
-	.read_clock = mlx5_txpp_read_clock,
-	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
-	.vlan_filter_set = mlx5_vlan_filter_set,
-	.rx_queue_setup = mlx5_rx_queue_setup,
-	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
-	.tx_queue_setup = mlx5_tx_queue_setup,
-	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
-	.rx_queue_release = mlx5_rx_queue_release,
-	.tx_queue_release = mlx5_tx_queue_release,
-	.rx_queue_start = mlx5_rx_queue_start,
-	.rx_queue_stop = mlx5_rx_queue_stop,
-	.tx_queue_start = mlx5_tx_queue_start,
-	.tx_queue_stop = mlx5_tx_queue_stop,
-	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
-	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
-	.mac_addr_remove = mlx5_mac_addr_remove,
-	.mac_addr_add = mlx5_mac_addr_add,
-	.mac_addr_set = mlx5_mac_addr_set,
-	.set_mc_addr_list = mlx5_set_mc_addr_list,
-	.mtu_set = mlx5_dev_set_mtu,
-	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
-	.vlan_offload_set = mlx5_vlan_offload_set,
-	.reta_update = mlx5_dev_rss_reta_update,
-	.reta_query = mlx5_dev_rss_reta_query,
-	.rss_hash_update = mlx5_rss_hash_update,
-	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
-	.filter_ctrl = mlx5_dev_filter_ctrl,
-	.rxq_info_get = mlx5_rxq_info_get,
-	.txq_info_get = mlx5_txq_info_get,
-	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
-	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
-	.rx_queue_intr_enable = mlx5_rx_intr_enable,
-	.rx_queue_intr_disable = mlx5_rx_intr_disable,
-	.is_removed = mlx5_is_removed,
-	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
-	.get_module_info = mlx5_get_module_info,
-	.get_module_eeprom = mlx5_get_module_eeprom,
-	.hairpin_cap_get = mlx5_hairpin_cap_get,
-	.mtr_ops_get = mlx5_flow_meter_ops_get,
-	.hairpin_bind = mlx5_hairpin_bind,
-	.hairpin_unbind = mlx5_hairpin_unbind,
-	.hairpin_get_peer_ports = mlx5_hairpin_get_peer_ports,
-	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
-	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
-	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
-};
-
-/* Available operations from secondary process. */
-const struct eth_dev_ops mlx5_os_dev_sec_ops = {
-	.stats_get = mlx5_stats_get,
-	.stats_reset = mlx5_stats_reset,
-	.xstats_get = mlx5_xstats_get,
-	.xstats_reset = mlx5_xstats_reset,
-	.xstats_get_names = mlx5_xstats_get_names,
-	.fw_version_get = mlx5_fw_version_get,
-	.dev_infos_get = mlx5_dev_infos_get,
-	.read_clock = mlx5_txpp_read_clock,
-	.rx_queue_start = mlx5_rx_queue_start,
-	.rx_queue_stop = mlx5_rx_queue_stop,
-	.tx_queue_start = mlx5_tx_queue_start,
-	.tx_queue_stop = mlx5_tx_queue_stop,
-	.rxq_info_get = mlx5_rxq_info_get,
-	.txq_info_get = mlx5_txq_info_get,
-	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
-	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
-	.get_module_info = mlx5_get_module_info,
-	.get_module_eeprom = mlx5_get_module_eeprom,
-};
-
-/* Available operations in flow isolated mode. */
-const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
-	.dev_configure = mlx5_dev_configure,
-	.dev_start = mlx5_dev_start,
-	.dev_stop = mlx5_dev_stop,
-	.dev_set_link_down = mlx5_set_link_down,
-	.dev_set_link_up = mlx5_set_link_up,
-	.dev_close = mlx5_dev_close,
-	.promiscuous_enable = mlx5_promiscuous_enable,
-	.promiscuous_disable = mlx5_promiscuous_disable,
-	.allmulticast_enable = mlx5_allmulticast_enable,
-	.allmulticast_disable = mlx5_allmulticast_disable,
-	.link_update = mlx5_link_update,
-	.stats_get = mlx5_stats_get,
-	.stats_reset = mlx5_stats_reset,
-	.xstats_get = mlx5_xstats_get,
-	.xstats_reset = mlx5_xstats_reset,
-	.xstats_get_names = mlx5_xstats_get_names,
-	.fw_version_get = mlx5_fw_version_get,
-	.dev_infos_get = mlx5_dev_infos_get,
-	.read_clock = mlx5_txpp_read_clock,
-	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
-	.vlan_filter_set = mlx5_vlan_filter_set,
-	.rx_queue_setup = mlx5_rx_queue_setup,
-	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
-	.tx_queue_setup = mlx5_tx_queue_setup,
-	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
-	.rx_queue_release = mlx5_rx_queue_release,
-	.tx_queue_release = mlx5_tx_queue_release,
-	.rx_queue_start = mlx5_rx_queue_start,
-	.rx_queue_stop = mlx5_rx_queue_stop,
-	.tx_queue_start = mlx5_tx_queue_start,
-	.tx_queue_stop = mlx5_tx_queue_stop,
-	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
-	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
-	.mac_addr_remove = mlx5_mac_addr_remove,
-	.mac_addr_add = mlx5_mac_addr_add,
-	.mac_addr_set = mlx5_mac_addr_set,
-	.set_mc_addr_list = mlx5_set_mc_addr_list,
-	.mtu_set = mlx5_dev_set_mtu,
-	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
-	.vlan_offload_set = mlx5_vlan_offload_set,
-	.filter_ctrl = mlx5_dev_filter_ctrl,
-	.rxq_info_get = mlx5_rxq_info_get,
-	.txq_info_get = mlx5_txq_info_get,
-	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
-	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
-	.rx_queue_intr_enable = mlx5_rx_intr_enable,
-	.rx_queue_intr_disable = mlx5_rx_intr_disable,
-	.is_removed = mlx5_is_removed,
-	.get_module_info = mlx5_get_module_info,
-	.get_module_eeprom = mlx5_get_module_eeprom,
-	.hairpin_cap_get = mlx5_hairpin_cap_get,
-	.mtr_ops_get = mlx5_flow_meter_ops_get,
-	.hairpin_bind = mlx5_hairpin_bind,
-	.hairpin_unbind = mlx5_hairpin_unbind,
-	.hairpin_get_peer_ports = mlx5_hairpin_get_peer_ports,
-	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
-	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
-	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
-};
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 19d8497..45791cc 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1512,6 +1512,156 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	return 0;
 }
 
+const struct eth_dev_ops mlx5_dev_ops = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.dev_close = mlx5_dev_close,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.read_clock = mlx5_txpp_read_clock,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.rx_queue_start = mlx5_rx_queue_start,
+	.rx_queue_stop = mlx5_rx_queue_stop,
+	.tx_queue_start = mlx5_tx_queue_start,
+	.tx_queue_stop = mlx5_tx_queue_stop,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.mtu_set = mlx5_dev_set_mtu,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.reta_update = mlx5_dev_rss_reta_update,
+	.reta_query = mlx5_dev_rss_reta_query,
+	.rss_hash_update = mlx5_rss_hash_update,
+	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+	.hairpin_bind = mlx5_hairpin_bind,
+	.hairpin_unbind = mlx5_hairpin_unbind,
+	.hairpin_get_peer_ports = mlx5_hairpin_get_peer_ports,
+	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
+	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
+	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+};
+
+/* Available operations from secondary process. */
+const struct eth_dev_ops mlx5_dev_sec_ops = {
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.read_clock = mlx5_txpp_read_clock,
+	.rx_queue_start = mlx5_rx_queue_start,
+	.rx_queue_stop = mlx5_rx_queue_stop,
+	.tx_queue_start = mlx5_tx_queue_start,
+	.tx_queue_stop = mlx5_tx_queue_stop,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+};
+
+/* Available operations in flow isolated mode. */
+const struct eth_dev_ops mlx5_dev_ops_isolate = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.dev_close = mlx5_dev_close,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.read_clock = mlx5_txpp_read_clock,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.rx_queue_start = mlx5_rx_queue_start,
+	.rx_queue_stop = mlx5_rx_queue_stop,
+	.tx_queue_start = mlx5_tx_queue_start,
+	.tx_queue_stop = mlx5_tx_queue_stop,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.mtu_set = mlx5_dev_set_mtu,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+	.hairpin_bind = mlx5_hairpin_bind,
+	.hairpin_unbind = mlx5_hairpin_unbind,
+	.hairpin_get_peer_ports = mlx5_hairpin_get_peer_ports,
+	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
+	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
+	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+};
+
 /**
  * Verify and store value for device argument.
  *
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 3f0211a..b1385b8 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -126,9 +126,9 @@ struct mlx5_local_data {
 extern struct mlx5_shared_data *mlx5_shared_data;
 
 /* Dev ops structs */
-extern const struct eth_dev_ops mlx5_os_dev_ops;
-extern const struct eth_dev_ops mlx5_os_dev_sec_ops;
-extern const struct eth_dev_ops mlx5_os_dev_ops_isolate;
+extern const struct eth_dev_ops mlx5_dev_ops;
+extern const struct eth_dev_ops mlx5_dev_sec_ops;
+extern const struct eth_dev_ops mlx5_dev_ops_isolate;
 
 struct mlx5_counter_ctrl {
 	/* Name of the counter. */
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index f009178..e436ccf 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -6418,9 +6418,9 @@ mlx5_flow_isolate(struct rte_eth_dev *dev,
 	}
 	priv->isolated = !!enable;
 	if (enable)
-		dev->dev_ops = &mlx5_os_dev_ops_isolate;
+		dev->dev_ops = &mlx5_dev_ops_isolate;
 	else
-		dev->dev_ops = &mlx5_os_dev_ops;
+		dev->dev_ops = &mlx5_dev_ops;
 
 	dev->rx_descriptor_status = mlx5_rx_descriptor_status;
 	dev->tx_descriptor_status = mlx5_tx_descriptor_status;
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 4374b05..f459f19 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -585,7 +585,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	/* Initialize burst functions to prevent crashes before link-up. */
 	eth_dev->rx_pkt_burst = removed_rx_burst;
 	eth_dev->tx_pkt_burst = removed_tx_burst;
-	eth_dev->dev_ops = &mlx5_os_dev_ops;
+	eth_dev->dev_ops = &mlx5_dev_ops;
 	eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
 	eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
 	eth_dev->rx_queue_count = mlx5_rx_queue_count;
@@ -1178,111 +1178,3 @@ mlx5_os_get_pdn(void *pd, uint32_t *pdn)
 }
 
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
-
-const struct eth_dev_ops mlx5_os_dev_ops = {
-	.dev_configure = mlx5_dev_configure,
-	.dev_start = mlx5_dev_start,
-	.dev_stop = mlx5_dev_stop,
-	.dev_close = mlx5_dev_close,
-	.mtu_set = mlx5_dev_set_mtu,
-	.link_update = mlx5_link_update,
-	.stats_get = mlx5_stats_get,
-	.stats_reset = mlx5_stats_reset,
-	.dev_infos_get = mlx5_dev_infos_get,
-	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
-	.promiscuous_enable = mlx5_promiscuous_enable,
-	.promiscuous_disable = mlx5_promiscuous_disable,
-	.allmulticast_enable = mlx5_allmulticast_enable,
-	.allmulticast_disable = mlx5_allmulticast_disable,
-	.xstats_get = mlx5_xstats_get,
-	.xstats_reset = mlx5_xstats_reset,
-	.xstats_get_names = mlx5_xstats_get_names,
-	.fw_version_get = mlx5_fw_version_get,
-	.read_clock = mlx5_read_clock,
-	.vlan_filter_set = mlx5_vlan_filter_set,
-	.rx_queue_setup = mlx5_rx_queue_setup,
-	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
-	.tx_queue_setup = mlx5_tx_queue_setup,
-	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
-	.rx_queue_release = mlx5_rx_queue_release,
-	.tx_queue_release = mlx5_tx_queue_release,
-	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
-	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
-	.mac_addr_remove = mlx5_mac_addr_remove,
-	.mac_addr_add = mlx5_mac_addr_add,
-	.mac_addr_set = mlx5_mac_addr_set,
-	.set_mc_addr_list = mlx5_set_mc_addr_list,
-	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
-	.vlan_offload_set = mlx5_vlan_offload_set,
-	.reta_update = mlx5_dev_rss_reta_update,
-	.reta_query = mlx5_dev_rss_reta_query,
-	.rss_hash_update = mlx5_rss_hash_update,
-	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
-	.filter_ctrl = mlx5_dev_filter_ctrl,
-	.rxq_info_get = mlx5_rxq_info_get,
-	.txq_info_get = mlx5_txq_info_get,
-	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
-	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
-	.rx_queue_intr_enable = mlx5_rx_intr_enable,
-	.rx_queue_intr_disable = mlx5_rx_intr_disable,
-	.is_removed = mlx5_is_removed,
-	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
-	.get_module_info = mlx5_get_module_info,
-	.get_module_eeprom = mlx5_get_module_eeprom,
-	.hairpin_cap_get = mlx5_hairpin_cap_get,
-	.mtr_ops_get = mlx5_flow_meter_ops_get,
-};
-
-/* Available operations from secondary process. */
-const struct eth_dev_ops mlx5_os_dev_sec_ops = {0};
-
-/* Available operations in flow isolated mode. */
-const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
-	.dev_configure = mlx5_dev_configure,
-	.dev_start = mlx5_dev_start,
-	.dev_stop = mlx5_dev_stop,
-	.dev_close = mlx5_dev_close,
-	.mtu_set = mlx5_dev_set_mtu,
-	.link_update = mlx5_link_update,
-	.stats_get = mlx5_stats_get,
-	.stats_reset = mlx5_stats_reset,
-	.dev_infos_get = mlx5_dev_infos_get,
-	.dev_set_link_down = mlx5_set_link_down,
-	.dev_set_link_up = mlx5_set_link_up,
-	.promiscuous_enable = mlx5_promiscuous_enable,
-	.promiscuous_disable = mlx5_promiscuous_disable,
-	.allmulticast_enable = mlx5_allmulticast_enable,
-	.allmulticast_disable = mlx5_allmulticast_disable,
-	.xstats_get = mlx5_xstats_get,
-	.xstats_reset = mlx5_xstats_reset,
-	.xstats_get_names = mlx5_xstats_get_names,
-	.fw_version_get = mlx5_fw_version_get,
-	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
-	.vlan_filter_set = mlx5_vlan_filter_set,
-	.rx_queue_setup = mlx5_rx_queue_setup,
-	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
-	.tx_queue_setup = mlx5_tx_queue_setup,
-	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
-	.rx_queue_release = mlx5_rx_queue_release,
-	.tx_queue_release = mlx5_tx_queue_release,
-	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
-	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
-	.mac_addr_remove = mlx5_mac_addr_remove,
-	.mac_addr_add = mlx5_mac_addr_add,
-	.mac_addr_set = mlx5_mac_addr_set,
-	.set_mc_addr_list = mlx5_set_mc_addr_list,
-	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
-	.vlan_offload_set = mlx5_vlan_offload_set,
-	.filter_ctrl = mlx5_dev_filter_ctrl,
-	.rxq_info_get = mlx5_rxq_info_get,
-	.txq_info_get = mlx5_txq_info_get,
-	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
-	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
-	.rx_queue_intr_enable = mlx5_rx_intr_enable,
-	.rx_queue_intr_disable = mlx5_rx_intr_disable,
-	.is_removed = mlx5_is_removed,
-	.get_module_info = mlx5_get_module_info,
-	.get_module_eeprom = mlx5_get_module_eeprom,
-	.hairpin_cap_get = mlx5_hairpin_cap_get,
-	.mtr_ops_get = mlx5_flow_meter_ops_get,
-};
-- 
2.8.4

[dpdk-dev] [PATCH v1 70/72] common/mlx5: fix Windows warnings on missing enum

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

This commit replaces included file mlx5_glue.h with file mlx5_prm.h
in file mlx5_common_mp.h. The new inclusion defines 'enum ibv_wq_state'
which is used in file mlx5_common_mp.h and causes Windows compilation
warnings if not declared in advance.

Fixes: 9d60f54569fd ("common/mlx5: remove inclusion of Verbs header files")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mp.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_common_mp.h b/drivers/common/mlx5/mlx5_common_mp.h
index 6829141..dc6563c 100644
--- a/drivers/common/mlx5/mlx5_common_mp.h
+++ b/drivers/common/mlx5/mlx5_common_mp.h
@@ -6,7 +6,7 @@
 #ifndef RTE_PMD_MLX5_COMMON_MP_H_
 #define RTE_PMD_MLX5_COMMON_MP_H_
 
-#include <mlx5_glue.h>
+#include <mlx5_prm.h>
 #include <rte_eal.h>
 #include <rte_string_fns.h>
 
-- 
2.8.4

[dpdk-dev] [PATCH v1 71/72] net/mlx5: fix Windows warnings on get_if_name

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Windows warns on missing function prototype get_if_name. To fix it -
move the prototype to shared file mlx5.h and add missing definition
IF_NAMESIZE to Windows mlx5_os.h file.

Fixes: e9c0b96e3526 ("net/mlx5: move Linux ifname function")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.h   | 4 ----
 drivers/net/mlx5/mlx5.h            | 1 +
 drivers/net/mlx5/windows/mlx5_os.h | 3 +++
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.h b/drivers/net/mlx5/linux/mlx5_os.h
index 759def2..e9cd511 100644
--- a/drivers/net/mlx5/linux/mlx5_os.h
+++ b/drivers/net/mlx5/linux/mlx5_os.h
@@ -17,8 +17,4 @@ enum {
 #define PCI_DRV_FLAGS  (RTE_PCI_DRV_INTR_LSC | \
 			RTE_PCI_DRV_INTR_RMV | \
 			RTE_PCI_DRV_PROBE_AGAIN)
-
-/* mlx5_ethdev_os.c */
-
-int mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]);
 #endif /* RTE_PMD_MLX5_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index b1385b8..6475fcc 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -976,6 +976,7 @@ int mlx5_dev_configure_rss_reta(struct rte_eth_dev *dev);
 
 /* mlx5_ethdev_os.c */
 
+int mlx5_get_ifname(const struct rte_eth_dev *dev, char (*ifname)[IF_NAMESIZE]);
 unsigned int mlx5_ifindex(const struct rte_eth_dev *dev);
 int mlx5_get_mac(struct rte_eth_dev *dev, uint8_t (*mac)[RTE_ETHER_ADDR_LEN]);
 int mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu);
diff --git a/drivers/net/mlx5/windows/mlx5_os.h b/drivers/net/mlx5/windows/mlx5_os.h
index 563cfa4..d21d4f2 100644
--- a/drivers/net/mlx5/windows/mlx5_os.h
+++ b/drivers/net/mlx5/windows/mlx5_os.h
@@ -21,4 +21,7 @@ enum {
 #ifndef ETOOMANYREFS
 #define ETOOMANYREFS 109     /* Too many references: cannot splice */
 #endif
+#ifndef IF_NAMESIZE
+#define IF_NAMESIZE 128
+#endif
 #endif /* RTE_PMD_MLX5_OS_H_ */
-- 
2.8.4

[dpdk-dev] [PATCH v1 72/72] mlx5: build pmd only with the clang compiler

[Ophir Munk]

From: Tal Shnaiderman <talshn@nvidia.com>

Currently the mlx5 PMD is supported on Windows for clang
compiler only, adding restriction in meson.build files
until mingw support will be added as well.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/meson.build | 4 ++--
 drivers/net/mlx5/meson.build    | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build
index f8515d2..3047b45 100644
--- a/drivers/common/mlx5/meson.build
+++ b/drivers/common/mlx5/meson.build
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2019 Mellanox Technologies, Ltd
 
-if not (is_linux or is_windows)
+if not (is_linux or (is_windows and is_ms_linker))
 	build = false
-	reason = 'only supported on Linux and Windows'
+	reason = 'only supported on Linux and Windows build with clang'
 	subdir_done()
 endif
 
diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index f75a169..a18d08e 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -2,9 +2,9 @@
 # Copyright 2018 6WIND S.A.
 # Copyright 2018 Mellanox Technologies, Ltd
 
-if not (is_linux or is_windows)
+if not (is_linux or (is_windows and is_ms_linker))
 	build = false
-	reason = 'only supported on Linux and Windows'
+	reason = 'only supported on Linux and Windows build with clang'
 	subdir_done()
 endif
 
-- 
2.8.4

Re: [dpdk-dev] [PATCH v1 37/72] mlx5/windows: add mlx5_os header file under net

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> File drivers/net/mlx5/windows/mlx5_os.h is added. It includes specific
> Windows definitions such as PCI driver flags link state changes
> interrupts, link removal interrupts.

The description does not match the content below.

> Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> ---
> --- /dev/null
> +++ b/drivers/net/mlx5/windows/mlx5_os.h
> @@ -0,0 +1,19 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2020 Mellanox Technologies, Ltd
> + */
> +
> +#ifndef RTE_PMD_MLX5_OS_H_
> +#define RTE_PMD_MLX5_OS_H_
> +/* sysfs equivalent under Windows */
> +enum {
> +	SYSFS_NAME_MAX = 64,
> +	SYSFS_PATH_MAX = 256
> +};
> +enum {
> +	DEV_SYSFS_NAME_MAX = SYSFS_NAME_MAX + 1,
> +	DEV_SYSFS_PATH_MAX = SYSFS_PATH_MAX + 1
> +};

There is no sysfs on Windows, so why do you need these definitions?

> +
> +#define PCI_DRV_FLAGS 0
> +
> +#endif /* RTE_PMD_MLX5_OS_H_ */

Re: [dpdk-dev] [PATCH v1 38/72] net/mlx5/windows: add pthread initializer definition

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> PTHREAD_MUTEX_INITIALIZER is a Linux macro defined in
> /usr/include/pthread.h. It is used by mlx5 PMD but it is not included
> in DPDK Windows pthread implementation. Therefore define it privately
> in windows/mlx5_os.h file.

You should not define EAL stuff privately in the PMD.
I remember Microsoft discouraged the definition of mutex initializer.
Instead we have pthread_mutex_init().

Re: [dpdk-dev] [PATCH v1 39/72] net/mlx5/windows: define epoll API to do nothing

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> This is a hack commit. Linux epoll mechanism is not implemented in
> Windows.  Therefore define rte_intr_free_epoll_fd API to do nothing.

Yes this is hack.
Better to have a stub in EAL.

Re: [dpdk-dev] [PATCH v1 40/72] net/mlx5/windows: define errno ETOOMANYREFS

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> Errno ETOOMANYREFS may not be supported under Windows. In such cases
> add its definition.

Please add it for everybody in EAL.

In which case it is not defined in Windows?

Re: [dpdk-dev] [PATCH v1 52/72] net/mlx5/windows: support is removed

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> This commit implements mlx5_is_removed() API. A new glue call
> 'init_shutdown_event' is added to support the new API.
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>

Please find a less confusing title, maybe using the words "hotplug"
and "query".

Re: [dpdk-dev] [PATCH v1 54/72] net/mlx5: exclude rte_intr_callback_register call

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> Exclude call to rte_intr_callback_register under Windows using
> definition RTE_EXEC_ENV_WINDOWS.

Rather than describing what is done, it is more interesting to know why.
Note: this is a general comment which may apply for other commits.

Re: [dpdk-dev] [PATCH v1 63/72] drivers/net: enable Windows net/mlx5 compilation

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> Add mlx5 as the first driver to be compiled under Windows (file
> drivers/net/meson.build).

It would be more interesting to have this patch earlier,
probably merged with the addition of the exports file.

I think the series should start with fixes and stubs,
then compile the driver skeleton,
and terminates with adding the features.

Re: [dpdk-dev] [PATCH v1 71/72] net/mlx5: fix Windows warnings on get_if_name

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Windows warns on missing function prototype get_if_name. To fix it -
> move the prototype to shared file mlx5.h and add missing definition
> IF_NAMESIZE to Windows mlx5_os.h file.
> 
> Fixes: e9c0b96e3526 ("net/mlx5: move Linux ifname function")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
[...]
> --- a/drivers/net/mlx5/windows/mlx5_os.h
> +++ b/drivers/net/mlx5/windows/mlx5_os.h
> @@ -21,4 +21,7 @@ enum {
>  #ifndef ETOOMANYREFS
>  #define ETOOMANYREFS 109     /* Too many references: cannot splice */
>  #endif
> +#ifndef IF_NAMESIZE
> +#define IF_NAMESIZE 128
> +#endif

Should be in EAL.

Re: [dpdk-dev] [PATCH v1 72/72] mlx5: build pmd only with the clang compiler

[Thomas Monjalon]

28/10/2020 00:23, Ophir Munk:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Currently the mlx5 PMD is supported on Windows for clang
> compiler only, adding restriction in meson.build files
> until mingw support will be added as well.

I should be squashed with the patch adding Windows compilation.

Re: [dpdk-dev] [PATCH v1 40/72] net/mlx5/windows: define errno ETOOMANYREFS

[Tal Shnaiderman]

> Subject: Re: [PATCH v1 40/72] net/mlx5/windows: define errno
> ETOOMANYREFS
> 
> External email: Use caution opening links or attachments
> 
> 
> 28/10/2020 00:23, Ophir Munk:
> > Errno ETOOMANYREFS may not be supported under Windows. In such
> cases
> > add its definition.
> 
> Please add it for everybody in EAL.
> 

Hi Dmitry,

Are you aware of the errno existence in Windows?

If not, any suggestion on an EAL place to add it? Create an errno.h file?

> In which case it is not defined in Windows?
> 
> 

Re: [dpdk-dev] [PATCH v1 58/72] net/mlx5/windws: spawn eth devices

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:23:21PM +0000, Ophir Munk wrote:
> This commit implements mlx5_dev_spawn() API which allocates an eth
> device (struct rte_eth_dev) for each PCI device. When working with
> representors virtual functions (as in Linux), one PCI device may spawn
> several eth devices: the master device for the main physical function
> (PF) and several representors for the virtual functions (VFs).  However,
> currently Windows does not work in switch dev mode, therefore, no VFs
> are created and no representors are spawned. In this case one eth device
> is created per one PCI main port.  In addition to device creation - the
> device configuration must be correctly set. The device arguments
> (devargs - set by the user) are parsed but they may be overridden by
> Windows limitations or hardware configurations. Some associated network
> parameters are stored in eth device (e.g. ifindex, MAC address, MTU) and
> some callback (e.g. burst functions) are set.
> 
> Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> ---
>  drivers/common/mlx5/windows/mlx5_win_defs.h |   6 +
>  drivers/net/mlx5/windows/mlx5_os.c          | 481 +++++++++++++++++++++++++++-
>  2 files changed, 482 insertions(+), 5 deletions(-)
> 
>  
> +static int
> +mlx5_alloc_shared_dr(struct mlx5_priv *priv)
> +{
> +	struct mlx5_dev_ctx_shared *sh = priv->sh;
> +	char s[MLX5_HLIST_NAMESIZE];

This array is not used for now, you can remove it.

> +	int err = 0;
> +
> +	if (!sh->flow_tbls)
> +		err = mlx5_alloc_table_hash_list(priv);
> +	else
> +		DRV_LOG(DEBUG, "sh->flow_tbls[%p] already created, reuse\n",
> +			(void *)sh->flow_tbls);
> +	return err;
> +}
> +
> @@ -231,17 +310,409 @@ mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn,
>   *   Device configuration parameters.
>   *
>   * @return
> - *   NULL pointer. Operation is not supported and rte_errno is set to ENOTSUP.
> + *   A valid Ethernet device object on success, NULL otherwise and rte_errno
> + *   is set. The following errors are defined:
> + *
> + *   EEXIST: device is already spawned
>   */
>  static struct rte_eth_dev *
>  mlx5_dev_spawn(struct rte_device *dpdk_dev,
>  	       struct mlx5_dev_spawn_data *spawn,
>  	       struct mlx5_dev_config *config)
>  {
> +	mlx5_malloc_mem_select(config->sys_mem_en);
> +	sh = mlx5_alloc_shared_dev_ctx(spawn, config);
> +	if (!sh)
> +		return NULL;

Should we set err to rte_errno here? Also maybe "goto error" instead of "return NULL" for consistency.

> +	config->devx = sh->devx;
> +	/* Initialize the shutdown event in mlx5_dev_spawn to
> +	 * support mlx5_is_removed for Windows.
> +	 */
> +	 * Allocate the buffer for flow creating, just once.
> +	 * The allocation must be done before any flow creating.
> +	 */
> +	mlx5_flow_alloc_intermediate(eth_dev);
Where is this function defined? Should it be added with the flow support patch?

> +	/* Query availability of metadata reg_c's. */
> +	err = mlx5_flow_discover_mreg_c(eth_dev);
> +	if (err < 0) {
> +		err = -err;
> +		goto error;
> +	}
> +	return eth_dev;
> +error:
> +	if (priv) {

Should we also call mlx5_os_free_shared_dr()? If the call to mlx5_alloc_shared_dr() above succeeds.

> +		if (priv->mreg_cp_tbl)
> +			mlx5_hlist_destroy(priv->mreg_cp_tbl, NULL, NULL);
> +		if (priv->qrss_id_pool)
> +			mlx5_flow_id_pool_release(priv->qrss_id_pool);
> +		if (own_domain_id)
> +			claim_zero(rte_eth_switch_domain_free(priv->domain_id));
> +		mlx5_free(priv);
> +		if (eth_dev != NULL)
> +			eth_dev->data->dev_private = NULL;
> +	}
> +	if (eth_dev != NULL) {
> +		/* mac_addrs must not be freed alone because part of
> +		 * dev_private
> +		 **/
> +		eth_dev->data->mac_addrs = NULL;

Looks like rte_eth_dev_release_port() frees mac_addrs as well. Why is it set to NULL here?
Should this line be removed or moved after the _release_port() call?

> +		rte_eth_dev_release_port(eth_dev);
> +	}
> +	if (sh)
> +		mlx5_free_shared_dev_ctx(sh);
> +	MLX5_ASSERT(err > 0);
> +	rte_errno = err;
>  	return NULL;
>  }
>  
> -- 
> 2.8.4

Re: [dpdk-dev] [PATCH v1 57/72] net/mlx5/windows: initial probing implementation

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:23:20PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> This commit implements mlx5_os_pci_probe API under Windows. It does all
> required initializations then it gets the PCI device list using glue API
> get_device_list().  Next, all non MLX5 matched devices are filtered out.
> The supported NIC types are: CONNECTX4VF, CONNECTX4LXVF, CONNECTX5VF,
> CONNECTX5EXVF, CONNECTX5BFVF, CONNECTX6VF, MELLANOX_CONNECTX6DXVF.  Each
> device in the list is assigned with default configuration parameters,
> most of them are 0. The default dv_flow_en parameter value is 1 (which
> means Windows match and action flows are based on DV code). Next for
> each PCI device call mlx5_dev_spawn() to create an eth device (struct
> rte_ethdev). The implementation of device spawn is in the follow up
> commit.  Finally, the device list is free.
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> ---
>  drivers/net/mlx5/windows/mlx5_os.c | 278 +++++++++++++++++++++++++++++++++++++
>  1 file changed, 278 insertions(+)
> 
> +static int
> +mlx5_init_shared_data(void)
> +{
> +	const struct rte_memzone *mz;
> +	int ret = 0;
> +
> +	rte_spinlock_lock(&mlx5_shared_data_lock);
> +	if (mlx5_shared_data == NULL) {

Maybe: Invert the check to reduce indentation. You can also move it before taking the lock: if (mlx5_shared_data) { return 0; }

> +		if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> +			/* Allocate shared memory. */
> +			mz = rte_memzone_reserve(MZ_MLX5_PMD_SHARED_DATA,
> +						 sizeof(*mlx5_shared_data),
> +						 SOCKET_ID_ANY, 0);
> +			if (mz == NULL) {
> +				DRV_LOG(ERR,
> +					"Cannot allocate mlx5 shared data");
> +				ret = -rte_errno;
> +				goto error;
> +			}
> +			mlx5_shared_data = mz->addr;
> +			memset(mlx5_shared_data, 0, sizeof(*mlx5_shared_data));
> +			rte_spinlock_init(&mlx5_shared_data->lock);
> +		} else {
> +			/* Lookup allocated shared memory. */
> +			mz = rte_memzone_lookup(MZ_MLX5_PMD_SHARED_DATA);
> +			if (mz == NULL) {
> +				DRV_LOG(ERR,
> +					"Cannot attach mlx5 shared data");
> +				ret = -rte_errno;
> +				goto error;
> +			}
> +			mlx5_shared_data = mz->addr;
> +			memset(&mlx5_local_data, 0, sizeof(mlx5_local_data));
> +		}
> +	}
> +error:
> +	rte_spinlock_unlock(&mlx5_shared_data_lock);
> +	return ret;
> +}

> +	dev_config.dv_flow_en = 1;
> +	dev_config.decap_en = 0;
> +	dev_config.log_hp_size = MLX5_ARG_UNSET;
> +	list[ns].eth_dev = mlx5_dev_spawn(&pci_dev->device,
> +					  &list[ns],
> +					  &dev_config);
> +	if (!list[ns].eth_dev)

mlx5_dev_spawn() updates rte_errno, we should ret to rte_errno before exiting here.

> +		goto exit;
> +	restore = list[ns].eth_dev->data->dev_flags;
> +	rte_eth_copy_pci_info(list[ns].eth_dev, pci_dev);
> +	/* Restore non-PCI flags cleared by the above call. */
> +	list[ns].eth_dev->data->dev_flags |= restore;
> +	rte_eth_dev_probing_finish(list[ns].eth_dev);
> +	ret = 0;
> 2.8.4

Re: [dpdk-dev] [PATCH v1 59/72] net/mlx5/windows: support VF PCI address

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:23:22PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Support VF BDF scanning by checking both the BDF and raw BDF provided by
> DevX. In Linux a PCI address is formatted as: domain, bus, device,
> function (DBDF).  This is right for both a PF and a VF. In Windows a PF
> also has a DBDF format, but the domain is always 0, while a VF has a
> special "domain" called "Virtual PCI Bus, Serial" (for example: "Virtual
> PCI Bus Slot 2 Serial 2") or segment.  The full VF format under Windows
> is called raw DBF.  Windows special domain must be considered and DevX
> must be called to support it.
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
>  drivers/net/mlx5/windows/mlx5_os.c | 67 ++++++++++++++++++++++++++++++++++++--
>  1 file changed, 64 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
> index f9b469f..4374b05 100644
> --- a/drivers/net/mlx5/windows/mlx5_os.c
> +++ b/drivers/net/mlx5/windows/mlx5_os.c
> @@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
>  }
> + * @return
> + *   1 on Device match, 0 on mismatch, rte_errno code on failure.
> + */
> +static int
> +mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
> +				struct rte_pci_addr *addr)
> +{
> +	err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
> +	if (err) {
> +		DRV_LOG(ERR, "query_device failed");
> +		rte_errno = err;
> +		return 0;

Return rte_errno.

> +	}
> +	if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
> +		return 1;
> +	return 0;
> +}
> +
> -- 
> 2.8.4

Re: [dpdk-dev] [PATCH v1 48/72] net/mlx5/windows: support link update

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:23:11PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Add support for mlx5_link_update() to get link speed and link state.
> Other parameters are currently hard-coded.
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
>  drivers/net/mlx5/windows/mlx5_ethdev_os.c | 39 +++++++++++++++++++++++++++++++
>  1 file changed, 39 insertions(+)
> 
> diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> index 4925fd8..0c45101 100644
> --- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> +++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> @@ -171,6 +171,45 @@ mlx5_os_read_dev_counters(struct rte_eth_dev *dev, uint64_t *stats)
>  }
>  
>  /**
> + * @return
> + *   0 if link status was not updated, positive if it was, a negative errno
> + *   value otherwise and rte_errno is set.
> + */
> +int
> +mlx5_link_update(struct rte_eth_dev *dev, int wait_to_complete)
> +{
> +	RTE_SET_USED(wait_to_complete);
> +	struct mlx5_priv *priv;
> +	mlx5_context_st *context_obj;
> +	struct rte_eth_link dev_link;
> +	int ret;
> +
> +	ret = 0;
> +	if (!dev) {
> +		rte_errno = EINVAL;
> +		return rte_errno;

Should this be "return -rte_errno", as per the function descriptiona above?

> +	}
> +	priv = dev->data->dev_private;
> +	context_obj = (mlx5_context_st *)priv->sh->ctx;
> 2.8.4

Re: [dpdk-dev] [PATCH v1 42/72] common/mlx5/windows: add DevX UAR getters

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:23:05PM +0000, Ophir Munk wrote:
> The following getters are added: mlx5_os_get_devx_uar_mmap_offset,
> mlx5_os_get_devx_uar_base_addr, mlx5_os_get_devx_uar_reg_addr,
> mlx5_os_get_devx_uar_page_id.  This commit is the Windows equivalent of
> the Linux implementation in (1).
> 
> (1)
> commit 8638e19a10aa ("net/mlx5: remove more DV dependencies")
> 
> Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> ---
>  drivers/common/mlx5/windows/mlx5_common_os.h | 74 ++++++++++++++++++++++++++++
>  1 file changed, 74 insertions(+)
> 
> diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
> + * Get mmap offset. Given a pointer to an DevX UAR object of type
> + * 'struct mlx5dv_devx_uar *' - return its mmap offset.
> + *
> + * @param[in] uar
> + *    Pointer to UAR object.
> + *
> + * @return
> + *    The mmap offset if uar is valid, 0 otherwise.
> + */
> +static inline off_t
> +mlx5_os_get_devx_uar_mmap_offset(void *uar)
> +{
> +	if (!uar)
> +		return 0;
> +	return 0;

Should we return uar->mmap_off here?

> +}
> +
> +/**
> + * Get base addr pointer. Given a pointer to an UAR object of type
> + * 'struct mlx5dv_devx_uar *' - return its base address.
> + *

Re: [dpdk-dev] [PATCH v1 34/72] net/mlx5/windows: implement mlx5 mac addr add

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:22:57PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Get the list of MAC addresses and verify if the input mac parameter
> already exists. If not - return -ENOTSUP (as Windows does not support
> adding new MAC addresses). If the MAC address exists (EEXIST) return 0
> (the equivalent of Linux implementation of this API).
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> ---
>  drivers/net/mlx5/windows/mlx5_os.c | 36 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 36 insertions(+)
> 
> diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
> index b0cc9f3..6e27474 100644
> --- a/drivers/net/mlx5/windows/mlx5_os.c
> +++ b/drivers/net/mlx5/windows/mlx5_os.c
> @@ -183,6 +183,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
>  }
>  
> + *
> + * @return
> + *   0 on success, a negative errno value otherwise
> + */
> +int
> +mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
> +		     uint32_t index)
> +{
> +	(void)index;
> +	struct rte_ether_addr lmac;
> +
> +	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
> +		DRV_LOG(ERR,
> +			"port %u cannot get MAC address, is mlx5_en"
> +			" loaded? (errno: %s)",
> +			dev->data->port_id, strerror(rte_errno));
> +		return rte_errno;

Should it be "return -rte_errno" here?

> +	}
> +	if (memcmp(&lmac, mac, sizeof(struct rte_ether_addr))) {
> +		DRV_LOG(ERR,
> +			"adding new mac address to device is unsupported");
> +		return -ENOTSUP;
> +	}
> +	return 0;
> +}
> +
> +/**
>   * Modify a VF MAC address
>   * Currently it has no support under Windows.
>   *
> -- 
> 2.8.4

Re: [dpdk-dev] [PATCH v1 27/72] common/mlx5/windows: add OS alloc/dealloc pd

[Narcisa Ana Maria Vasile]

On Tue, Oct 27, 2020 at 11:22:50PM +0000, Ophir Munk wrote:
> From: Tal Shnaiderman <talshn@nvidia.com>
> 
> Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd(). They
> are equivalent to the Linux implementation.
> 
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> Acked-by: Matan Azrad <matan@nvidia.com>
> ---
>  drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
>  drivers/common/mlx5/windows/mlx5_common_os.c    | 47 +++++++++++++++++++++++++
>  drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
>  drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
>  4 files changed, 58 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
> + * @return
> + *    The mlx5_pd if pd is valid, NULL and errno otherwise.
> + */
> +void *
> +mlx5_os_alloc_pd(void *ctx)
> +{
> +	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
> +		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
> +	if (!ppd)
> +		return NULL;
> +
> +	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
> +	if (!obj)

Free ppd here, to avoid memory leak.

> +		return NULL;
> +
> +	ppd->obj = obj;
> +	ppd->pdn = obj->id;
> +	ppd->devx_ctx = ctx;
> +	return ppd;
> +}
> +

Re: [dpdk-dev] [PATCH v1 59/72] net/mlx5/windows: support VF PCI address

[Tal Shnaiderman]

> Subject: Re: [dpdk-dev] [PATCH v1 59/72] net/mlx5/windows: support VF PCI
> address
> 
> On Tue, Oct 27, 2020 at 11:23:22PM +0000, Ophir Munk wrote:
> > From: Tal Shnaiderman <talshn@nvidia.com>
> >
> > Support VF BDF scanning by checking both the BDF and raw BDF provided
> > by DevX. In Linux a PCI address is formatted as: domain, bus, device,
> > function (DBDF).  This is right for both a PF and a VF. In Windows a
> > PF also has a DBDF format, but the domain is always 0, while a VF has
> > a special "domain" called "Virtual PCI Bus, Serial" (for example:
> > "Virtual PCI Bus Slot 2 Serial 2") or segment.  The full VF format
> > under Windows is called raw DBF.  Windows special domain must be
> > considered and DevX must be called to support it.
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > ---
> >  drivers/net/mlx5/windows/mlx5_os.c | 67
> > ++++++++++++++++++++++++++++++++++++--
> >  1 file changed, 64 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/net/mlx5/windows/mlx5_os.c
> > b/drivers/net/mlx5/windows/mlx5_os.c
> > index f9b469f..4374b05 100644
> > --- a/drivers/net/mlx5/windows/mlx5_os.c
> > +++ b/drivers/net/mlx5/windows/mlx5_os.c
> > @@ -901,6 +901,68 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int
> > enable)  }
> > + * @return
> > + *   1 on Device match, 0 on mismatch, rte_errno code on failure.
> > + */
> > +static int
> > +mlx5_match_devx_devices_to_addr(struct devx_device_bdf *devx_bdf,
> > +				struct rte_pci_addr *addr)
> > +{
> > +	err = mlx5_glue->query_device(devx_bdf, &mlx5_dev);
> > +	if (err) {
> > +		DRV_LOG(ERR, "query_device failed");
> > +		rte_errno = err;
> > +		return 0;
> 
> Return rte_errno.

Actually updating rte_errno is enough in the function, but the caller need to check it, I'll fix in v2.

> 
> > +	}
> > +	if (mlx5_match_devx_bdf_to_addr(&mlx5_dev.raw_bdf, addr))
> > +		return 1;
> > +	return 0;
> > +}
> > +
> > --
> > 2.8.4

Re: [dpdk-dev] [PATCH v1 48/72] net/mlx5/windows: support link update

[Tal Shnaiderman]

> Subject: Re: [dpdk-dev] [PATCH v1 48/72] net/mlx5/windows: support link
> update
> 
> On Tue, Oct 27, 2020 at 11:23:11PM +0000, Ophir Munk wrote:
> > From: Tal Shnaiderman <talshn@nvidia.com>
> >
> > Add support for mlx5_link_update() to get link speed and link state.
> > Other parameters are currently hard-coded.
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > ---
> >  drivers/net/mlx5/windows/mlx5_ethdev_os.c | 39
> > +++++++++++++++++++++++++++++++
> >  1 file changed, 39 insertions(+)
> >
> > diff --git a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> > b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> > index 4925fd8..0c45101 100644
> > --- a/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> > +++ b/drivers/net/mlx5/windows/mlx5_ethdev_os.c
> > @@ -171,6 +171,45 @@ mlx5_os_read_dev_counters(struct rte_eth_dev
> > *dev, uint64_t *stats)  }
> >
> >  /**
> > + * @return
> > + *   0 if link status was not updated, positive if it was, a negative errno
> > + *   value otherwise and rte_errno is set.
> > + */
> > +int
> > +mlx5_link_update(struct rte_eth_dev *dev, int wait_to_complete) {
> > +	RTE_SET_USED(wait_to_complete);
> > +	struct mlx5_priv *priv;
> > +	mlx5_context_st *context_obj;
> > +	struct rte_eth_link dev_link;
> > +	int ret;
> > +
> > +	ret = 0;
> > +	if (!dev) {
> > +		rte_errno = EINVAL;
> > +		return rte_errno;
> 
> Should this be "return -rte_errno", as per the function description above?

Right, will fix in v2.

> 
> > +	}
> > +	priv = dev->data->dev_private;
> > +	context_obj = (mlx5_context_st *)priv->sh->ctx;
> > 2.8.4

Re: [dpdk-dev] [PATCH v1 42/72] common/mlx5/windows: add DevX UAR getters

[Tal Shnaiderman]

> Subject: Re: [dpdk-dev] [PATCH v1 42/72] common/mlx5/windows: add
> DevX UAR getters
> 
> On Tue, Oct 27, 2020 at 11:23:05PM +0000, Ophir Munk wrote:
> > The following getters are added: mlx5_os_get_devx_uar_mmap_offset,
> > mlx5_os_get_devx_uar_base_addr, mlx5_os_get_devx_uar_reg_addr,
> > mlx5_os_get_devx_uar_page_id.  This commit is the Windows equivalent
> > of the Linux implementation in (1).
> >
> > (1)
> > commit 8638e19a10aa ("net/mlx5: remove more DV dependencies")
> >
> > Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> > ---
> >  drivers/common/mlx5/windows/mlx5_common_os.h | 74
> > ++++++++++++++++++++++++++++
> >  1 file changed, 74 insertions(+)
> >
> > diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h
> > b/drivers/common/mlx5/windows/mlx5_common_os.h
> > + * Get mmap offset. Given a pointer to an DevX UAR object of type
> > + * 'struct mlx5dv_devx_uar *' - return its mmap offset.
> > + *
> > + * @param[in] uar
> > + *    Pointer to UAR object.
> > + *
> > + * @return
> > + *    The mmap offset if uar is valid, 0 otherwise.
> > + */
> > +static inline off_t
> > +mlx5_os_get_devx_uar_mmap_offset(void *uar) {
> > +	if (!uar)
> > +		return 0;
> > +	return 0;
> 
> Should we return uar->mmap_off here?

In Windows, this function will always return 0 as mmap offset is unneeded, I'll update the function docu and change it to reflect it, thanks.

> 
> > +}
> > +
> > +/**
> > + * Get base addr pointer. Given a pointer to an UAR object of type
> > + * 'struct mlx5dv_devx_uar *' - return its base address.
> > + *

Re: [dpdk-dev] [PATCH v1 34/72] net/mlx5/windows: implement mlx5 mac addr add

[Tal Shnaiderman]

> Subject: Re: [dpdk-dev] [PATCH v1 34/72] net/mlx5/windows: implement
> mlx5 mac addr add
> 
> On Tue, Oct 27, 2020 at 11:22:57PM +0000, Ophir Munk wrote:
> > From: Tal Shnaiderman <talshn@nvidia.com>
> >
> > Get the list of MAC addresses and verify if the input mac parameter
> > already exists. If not - return -ENOTSUP (as Windows does not support
> > adding new MAC addresses). If the MAC address exists (EEXIST) return 0
> > (the equivalent of Linux implementation of this API).
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > ---
> >  drivers/net/mlx5/windows/mlx5_os.c | 36
> > ++++++++++++++++++++++++++++++++++++
> >  1 file changed, 36 insertions(+)
> >
> > diff --git a/drivers/net/mlx5/windows/mlx5_os.c
> > b/drivers/net/mlx5/windows/mlx5_os.c
> > index b0cc9f3..6e27474 100644
> > --- a/drivers/net/mlx5/windows/mlx5_os.c
> > +++ b/drivers/net/mlx5/windows/mlx5_os.c
> > @@ -183,6 +183,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev
> *dev,
> > uint32_t index)  }
> >
> > + *
> > + * @return
> > + *   0 on success, a negative errno value otherwise
> > + */
> > +int
> > +mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr
> *mac,
> > +		     uint32_t index)
> > +{
> > +	(void)index;
> > +	struct rte_ether_addr lmac;
> > +
> > +	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
> > +		DRV_LOG(ERR,
> > +			"port %u cannot get MAC address, is mlx5_en"
> > +			" loaded? (errno: %s)",
> > +			dev->data->port_id, strerror(rte_errno));
> > +		return rte_errno;
> 
> Should it be "return -rte_errno" here?

Actually, mlx5_get_mac return a negative value in rte_errno in case of failure.

> 
> > +	}
> > +	if (memcmp(&lmac, mac, sizeof(struct rte_ether_addr))) {
> > +		DRV_LOG(ERR,
> > +			"adding new mac address to device is unsupported");
> > +		return -ENOTSUP;
> > +	}
> > +	return 0;
> > +}
> > +
> > +/**
> >   * Modify a VF MAC address
> >   * Currently it has no support under Windows.
> >   *
> > --
> > 2.8.4

Re: [dpdk-dev] [PATCH v1 27/72] common/mlx5/windows: add OS alloc/dealloc pd

[Tal Shnaiderman]

> Subject: Re: [dpdk-dev] [PATCH v1 27/72] common/mlx5/windows: add OS
> alloc/dealloc pd
> 
> On Tue, Oct 27, 2020 at 11:22:50PM +0000, Ophir Munk wrote:
> > From: Tal Shnaiderman <talshn@nvidia.com>
> >
> > Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd().
> > They are equivalent to the Linux implementation.
> >
> > Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> > Acked-by: Matan Azrad <matan@nvidia.com>
> > ---
> >  drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
> >  drivers/common/mlx5/windows/mlx5_common_os.c    | 47
> +++++++++++++++++++++++++
> >  drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
> >  drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
> >  4 files changed, 58 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def
> > b/drivers/common/mlx5/rte_common_mlx5_exports.def
> > + * @return
> > + *    The mlx5_pd if pd is valid, NULL and errno otherwise.
> > + */
> > +void *
> > +mlx5_os_alloc_pd(void *ctx)
> > +{
> > +	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
> > +		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
> > +	if (!ppd)
> > +		return NULL;
> > +
> > +	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
> > +	if (!obj)
> 
> Free ppd here, to avoid memory leak.

Right, will do in v2.

> 
> > +		return NULL;
> > +
> > +	ppd->obj = obj;
> > +	ppd->pdn = obj->id;
> > +	ppd->devx_ctx = ctx;
> > +	return ppd;
> > +}
> > +

[dpdk-dev] [PATCH v2 00/33] mlx5 Windows support - part #5

[Tal Shnaiderman]

This patch series is part of preparing mlx5 PMD to compile and run under Windows OS. Part #5

v1: initial version
v2: Fixes after CR and rebase, also patchset was split and part of the commits sent in v1 will be sent later on.

Ophir Munk (21):
  net/mlx5/linux: extend device attributes getter
  net/mlx5: remove Linux files from Windows compilation
  net/mlx5: fix freeing packet pacing
  net/mlx5: replace Linux sleep with rte sleep
  net/mlx5: define mprq functions as static inline
  net/mlx5: do not define static_assert in Windows
  net/mlx5: move static_assert calls to global scope
  net/mlx5: wrap glue alloc/dealloc PD with OS calls
  net/mlx5: fix adding destroy flow action wrapper
  common/mlx5: add definition HAVE_INFINIBAND_VERBS_H
  common/mlx5/linux: handle memory allocations with alignment
  common/mlx5/windows: handle memory allocations with alignment
  mlx5/windows: add initialization routine for external lib
  mlx5/windows: generate file mlx5_autoconf.h
  common/mlx5/windows: add getter functions
  net/mlx5: update MR prototypes for DevX
  common/mlx5/windows: add OS reg/dereg MR
  drivers/common: enable Windows common mlx5 compilation
  net/mlx5/windows: implement device attribute getter
  net/mlx5/windows: add mlx5_os.c stubs
  net/mlx5: refactor eth dev ops for Windows

Tal Shnaiderman (12):
  net/mlx5: fix folding constant array error
  net/mlx5: wrap glue reg/dereg UMEM with OS calls
  common/mlx5/linux: wrap event channel APIs with OS calls
  common/mlx5: add Windows exports file
  common/mlx5: extend DevX query hca attributes command
  common/mlx5: add DevX alloc PD command
  common/mlx5/windows: add glue functions APIs
  mlx5/windows: add mlx5 meson file
  common/mlx5/windows: extend PRM match_param_bits struct
  common/mlx5/windows: add OS alloc/dealloc pd
  common/mlx5/windows: add OS umem reg/dereg API
  net/mlx5/windows: implement mlx5 mac addr add

 drivers/common/meson.build                      |   8 +-
 drivers/common/mlx5/linux/meson.build           |   6 +-
 drivers/common/mlx5/linux/mlx5_common_os.h      |  83 +++++
 drivers/common/mlx5/meson.build                 |   4 +-
 drivers/common/mlx5/mlx5_common.c               |   4 +-
 drivers/common/mlx5/mlx5_common_mr.h            |   1 +
 drivers/common/mlx5/mlx5_devx_cmds.c            |  46 +++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  12 +-
 drivers/common/mlx5/mlx5_malloc.c               |  14 +-
 drivers/common/mlx5/mlx5_prm.h                  |  87 +++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  69 ++++
 drivers/common/mlx5/version.map                 |   1 +
 drivers/common/mlx5/windows/meson.build         |  46 +++
 drivers/common/mlx5/windows/mlx5_common_os.c    | 210 ++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    | 154 +++++++++
 drivers/common/mlx5/windows/mlx5_glue.c         | 307 ++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 ++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  40 +++
 drivers/net/mlx5/linux/mlx5_os.c                |   9 +-
 drivers/net/mlx5/linux/mlx5_verbs.c             |   2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h             |   8 +-
 drivers/net/mlx5/meson.build                    |  16 +-
 drivers/net/mlx5/mlx5.c                         |  18 +-
 drivers/net/mlx5/mlx5.h                         |  11 +
 drivers/net/mlx5/mlx5_defs.h                    |   7 +-
 drivers/net/mlx5/mlx5_devx.c                    |  24 +-
 drivers/net/mlx5/mlx5_flow.c                    |  17 +-
 drivers/net/mlx5/mlx5_flow_dv.c                 |   6 +-
 drivers/net/mlx5/mlx5_rxq.c                     |  71 -----
 drivers/net/mlx5/mlx5_rxtx.c                    |  98 +++---
 drivers/net/mlx5/mlx5_rxtx.h                    |  73 ++++-
 drivers/net/mlx5/mlx5_trigger.c                 |   3 +-
 drivers/net/mlx5/mlx5_txpp.c                    |  34 +-
 drivers/net/mlx5/windows/meson.build            |   8 +
 drivers/net/mlx5/windows/mlx5_os.c              | 406 ++++++++++++++++++++++++
 drivers/vdpa/mlx5/mlx5_vdpa_event.c             |   7 +-
 37 files changed, 1783 insertions(+), 210 deletions(-)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def
 create mode 100644 drivers/common/mlx5/windows/meson.build
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 01/33] net/mlx5: fix folding constant array error

[Tal Shnaiderman]

Before this commit the PMD used:
   const int elt_n = 8
   const int *stack[elt_n];

In Windows clang compiler complains:
net/mlx5/mlx5_flow.c:215:19: error: variable length array folded
to constant array as an extension [-Werror,-Wgnu-folding-constant]

Fix it by using a constant macro definition instead of a variable:
   #define MLX5_RSS_EXP_ELT_N 8
   const int *stack[MLX5_RSS_EXP_ELT_N];

Fixes: c7870bfe09dc ("ethdev: move RSS expansion code to mlx5 driver")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 82e24d7067..bf86aaaa39 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -212,6 +212,8 @@ mlx5_flow_expand_rss_item_complete(const struct rte_flow_item *item)
 	return ret;
 }
 
+#define MLX5_RSS_EXP_ELT_N 8
+
 /**
  * Expand RSS flows into several possible flows according to the RSS hash
  * fields requested and the driver capabilities.
@@ -242,13 +244,12 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		     const struct mlx5_flow_expand_node graph[],
 		     int graph_root_index)
 {
-	const int elt_n = 8;
 	const struct rte_flow_item *item;
 	const struct mlx5_flow_expand_node *node = &graph[graph_root_index];
 	const int *next_node;
-	const int *stack[elt_n];
+	const int *stack[MLX5_RSS_EXP_ELT_N];
 	int stack_pos = 0;
-	struct rte_flow_item flow_items[elt_n];
+	struct rte_flow_item flow_items[MLX5_RSS_EXP_ELT_N];
 	unsigned int i;
 	size_t lsize;
 	size_t user_pattern_size = 0;
@@ -261,10 +262,10 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 
 	memset(&missed_item, 0, sizeof(missed_item));
 	lsize = offsetof(struct mlx5_flow_expand_rss, entry) +
-		elt_n * sizeof(buf->entry[0]);
+		MLX5_RSS_EXP_ELT_N * sizeof(buf->entry[0]);
 	if (lsize <= size) {
 		buf->entry[0].priority = 0;
-		buf->entry[0].pattern = (void *)&buf->entry[elt_n];
+		buf->entry[0].pattern = (void *)&buf->entry[MLX5_RSS_EXP_ELT_N];
 		buf->entries = 0;
 		addr = buf->entry[0].pattern;
 	}
@@ -367,7 +368,7 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		/* Go deeper. */
 		if (node->next) {
 			next_node = node->next;
-			if (stack_pos++ == elt_n) {
+			if (stack_pos++ == MLX5_RSS_EXP_ELT_N) {
 				rte_errno = E2BIG;
 				return -rte_errno;
 			}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 02/33] net/mlx5/linux: extend device attributes getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 5 +++++
 drivers/net/mlx5/mlx5.h          | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 917d6be7b8..a6c1eba640 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -116,7 +116,12 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	device_attr->max_qp_wr = attr_ex.orig_attr.max_qp_wr;
 	device_attr->max_sge = attr_ex.orig_attr.max_sge;
 	device_attr->max_cq = attr_ex.orig_attr.max_cq;
+	device_attr->max_cqe = attr_ex.orig_attr.max_cqe;
+	device_attr->max_mr = attr_ex.orig_attr.max_mr;
+	device_attr->max_pd = attr_ex.orig_attr.max_pd;
 	device_attr->max_qp = attr_ex.orig_attr.max_qp;
+	device_attr->max_srq = attr_ex.orig_attr.max_srq;
+	device_attr->max_srq_wr = attr_ex.orig_attr.max_srq_wr;
 	device_attr->raw_packet_caps = attr_ex.raw_packet_caps;
 	device_attr->max_rwq_indirection_table_size =
 		attr_ex.rss_caps.max_rwq_indirection_table_size;
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 121d726405..2186677810 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -83,6 +83,11 @@ struct mlx5_dev_attr {
 	int		max_sge;
 	int		max_cq;
 	int		max_qp;
+	int		max_cqe;
+	uint32_t	max_pd;
+	uint32_t	max_mr;
+	uint32_t	max_srq;
+	uint32_t	max_srq_wr;
 	uint32_t	raw_packet_caps;
 	uint32_t	max_rwq_indirection_table_size;
 	uint32_t	max_tso;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 03/33] net/mlx5: remove Linux files from Windows compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit removes Linux files flow_verbs.c and mlx5_rxtx_vec.c
from Windows compilation.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/meson.build | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index e7495a764a..f2fafbdd05 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -15,7 +15,6 @@ sources = files(
 	'mlx5_flow.c',
 	'mlx5_flow_meter.c',
 	'mlx5_flow_dv.c',
-	'mlx5_flow_verbs.c',
         'mlx5_flow_age.c',
 	'mlx5_mac.c',
 	'mlx5_mr.c',
@@ -31,11 +30,18 @@ sources = files(
 	'mlx5_utils.c',
 	'mlx5_devx.c',
 )
-if (dpdk_conf.has('RTE_ARCH_X86_64')
-	or dpdk_conf.has('RTE_ARCH_ARM64')
-	or dpdk_conf.has('RTE_ARCH_PPC_64'))
-	sources += files('mlx5_rxtx_vec.c')
+
+if is_linux
+	sources += files(
+		'mlx5_flow_verbs.c',
+	)
+	if (dpdk_conf.has('RTE_ARCH_X86_64')
+		or dpdk_conf.has('RTE_ARCH_ARM64')
+		or dpdk_conf.has('RTE_ARCH_PPC_64'))
+		sources += files('mlx5_rxtx_vec.c')
+	endif
 endif
+
 cflags_options = [
 	'-std=c11',
 	'-Wno-strict-prototypes',
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 04/33] net/mlx5: fix freeing packet pacing

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Packet pacing is allocated under condition #ifdef HAVE_MLX5DV_PP_ALLOC.
In a similar way - free packet pacing index under the same condition.
This update is required to successfully compile under operating systems
which do not support packet pacing.

Fixes: aef1e20ebeb2 ("net/mlx5: allocate packet pacing context")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_txpp.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 2438bf1f1d..21675ab17a 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -57,11 +57,16 @@ mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 static void
 mlx5_txpp_free_pp_index(struct mlx5_dev_ctx_shared *sh)
 {
+#ifdef HAVE_MLX5DV_PP_ALLOC
 	if (sh->txpp.pp) {
 		mlx5_glue->dv_free_pp(sh->txpp.pp);
 		sh->txpp.pp = NULL;
 		sh->txpp.pp_id = 0;
 	}
+#else
+	RTE_SET_USED(sh);
+	DRV_LOG(ERR, "Freeing pacing index is not supported.");
+#endif
 }
 
 /* Allocate Packet Pacing index from kernel via mlx5dv call. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 05/33] net/mlx5: replace Linux sleep with rte sleep

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Replace Linux API usleep() and nanosleep() with rte_delay_us_sleep().
The replacement occurs in shared files compiled under different
operating systems.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_trigger.c | 3 ++-
 drivers/net/mlx5/mlx5_txpp.c    | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 52a8a252d4..8e9df6abcf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -21,6 +21,7 @@
 #include <rte_spinlock.h>
 #include <rte_string_fns.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
@@ -1330,7 +1331,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_flex_parser_ecpri_release(dev);
 	if (priv->rxqs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
@@ -1338,7 +1339,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->txqs_n); ++i)
 			mlx5_txq_release(dev, i);
 		priv->txqs_n = 0;
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index bd029154f8..646f29b923 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -9,6 +9,7 @@
 #include <rte_ethdev_driver.h>
 #include <rte_interrupts.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_malloc.h>
 
@@ -1171,7 +1172,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	rte_wmb();
 	/* Disable datapath on secondary process. */
 	mlx5_mp_os_req_stop_rxtx(dev);
-	usleep(1000 * priv->rxqs_n);
+	rte_delay_us_sleep(1000 * priv->rxqs_n);
 	DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id);
 	mlx5_flow_stop_default(dev);
 	/* Control flows for default traffic can be removed firstly. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 21675ab17a..a1ec294942 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -769,15 +769,11 @@ mlx5_txpp_init_timestamp(struct mlx5_dev_ctx_shared *sh)
 	sh->txpp.ts_p = 0;
 	sh->txpp.ts_n = 0;
 	for (wait = 0; wait < MLX5_TXPP_WAIT_INIT_TS; wait++) {
-		struct timespec onems;
-
 		mlx5_txpp_update_timestamp(sh);
 		if (wq->sq_ci)
 			return;
 		/* Wait one millisecond and try again. */
-		onems.tv_sec = 0;
-		onems.tv_nsec = NS_PER_S / MS_PER_S;
-		nanosleep(&onems, 0);
+		rte_delay_us_sleep(US_PER_S / MS_PER_S);
 	}
 	DRV_LOG(ERR, "Unable to initialize timestamp.");
 	sh->txpp.sync_lost = 1;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 06/33] net/mlx5: define mprq functions as static inline

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Functions mlx5_check_mprq_support(), mlx5_rxq_mprq_enabled(),
mlx5_mprq_enabled() are moved from source file mlx5_rxq.c to header file
mlx5_rxtx.h and their type is updated to 'static __rte_always_inline'.
Previously the functions were declared as 'inline' in the source file
which was reported as 'unresolved external symbol' error by some Windows
linkers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxq.c  | 71 ------------------------------------------
 drivers/net/mlx5/mlx5_rxtx.h | 73 ++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 70 insertions(+), 74 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index da7a8b3cd7..c89e55b1c0 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -48,77 +48,6 @@ static_assert(MLX5_RSS_HASH_KEY_LEN ==
 	      (unsigned int)sizeof(rss_hash_default_key),
 	      "wrong RSS default key size.");
 
-/**
- * Check whether Multi-Packet RQ can be enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   1 if supported, negative errno value if not.
- */
-inline int
-mlx5_check_mprq_support(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	if (priv->config.mprq.enabled &&
-	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
-		return 1;
-	return -ENOTSUP;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the Rx queue.
- *
- *  @param rxq
- *     Pointer to receive queue structure.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
-{
-	return rxq->strd_num_n > 0;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_mprq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	uint32_t i;
-	uint16_t n = 0;
-	uint16_t n_ibv = 0;
-
-	if (mlx5_check_mprq_support(dev) < 0)
-		return 0;
-	/* All the configured queues should be enabled. */
-	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
-
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
-			continue;
-		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
-			++n;
-	}
-	/* Multi-Packet RQ can't be partially configured. */
-	MLX5_ASSERT(n == 0 || n == n_ibv);
-	return n == n_ibv;
-}
-
 /**
  * Calculate the number of CQEs in CQ for the Rx queue.
  *
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 7989a50403..1e9345af61 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -311,9 +311,6 @@ struct mlx5_txq_ctrl {
 
 extern uint8_t rss_hash_default_key[];
 
-int mlx5_check_mprq_support(struct rte_eth_dev *dev);
-int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq);
-int mlx5_mprq_enabled(struct rte_eth_dev *dev);
 unsigned int mlx5_rxq_cqe_num(struct mlx5_rxq_data *rxq_data);
 int mlx5_mprq_free_mp(struct rte_eth_dev *dev);
 int mlx5_mprq_alloc_mp(struct rte_eth_dev *dev);
@@ -920,4 +917,74 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Multi-Packet RQ can be enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   1 if supported, negative errno value if not.
+ */
+static __rte_always_inline int
+mlx5_check_mprq_support(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	if (priv->config.mprq.enabled &&
+	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
+		return 1;
+	return -ENOTSUP;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the Rx queue.
+ *
+ *  @param rxq
+ *     Pointer to receive queue structure.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
+{
+	return rxq->strd_num_n > 0;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_mprq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint32_t i;
+	uint16_t n = 0;
+	uint16_t n_ibv = 0;
+
+	if (mlx5_check_mprq_support(dev) < 0)
+		return 0;
+	/* All the configured queues should be enabled. */
+	for (i = 0; i < priv->rxqs_n; ++i) {
+		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
+			(rxq, struct mlx5_rxq_ctrl, rxq);
+
+		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+			continue;
+		n_ibv++;
+		if (mlx5_rxq_mprq_enabled(rxq))
+			++n;
+	}
+	/* Multi-Packet RQ can't be partially configured. */
+	MLX5_ASSERT(n == 0 || n == n_ibv);
+	return n == n_ibv;
+}
 #endif /* RTE_PMD_MLX5_RXTX_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 07/33] net/mlx5: do not define static_assert in Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

In Linux 'static_assert' is defined in file mlx5_defs.h:

 #ifndef HAVE_STATIC_ASSERT
    #define static_assert _Static_assert
 #endif

The same definition can originate from Linux file /usr/include/assert.h.

In Windows static_assert is used while _Static_assert is unknown.
Therefore update the definition condition to be:

 #if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
    #define static_assert _Static_assert
 #endif

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_defs.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index aa55db3750..85a0979653 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -199,8 +199,11 @@
 /* Maximum number of shared actions supported by rte_flow */
 #define MLX5_MAX_SHARED_ACTIONS 2
 
-/* Definition of static_assert found in /usr/include/assert.h */
-#ifndef HAVE_STATIC_ASSERT
+/*
+ * Linux definition of static_assert is found in /usr/include/assert.h.
+ * Windows does not require a redefinition.
+ */
+#if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
 #define static_assert _Static_assert
 #endif
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 08/33] net/mlx5: move static_assert calls to global scope

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Some Windows compilers consider static_assert() as calls to another
function rather than a compiler directive which allows checking type
information at compile time.  This only occurs if the static_assert call
appears inside another function scope. To solve it move the
static_assert calls to global scope in the files where they are used.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxtx.c | 98 ++++++++++++++++++++++----------------------
 drivers/net/mlx5/mlx5_txpp.c |  5 ++-
 2 files changed, 53 insertions(+), 50 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index d12d746c2f..65a1f997e9 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -79,6 +79,56 @@ static uint16_t mlx5_tx_burst_##func(void *txq, \
 
 #define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx},
 
+/* static asserts */
+static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
+static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
+		(2 * RTE_ETHER_ADDR_LEN),
+		"invalid Data Segment data size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert((sizeof(struct rte_vlan_hdr) +
+			sizeof(struct rte_ether_hdr)) ==
+		MLX5_ESEG_MIN_INLINE_SIZE,
+		"invalid min inline data size");
+static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
+		MLX5_DSEG_MAX, "invalid WQE max size");
+static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Control Segment size");
+static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Ethernet Segment size");
+static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Data Segment size");
+static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
+		"invalid WQE size");
+
 static __rte_always_inline uint32_t
 rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe,
 				   volatile struct mlx5_mini_cqe8 *mcqe);
@@ -2070,8 +2120,6 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq,
 	bool ring_doorbell = false;
 	int ret;
 
-	static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
-	static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
 	do {
 		volatile struct mlx5_cqe *cqe;
 
@@ -2381,15 +2429,6 @@ mlx5_tx_eseg_dmin(struct mlx5_txq_data *__rte_restrict txq __rte_unused,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = RTE_BE16(MLX5_ESEG_MIN_INLINE_SIZE);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2474,15 +2513,6 @@ mlx5_tx_eseg_data(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = rte_cpu_to_be_16(inlen);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2697,15 +2727,6 @@ mlx5_tx_eseg_mdat(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	MLX5_ASSERT(inlen >= MLX5_ESEG_MIN_INLINE_SIZE);
 	pdst = (uint8_t *)&es->inline_data;
 	if (MLX5_TXOFF_CONFIG(VLAN) && vlan) {
@@ -2952,9 +2973,6 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *__rte_restrict txq,
 	uint8_t *pdst;
 
 	MLX5_ASSERT(len > MLX5_ESEG_MIN_INLINE_SIZE);
-	static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
-				 (2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Data Segment data size");
 	if (!MLX5_TXOFF_CONFIG(MPW)) {
 		/* Store the descriptor byte counter for eMPW sessions. */
 		dseg->bcount = rte_cpu_to_be_32
@@ -4070,7 +4088,6 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4247,7 +4264,6 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4561,10 +4577,6 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq,
 			    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
 				vlan = sizeof(struct rte_vlan_hdr);
 				inlen += vlan;
-				static_assert((sizeof(struct rte_vlan_hdr) +
-					       sizeof(struct rte_ether_hdr)) ==
-					       MLX5_ESEG_MIN_INLINE_SIZE,
-					       "invalid min inline data size");
 			}
 			/*
 			 * If inlining is enabled at configuration time
@@ -5567,16 +5579,6 @@ mlx5_select_tx_function(struct rte_eth_dev *dev)
 	uint64_t tx_offloads = dev->data->dev_conf.txmode.offloads;
 	unsigned int diff = 0, olx = 0, i, m;
 
-	static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
-		      MLX5_DSEG_MAX, "invalid WQE max size");
-	static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Control Segment size");
-	static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Ethernet Segment size");
-	static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Data Segment size");
-	static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
-		      "invalid WQE size");
 	MLX5_ASSERT(priv);
 	if (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) {
 		/* We should support Multi-Segment Packets. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index a1ec294942..d61e43e55d 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -18,6 +18,9 @@
 #include "mlx5_rxtx.h"
 #include "mlx5_common_os.h"
 
+static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
+		"Wrong timestamp CQE part size");
+
 static const char * const mlx5_txpp_stat_names[] = {
 	"tx_pp_missed_interrupt_errors", /* Missed service interrupt. */
 	"tx_pp_rearm_queue_errors", /* Rearm Queue errors. */
@@ -741,8 +744,6 @@ mlx5_txpp_update_timestamp(struct mlx5_dev_ctx_shared *sh)
 	uint64_t ts;
 	uint16_t ci;
 
-	static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
-		      "Wrong timestamp CQE part size");
 	mlx5_atomic_read_cqe((rte_int128_t *)&cqe->timestamp, &to.u128);
 	if (to.cts.op_own >> 4) {
 		DRV_LOG(DEBUG, "Clock Queue error sync lost.");
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 09/33] net/mlx5: wrap glue alloc/dealloc PD with OS calls

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Wrap glue calls alloc_pd() and dealloc_pd() with generic OS calls.  In
Linux - protection domain allocations are implemented by Verbs glue API
while in Windows it is by DevX API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 11 +++++++++++
 drivers/net/mlx5/mlx5.c                    |  6 +++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index 3420937859..c9f8d7cbc1 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -201,4 +201,15 @@ mlx5_os_get_devx_uar_page_id(void *uar)
 #endif
 }
 
+static inline void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	return mlx5_glue->alloc_pd(ctx);
+}
+
+static inline int
+mlx5_os_dealloc_pd(void *pd)
+{
+	return mlx5_glue->dealloc_pd(pd);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 8e9df6abcf..7d1b4fbdc0 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -932,7 +932,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sh->port[i].ih_port_id = RTE_MAX_ETHPORTS;
 		sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS;
 	}
-	sh->pd = mlx5_glue->alloc_pd(sh->ctx);
+	sh->pd = mlx5_os_alloc_pd(sh->ctx);
 	if (sh->pd == NULL) {
 		DRV_LOG(ERR, "PD allocation failure");
 		err = ENOMEM;
@@ -1032,7 +1032,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	if (sh->tx_uar)
 		mlx5_glue->devx_free_uar(sh->tx_uar);
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->ctx)
 		claim_zero(mlx5_glue->close_device(sh->ctx));
 	mlx5_free(sh);
@@ -1100,7 +1100,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		sh->tx_uar = NULL;
 	}
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->tis)
 		claim_zero(mlx5_devx_cmd_destroy(sh->tis));
 	if (sh->td)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 10/33] net/mlx5: wrap glue reg/dereg UMEM with OS calls

[Tal Shnaiderman]

Wrap glue calls for UMEM registration and deregistration with generic OS
calls since each OS (Linux or Windows) has a different glue API
parameters.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 12 ++++++++++++
 drivers/common/mlx5/mlx5_common.c          |  4 ++--
 drivers/net/mlx5/mlx5.c                    |  2 +-
 drivers/net/mlx5/mlx5_devx.c               | 16 ++++++++--------
 drivers/net/mlx5/mlx5_flow.c               |  4 ++--
 drivers/net/mlx5/mlx5_txpp.c               | 12 ++++++------
 6 files changed, 31 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index c9f8d7cbc1..f8b215cc29 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -212,4 +212,16 @@ mlx5_os_dealloc_pd(void *pd)
 {
 	return mlx5_glue->dealloc_pd(pd);
 }
+
+static inline void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	return mlx5_glue->devx_umem_reg(ctx, addr, size, access);
+}
+
+static inline int
+mlx5_os_umem_dereg(void *pumem)
+{
+	return mlx5_glue->devx_umem_dereg(pumem);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c
index 044513223c..a00ffcb833 100644
--- a/drivers/common/mlx5/mlx5_common.c
+++ b/drivers/common/mlx5/mlx5_common.c
@@ -148,7 +148,7 @@ mlx5_alloc_dbr_page(void *ctx)
 		return NULL;
 	}
 	/* Register allocated memory. */
-	page->umem = mlx5_glue->devx_umem_reg(ctx, page->dbrs,
+	page->umem = mlx5_os_umem_reg(ctx, page->dbrs,
 					      MLX5_DBR_PAGE_SIZE, 0);
 	if (!page->umem) {
 		DRV_LOG(ERR, "cannot umem reg dbr page");
@@ -232,7 +232,7 @@ mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id,
 		/* Page not used, free it and remove from list. */
 		LIST_REMOVE(page, next);
 		if (page->umem)
-			ret = -mlx5_glue->devx_umem_dereg(page->umem);
+			ret = -mlx5_os_umem_dereg(page->umem);
 		mlx5_free(page);
 	} else {
 		/* Mark in bitmap that this door-bell is not in use. */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 7d1b4fbdc0..84123f8e3d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -482,7 +482,7 @@ mlx5_flow_destroy_counter_stat_mem_mng(struct mlx5_counter_stats_mem_mng *mng)
 
 	LIST_REMOVE(mng, next);
 	claim_zero(mlx5_devx_cmd_destroy(mng->dm));
-	claim_zero(mlx5_glue->devx_umem_dereg(mng->umem));
+	claim_zero(mlx5_os_umem_dereg(mng->umem));
 	mlx5_free(mem);
 }
 
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index de9b204075..235fd5798d 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -155,7 +155,7 @@ mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->rq_dbrec_page;
 
 	if (rxq_ctrl->wq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->wq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->wq_umem);
 		rxq_ctrl->wq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.wqes) {
@@ -182,7 +182,7 @@ mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->cq_dbrec_page;
 
 	if (rxq_ctrl->cq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->cq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->cq_umem);
 		rxq_ctrl->cq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.cqes) {
@@ -375,7 +375,7 @@ mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	if (!buf)
 		return NULL;
 	rxq_data->wqes = buf;
-	rxq_ctrl->wq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	rxq_ctrl->wq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						     buf, wq_size, 0);
 	if (!rxq_ctrl->wq_umem)
 		goto error;
@@ -497,7 +497,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	rxq_data->cqes = (volatile struct mlx5_cqe (*)[])(uintptr_t)buf;
-	rxq_ctrl->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx, buf,
+	rxq_ctrl->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, buf,
 						     cq_size,
 						     IBV_ACCESS_LOCAL_WRITE);
 	if (!rxq_ctrl->cq_umem) {
@@ -1127,7 +1127,7 @@ mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 		txq_obj->sq_devx = NULL;
 	}
 	if (txq_obj->sq_umem) {
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->sq_umem));
 		txq_obj->sq_umem = NULL;
 	}
 	if (txq_obj->sq_buf) {
@@ -1155,7 +1155,7 @@ mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
 	if (txq_obj->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->cq_umem));
 	if (txq_obj->cq_buf)
 		mlx5_free(txq_obj->cq_buf);
 	if (txq_obj->cq_dbrec_page)
@@ -1243,7 +1243,7 @@ mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		return 0;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	txq_obj->cq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						(void *)txq_obj->cq_buf,
 						cqe_n * sizeof(struct mlx5_cqe),
 						IBV_ACCESS_LOCAL_WRITE);
@@ -1342,7 +1342,7 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->sq_umem = mlx5_glue->devx_umem_reg
+	txq_obj->sq_umem = mlx5_os_umem_reg
 					(priv->sh->ctx,
 					 (void *)txq_obj->sq_buf,
 					 wqe_n * sizeof(struct mlx5_wqe),
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bf86aaaa39..66491bbf78 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -6446,7 +6446,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	}
 	mem_mng = (struct mlx5_counter_stats_mem_mng *)(mem + size) - 1;
 	size = sizeof(*raw_data) * MLX5_COUNTERS_PER_POOL * raws_n;
-	mem_mng->umem = mlx5_glue->devx_umem_reg(sh->ctx, mem, size,
+	mem_mng->umem = mlx5_os_umem_reg(sh->ctx, mem, size,
 						 IBV_ACCESS_LOCAL_WRITE);
 	if (!mem_mng->umem) {
 		rte_errno = errno;
@@ -6465,7 +6465,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	mkey_attr.relaxed_ordering_read = sh->cmng.relaxed_ordering_read;
 	mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr);
 	if (!mem_mng->dm) {
-		mlx5_glue->devx_umem_dereg(mem_mng->umem);
+		mlx5_os_umem_dereg(mem_mng->umem);
 		rte_errno = errno;
 		mlx5_free(mem);
 		return -rte_errno;
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index d61e43e55d..749529c410 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -131,13 +131,13 @@ mlx5_txpp_destroy_send_queue(struct mlx5_txpp_wq *wq)
 	if (wq->sq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->sq));
 	if (wq->sq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->sq_umem));
 	if (wq->sq_buf)
 		mlx5_free((void *)(uintptr_t)wq->sq_buf);
 	if (wq->cq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->cq));
 	if (wq->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->cq_umem));
 	if (wq->cq_buf)
 		mlx5_free((void *)(uintptr_t)wq->cq_buf);
 	memset(wq, 0, sizeof(*wq));
@@ -268,7 +268,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -318,7 +318,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -506,7 +506,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -562,7 +562,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 11/33] net/mlx5: fix adding destroy flow action wrapper

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Glue function destroy_flow_action() was wrapped by OS specific operation
mlx5_flow_os_destroy_flow_action(). It was skipped in file mlx5.c.

Fixes: b293fbf9672b ("net/mlx5: add OS specific flow actions operations")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_flow_dv.c | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 84123f8e3d..ea5cf80ac1 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -38,6 +38,7 @@
 #include "mlx5_autoconf.h"
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
+#include "mlx5_flow_os.h"
 #include "rte_pmd_mlx5.h"
 
 /* Device parameter to enable RX completion queue compression. */
@@ -415,7 +416,7 @@ mlx5_flow_aso_age_mng_close(struct mlx5_dev_ctx_shared *sh)
 			for (j = 0; j < MLX5_COUNTERS_PER_POOL; ++j)
 				if (pool->actions[j].dr_action)
 					claim_zero
-						(mlx5_glue->destroy_flow_action
+						(mlx5_flow_os_destroy_flow_action
 						  (pool->actions[j].dr_action));
 			mlx5_free(pool);
 		}
@@ -523,7 +524,7 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh)
 
 				if (cnt->action)
 					claim_zero
-					 (mlx5_glue->destroy_flow_action
+					 (mlx5_flow_os_destroy_flow_action
 					  (cnt->action));
 				if (fallback && MLX5_POOL_GET_CNT
 				    (pool, j)->dcs_when_free)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index c31737652c..78b5a6b338 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -11117,11 +11117,11 @@ flow_dv_sample_remove_cb(struct mlx5_cache_list *list __rte_unused,
 	struct mlx5_priv *priv = dev->data->dev_private;
 
 	if (cache_resource->verbs_action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 				(cache_resource->verbs_action));
 	if (cache_resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB) {
 		if (cache_resource->default_miss)
-			claim_zero(mlx5_glue->destroy_flow_action
+			claim_zero(mlx5_flow_os_destroy_flow_action
 			  (cache_resource->default_miss));
 	}
 	if (cache_resource->normal_path_tbl)
@@ -11174,7 +11174,7 @@ flow_dv_dest_array_remove_cb(struct mlx5_cache_list *list __rte_unused,
 
 	MLX5_ASSERT(cache_resource->action);
 	if (cache_resource->action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 					(cache_resource->action));
 	for (; i < cache_resource->num_of_dest; i++)
 		flow_dv_sample_sub_actions_release(dev,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 12/33] common/mlx5: add definition HAVE_INFINIBAND_VERBS_H

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add a Verbs file presence indication. Under Linux it is required that
file infiniband/verbs.h is installed for building DPDK.  Other
operating systems (e.g. Windows) ignore Verbs completely.  This commit
adds definition HAVE_INFINIBAND_VERBS_H (file mlx5_autoconf.h) to
indicate whether DPDK compiles with Verbs or not.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/meson.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build
index 63b78e4bce..580419e6d9 100644
--- a/drivers/common/mlx5/linux/meson.build
+++ b/drivers/common/mlx5/linux/meson.build
@@ -181,8 +181,10 @@ has_sym_args = [
 	[ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h',
 	'mlx5dv_dr_action_create_dest_array'],
 	[ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ],
-        [ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
-        'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
+	'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_INFINIBAND_VERBS_H', 'infiniband/verbs.h',
+	'INFINIBAND_VERBS_H' ],
 ]
 config = configuration_data()
 foreach arg:has_sym_args
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 13/33] common/mlx5/linux: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_malloc() API has an alignment parameter for system memory
allocations. malloc() is called for non-aligned allocations and
posix_memalign() is called for aligned allocations. When calling
mlx5_free() there is no distinction whether the memory was originally
allocated with or without alignment. Freeing a memory may be handled
differently by operating systems. Therefore this commit wraps these APIs
with OS specific calls: mlx5_os_malloc(), mlx5_os_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 38 ++++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_malloc.c          | 14 +++++------
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index f8b215cc29..bd44ecb7d6 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -6,6 +6,7 @@
 #define RTE_PMD_MLX5_COMMON_OS_H_
 
 #include <stdio.h>
+#include <malloc.h>
 
 #include <rte_pci.h>
 #include <rte_debug.h>
@@ -16,6 +17,7 @@
 
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
+#include "mlx5_malloc.h"
 
 /**
  * Get device name. Given an ibv_device pointer - return a
@@ -224,4 +226,40 @@ mlx5_os_umem_dereg(void *pumem)
 {
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
+
+/**
+ * Memory allocation optionally with alignment.
+ *
+ * @param[in] align
+ *    Alignment size (may be zero)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	void *buf;
+
+	if (posix_memalign(&buf, align, size))
+		return NULL;
+	return buf;
+}
+
+/**
+ * This API de-allocates a memory that originally could have been
+ * allocated aligned or non-aligned. In Linux it is a wrapper
+ * around free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	free(addr);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_malloc.c b/drivers/common/mlx5/mlx5_malloc.c
index 5a3267f730..9d30cedbaa 100644
--- a/drivers/common/mlx5/mlx5_malloc.c
+++ b/drivers/common/mlx5/mlx5_malloc.c
@@ -9,6 +9,7 @@
 #include <string.h>
 
 #include "mlx5_common_utils.h"
+#include "mlx5_common_os.h"
 #include "mlx5_malloc.h"
 
 struct mlx5_sys_mem {
@@ -148,14 +149,11 @@ static void *
 mlx5_alloc_align(size_t size, unsigned int align, unsigned int zero)
 {
 	void *buf;
-	int ret;
-
-	ret = posix_memalign(&buf, align, size);
-	if (ret) {
-		DRV_LOG(ERR,
-			"Couldn't allocate buf size=%zu align=%u. Err=%d\n",
-			size, align, ret);
 
+	buf = mlx5_os_malloc(align, size);
+	if (!buf) {
+		DRV_LOG(ERR, "Couldn't allocate buf size=%zu align=%u.",
+			size, align);
 		return NULL;
 	}
 	if (zero)
@@ -264,7 +262,7 @@ mlx5_free(void *addr)
 		__atomic_add_fetch(&mlx5_sys_mem.free_sys, 1,
 				   __ATOMIC_RELAXED);
 #endif
-		free(addr);
+		mlx5_os_free(addr);
 	} else {
 #ifdef RTE_LIBRTE_MLX5_DEBUG
 		__atomic_add_fetch(&mlx5_sys_mem.free_rte, 1,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 14/33] common/mlx5/windows: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows equivalent of the Linux implementation.  The
APIs included in this commit: mlx5_os_malloc(), mlx5_os_free(). For
memory allocations (with or without alignment) we always call
_aligned_malloc(). Even if zero alignment was requested in the first
place - we always select a minimal alignment value. In this way when the
memory is free - it is always safe to call _aligned_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 51 ++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
new file mode 100644
index 0000000000..cce7c88c6a
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_COMMON_OS_H_
+#define RTE_PMD_MLX5_COMMON_OS_H_
+
+#include <stdio.h>
+
+#include "mlx5_autoconf.h"
+#include "mlx5_glue.h"
+#include "mlx5_malloc.h"
+
+/**
+ * This API allocates aligned or non-aligned memory.  The free can be on either
+ * aligned or nonaligned memory.  To be protected - even though there may be no
+ * alignment - in Windows this API will unconditioanlly call _aligned_malloc()
+ * with at least a minimal alignment size.
+ *
+ * @param[in] align
+ *    The alignment value, which must be an integer power of 2 (or 0 for
+ *    non-alignment)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	if (align < MLX5_MALLOC_ALIGNMENT)
+		align = MLX5_MALLOC_ALIGNMENT;
+	return _aligned_malloc(size, align);
+}
+
+/**
+ * This API de-allocates a memory that originally could have been allocated
+ * aligned or non-aligned. In Windows since the allocation was with
+ * _aligned_malloc() - it is safe to always call _aligned_free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	_aligned_free(addr);
+}
+#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 15/33] common/mlx5/linux: wrap event channel APIs with OS calls

[Tal Shnaiderman]

Wrap the API to create/destroy event channel and to subscribe an event
with OS calls. In Linux those calls are implemented by glue functions
while in Windows they are not supported.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 22 ++++++++++++++++++++++
 drivers/net/mlx5/mlx5_devx.c               |  8 ++++----
 drivers/net/mlx5/mlx5_txpp.c               |  6 +++---
 drivers/vdpa/mlx5/mlx5_vdpa_event.c        |  7 ++++---
 4 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index bd44ecb7d6..63f070d9c4 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -227,6 +227,28 @@ mlx5_os_umem_dereg(void *pumem)
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
 
+static inline void *
+mlx5_os_devx_create_event_channel(void *ctx, int flags)
+{
+	return mlx5_glue->devx_create_event_channel(ctx, flags);
+}
+
+static inline void
+mlx5_os_devx_destroy_event_channel(void *eventc)
+{
+	mlx5_glue->devx_destroy_event_channel(eventc);
+}
+
+static inline int
+mlx5_os_devx_subscribe_devx_event(void *eventc,
+				  void *obj,
+				  uint16_t events_sz, uint16_t events_num[],
+				  uint64_t cookie)
+{
+	return mlx5_glue->devx_subscribe_devx_event(eventc, obj, events_sz,
+						    events_num, cookie);
+}
+
 /**
  * Memory allocation optionally with alignment.
  *
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 235fd5798d..84a5c55ee0 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -216,7 +216,7 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_obj *rxq_obj)
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->devx_cq));
 		if (rxq_obj->devx_channel)
-			mlx5_glue->devx_destroy_event_channel
+			mlx5_os_devx_destroy_event_channel
 							(rxq_obj->devx_channel);
 		mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
 		mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
@@ -533,7 +533,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cqe_n = log_cqe_n;
 	rxq_data->cqn = cq_obj->id;
 	if (rxq_ctrl->obj->devx_channel) {
-		ret = mlx5_glue->devx_subscribe_devx_event
+		ret = mlx5_os_devx_subscribe_devx_event
 						(rxq_ctrl->obj->devx_channel,
 						 cq_obj->obj,
 						 sizeof(event_nums),
@@ -644,7 +644,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		int devx_ev_flag =
 			  MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA;
 
-		tmpl->devx_channel = mlx5_glue->devx_create_event_channel
+		tmpl->devx_channel = mlx5_os_devx_create_event_channel
 								(priv->sh->ctx,
 								 devx_ev_flag);
 		if (!tmpl->devx_channel) {
@@ -686,7 +686,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	if (tmpl->devx_cq)
 		claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq));
 	if (tmpl->devx_channel)
-		mlx5_glue->devx_destroy_event_channel(tmpl->devx_channel);
+		mlx5_os_devx_destroy_event_channel(tmpl->devx_channel);
 	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
 	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	rte_errno = ret; /* Restore rte_errno. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 749529c410..726bdc6ae1 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -37,7 +37,7 @@ static void
 mlx5_txpp_destroy_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	if (sh->txpp.echan) {
-		mlx5_glue->devx_destroy_event_channel(sh->txpp.echan);
+		mlx5_os_devx_destroy_event_channel(sh->txpp.echan);
 		sh->txpp.echan = NULL;
 	}
 }
@@ -47,7 +47,7 @@ static int
 mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	MLX5_ASSERT(!sh->txpp.echan);
-	sh->txpp.echan = mlx5_glue->devx_create_event_channel(sh->ctx,
+	sh->txpp.echan = mlx5_os_devx_create_event_channel(sh->ctx,
 			MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!sh->txpp.echan) {
 		rte_errno = errno;
@@ -937,7 +937,7 @@ mlx5_txpp_start_service(struct mlx5_dev_ctx_shared *sh)
 		return -rte_errno;
 	}
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(sh->txpp.echan,
+	ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan,
 						   sh->txpp.rearm_queue.cq->obj,
 						   sizeof(event_nums),
 						   event_nums, 0);
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
index 3aeaeb893f..fd6150928b 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
@@ -15,6 +15,7 @@
 #include <rte_alarm.h>
 
 #include <mlx5_common.h>
+#include <mlx5_common_os.h>
 #include <mlx5_glue.h>
 
 #include "mlx5_vdpa_utils.h"
@@ -43,7 +44,7 @@ mlx5_vdpa_event_qp_global_release(struct mlx5_vdpa_priv *priv)
 		       sizeof(out.buf)) >=
 		       (ssize_t)sizeof(out.event_resp.cookie))
 			;
-		mlx5_glue->devx_destroy_event_channel(priv->eventc);
+		mlx5_os_devx_destroy_event_channel(priv->eventc);
 		priv->eventc = NULL;
 	}
 #endif
@@ -63,7 +64,7 @@ mlx5_vdpa_event_qp_global_prepare(struct mlx5_vdpa_priv *priv)
 		DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno);
 		return -1;
 	}
-	priv->eventc = mlx5_glue->devx_create_event_channel(priv->ctx,
+	priv->eventc = mlx5_os_devx_create_event_channel(priv->ctx,
 			   MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!priv->eventc) {
 		rte_errno = errno;
@@ -176,7 +177,7 @@ mlx5_vdpa_cq_create(struct mlx5_vdpa_priv *priv, uint16_t log_desc_n,
 	cq->cq_ci = 0;
 	rte_spinlock_init(&cq->sl);
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
+	ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
 						   sizeof(event_nums),
 						   event_nums,
 						   (uint64_t)(uintptr_t)cq);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 16/33] common/mlx5: add Windows exports file

[Tal Shnaiderman]

File drivers/common/mlx5/rte_common_mlx5_exports.def contains mlx5
Windows exported symbols under common/mlx5 directory (DLL file
name librte_common_mlx5*.dll). It is the equivalent of Linux map
file rte_common_mlx5_version.map but the list of symbols may be
different between the two operating systems.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def | 62 +++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
new file mode 100644
index 0000000000..23984b7abd
--- /dev/null
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -0,0 +1,62 @@
+EXPORTS
+	haswell_broadwell_cpu
+
+	mlx5_common_init
+
+	mlx5_create_mr_ext
+
+	mlx5_devx_cmd_create_cq
+	mlx5_devx_cmd_create_flex_parser
+	mlx5_devx_cmd_create_qp
+	mlx5_devx_cmd_create_rq
+	mlx5_devx_cmd_create_rqt
+	mlx5_devx_cmd_create_sq
+	mlx5_devx_cmd_create_tir
+	mlx5_devx_cmd_create_td
+	mlx5_devx_cmd_create_tis
+	mlx5_devx_cmd_create_virtq
+	mlx5_devx_cmd_destroy
+	mlx5_devx_cmd_flow_counter_alloc
+	mlx5_devx_cmd_flow_counter_query
+	mlx5_devx_cmd_flow_dump
+	mlx5_devx_cmd_mkey_create
+	mlx5_devx_cmd_modify_qp_state
+	mlx5_devx_cmd_modify_rq
+	mlx5_devx_cmd_modify_rqt
+	mlx5_devx_cmd_modify_sq
+	mlx5_devx_cmd_modify_tir
+	mlx5_devx_cmd_modify_virtq
+	mlx5_devx_cmd_qp_query_tis_td
+	mlx5_devx_cmd_query_hca_attr
+	mlx5_devx_cmd_query_parse_samples
+	mlx5_devx_cmd_query_virtq
+	mlx5_devx_cmd_register_read
+	mlx5_devx_get_out_command_status
+	mlx5_devx_cmd_create_flow_hit_aso_obj
+
+	mlx5_get_dbr
+
+	mlx5_malloc_mem_select
+	mlx5_mr_btree_init
+	mlx5_mr_btree_free
+	mlx5_mr_btree_dump
+	mlx5_mr_addr2mr_bh
+	mlx5_mr_release_cache
+	mlx5_mr_dump_cache
+	mlx5_mr_rebuild_cache
+	mlx5_mr_insert_cache
+	mlx5_mr_lookup_cache
+	mlx5_mr_lookup_list
+	mlx5_mr_create_primary
+	mlx5_mr_flush_local_cache
+	mlx5_mp_req_queue_state_modify
+	mlx5_mr_free
+
+	mlx5_pci_driver_register
+
+	mlx5_release_dbr
+
+	mlx5_malloc
+	mlx5_realloc
+	mlx5_free
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 17/33] common/mlx5: extend DevX query hca attributes command

[Tal Shnaiderman]

Extend DevX API mlx5_devx_cmd_query_hca_attr() to report on max number
of available objects including: CQ, QP, PD, SRQ.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 11 +++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++++++++-
 drivers/common/mlx5/mlx5_prm.h       |  1 +
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 9c1d1883ea..36d3a421aa 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -720,6 +720,14 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->flow_hit_aso = !!(MLX5_GET64(cmd_hca_cap, hcattr,
 					   general_obj_types) &
 				MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
+	attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
+	attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
+	attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
+	attr->log_max_qp_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp_sz);
+	attr->log_max_mrw_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_mrw_sz);
+	attr->log_max_pd = MLX5_GET(cmd_hca_cap, hcattr, log_max_pd);
+	attr->log_max_srq = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq);
+	attr->log_max_srq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq_sz);
 	if (attr->qos.sup) {
 		MLX5_SET(query_hca_cap_in, in, op_mod,
 			 MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
@@ -834,6 +842,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->tunnel_stateless_gtp = MLX5_GET
 					(per_protocol_networking_offload_caps,
 					 hcattr, tunnel_stateless_gtp);
+	attr->rss_ind_tbl_cap = MLX5_GET
+					(per_protocol_networking_offload_caps,
+					 hcattr, rss_ind_tbl_cap);
 	if (attr->wqe_inline_mode != MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
 		return 0;
 	if (attr->eth_virt) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 726e9f5192..5056d86ed9 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -117,6 +117,15 @@ struct mlx5_hca_attr {
 	uint32_t log_max_ft_sampler_num:8;
 	struct mlx5_hca_qos_attr qos;
 	struct mlx5_hca_vdpa_attr vdpa;
+	int log_max_qp_sz;
+	int log_max_cq_sz;
+	int log_max_qp;
+	int log_max_cq;
+	uint32_t log_max_pd;
+	uint32_t log_max_mrw_sz;
+	uint32_t log_max_srq;
+	uint32_t log_max_srq_sz;
+	uint32_t rss_ind_tbl_cap;
 };
 
 struct mlx5_devx_wq_attr {
@@ -495,7 +504,6 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_virtio_q_counters(void *ctx);
 __rte_internal
 int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 				  struct mlx5_devx_virtio_q_couners_attr *attr);
-
 __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 58d180486e..585e9d1c05 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -665,6 +665,7 @@ typedef uint8_t u8;
 #define MLX5_GET64(typ, p, fld) rte_be_to_cpu_64(*((rte_be64_t *)(p) + \
 						   __mlx5_64_off(typ, fld)))
 #define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
+#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
 
 struct mlx5_ifc_fte_match_set_misc_bits {
 	u8 gre_c_present[0x1];
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 18/33] common/mlx5: add DevX alloc PD command

[Tal Shnaiderman]

Add a new DevX API mlx5_devx_cmd_alloc_pd() that creates a new protection
domain (PD).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c            | 35 ++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  2 ++
 drivers/common/mlx5/mlx5_prm.h                  | 36 +++++++++++++++++++++++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  1 +
 drivers/common/mlx5/version.map                 |  1 +
 5 files changed, 75 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 36d3a421aa..12f51a940c 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2055,3 +2055,38 @@ mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx, uint32_t pd)
 	flow_hit_aso_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
 	return flow_hit_aso_obj;
 }
+
+/*
+ * Create PD using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_alloc_pd(void *ctx)
+{
+	struct mlx5_devx_obj *ppd =
+		mlx5_malloc(MLX5_MEM_ZERO, sizeof(*ppd), 0, SOCKET_ID_ANY);
+	u32 in[MLX5_ST_SZ_DW(alloc_pd_in)] = {0};
+	u32 out[MLX5_ST_SZ_DW(alloc_pd_out)] = {0};
+
+	if (!ppd) {
+		DRV_LOG(ERR, "Failed to allocate PD data.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(alloc_pd_in, in, opcode, MLX5_CMD_OP_ALLOC_PD);
+	ppd->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+				out, sizeof(out));
+	if (!ppd->obj) {
+		mlx5_free(ppd);
+		DRV_LOG(ERR, "Failed to allocate PD Obj using DevX.");
+		rte_errno = errno;
+		return NULL;
+	}
+	ppd->id = MLX5_GET(alloc_pd_out, out, pd);
+	return ppd;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 5056d86ed9..8277fdbc39 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -508,4 +508,6 @@ __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
 
+__rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_alloc_pd(void *ctx);
 #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 585e9d1c05..4ddf865743 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -844,6 +844,8 @@ enum {
 	MLX5_CMD_OP_SUSPEND_QP = 0x50F,
 	MLX5_CMD_OP_RESUME_QP = 0x510,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
+	MLX5_CMD_OP_ALLOC_PD = 0x800,
+	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ACCESS_REGISTER = 0x805,
 	MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -2772,6 +2774,40 @@ struct mlx5_ifc_init2init_qp_in_bits {
 	u8 reserved_at_800[0x80];
 };
 
+struct mlx5_ifc_dealloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x8];
+	u8 pd[0x18];
+	u8 reserved_3[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x8];
+	u8 pd[0x18];
+	u8 reserved_2[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x40];
+};
+
 #ifdef PEDANTIC
 #pragma GCC diagnostic ignored "-Wpedantic"
 #endif
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 23984b7abd..15aafc3809 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -5,6 +5,7 @@ EXPORTS
 
 	mlx5_create_mr_ext
 
+	mlx5_devx_cmd_alloc_pd
 	mlx5_devx_cmd_create_cq
 	mlx5_devx_cmd_create_flex_parser
 	mlx5_devx_cmd_create_qp
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 17dd11f635..fb3952bbc6 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -10,6 +10,7 @@ INTERNAL {
 
 	mlx5_dev_to_pci_addr;
 
+	mlx5_devx_cmd_alloc_pd;
 	mlx5_devx_cmd_create_cq;
 	mlx5_devx_cmd_create_flex_parser;
 	mlx5_devx_cmd_create_qp;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 19/33] common/mlx5/windows: add glue functions APIs

[Tal Shnaiderman]

Windows glue functions are added to file mlx5/windows/mlx5_glue.c.
The following APIs are supported:
get_device_list, free_device_list, open_device, close_device,
query_device, query_hca_iseg, devx_obj_create, devx_obj_destroy,
devx_obj_query, devx_obj_modify, devx_general_cmd, devx_umem_reg,
devx_umem_dereg, devx_alloc_uar, devx_free_uar, devx_fs_rule_add,
devx_fs_rule_del, devx_query_eqn
New added files:
mlx5_win_defs.h - this file imports missing definitions from Linux
rdma-core library and Linux OS.
mlx5_win_ext.h - this file contains structs that enable a unified
Linux/Windows API. Each struct has an equivalent (but different) Linux
struct. By calling with 'void *' pointers - the Linux/Windows API is
identical.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |   1 +
 drivers/common/mlx5/windows/mlx5_glue.c         | 304 ++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 +++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  34 +++
 5 files changed, 422 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 15aafc3809..2f6aab133b 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -36,6 +36,7 @@ EXPORTS
 	mlx5_devx_cmd_create_flow_hit_aso_obj
 
 	mlx5_get_dbr
+	mlx5_glue
 
 	mlx5_malloc_mem_select
 	mlx5_mr_btree_init
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
new file mode 100644
index 0000000000..7f8a00aaa7
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -0,0 +1,304 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+
+#include "mlx5_glue.h"
+#include "mlx5_common_utils.h"
+#include "mlx5_win_ext.h"
+
+/*
+ * The returned value of this API is an array of pointers to mlx5
+ * devices under Windows. The interesting parameters of a device:
+ * Device PCI parameters: domain, bus, device id, function.
+ * Device name.
+ */
+static void *
+mlx5_glue_devx_get_device_list(int *num_devices)
+{
+	struct devx_device_bdf *devx_bdf_devs = NULL;
+	size_t n_devx_devx = 0;
+	int32_t ret = devx_get_device_list(&n_devx_devx, &devx_bdf_devs);
+
+	if (ret) {
+		errno = ret;
+		*num_devices = 0;
+		return NULL;
+	}
+	*num_devices = (int)n_devx_devx;
+	return devx_bdf_devs;
+}
+
+static void
+mlx5_glue_devx_free_device_list(void *list)
+{
+	if (!list) {
+		errno = EINVAL;
+		return;
+	}
+	devx_free_device_list(list);
+}
+
+static int
+mlx5_glue_devx_close_device(void *ctx)
+{
+	mlx5_context_st *mlx5_ctx;
+	int rc;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (mlx5_context_st *)ctx;
+	rc = devx_close_device(mlx5_ctx->devx_ctx);
+	free(mlx5_ctx);
+	return rc;
+}
+
+static void *
+mlx5_glue_devx_open_device(void *device)
+{
+	struct mlx5_context *mlx5_ctx;
+
+	if (!device) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = malloc((sizeof(struct mlx5_context)));
+	if (!mlx5_ctx) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(mlx5_ctx, 0, sizeof(*mlx5_ctx));
+	mlx5_ctx->devx_ctx = devx_open_device(device);
+	if (DEVX_IS_ERR(mlx5_ctx->devx_ctx)) {
+		errno = -DEVX_PTR_ERR(mlx5_ctx->devx_ctx);
+		free(mlx5_ctx);
+		return NULL;
+	}
+	return mlx5_ctx;
+}
+
+static int
+mlx5_glue_devx_query_device(void *device_bdf, void *dev_inf)
+{
+	struct devx_device_bdf *dev_bdf;
+	struct devx_device *mlx5_dev;
+
+	if (!device_bdf)
+		return -EINVAL;
+	dev_bdf = (struct devx_device_bdf *)device_bdf;
+	mlx5_dev = (struct devx_device *)dev_inf;
+	int err = devx_query_device(dev_bdf, mlx5_dev);
+	if (err)
+		return -E_FAIL;
+	return 0;
+}
+
+static void *
+mlx5_glue_devx_query_hca_iseg_mapping(void *ctx, uint32_t *cb_iseg)
+{
+	struct mlx5_context *mlx5_ctx;
+	void *pv_iseg;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	err = devx_query_hca_iseg_mapping(mlx5_ctx->devx_ctx,
+						cb_iseg, &pv_iseg);
+	if (err) {
+		errno = err;
+		return NULL;
+	}
+	return pv_iseg;
+}
+
+static void *
+mlx5_glue_devx_obj_create(void *ctx,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	devx_obj = malloc((sizeof(*devx_obj)));
+	if (!devx_obj) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(devx_obj, 0, sizeof(*devx_obj));
+	devx_obj->devx_ctx = GET_DEVX_CTX(ctx);
+	devx_obj->obj = devx_obj_create(devx_obj->devx_ctx,
+					in, inlen, out, outlen);
+	if (DEVX_IS_ERR(devx_obj->obj)) {
+		errno = -DEVX_PTR_ERR(devx_obj->obj);
+		free(devx_obj);
+		return NULL;
+	}
+	return devx_obj;
+}
+
+static int
+mlx5_glue_devx_obj_destroy(void *obj)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!obj)
+		return -EINVAL;
+	devx_obj = obj;
+	int rc = devx_obj_destroy(devx_obj->obj);
+	free(devx_obj);
+	return rc;
+}
+
+static int
+mlx5_glue_devx_general_cmd(void *ctx,
+			   void *in, size_t inlen,
+			   void *out, size_t outlen)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_cmd(GET_DEVX_CTX(ctx), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_query(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_modify(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_umem_dereg(void *pumem)
+{
+	struct devx_obj_handle *umem;
+
+	if (!pumem)
+		return -EINVAL;
+	umem = pumem;
+	return devx_umem_unreg(umem);
+}
+
+static void *
+mlx5_glue_devx_umem_reg(void *ctx, void *addr, size_t size,
+				  uint32_t access, uint32_t *id)
+{
+	struct devx_obj_handle *umem_hdl;
+	int w_access = DEVX_UMEM_ACCESS_READ;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	if (access)
+		w_access |= DEVX_UMEM_ACCESS_WRITE;
+
+	umem_hdl = devx_umem_reg(GET_DEVX_CTX(ctx), addr,
+					size, w_access, id);
+	if (DEVX_IS_ERR(umem_hdl)) {
+		errno = -DEVX_PTR_ERR(umem_hdl);
+		return NULL;
+	}
+	return umem_hdl;
+}
+
+static void *
+mlx5_glue_devx_alloc_uar(void *ctx,
+		uint32_t flags)
+{
+	devx_uar_handle *uar;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	uar = devx_alloc_uar(GET_DEVX_CTX(ctx), flags);
+	if (DEVX_IS_ERR(uar)) {
+		errno = -DEVX_PTR_ERR(uar);
+		return NULL;
+	}
+	return uar;
+}
+
+static int
+mlx5_glue_devx_query_eqn(void *ctx,
+		uint32_t cpus, uint32_t *eqn)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_query_eqn(GET_DEVX_CTX(ctx), cpus, eqn);
+}
+
+static void
+mlx5_glue_devx_free_uar(void *uar)
+{
+	devx_free_uar((devx_uar_handle *)uar);
+}
+
+static void*
+mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
+
+{
+	struct devx_obj_handle *rule_hdl = NULL;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	rule_hdl = devx_fs_rule_add(GET_DEVX_CTX(ctx), in, inlen);
+	if (DEVX_IS_ERR(rule_hdl)) {
+		errno = -DEVX_PTR_ERR(rule_hdl);
+		return NULL;
+	}
+	return rule_hdl;
+}
+
+static int
+mlx5_glue_devx_fs_rule_del(void *flow)
+{
+	return devx_fs_rule_del(flow);
+}
+
+alignas(RTE_CACHE_LINE_SIZE)
+const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+	.version = MLX5_GLUE_VERSION,
+	.get_device_list = mlx5_glue_devx_get_device_list,
+	.free_device_list = mlx5_glue_devx_free_device_list,
+	.open_device = mlx5_glue_devx_open_device,
+	.close_device = mlx5_glue_devx_close_device,
+	.query_device = mlx5_glue_devx_query_device,
+	.query_hca_iseg = mlx5_glue_devx_query_hca_iseg_mapping,
+	.devx_obj_create = mlx5_glue_devx_obj_create,
+	.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
+	.devx_obj_query = mlx5_glue_devx_obj_query,
+	.devx_obj_modify = mlx5_glue_devx_obj_modify,
+	.devx_general_cmd = mlx5_glue_devx_general_cmd,
+	.devx_umem_reg = mlx5_glue_devx_umem_reg,
+	.devx_umem_dereg = mlx5_glue_devx_umem_dereg,
+	.devx_alloc_uar = mlx5_glue_devx_alloc_uar,
+	.devx_free_uar = mlx5_glue_devx_free_uar,
+	.devx_fs_rule_add = mlx5_glue_devx_fs_rule_add,
+	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
+	.devx_query_eqn = mlx5_glue_devx_query_eqn,
+};
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
new file mode 100644
index 0000000000..f2261ec7ac
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_GLUE_H_
+#define MLX5_GLUE_H_
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include <rte_byteorder.h>
+#include <mlx5_win_defs.h>
+
+#ifndef MLX5_GLUE_VERSION
+#define MLX5_GLUE_VERSION ""
+#endif
+
+/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
+struct mlx5_glue {
+	const char *version;
+	void *(*devx_obj_create)(void *ctx,
+				 void *in, size_t inlen,
+				 void *out, size_t outlen);
+	int (*devx_obj_destroy)(void *obj);
+	int (*devx_obj_query)(void *obj,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen);
+	int (*devx_obj_modify)(void *obj,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_general_cmd)(void *ctx,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_umem_dereg)(void *umem);
+	void *(*devx_umem_reg)(void *ctx,
+			void *addr, size_t size,
+			uint32_t access, uint32_t *id);
+	void *(*devx_alloc_uar)(void *ctx,
+			uint32_t flags);
+	void (*devx_free_uar)(void *uar);
+	void *(*get_device_list)(int *num_devices);
+	void (*free_device_list)(void *list);
+	void *(*open_device)(void *device);
+	int (*close_device)(void *ctx);
+	int (*query_device)(void *device_bdf, void *dev_inf);
+	void* (*query_hca_iseg)(void *ctx, uint32_t *cb_iseg);
+	int (*devx_obj_query_async)(void *obj,
+				    const void *in, size_t inlen,
+				    size_t outlen, uint64_t wr_id,
+				    void *cmd_comp);
+	void *(*devx_fs_rule_add)(void *ctx, void *in, uint32_t inlen);
+	int (*devx_fs_rule_del)(void *flow);
+	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
+};
+
+extern const struct mlx5_glue *mlx5_glue;
+
+#endif /* MLX5_GLUE_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
new file mode 100644
index 0000000000..72a3131f5e
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_DEFS_H__
+#define __MLX5_WIN_DEFS_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum {
+	MLX5_CQE_OWNER_MASK	= 1,
+	MLX5_CQE_REQ		= 0,
+	MLX5_CQE_RESP_WR_IMM	= 1,
+	MLX5_CQE_RESP_SEND	= 2,
+	MLX5_CQE_RESP_SEND_IMM	= 3,
+	MLX5_CQE_RESP_SEND_INV	= 4,
+	MLX5_CQE_RESIZE_CQ	= 5,
+	MLX5_CQE_NO_PACKET	= 6,
+	MLX5_CQE_REQ_ERR	= 13,
+	MLX5_CQE_RESP_ERR	= 14,
+	MLX5_CQE_INVALID	= 15,
+};
+#endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
new file mode 100644
index 0000000000..0e74910e9d
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_ETX_H__
+#define __MLX5_WIN_ETX_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "mlx5devx.h"
+
+typedef struct mlx5_context {
+	devx_device_ctx        *devx_ctx;
+	struct devx_device mlx5_dev;
+
+} mlx5_context_st;
+
+typedef struct {
+	devx_device_ctx *devx_ctx;
+	struct devx_obj_handle *obj;
+} mlx5_devx_obj_st;
+
+struct mlx5_devx_umem {
+	void                   *addr;
+	struct devx_obj_handle *umem_hdl;
+	uint32_t                umem_id;
+};
+
+#define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
+#define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
+
+#endif /* __MLX5_WIN_ETX_H__ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 20/33] mlx5/windows: add mlx5 meson file

[Tal Shnaiderman]

File drivers/common/mlx5/windows/meson.build is added to enable mlx5
source files compilation under common windows directory.
A Devx SDK tool must be installed to export two external H files:
mlx5devx.h and mlx5_ifc_devx.h. The installation is based on environment
variable DEVX_INC_PATH. In addition a DLL lib file is installed based on
environment variable DEVX_LIB_PATH. The meson file is using the
environment variables for compilation and linkage.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/meson.build         |  4 ++--
 drivers/common/mlx5/windows/meson.build | 42 +++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 2 deletions(-)
 create mode 100644 drivers/common/mlx5/windows/meson.build

diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build
index 3dacc6f683..3047b455cf 100644
--- a/drivers/common/mlx5/meson.build
+++ b/drivers/common/mlx5/meson.build
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2019 Mellanox Technologies, Ltd
 
-if not is_linux
+if not (is_linux or (is_windows and is_ms_linker))
 	build = false
-	reason = 'only supported on Linux'
+	reason = 'only supported on Linux and Windows build with clang'
 	subdir_done()
 endif
 
diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2b2aa3b097
--- /dev/null
+++ b/drivers/common/mlx5/windows/meson.build
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2019 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+
+allow_experimental_apis = true
+sources += files(
+	'mlx5_glue.c',
+)
+
+res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
+res_inc = run_command(python3, '-c', 'import os; print(os.environ["DEVX_INC_PATH"])')
+
+if (res_lib.returncode() != 0 or res_inc.returncode() != 0)
+	build = false
+	reason = 'DevX environment variables are not set, DEVX_LIB_PATH and DEVX_INC_PATH vars must be exported'
+	subdir_done()
+endif
+
+devx_lib_dir = res_lib.stdout().strip()
+devx_inc_dir = res_inc.stdout().strip()
+
+ext_deps += cc.find_library('mlx5devx', dirs: devx_lib_dir, required: true)
+includes += include_directories(devx_inc_dir)
+cflags_options = [
+	'-std=c11',
+	'-Wno-strict-prototypes',
+	'-D_BSD_SOURCE',
+	'-D_DEFAULT_SOURCE',
+	'-D_XOPEN_SOURCE=600'
+]
+foreach option:cflags_options
+	if cc.has_argument(option)
+		cflags += option
+	endif
+endforeach
+if get_option('buildtype').contains('debug')
+	cflags += [ '-pedantic', '-DPEDANTIC' ]
+else
+	cflags += [ '-UPEDANTIC' ]
+endif
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 21/33] mlx5/windows: add initialization routine for external lib

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add function mlx5_glue_constructor() as an initialization routine for
run-time dependency on external lib.  Currently the routine has an empty
body. It is used for compatibility with Linux.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build      |  1 +
 drivers/common/mlx5/windows/mlx5_common_os.c | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 2b2aa3b097..9e74ebdea3 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -6,6 +6,7 @@ includes += include_directories('.')
 allow_experimental_apis = true
 sources += files(
 	'mlx5_glue.c',
+	'mlx5_common_os.c',
 )
 
 res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
new file mode 100644
index 0000000000..5707fb60c3
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <unistd.h>
+#include <string.h>
+#include <stdio.h>
+
+#include <rte_errno.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "mlx5_devx_cmds.h"
+#include "mlx5_common_utils.h"
+#include <mlx5_common.h>
+
+/**
+ * Initialization routine for run-time dependency on external lib
+ */
+void
+mlx5_glue_constructor(void)
+{
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 22/33] mlx5/windows: generate file mlx5_autoconf.h

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

File mlx5_autoconf.h is generated under Windows to maintain
compatibility with the Linux build system.  This file is included in
Linux/Windows shared source files therefore it is required. Currently
the file is created empty.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 9e74ebdea3..5ac5724b40 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -41,3 +41,6 @@ else
 	cflags += [ '-UPEDANTIC' ]
 endif
 
+# Generate an empty mlx5_autoconf.h file for compatibility with Linux
+config = configuration_data()
+configure_file(output : 'mlx5_autoconf.h', configuration : config)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 23/33] common/mlx5/windows: extend PRM match_param_bits struct

[Tal Shnaiderman]

Add reserved size to PRM 'struct mlx5_ifc_fte_match_param_bits' for non
Linux OS.  Windows drivers require this extension since their expected
size should match the actual struct size. Linux drivers do not require
this extension and already use calculations based on the shorter size.
Use a static assert to verify that the PRM fte_match_param struct size
remains correct.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h          | 7 +++++++
 drivers/common/mlx5/windows/mlx5_glue.c | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 4ddf865743..27a4e4f0f1 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -815,6 +815,13 @@ struct mlx5_ifc_fte_match_param_bits {
 	struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
 	struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
 	struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
+/*
+ * Add reserved bit to match the struct size with the size defined in PRM.
+ * This extension is not required in Linux.
+ */
+#ifndef HAVE_INFINIBAND_VERBS_H
+	u8 reserved_0[0x400];
+#endif
 };
 
 enum {
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 7f8a00aaa7..3896cf0932 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -256,6 +256,9 @@ mlx5_glue_devx_free_uar(void *uar)
 	devx_free_uar((devx_uar_handle *)uar);
 }
 
+static_assert(MLX5_ST_SZ_BYTES(fte_match_param) == 0x200,
+	"PRM size of fte_match_param is broken! cannot compile Windows!");
+
 static void*
 mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 24/33] common/mlx5/windows: add getter functions

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add file mlx5/windows/mlx5_common_os.h the equivalent of Linux file
mlx5/linux/mlx5_common_os.h. It contains getters functions
mlx5_os_get_dev_device_name, mlx5_os_get_ctx_device_name,
mlx5_os_get_ctx_device_path, mlx5_os_get_umem_id,
mlx5_os_get_devx_channel_fd.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 91 ++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index cce7c88c6a..2abdb2c3ae 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -48,4 +48,95 @@ mlx5_os_free(void *addr)
 {
 	_aligned_free(addr);
 }
+
+/**
+ * Get fd. Given a pointer to DevX channel object of type
+ * 'struct mlx5dv_devx_event_channel*' - return its fd.
+ * Under Windows it is a stub.
+ *
+ * @param[in] channel
+ *    Pointer to channel object.
+ *
+ * @return
+ *    0
+ */
+static inline int
+mlx5_os_get_devx_channel_fd(void *channel)
+{
+	if (!channel)
+		return 0;
+	return 0;
+}
+
+/**
+ * Get device name. Given a device pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] dev
+ *   Pointer to device.
+ *
+ * @return
+ *   Pointer to device name if dev is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_dev_device_name(void *dev)
+{
+	if (!dev)
+		return NULL;
+	return ((struct devx_device *)dev)->name;
+}
+
+/**
+ * Get device name. Given a context pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device name if ctx is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_ctx_device_name(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.name;
+}
+
+/**
+ * Get a device path name. Given acontext pointer - return a
+ * pointer to the corresponding device path name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device path name if ctx is valid, NULL otherwise.
+ */
+
+static inline const char *
+mlx5_os_get_ctx_device_path(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.dev_pnp_id;
+}
+
+/**
+ * Get umem id. Given a pointer to umem object of type return its id.
+ *
+ * @param[in] umem
+ *    Pointer to umem object.
+ *
+ * @return
+ *    The umem id if umem is valid, 0 otherwise.
+ */
+static inline uint32_t
+mlx5_os_get_umem_id(void *umem)
+{
+	if (!umem)
+		return 0;
+	return ((struct mlx5_devx_umem *)umem)->umem_id;
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 25/33] common/mlx5/windows: add OS alloc/dealloc pd

[Tal Shnaiderman]

Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd(). They
are equivalent to the Linux implementation in [1].

[1]
("net/mlx5: wrap glue alloc/dealloc PD with OS calls")

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
 drivers/common/mlx5/windows/mlx5_common_os.c    | 48 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 2f6aab133b..5db1eaa48c 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -61,4 +61,5 @@ EXPORTS
 	mlx5_malloc
 	mlx5_realloc
 	mlx5_free
-
+	mlx5_os_alloc_pd
+	mlx5_os_dealloc_pd
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 5707fb60c3..85537b6739 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -22,3 +22,51 @@ void
 mlx5_glue_constructor(void)
 {
 }
+
+/**
+ * Allocate PD. Given a devx context object
+ * return an mlx5-pd object.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *    The mlx5_pd if pd is valid, NULL and errno otherwise.
+ */
+void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
+		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
+	if (!ppd)
+		return NULL;
+
+	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
+	if (!obj) {
+		mlx5_free(ppd);
+		return NULL;
+	}
+	ppd->obj = obj;
+	ppd->pdn = obj->id;
+	ppd->devx_ctx = ctx;
+	return ppd;
+}
+
+/**
+ * Release PD. Releases a given mlx5_pd object
+ *
+ * @param[in] pd
+ *   Pointer to mlx5_pd.
+ *
+ * @return
+ *    Zero if pd is released successfully, negative number otherwise.
+ */
+int
+mlx5_os_dealloc_pd(void *pd)
+{
+	if (!pd)
+		return -EINVAL;
+	mlx5_devx_cmd_destroy(((struct mlx5_pd *)pd)->obj);
+	mlx5_free(pd);
+	return 0;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index 2abdb2c3ae..f47351ec41 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -139,4 +139,7 @@ mlx5_os_get_umem_id(void *umem)
 		return 0;
 	return ((struct mlx5_devx_umem *)umem)->umem_id;
 }
+
+void *mlx5_os_alloc_pd(void *ctx);
+int mlx5_os_dealloc_pd(void *pd);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index 0e74910e9d..8e697b30c1 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -28,6 +28,12 @@ struct mlx5_devx_umem {
 	uint32_t                umem_id;
 };
 
+struct mlx5_pd {
+	void                   *obj;
+	uint32_t                pdn;
+	devx_device_ctx        *devx_ctx;
+};
+
 #define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
 #define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 26/33] common/mlx5/windows: add OS umem reg/dereg API

[Tal Shnaiderman]

Implement Windows API mlx5_os_umem_reg() and mlx5_os_umem_dereg(). They
are equivalent to the Linux implementation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 60 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  2 +
 3 files changed, 64 insertions(+)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 5db1eaa48c..1eabe29a58 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,3 +63,5 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_umem_reg
+	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 85537b6739..cbb09906bd 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -70,3 +70,63 @@ mlx5_os_dealloc_pd(void *pd)
 	mlx5_free(pd);
 	return 0;
 }
+
+/**
+ * Register umem.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ * @param[in] addr
+ *   Pointer to memory start address.
+ * @param[in] size
+ *   Size of the memory to register.
+ * @param[out] access
+ *   UMEM access type
+ *
+ * @return
+ *   umem on successful registration, NULL and errno otherwise
+ */
+void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	struct mlx5_devx_umem *umem;
+
+	umem = mlx5_malloc(MLX5_MEM_ZERO,
+		(sizeof(*umem)), 0, SOCKET_ID_ANY);
+	if (!umem) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	umem->umem_hdl = mlx5_glue->devx_umem_reg(ctx, addr, size, access,
+		&umem->umem_id);
+	if (!umem->umem_hdl) {
+		mlx5_free(umem);
+		return NULL;
+	}
+	umem->addr = addr;
+	return umem;
+}
+
+/**
+ * Deregister umem.
+ *
+ * @param[in] pumem
+ *   Pointer to umem.
+ *
+ * @return
+ *   0 on successful release, negative number otherwise
+ */
+int
+mlx5_os_umem_dereg(void *pumem)
+{
+	struct mlx5_devx_umem *umem;
+	int err = 0;
+
+	if (!pumem)
+		return err;
+	umem = pumem;
+	if (umem->umem_hdl)
+		err = mlx5_glue->devx_umem_dereg(umem->umem_hdl);
+	mlx5_free(umem);
+	return err;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index f47351ec41..decb5acd45 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -142,4 +142,6 @@ mlx5_os_get_umem_id(void *umem)
 
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
+void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
+int mlx5_os_umem_dereg(void *pumem);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 27/33] net/mlx5: update MR prototypes for DevX

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well.  Rename 'struct mlx5_verbs_ops' as 'struct mlx5_mr_ops' and
move it to shared file mlx5.h.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c    | 4 ++--
 drivers/net/mlx5/linux/mlx5_verbs.c | 2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h | 8 +-------
 drivers/net/mlx5/mlx5.h             | 6 ++++++
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a6c1eba640..4f68c74267 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -2478,8 +2478,8 @@ void
 mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
 		      mlx5_dereg_mr_t *dereg_mr_cb)
 {
-	*reg_mr_cb = mlx5_verbs_ops.reg_mr;
-	*dereg_mr_cb = mlx5_verbs_ops.dereg_mr;
+	*reg_mr_cb = mlx5_mr_verbs_ops.reg_mr;
+	*dereg_mr_cb = mlx5_mr_verbs_ops.dereg_mr;
 }
 
 /**
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c
index 9161fa3b7d..b52ae2e6c1 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.c
+++ b/drivers/net/mlx5/linux/mlx5_verbs.c
@@ -62,7 +62,7 @@ mlx5_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
 }
 
 /* verbs operations. */
-const struct mlx5_verbs_ops mlx5_verbs_ops = {
+const struct mlx5_mr_ops mlx5_mr_verbs_ops = {
 	.reg_mr = mlx5_reg_mr,
 	.dereg_mr = mlx5_dereg_mr,
 };
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.h b/drivers/net/mlx5/linux/mlx5_verbs.h
index 0670f6c47e..76a79bf4f4 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.h
+++ b/drivers/net/mlx5/linux/mlx5_verbs.h
@@ -7,16 +7,10 @@
 
 #include "mlx5.h"
 
-struct mlx5_verbs_ops {
-	mlx5_reg_mr_t reg_mr;
-	mlx5_dereg_mr_t dereg_mr;
-};
-
 int mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
 
 /* Verbs ops struct */
-extern const struct mlx5_verbs_ops mlx5_verbs_ops;
+extern const struct mlx5_mr_ops mlx5_mr_verbs_ops;
 extern struct mlx5_obj_ops ibv_obj_ops;
-
 #endif /* RTE_PMD_MLX5_VERBS_H_ */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 2186677810..a9db597f2b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -917,6 +917,12 @@ struct mlx5_obj_ops {
 
 #define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields)
 
+/* MR operations structure. */
+struct mlx5_mr_ops {
+	mlx5_reg_mr_t reg_mr;
+	mlx5_dereg_mr_t dereg_mr;
+};
+
 struct mlx5_priv {
 	struct rte_eth_dev_data *dev_data;  /* Pointer to device data. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared device context. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 28/33] common/mlx5/windows: add OS reg/dereg MR

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commits implements Windows API for MR registration and
deregistration. It is based on DevX.  Is support the relaxed ordering
flow in Windows by checking the capabilities and machine type.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.h            |  1 +
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 78 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  7 +++
 4 files changed, 88 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index da0a0f0c79..5cc3f097c2 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -28,6 +28,7 @@ struct mlx5_pmd_mr {
 	void		     *addr;
 	size_t		     len;
 	void		     *obj;  /* verbs mr object or devx umem object. */
+	struct mlx5_devx_obj *mkey; /* devx mkey object. */
 };
 
 /**
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 1eabe29a58..65ae47ae6a 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,5 +63,7 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_dereg_mr
+	mlx5_os_reg_mr
 	mlx5_os_umem_reg
 	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index cbb09906bd..84ed561dc0 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -14,6 +14,8 @@
 #include "mlx5_devx_cmds.h"
 #include "mlx5_common_utils.h"
 #include <mlx5_common.h>
+#include "mlx5_common_os.h"
+#include "mlx5_malloc.h"
 
 /**
  * Initialization routine for run-time dependency on external lib
@@ -130,3 +132,79 @@ mlx5_os_umem_dereg(void *pumem)
 	mlx5_free(umem);
 	return err;
 }
+
+/**
+ * Register mr. Given protection doamin pointer, pointer to addr and length
+ * register the memory region.
+ *
+ * @param[in] pd
+ *   Pointer to protection domain context (type mlx5_pd).
+ * @param[in] addr
+ *   Pointer to memory start address (type devx_device_ctx).
+ * @param[in] length
+ *   Lengtoh of the memory to register.
+ * @param[out] pmd_mr
+ *   pmd_mr struct set with lkey, address, length, pointer to mr object, mkey
+ *
+ * @return
+ *   0 on successful registration, -1 otherwise
+ */
+int
+mlx5_os_reg_mr(void *pd,
+	       void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr)
+{
+	struct mlx5_devx_mkey_attr mkey_attr;
+	struct mlx5_pd *mlx5_pd = (struct mlx5_pd *)pd;
+	struct mlx5_hca_attr attr;
+
+	if (!pd || !addr) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+	if (mlx5_devx_cmd_query_hca_attr(mlx5_pd->devx_ctx, &attr))
+		return -1;
+	pmd_mr->addr = addr;
+	pmd_mr->len = length;
+	pmd_mr->obj = mlx5_os_umem_reg(mlx5_pd->devx_ctx, pmd_mr->addr,
+				       pmd_mr->len, IBV_ACCESS_LOCAL_WRITE);
+	if (!pmd_mr->obj)
+		return -1;
+	mkey_attr.addr = (uintptr_t)addr;
+	mkey_attr.size = length;
+	mkey_attr.umem_id = ((struct mlx5_devx_umem *)(pmd_mr->obj))->umem_id;
+	mkey_attr.pd = mlx5_pd->pdn;
+	mkey_attr.log_entity_size = 0;
+	mkey_attr.pg_access = 0;
+	mkey_attr.klm_array = NULL;
+	mkey_attr.klm_num = 0;
+	mkey_attr.relaxed_ordering_read = 0;
+	mkey_attr.relaxed_ordering_write = 0;
+	if (!haswell_broadwell_cpu) {
+		mkey_attr.relaxed_ordering_write = attr.relaxed_ordering_write;
+		mkey_attr.relaxed_ordering_read = attr.relaxed_ordering_read;
+	}
+	pmd_mr->mkey = mlx5_devx_cmd_mkey_create(mlx5_pd->devx_ctx, &mkey_attr);
+	if (!pmd_mr->mkey) {
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+		return -1;
+	}
+	pmd_mr->lkey = pmd_mr->mkey->id;
+	return 0;
+}
+
+/**
+ * De-register mr.
+ *
+ * @param[in] pmd_mr
+ *  Pointer to PMD mr object
+ */
+void
+mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
+{
+	if (pmd_mr && pmd_mr->mkey)
+		claim_zero(mlx5_glue->devx_obj_destroy(pmd_mr->mkey->obj));
+	if (pmd_mr && pmd_mr->obj)
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index decb5acd45..ba166412cc 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -7,9 +7,13 @@
 
 #include <stdio.h>
 
+#include <rte_errno.h>
+
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
 #include "mlx5_malloc.h"
+#include "mlx5_common_mr.h"
+#include "mlx5_win_ext.h"
 
 /**
  * This API allocates aligned or non-aligned memory.  The free can be on either
@@ -144,4 +148,7 @@ void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
 int mlx5_os_umem_dereg(void *pumem);
+int mlx5_os_reg_mr(void *pd,
+		   void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr);
+void mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 29/33] drivers/common: enable Windows common mlx5 compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Prior to this commit meson did not progress beyond directory
drivers/common for Windows compilations.  This commit enables drivers
compilation under this directory.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/meson.build | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index d78882f05e..be97424548 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -1,10 +1,8 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Cavium, Inc
 
-if is_windows
-	subdir_done()
+if not is_windows
+	std_deps = ['eal']
+	drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
 endif
-
-std_deps = ['eal']
-drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
 config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON'
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 30/33] net/mlx5/windows: implement device attribute getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows implementation of mlx5_os_get_dev_attr() API.
It follows the commit in [1]. A new file named mlx5_os.c is added under
windows directory as its Linux counterpart file: linux/mlx5_os.c.

[1].
commit e85f623e13ea ("net/mlx5: remove attributes dependency on Verbs")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h       | 43 +++++++++++++++++++++
 drivers/net/mlx5/windows/meson.build |  8 ++++
 drivers/net/mlx5/windows/mlx5_os.c   | 75 ++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 27a4e4f0f1..d62d2558eb 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -2217,6 +2217,49 @@ struct mlx5_ifc_cqc_bits {
 	u8 dbr_addr[0x40];
 };
 
+struct mlx5_ifc_health_buffer_bits {
+	u8         reserved_0[0x100];
+	u8         assert_existptr[0x20];
+	u8         assert_callra[0x20];
+	u8         reserved_1[0x40];
+	u8         fw_version[0x20];
+	u8         hw_id[0x20];
+	u8         reserved_2[0x20];
+	u8         irisc_index[0x8];
+	u8         synd[0x8];
+	u8         ext_synd[0x10];
+};
+
+struct mlx5_ifc_initial_seg_bits {
+	u8         fw_rev_minor[0x10];
+	u8         fw_rev_major[0x10];
+	u8         cmd_interface_rev[0x10];
+	u8         fw_rev_subminor[0x10];
+	u8         reserved_0[0x40];
+	u8         cmdq_phy_addr_63_32[0x20];
+	u8         cmdq_phy_addr_31_12[0x14];
+	u8         reserved_1[0x2];
+	u8         nic_interface[0x2];
+	u8         log_cmdq_size[0x4];
+	u8         log_cmdq_stride[0x4];
+	u8         command_doorbell_vector[0x20];
+	u8         reserved_2[0xf00];
+	u8         initializing[0x1];
+	u8         nic_interface_supported[0x7];
+	u8         reserved_4[0x18];
+	struct mlx5_ifc_health_buffer_bits health_buffer;
+	u8         no_dram_nic_offset[0x20];
+	u8         reserved_5[0x6de0];
+	u8         internal_timer_h[0x20];
+	u8         internal_timer_l[0x20];
+	u8         reserved_6[0x20];
+	u8         reserved_7[0x1f];
+	u8         clear_int[0x1];
+	u8         health_syndrome[0x8];
+	u8         health_counter[0x18];
+	u8         reserved_8[0x17fc0];
+};
+
 struct mlx5_ifc_create_cq_out_bits {
 	u8 status[0x8];
 	u8 reserved_at_8[0x18];
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2ea0792a40
--- /dev/null
+++ b/drivers/net/mlx5/windows/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2020 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+sources += files(
+	'mlx5_os.c',
+)
+
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
new file mode 100644
index 0000000000..e0646670a3
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <rte_windows.h>
+
+#include <mlx5_glue.h>
+#include <mlx5_devx_cmds.h>
+#include <mlx5_common.h>
+
+#include "mlx5_defs.h"
+#include "mlx5.h"
+#include "mlx5_autoconf.h"
+
+/**
+ * Get mlx5 device attributes.
+ *
+ * @param ctx
+ *   Pointer to device context.
+ *
+ * @param device_attr
+ *   Pointer to mlx5 device attributes.
+ *
+ * @return
+ *   0 on success, non zero error number otherwise
+ */
+int
+mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
+{
+	struct mlx5_context *mlx5_ctx;
+	struct mlx5_hca_attr hca_attr;
+	void *pv_iseg = NULL;
+	u32 cb_iseg = 0;
+	int err = 0;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	memset(device_attr, 0, sizeof(*device_attr));
+	err = mlx5_devx_cmd_query_hca_attr(mlx5_ctx, &hca_attr);
+	if (err) {
+		DRV_LOG(ERR, "Failed to get device hca_cap");
+		return err;
+	}
+	device_attr->max_cq = 1 << hca_attr.log_max_cq;
+	device_attr->max_qp = 1 << hca_attr.log_max_qp;
+	device_attr->max_qp_wr = 1 << hca_attr.log_max_qp_sz;
+	device_attr->max_cqe = 1 << hca_attr.log_max_cq_sz;
+	device_attr->max_mr = 1 << hca_attr.log_max_mrw_sz;
+	device_attr->max_pd = 1 << hca_attr.log_max_pd;
+	device_attr->max_srq = 1 << hca_attr.log_max_srq;
+	device_attr->max_srq_wr = 1 << hca_attr.log_max_srq_sz;
+	if (hca_attr.rss_ind_tbl_cap) {
+		device_attr->max_rwq_indirection_table_size =
+			1 << hca_attr.rss_ind_tbl_cap;
+	}
+	pv_iseg = mlx5_glue->query_hca_iseg(mlx5_ctx, &cb_iseg);
+	if (pv_iseg == NULL) {
+		DRV_LOG(ERR, "Failed to get device hca_iseg");
+		return errno;
+	}
+	if (!err) {
+		snprintf(device_attr->fw_ver, 64, "%x.%x.%04x",
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_major),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_minor),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_subminor));
+	}
+	return err;
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 31/33] net/mlx5/windows: add mlx5_os.c stubs

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_os_set_nonblock_channel_fd
mlx5_os_dev_shared_handler_install
mlx5_os_dev_shared_handler_uninstall
mlx5_os_read_dev_stat
mlx5_os_mac_addr_flush
mlx5_os_mac_addr_remove
mlx5_os_vf_mac_addr_modify
mlx5_os_set_promisc
mlx5_os_set_allmulti

Set struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops with NULL
pointers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 187 +++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index e0646670a3..bef10aba30 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -13,10 +13,18 @@
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
 #include <mlx5_common.h>
+#include <mlx5_common_mp.h>
+#include <mlx5_common_mr.h>
+#include <mlx5_malloc.h>
 
 #include "mlx5_defs.h"
 #include "mlx5.h"
+#include "mlx5_common_os.h"
+#include "mlx5_utils.h"
+#include "mlx5_rxtx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_mr.h"
+#include "mlx5_flow.h"
 
 /**
  * Get mlx5 device attributes.
@@ -73,3 +81,182 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	}
 	return err;
 }
+
+/**
+ * Set the completion channel file descriptor interrupt as non-blocking.
+ * Currently it has no support under Windows.
+ *
+ * @param[in] rxq_obj
+ *   Pointer to RQ channel object, which includes the channel fd
+ *
+ * @param[out] fd
+ *   The file descriptor (representing the intetrrupt) used in this channel.
+ *
+ * @return
+ *   0 on successfully setting the fd to non-blocking, non-zero otherwise.
+ */
+int
+mlx5_os_set_nonblock_channel_fd(int fd)
+{
+	(void)fd;
+	DRV_LOG(WARNING, "mlx5_os_set_nonblock_channel_fd is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param sh
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "mlx5_os_dev_shared_handler_install is not support");
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "mlx5_os_dev_shared_handler_uninstall is not support");
+}
+
+/**
+ * Read statistics by a named counter.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ * @param[in] ctr_name
+ *   Pointer to the name of the statistic counter to read
+ * @param[out] stat
+ *   Pointer to read statistic value.
+ * @return
+ *   0 on success and stat is valud, 1 if failed to read the value
+ *   rte_errno is set.
+ *
+ */
+int
+mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name,
+		      uint64_t *stat)
+{
+	RTE_SET_USED(priv);
+	RTE_SET_USED(ctr_name);
+	RTE_SET_USED(stat);
+	DRV_LOG(WARNING, "mlx5_os_read_dev_stat is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * Flush device MAC addresses
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ */
+void
+mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
+{
+	(void)dev;
+	DRV_LOG(WARNING, "mlx5_os_mac_addr_flush is not support");
+}
+
+/**
+ * Remove a MAC address from device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param index
+ *   MAC address index.
+ */
+void
+mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+	(void)dev;
+	(void)(index);
+	DRV_LOG(WARNING, "mlx5_os_mac_addr_remove is not support");
+}
+
+/**
+ * Modify a VF MAC address
+ * Currently it has no support under Windows.
+ *
+ * @param priv
+ *   Pointer to device private data.
+ * @param mac_addr
+ *   MAC address to modify into.
+ * @param iface_idx
+ *   Net device interface index
+ * @param vf_index
+ *   VF index
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_vf_mac_addr_modify(struct mlx5_priv *priv,
+			   unsigned int iface_idx,
+			   struct rte_ether_addr *mac_addr,
+			   int vf_index)
+{
+	(void)priv;
+	(void)iface_idx;
+	(void)mac_addr;
+	(void)vf_index;
+	DRV_LOG(WARNING, "mlx5_os_vf_mac_addr_modify is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * Set device promiscuous mode
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - promiscuous is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_promisc(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "mlx5_os_set_promisc is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * Set device allmulti mode
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - all multicase is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "mlx5_os_set_allmulti is not support");
+	return -ENOTSUP;
+}
+
+const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 32/33] net/mlx5/windows: implement mlx5 mac addr add

[Tal Shnaiderman]

Get the list of MAC addresses and verify if the input mac parameter
already exists. If not - return -ENOTSUP (as Windows does not support
adding new MAC addresses). If the MAC address exists (EEXIST) return 0
(the equivalent of Linux implementation of this API).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index bef10aba30..ccf0bc3be9 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -188,6 +188,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
 	DRV_LOG(WARNING, "mlx5_os_mac_addr_remove is not support");
 }
 
+/**
+ * Adds a MAC address to the device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param mac_addr
+ *   MAC address to register.
+ * @param index
+ *   MAC address index.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
+		     uint32_t index)
+{
+	(void)index;
+	struct rte_ether_addr lmac;
+
+	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
+		DRV_LOG(ERR,
+			"port %u cannot get MAC address, is mlx5_en"
+			" loaded? (errno: %s)",
+			dev->data->port_id, strerror(rte_errno));
+		return rte_errno;
+	}
+	if (!rte_is_same_ether_addr(&lmac, mac)) {
+		DRV_LOG(ERR,
+			"adding new mac address to device is unsupported");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
 /**
  * Modify a VF MAC address
  * Currently it has no support under Windows.
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v2 33/33] net/mlx5: refactor eth dev ops for Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

There are two types of eth_dev_ops used under Windows: primary and
isolate mode. Their function calls initialization is added to the OS
specific file mlx5_os.c. Secondary process eth_dev_ops is nullified.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 108 +++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index ccf0bc3be9..43665bff7f 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -296,3 +296,111 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
+
+const struct eth_dev_ops mlx5_os_dev_ops = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.read_clock = mlx5_read_clock,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.reta_update = mlx5_dev_rss_reta_update,
+	.reta_query = mlx5_dev_rss_reta_query,
+	.rss_hash_update = mlx5_rss_hash_update,
+	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
+
+/* Available operations from secondary process. */
+const struct eth_dev_ops mlx5_os_dev_sec_ops = {0};
+
+/* Available operations in flow isolated mode. */
+const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
-- 
2.16.1.windows.4

Re: [dpdk-dev] [PATCH v2 20/33] mlx5/windows: add mlx5 meson file

[Dmitry Kozlyuk]

On Thu, 10 Dec 2020 17:06:35 +0200, Tal Shnaiderman wrote:
> File drivers/common/mlx5/windows/meson.build is added to enable mlx5
> source files compilation under common windows directory.
> A Devx SDK tool must be installed to export two external H files:
> mlx5devx.h and mlx5_ifc_devx.h. The installation is based on environment
> variable DEVX_INC_PATH. In addition a DLL lib file is installed based on
> environment variable DEVX_LIB_PATH. The meson file is using the
> environment variables for compilation and linkage.

I installed WinOF-2 for Windows Server 2019 from this page:

https://www.mellanox.com/products/adapter-software/ethernet/windows/winof-2

Then I installed DevX SDK from C:\Program Files\Mellanox\DevX SDK
and set environment variables:

$env:DEVX_INC_PATH = "C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc"
$env:DEVX_LIB_PATH = "C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\lib"

Meson complained no more, but the build failed, log is below.
What am I missing? Note: I'm building the whole series.

Another question, why there's a clang restriction?

[149/203] Compiling C object drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_common.c.obj.
FAILED: drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_common.c.obj
clang @drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_common.c.obj.rsp
In file included from ../../drivers/common/mlx5/mlx5_common.c:13:
In file included from ..\..\drivers\common/mlx5/mlx5_common.h:18:
..\..\drivers\common/mlx5/mlx5_prm.h:363:31: error: field has incomplete type 'struct mlx5_wqe_srq_next_seg'
        struct mlx5_wqe_srq_next_seg next_seg;
                                     ^
..\..\drivers\common/mlx5/mlx5_prm.h:363:9: note: forward declaration of 'struct mlx5_wqe_srq_next_seg'
        struct mlx5_wqe_srq_next_seg next_seg;
               ^
..\..\drivers\common/mlx5/mlx5_prm.h:364:27: error: field has incomplete type 'struct mlx5_wqe_data_seg'
        struct mlx5_wqe_data_seg dseg;
                                 ^
..\..\drivers\common/mlx5/mlx5_prm.h:364:9: note: forward declaration of 'struct mlx5_wqe_data_seg'
        struct mlx5_wqe_data_seg dseg;
               ^
In file included from ../../drivers/common/mlx5/mlx5_common.c:14:
In file included from ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:15:
In file included from ..\..\drivers\common/mlx5\mlx5_common_mr.h:19:
..\..\drivers\common/mlx5/mlx5_common_mp.h:29:7: warning: forward references to 'enum' types are a Microsoft extension [-Wmicrosoft-enum-forward-reference]
        enum ibv_wq_state state; /* WQ requested state. */
             ^
In file included from ../../drivers/common/mlx5/mlx5_common.c:14:
In file included from ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:16:
In file included from ..\..\drivers\common/mlx5\windows/mlx5_win_ext.h:12:
In file included from C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc\mlx5devx.h:26:
C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:45: error: field has incomplete type 'struct mlx5_ifc_dest_format_struct_bits'
    struct mlx5_ifc_dest_format_struct_bits dest;
                                            ^
C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:12: note: forward declaration of 'struct mlx5_ifc_dest_format_struct_bits'
    struct mlx5_ifc_dest_format_struct_bits dest;
           ^
../../drivers/common/mlx5/mlx5_common.c:323:15: warning: implicit declaration of function 'mlx5_os_get_devx_uar_base_addr' is invalid in C99 [-Wimplicit-function-declaration]
                base_addr = mlx5_os_get_devx_uar_base_addr(uar);
                            ^
../../drivers/common/mlx5/mlx5_common.c:323:13: warning: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion]
                base_addr = mlx5_os_get_devx_uar_base_addr(uar);
                          ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings and 3 errors generated.
[151/203] Compiling C object drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.obj.
FAILED: drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.obj
clang @drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.obj.rsp
In file included from ../../drivers/common/mlx5/mlx5_malloc.c:11:
In file included from ..\..\drivers\common/mlx5/mlx5_common_utils.h:8:
In file included from ..\..\drivers\common/mlx5/mlx5_common.h:18:
..\..\drivers\common/mlx5/mlx5_prm.h:363:31: error: field has incomplete type 'struct mlx5_wqe_srq_next_seg'
        struct mlx5_wqe_srq_next_seg next_seg;
                                     ^
..\..\drivers\common/mlx5/mlx5_prm.h:363:9: note: forward declaration of 'struct mlx5_wqe_srq_next_seg'
        struct mlx5_wqe_srq_next_seg next_seg;
               ^
..\..\drivers\common/mlx5/mlx5_prm.h:364:27: error: field has incomplete type 'struct mlx5_wqe_data_seg'
        struct mlx5_wqe_data_seg dseg;
                                 ^
..\..\drivers\common/mlx5/mlx5_prm.h:364:9: note: forward declaration of 'struct mlx5_wqe_data_seg'
        struct mlx5_wqe_data_seg dseg;
               ^
In file included from ../../drivers/common/mlx5/mlx5_malloc.c:12:
In file included from ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:15:
In file included from ..\..\drivers\common/mlx5\mlx5_common_mr.h:19:
..\..\drivers\common/mlx5/mlx5_common_mp.h:29:7: warning: forward references to 'enum' types are a Microsoft extension [-Wmicrosoft-enum-forward-reference]
        enum ibv_wq_state state; /* WQ requested state. */
             ^
In file included from ../../drivers/common/mlx5/mlx5_malloc.c:12:
In file included from ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:16:
In file included from ..\..\drivers\common/mlx5\windows/mlx5_win_ext.h:12:
In file included from C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc\mlx5devx.h:26:
C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:45: error: field has incomplete type 'struct mlx5_ifc_dest_format_struct_bits'
    struct mlx5_ifc_dest_format_struct_bits dest;
                                            ^
C:\Program Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:12: note: forward declaration of 'struct mlx5_ifc_dest_format_struct_bits'
    struct mlx5_ifc_dest_format_struct_bits dest;
           ^
1 warning and 3 errors generated.
[154/203] Compiling C object lib/76b5a35@@rte_hash@sta/librte_hash_rte_cuckoo_hash.c.obj.
ninja: build stopped: subcommand failed.

Re: [dpdk-dev] [PATCH v2 20/33] mlx5/windows: add mlx5 meson file

[Tal Shnaiderman]

> Subject: Re: [dpdk-dev] [PATCH v2 20/33] mlx5/windows: add mlx5 meson
> file
> 
> External email: Use caution opening links or attachments
> 
> 
> On Thu, 10 Dec 2020 17:06:35 +0200, Tal Shnaiderman wrote:
> > File drivers/common/mlx5/windows/meson.build is added to enable mlx5
> > source files compilation under common windows directory.
> > A Devx SDK tool must be installed to export two external H files:
> > mlx5devx.h and mlx5_ifc_devx.h. The installation is based on
> > environment variable DEVX_INC_PATH. In addition a DLL lib file is
> > installed based on environment variable DEVX_LIB_PATH. The meson file
> > is using the environment variables for compilation and linkage.
> 
> I installed WinOF-2 for Windows Server 2019 from this page:
> 
> https://www.mellanox.com/products/adapter-
> software/ethernet/windows/winof-2
> 
> Then I installed DevX SDK from C:\Program Files\Mellanox\DevX SDK and set
> environment variables:
> 
> $env:DEVX_INC_PATH = "C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc"
> $env:DEVX_LIB_PATH = "C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\lib"
> 
> Meson complained no more, but the build failed, log is below.
> What am I missing? Note: I'm building the whole series.

Thanks for testing Dmitry, this is an patches order issue on my side, the common
build patch should come in the next series.

I'll fix in v3.

> 
> Another question, why there's a clang restriction?
> 

On the current release we've ported and tested the PMD and SDK for clang, once we'll do the same for MinGW64 we'll remove this restriction. 

> [149/203] Compiling C object
> drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_common
> .c.obj.
> FAILED:
> drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_common
> .c.obj
> clang
> @drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_comm
> on.c.obj.rsp
> In file included from ../../drivers/common/mlx5/mlx5_common.c:13:
> In file included from ..\..\drivers\common/mlx5/mlx5_common.h:18:
> ..\..\drivers\common/mlx5/mlx5_prm.h:363:31: error: field has incomplete
> type 'struct mlx5_wqe_srq_next_seg'
>         struct mlx5_wqe_srq_next_seg next_seg;
>                                      ^
> ..\..\drivers\common/mlx5/mlx5_prm.h:363:9: note: forward declaration of
> 'struct mlx5_wqe_srq_next_seg'
>         struct mlx5_wqe_srq_next_seg next_seg;
>                ^
> ..\..\drivers\common/mlx5/mlx5_prm.h:364:27: error: field has incomplete
> type 'struct mlx5_wqe_data_seg'
>         struct mlx5_wqe_data_seg dseg;
>                                  ^
> ..\..\drivers\common/mlx5/mlx5_prm.h:364:9: note: forward declaration of
> 'struct mlx5_wqe_data_seg'
>         struct mlx5_wqe_data_seg dseg;
>                ^
> In file included from ../../drivers/common/mlx5/mlx5_common.c:14:
> In file included from
> ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:15:
> In file included from ..\..\drivers\common/mlx5\mlx5_common_mr.h:19:
> ..\..\drivers\common/mlx5/mlx5_common_mp.h:29:7: warning: forward
> references to 'enum' types are a Microsoft extension [-Wmicrosoft-enum-
> forward-reference]
>         enum ibv_wq_state state; /* WQ requested state. */
>              ^
> In file included from ../../drivers/common/mlx5/mlx5_common.c:14:
> In file included from
> ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:16:
> In file included from
> ..\..\drivers\common/mlx5\windows/mlx5_win_ext.h:12:
> In file included from C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc\mlx5devx.h:26:
> C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:45: error:
> field has incomplete type 'struct mlx5_ifc_dest_format_struct_bits'
>     struct mlx5_ifc_dest_format_struct_bits dest;
>                                             ^ C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:12: note:
> forward declaration of 'struct mlx5_ifc_dest_format_struct_bits'
>     struct mlx5_ifc_dest_format_struct_bits dest;
>            ^
> ../../drivers/common/mlx5/mlx5_common.c:323:15: warning: implicit
> declaration of function 'mlx5_os_get_devx_uar_base_addr' is invalid in C99
> [-Wimplicit-function-declaration]
>                 base_addr = mlx5_os_get_devx_uar_base_addr(uar);
>                             ^
> ../../drivers/common/mlx5/mlx5_common.c:323:13: warning: incompatible
> integer to pointer conversion assigning to 'void *' from 'int' [-Wint-
> conversion]
>                 base_addr = mlx5_os_get_devx_uar_base_addr(uar);
>                           ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 3 warnings and 3 errors generated.
> [151/203] Compiling C object
> drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.
> obj.
> FAILED:
> drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_malloc.c.
> obj
> clang
> @drivers/a715181@@rte_common_mlx5@sta/common_mlx5_mlx5_malloc.
> c.obj.rsp
> In file included from ../../drivers/common/mlx5/mlx5_malloc.c:11:
> In file included from ..\..\drivers\common/mlx5/mlx5_common_utils.h:8:
> In file included from ..\..\drivers\common/mlx5/mlx5_common.h:18:
> ..\..\drivers\common/mlx5/mlx5_prm.h:363:31: error: field has incomplete
> type 'struct mlx5_wqe_srq_next_seg'
>         struct mlx5_wqe_srq_next_seg next_seg;
>                                      ^
> ..\..\drivers\common/mlx5/mlx5_prm.h:363:9: note: forward declaration of
> 'struct mlx5_wqe_srq_next_seg'
>         struct mlx5_wqe_srq_next_seg next_seg;
>                ^
> ..\..\drivers\common/mlx5/mlx5_prm.h:364:27: error: field has incomplete
> type 'struct mlx5_wqe_data_seg'
>         struct mlx5_wqe_data_seg dseg;
>                                  ^
> ..\..\drivers\common/mlx5/mlx5_prm.h:364:9: note: forward declaration of
> 'struct mlx5_wqe_data_seg'
>         struct mlx5_wqe_data_seg dseg;
>                ^
> In file included from ../../drivers/common/mlx5/mlx5_malloc.c:12:
> In file included from
> ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:15:
> In file included from ..\..\drivers\common/mlx5\mlx5_common_mr.h:19:
> ..\..\drivers\common/mlx5/mlx5_common_mp.h:29:7: warning: forward
> references to 'enum' types are a Microsoft extension [-Wmicrosoft-enum-
> forward-reference]
>         enum ibv_wq_state state; /* WQ requested state. */
>              ^
> In file included from ../../drivers/common/mlx5/mlx5_malloc.c:12:
> In file included from
> ..\..\drivers\common/mlx5\windows\mlx5_common_os.h:16:
> In file included from
> ..\..\drivers\common/mlx5\windows/mlx5_win_ext.h:12:
> In file included from C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc\mlx5devx.h:26:
> C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:45: error:
> field has incomplete type 'struct mlx5_ifc_dest_format_struct_bits'
>     struct mlx5_ifc_dest_format_struct_bits dest;
>                                             ^ C:\Program
> Files\Mellanox\MLNX_WinOF2_DevX_SDK\inc/mlx5_ifc_devx.h:95:12: note:
> forward declaration of 'struct mlx5_ifc_dest_format_struct_bits'
>     struct mlx5_ifc_dest_format_struct_bits dest;
>            ^
> 1 warning and 3 errors generated.
> [154/203] Compiling C object
> lib/76b5a35@@rte_hash@sta/librte_hash_rte_cuckoo_hash.c.obj.
> ninja: build stopped: subcommand failed.

[dpdk-dev] [PATCH v3 00/32] mlx5 Windows support - part #5

[Tal Shnaiderman]

This patch series is part of preparing mlx5 PMD to compile and run under Windows OS. Part #5

v1: initial version
v2: Fixes after CR and rebase, also patchset was split and part of the commits sent in v1 will be sent later on.
v3: Remove common compilation as it depends on upcoming patches [DmitryK].

Ophir Munk (20):
  net/mlx5/linux: extend device attributes getter
  net/mlx5: remove Linux files from Windows compilation
  net/mlx5: fix freeing packet pacing
  net/mlx5: replace Linux sleep with rte sleep
  net/mlx5: define mprq functions as static inline
  net/mlx5: do not define static_assert in Windows
  net/mlx5: move static_assert calls to global scope
  net/mlx5: wrap glue alloc/dealloc PD with OS calls
  net/mlx5: fix adding destroy flow action wrapper
  common/mlx5: add definition HAVE_INFINIBAND_VERBS_H
  common/mlx5/linux: handle memory allocations with alignment
  common/mlx5/windows: handle memory allocations with alignment
  mlx5/windows: add initialization routine for external lib
  mlx5/windows: generate file mlx5_autoconf.h
  common/mlx5/windows: add getter functions
  net/mlx5: update MR prototypes for DevX
  common/mlx5/windows: add OS reg/dereg MR
  net/mlx5/windows: implement device attribute getter
  net/mlx5/windows: add mlx5_os.c stubs
  net/mlx5: refactor eth dev ops for Windows

Tal Shnaiderman (12):
  net/mlx5: fix folding constant array error
  net/mlx5: wrap glue reg/dereg UMEM with OS calls
  common/mlx5/linux: wrap event channel APIs with OS calls
  common/mlx5: add Windows exports file
  common/mlx5: extend DevX query hca attributes command
  common/mlx5: add DevX alloc PD command
  common/mlx5/windows: add glue functions APIs
  mlx5/windows: add mlx5 meson file
  common/mlx5/windows: extend PRM match_param_bits struct
  common/mlx5/windows: add OS alloc/dealloc pd
  common/mlx5/windows: add OS umem reg/dereg API
  net/mlx5/windows: implement mlx5 mac addr add

 drivers/common/mlx5/linux/meson.build           |   6 +-
 drivers/common/mlx5/linux/mlx5_common_os.h      |  83 +++++
 drivers/common/mlx5/mlx5_common.c               |   4 +-
 drivers/common/mlx5/mlx5_common_mr.h            |   1 +
 drivers/common/mlx5/mlx5_devx_cmds.c            |  46 +++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  12 +-
 drivers/common/mlx5/mlx5_malloc.c               |  14 +-
 drivers/common/mlx5/mlx5_prm.h                  |  87 +++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  69 ++++
 drivers/common/mlx5/version.map                 |   1 +
 drivers/common/mlx5/windows/meson.build         |  46 +++
 drivers/common/mlx5/windows/mlx5_common_os.c    | 210 ++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    | 154 +++++++++
 drivers/common/mlx5/windows/mlx5_glue.c         | 307 ++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 ++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  40 +++
 drivers/net/mlx5/linux/mlx5_os.c                |   9 +-
 drivers/net/mlx5/linux/mlx5_verbs.c             |   2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h             |   8 +-
 drivers/net/mlx5/meson.build                    |  16 +-
 drivers/net/mlx5/mlx5.c                         |  18 +-
 drivers/net/mlx5/mlx5.h                         |  11 +
 drivers/net/mlx5/mlx5_defs.h                    |   7 +-
 drivers/net/mlx5/mlx5_devx.c                    |  24 +-
 drivers/net/mlx5/mlx5_flow.c                    |  17 +-
 drivers/net/mlx5/mlx5_flow_dv.c                 |   6 +-
 drivers/net/mlx5/mlx5_rxq.c                     |  71 -----
 drivers/net/mlx5/mlx5_rxtx.c                    |  98 +++---
 drivers/net/mlx5/mlx5_rxtx.h                    |  73 ++++-
 drivers/net/mlx5/mlx5_trigger.c                 |   3 +-
 drivers/net/mlx5/mlx5_txpp.c                    |  34 +-
 drivers/net/mlx5/windows/meson.build            |   8 +
 drivers/net/mlx5/windows/mlx5_os.c              | 406 ++++++++++++++++++++++++
 drivers/vdpa/mlx5/mlx5_vdpa_event.c             |   7 +-
 35 files changed, 1778 insertions(+), 203 deletions(-)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def
 create mode 100644 drivers/common/mlx5/windows/meson.build
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 01/32] net/mlx5: fix folding constant array error

[Tal Shnaiderman]

Before this commit the PMD used:
   const int elt_n = 8
   const int *stack[elt_n];

In Windows clang compiler complains:
net/mlx5/mlx5_flow.c:215:19: error: variable length array folded
to constant array as an extension [-Werror,-Wgnu-folding-constant]

Fix it by using a constant macro definition instead of a variable:
   #define MLX5_RSS_EXP_ELT_N 8
   const int *stack[MLX5_RSS_EXP_ELT_N];

Fixes: c7870bfe09dc ("ethdev: move RSS expansion code to mlx5 driver")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 82e24d7067..bf86aaaa39 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -212,6 +212,8 @@ mlx5_flow_expand_rss_item_complete(const struct rte_flow_item *item)
 	return ret;
 }
 
+#define MLX5_RSS_EXP_ELT_N 8
+
 /**
  * Expand RSS flows into several possible flows according to the RSS hash
  * fields requested and the driver capabilities.
@@ -242,13 +244,12 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		     const struct mlx5_flow_expand_node graph[],
 		     int graph_root_index)
 {
-	const int elt_n = 8;
 	const struct rte_flow_item *item;
 	const struct mlx5_flow_expand_node *node = &graph[graph_root_index];
 	const int *next_node;
-	const int *stack[elt_n];
+	const int *stack[MLX5_RSS_EXP_ELT_N];
 	int stack_pos = 0;
-	struct rte_flow_item flow_items[elt_n];
+	struct rte_flow_item flow_items[MLX5_RSS_EXP_ELT_N];
 	unsigned int i;
 	size_t lsize;
 	size_t user_pattern_size = 0;
@@ -261,10 +262,10 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 
 	memset(&missed_item, 0, sizeof(missed_item));
 	lsize = offsetof(struct mlx5_flow_expand_rss, entry) +
-		elt_n * sizeof(buf->entry[0]);
+		MLX5_RSS_EXP_ELT_N * sizeof(buf->entry[0]);
 	if (lsize <= size) {
 		buf->entry[0].priority = 0;
-		buf->entry[0].pattern = (void *)&buf->entry[elt_n];
+		buf->entry[0].pattern = (void *)&buf->entry[MLX5_RSS_EXP_ELT_N];
 		buf->entries = 0;
 		addr = buf->entry[0].pattern;
 	}
@@ -367,7 +368,7 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		/* Go deeper. */
 		if (node->next) {
 			next_node = node->next;
-			if (stack_pos++ == elt_n) {
+			if (stack_pos++ == MLX5_RSS_EXP_ELT_N) {
 				rte_errno = E2BIG;
 				return -rte_errno;
 			}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 02/32] net/mlx5/linux: extend device attributes getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 5 +++++
 drivers/net/mlx5/mlx5.h          | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 917d6be7b8..a6c1eba640 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -116,7 +116,12 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	device_attr->max_qp_wr = attr_ex.orig_attr.max_qp_wr;
 	device_attr->max_sge = attr_ex.orig_attr.max_sge;
 	device_attr->max_cq = attr_ex.orig_attr.max_cq;
+	device_attr->max_cqe = attr_ex.orig_attr.max_cqe;
+	device_attr->max_mr = attr_ex.orig_attr.max_mr;
+	device_attr->max_pd = attr_ex.orig_attr.max_pd;
 	device_attr->max_qp = attr_ex.orig_attr.max_qp;
+	device_attr->max_srq = attr_ex.orig_attr.max_srq;
+	device_attr->max_srq_wr = attr_ex.orig_attr.max_srq_wr;
 	device_attr->raw_packet_caps = attr_ex.raw_packet_caps;
 	device_attr->max_rwq_indirection_table_size =
 		attr_ex.rss_caps.max_rwq_indirection_table_size;
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 121d726405..2186677810 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -83,6 +83,11 @@ struct mlx5_dev_attr {
 	int		max_sge;
 	int		max_cq;
 	int		max_qp;
+	int		max_cqe;
+	uint32_t	max_pd;
+	uint32_t	max_mr;
+	uint32_t	max_srq;
+	uint32_t	max_srq_wr;
 	uint32_t	raw_packet_caps;
 	uint32_t	max_rwq_indirection_table_size;
 	uint32_t	max_tso;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 03/32] net/mlx5: remove Linux files from Windows compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit removes Linux files flow_verbs.c and mlx5_rxtx_vec.c
from Windows compilation.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/meson.build | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index e7495a764a..f2fafbdd05 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -15,7 +15,6 @@ sources = files(
 	'mlx5_flow.c',
 	'mlx5_flow_meter.c',
 	'mlx5_flow_dv.c',
-	'mlx5_flow_verbs.c',
         'mlx5_flow_age.c',
 	'mlx5_mac.c',
 	'mlx5_mr.c',
@@ -31,11 +30,18 @@ sources = files(
 	'mlx5_utils.c',
 	'mlx5_devx.c',
 )
-if (dpdk_conf.has('RTE_ARCH_X86_64')
-	or dpdk_conf.has('RTE_ARCH_ARM64')
-	or dpdk_conf.has('RTE_ARCH_PPC_64'))
-	sources += files('mlx5_rxtx_vec.c')
+
+if is_linux
+	sources += files(
+		'mlx5_flow_verbs.c',
+	)
+	if (dpdk_conf.has('RTE_ARCH_X86_64')
+		or dpdk_conf.has('RTE_ARCH_ARM64')
+		or dpdk_conf.has('RTE_ARCH_PPC_64'))
+		sources += files('mlx5_rxtx_vec.c')
+	endif
 endif
+
 cflags_options = [
 	'-std=c11',
 	'-Wno-strict-prototypes',
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 04/32] net/mlx5: fix freeing packet pacing

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Packet pacing is allocated under condition #ifdef HAVE_MLX5DV_PP_ALLOC.
In a similar way - free packet pacing index under the same condition.
This update is required to successfully compile under operating systems
which do not support packet pacing.

Fixes: aef1e20ebeb2 ("net/mlx5: allocate packet pacing context")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_txpp.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 2438bf1f1d..21675ab17a 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -57,11 +57,16 @@ mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 static void
 mlx5_txpp_free_pp_index(struct mlx5_dev_ctx_shared *sh)
 {
+#ifdef HAVE_MLX5DV_PP_ALLOC
 	if (sh->txpp.pp) {
 		mlx5_glue->dv_free_pp(sh->txpp.pp);
 		sh->txpp.pp = NULL;
 		sh->txpp.pp_id = 0;
 	}
+#else
+	RTE_SET_USED(sh);
+	DRV_LOG(ERR, "Freeing pacing index is not supported.");
+#endif
 }
 
 /* Allocate Packet Pacing index from kernel via mlx5dv call. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 05/32] net/mlx5: replace Linux sleep with rte sleep

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Replace Linux API usleep() and nanosleep() with rte_delay_us_sleep().
The replacement occurs in shared files compiled under different
operating systems.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_trigger.c | 3 ++-
 drivers/net/mlx5/mlx5_txpp.c    | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 52a8a252d4..8e9df6abcf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -21,6 +21,7 @@
 #include <rte_spinlock.h>
 #include <rte_string_fns.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
@@ -1330,7 +1331,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_flex_parser_ecpri_release(dev);
 	if (priv->rxqs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
@@ -1338,7 +1339,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->txqs_n); ++i)
 			mlx5_txq_release(dev, i);
 		priv->txqs_n = 0;
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index bd029154f8..646f29b923 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -9,6 +9,7 @@
 #include <rte_ethdev_driver.h>
 #include <rte_interrupts.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_malloc.h>
 
@@ -1171,7 +1172,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	rte_wmb();
 	/* Disable datapath on secondary process. */
 	mlx5_mp_os_req_stop_rxtx(dev);
-	usleep(1000 * priv->rxqs_n);
+	rte_delay_us_sleep(1000 * priv->rxqs_n);
 	DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id);
 	mlx5_flow_stop_default(dev);
 	/* Control flows for default traffic can be removed firstly. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 21675ab17a..a1ec294942 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -769,15 +769,11 @@ mlx5_txpp_init_timestamp(struct mlx5_dev_ctx_shared *sh)
 	sh->txpp.ts_p = 0;
 	sh->txpp.ts_n = 0;
 	for (wait = 0; wait < MLX5_TXPP_WAIT_INIT_TS; wait++) {
-		struct timespec onems;
-
 		mlx5_txpp_update_timestamp(sh);
 		if (wq->sq_ci)
 			return;
 		/* Wait one millisecond and try again. */
-		onems.tv_sec = 0;
-		onems.tv_nsec = NS_PER_S / MS_PER_S;
-		nanosleep(&onems, 0);
+		rte_delay_us_sleep(US_PER_S / MS_PER_S);
 	}
 	DRV_LOG(ERR, "Unable to initialize timestamp.");
 	sh->txpp.sync_lost = 1;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 06/32] net/mlx5: define mprq functions as static inline

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Functions mlx5_check_mprq_support(), mlx5_rxq_mprq_enabled(),
mlx5_mprq_enabled() are moved from source file mlx5_rxq.c to header file
mlx5_rxtx.h and their type is updated to 'static __rte_always_inline'.
Previously the functions were declared as 'inline' in the source file
which was reported as 'unresolved external symbol' error by some Windows
linkers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxq.c  | 71 ------------------------------------------
 drivers/net/mlx5/mlx5_rxtx.h | 73 ++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 70 insertions(+), 74 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index da7a8b3cd7..c89e55b1c0 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -48,77 +48,6 @@ static_assert(MLX5_RSS_HASH_KEY_LEN ==
 	      (unsigned int)sizeof(rss_hash_default_key),
 	      "wrong RSS default key size.");
 
-/**
- * Check whether Multi-Packet RQ can be enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   1 if supported, negative errno value if not.
- */
-inline int
-mlx5_check_mprq_support(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	if (priv->config.mprq.enabled &&
-	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
-		return 1;
-	return -ENOTSUP;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the Rx queue.
- *
- *  @param rxq
- *     Pointer to receive queue structure.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
-{
-	return rxq->strd_num_n > 0;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_mprq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	uint32_t i;
-	uint16_t n = 0;
-	uint16_t n_ibv = 0;
-
-	if (mlx5_check_mprq_support(dev) < 0)
-		return 0;
-	/* All the configured queues should be enabled. */
-	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
-
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
-			continue;
-		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
-			++n;
-	}
-	/* Multi-Packet RQ can't be partially configured. */
-	MLX5_ASSERT(n == 0 || n == n_ibv);
-	return n == n_ibv;
-}
-
 /**
  * Calculate the number of CQEs in CQ for the Rx queue.
  *
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 7989a50403..1e9345af61 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -311,9 +311,6 @@ struct mlx5_txq_ctrl {
 
 extern uint8_t rss_hash_default_key[];
 
-int mlx5_check_mprq_support(struct rte_eth_dev *dev);
-int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq);
-int mlx5_mprq_enabled(struct rte_eth_dev *dev);
 unsigned int mlx5_rxq_cqe_num(struct mlx5_rxq_data *rxq_data);
 int mlx5_mprq_free_mp(struct rte_eth_dev *dev);
 int mlx5_mprq_alloc_mp(struct rte_eth_dev *dev);
@@ -920,4 +917,74 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Multi-Packet RQ can be enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   1 if supported, negative errno value if not.
+ */
+static __rte_always_inline int
+mlx5_check_mprq_support(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	if (priv->config.mprq.enabled &&
+	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
+		return 1;
+	return -ENOTSUP;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the Rx queue.
+ *
+ *  @param rxq
+ *     Pointer to receive queue structure.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
+{
+	return rxq->strd_num_n > 0;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_mprq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint32_t i;
+	uint16_t n = 0;
+	uint16_t n_ibv = 0;
+
+	if (mlx5_check_mprq_support(dev) < 0)
+		return 0;
+	/* All the configured queues should be enabled. */
+	for (i = 0; i < priv->rxqs_n; ++i) {
+		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
+			(rxq, struct mlx5_rxq_ctrl, rxq);
+
+		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+			continue;
+		n_ibv++;
+		if (mlx5_rxq_mprq_enabled(rxq))
+			++n;
+	}
+	/* Multi-Packet RQ can't be partially configured. */
+	MLX5_ASSERT(n == 0 || n == n_ibv);
+	return n == n_ibv;
+}
 #endif /* RTE_PMD_MLX5_RXTX_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 07/32] net/mlx5: do not define static_assert in Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

In Linux 'static_assert' is defined in file mlx5_defs.h:

 #ifndef HAVE_STATIC_ASSERT
    #define static_assert _Static_assert
 #endif

The same definition can originate from Linux file /usr/include/assert.h.

In Windows static_assert is used while _Static_assert is unknown.
Therefore update the definition condition to be:

 #if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
    #define static_assert _Static_assert
 #endif

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_defs.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index aa55db3750..85a0979653 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -199,8 +199,11 @@
 /* Maximum number of shared actions supported by rte_flow */
 #define MLX5_MAX_SHARED_ACTIONS 2
 
-/* Definition of static_assert found in /usr/include/assert.h */
-#ifndef HAVE_STATIC_ASSERT
+/*
+ * Linux definition of static_assert is found in /usr/include/assert.h.
+ * Windows does not require a redefinition.
+ */
+#if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
 #define static_assert _Static_assert
 #endif
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 08/32] net/mlx5: move static_assert calls to global scope

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Some Windows compilers consider static_assert() as calls to another
function rather than a compiler directive which allows checking type
information at compile time.  This only occurs if the static_assert call
appears inside another function scope. To solve it move the
static_assert calls to global scope in the files where they are used.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxtx.c | 98 ++++++++++++++++++++++----------------------
 drivers/net/mlx5/mlx5_txpp.c |  5 ++-
 2 files changed, 53 insertions(+), 50 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index d12d746c2f..65a1f997e9 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -79,6 +79,56 @@ static uint16_t mlx5_tx_burst_##func(void *txq, \
 
 #define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx},
 
+/* static asserts */
+static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
+static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
+		(2 * RTE_ETHER_ADDR_LEN),
+		"invalid Data Segment data size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert((sizeof(struct rte_vlan_hdr) +
+			sizeof(struct rte_ether_hdr)) ==
+		MLX5_ESEG_MIN_INLINE_SIZE,
+		"invalid min inline data size");
+static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
+		MLX5_DSEG_MAX, "invalid WQE max size");
+static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Control Segment size");
+static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Ethernet Segment size");
+static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Data Segment size");
+static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
+		"invalid WQE size");
+
 static __rte_always_inline uint32_t
 rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe,
 				   volatile struct mlx5_mini_cqe8 *mcqe);
@@ -2070,8 +2120,6 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq,
 	bool ring_doorbell = false;
 	int ret;
 
-	static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
-	static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
 	do {
 		volatile struct mlx5_cqe *cqe;
 
@@ -2381,15 +2429,6 @@ mlx5_tx_eseg_dmin(struct mlx5_txq_data *__rte_restrict txq __rte_unused,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = RTE_BE16(MLX5_ESEG_MIN_INLINE_SIZE);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2474,15 +2513,6 @@ mlx5_tx_eseg_data(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = rte_cpu_to_be_16(inlen);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2697,15 +2727,6 @@ mlx5_tx_eseg_mdat(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	MLX5_ASSERT(inlen >= MLX5_ESEG_MIN_INLINE_SIZE);
 	pdst = (uint8_t *)&es->inline_data;
 	if (MLX5_TXOFF_CONFIG(VLAN) && vlan) {
@@ -2952,9 +2973,6 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *__rte_restrict txq,
 	uint8_t *pdst;
 
 	MLX5_ASSERT(len > MLX5_ESEG_MIN_INLINE_SIZE);
-	static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
-				 (2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Data Segment data size");
 	if (!MLX5_TXOFF_CONFIG(MPW)) {
 		/* Store the descriptor byte counter for eMPW sessions. */
 		dseg->bcount = rte_cpu_to_be_32
@@ -4070,7 +4088,6 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4247,7 +4264,6 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4561,10 +4577,6 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq,
 			    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
 				vlan = sizeof(struct rte_vlan_hdr);
 				inlen += vlan;
-				static_assert((sizeof(struct rte_vlan_hdr) +
-					       sizeof(struct rte_ether_hdr)) ==
-					       MLX5_ESEG_MIN_INLINE_SIZE,
-					       "invalid min inline data size");
 			}
 			/*
 			 * If inlining is enabled at configuration time
@@ -5567,16 +5579,6 @@ mlx5_select_tx_function(struct rte_eth_dev *dev)
 	uint64_t tx_offloads = dev->data->dev_conf.txmode.offloads;
 	unsigned int diff = 0, olx = 0, i, m;
 
-	static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
-		      MLX5_DSEG_MAX, "invalid WQE max size");
-	static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Control Segment size");
-	static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Ethernet Segment size");
-	static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Data Segment size");
-	static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
-		      "invalid WQE size");
 	MLX5_ASSERT(priv);
 	if (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) {
 		/* We should support Multi-Segment Packets. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index a1ec294942..d61e43e55d 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -18,6 +18,9 @@
 #include "mlx5_rxtx.h"
 #include "mlx5_common_os.h"
 
+static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
+		"Wrong timestamp CQE part size");
+
 static const char * const mlx5_txpp_stat_names[] = {
 	"tx_pp_missed_interrupt_errors", /* Missed service interrupt. */
 	"tx_pp_rearm_queue_errors", /* Rearm Queue errors. */
@@ -741,8 +744,6 @@ mlx5_txpp_update_timestamp(struct mlx5_dev_ctx_shared *sh)
 	uint64_t ts;
 	uint16_t ci;
 
-	static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
-		      "Wrong timestamp CQE part size");
 	mlx5_atomic_read_cqe((rte_int128_t *)&cqe->timestamp, &to.u128);
 	if (to.cts.op_own >> 4) {
 		DRV_LOG(DEBUG, "Clock Queue error sync lost.");
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 09/32] net/mlx5: wrap glue alloc/dealloc PD with OS calls

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Wrap glue calls alloc_pd() and dealloc_pd() with generic OS calls.  In
Linux - protection domain allocations are implemented by Verbs glue API
while in Windows it is by DevX API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 11 +++++++++++
 drivers/net/mlx5/mlx5.c                    |  6 +++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index 3420937859..c9f8d7cbc1 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -201,4 +201,15 @@ mlx5_os_get_devx_uar_page_id(void *uar)
 #endif
 }
 
+static inline void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	return mlx5_glue->alloc_pd(ctx);
+}
+
+static inline int
+mlx5_os_dealloc_pd(void *pd)
+{
+	return mlx5_glue->dealloc_pd(pd);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 8e9df6abcf..7d1b4fbdc0 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -932,7 +932,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sh->port[i].ih_port_id = RTE_MAX_ETHPORTS;
 		sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS;
 	}
-	sh->pd = mlx5_glue->alloc_pd(sh->ctx);
+	sh->pd = mlx5_os_alloc_pd(sh->ctx);
 	if (sh->pd == NULL) {
 		DRV_LOG(ERR, "PD allocation failure");
 		err = ENOMEM;
@@ -1032,7 +1032,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	if (sh->tx_uar)
 		mlx5_glue->devx_free_uar(sh->tx_uar);
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->ctx)
 		claim_zero(mlx5_glue->close_device(sh->ctx));
 	mlx5_free(sh);
@@ -1100,7 +1100,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		sh->tx_uar = NULL;
 	}
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->tis)
 		claim_zero(mlx5_devx_cmd_destroy(sh->tis));
 	if (sh->td)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 10/32] net/mlx5: wrap glue reg/dereg UMEM with OS calls

[Tal Shnaiderman]

Wrap glue calls for UMEM registration and deregistration with generic OS
calls since each OS (Linux or Windows) has a different glue API
parameters.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 12 ++++++++++++
 drivers/common/mlx5/mlx5_common.c          |  4 ++--
 drivers/net/mlx5/mlx5.c                    |  2 +-
 drivers/net/mlx5/mlx5_devx.c               | 16 ++++++++--------
 drivers/net/mlx5/mlx5_flow.c               |  4 ++--
 drivers/net/mlx5/mlx5_txpp.c               | 12 ++++++------
 6 files changed, 31 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index c9f8d7cbc1..f8b215cc29 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -212,4 +212,16 @@ mlx5_os_dealloc_pd(void *pd)
 {
 	return mlx5_glue->dealloc_pd(pd);
 }
+
+static inline void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	return mlx5_glue->devx_umem_reg(ctx, addr, size, access);
+}
+
+static inline int
+mlx5_os_umem_dereg(void *pumem)
+{
+	return mlx5_glue->devx_umem_dereg(pumem);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c
index 044513223c..a00ffcb833 100644
--- a/drivers/common/mlx5/mlx5_common.c
+++ b/drivers/common/mlx5/mlx5_common.c
@@ -148,7 +148,7 @@ mlx5_alloc_dbr_page(void *ctx)
 		return NULL;
 	}
 	/* Register allocated memory. */
-	page->umem = mlx5_glue->devx_umem_reg(ctx, page->dbrs,
+	page->umem = mlx5_os_umem_reg(ctx, page->dbrs,
 					      MLX5_DBR_PAGE_SIZE, 0);
 	if (!page->umem) {
 		DRV_LOG(ERR, "cannot umem reg dbr page");
@@ -232,7 +232,7 @@ mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id,
 		/* Page not used, free it and remove from list. */
 		LIST_REMOVE(page, next);
 		if (page->umem)
-			ret = -mlx5_glue->devx_umem_dereg(page->umem);
+			ret = -mlx5_os_umem_dereg(page->umem);
 		mlx5_free(page);
 	} else {
 		/* Mark in bitmap that this door-bell is not in use. */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 7d1b4fbdc0..84123f8e3d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -482,7 +482,7 @@ mlx5_flow_destroy_counter_stat_mem_mng(struct mlx5_counter_stats_mem_mng *mng)
 
 	LIST_REMOVE(mng, next);
 	claim_zero(mlx5_devx_cmd_destroy(mng->dm));
-	claim_zero(mlx5_glue->devx_umem_dereg(mng->umem));
+	claim_zero(mlx5_os_umem_dereg(mng->umem));
 	mlx5_free(mem);
 }
 
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index de9b204075..235fd5798d 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -155,7 +155,7 @@ mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->rq_dbrec_page;
 
 	if (rxq_ctrl->wq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->wq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->wq_umem);
 		rxq_ctrl->wq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.wqes) {
@@ -182,7 +182,7 @@ mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->cq_dbrec_page;
 
 	if (rxq_ctrl->cq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->cq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->cq_umem);
 		rxq_ctrl->cq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.cqes) {
@@ -375,7 +375,7 @@ mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	if (!buf)
 		return NULL;
 	rxq_data->wqes = buf;
-	rxq_ctrl->wq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	rxq_ctrl->wq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						     buf, wq_size, 0);
 	if (!rxq_ctrl->wq_umem)
 		goto error;
@@ -497,7 +497,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	rxq_data->cqes = (volatile struct mlx5_cqe (*)[])(uintptr_t)buf;
-	rxq_ctrl->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx, buf,
+	rxq_ctrl->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, buf,
 						     cq_size,
 						     IBV_ACCESS_LOCAL_WRITE);
 	if (!rxq_ctrl->cq_umem) {
@@ -1127,7 +1127,7 @@ mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 		txq_obj->sq_devx = NULL;
 	}
 	if (txq_obj->sq_umem) {
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->sq_umem));
 		txq_obj->sq_umem = NULL;
 	}
 	if (txq_obj->sq_buf) {
@@ -1155,7 +1155,7 @@ mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
 	if (txq_obj->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->cq_umem));
 	if (txq_obj->cq_buf)
 		mlx5_free(txq_obj->cq_buf);
 	if (txq_obj->cq_dbrec_page)
@@ -1243,7 +1243,7 @@ mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		return 0;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	txq_obj->cq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						(void *)txq_obj->cq_buf,
 						cqe_n * sizeof(struct mlx5_cqe),
 						IBV_ACCESS_LOCAL_WRITE);
@@ -1342,7 +1342,7 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->sq_umem = mlx5_glue->devx_umem_reg
+	txq_obj->sq_umem = mlx5_os_umem_reg
 					(priv->sh->ctx,
 					 (void *)txq_obj->sq_buf,
 					 wqe_n * sizeof(struct mlx5_wqe),
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bf86aaaa39..66491bbf78 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -6446,7 +6446,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	}
 	mem_mng = (struct mlx5_counter_stats_mem_mng *)(mem + size) - 1;
 	size = sizeof(*raw_data) * MLX5_COUNTERS_PER_POOL * raws_n;
-	mem_mng->umem = mlx5_glue->devx_umem_reg(sh->ctx, mem, size,
+	mem_mng->umem = mlx5_os_umem_reg(sh->ctx, mem, size,
 						 IBV_ACCESS_LOCAL_WRITE);
 	if (!mem_mng->umem) {
 		rte_errno = errno;
@@ -6465,7 +6465,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	mkey_attr.relaxed_ordering_read = sh->cmng.relaxed_ordering_read;
 	mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr);
 	if (!mem_mng->dm) {
-		mlx5_glue->devx_umem_dereg(mem_mng->umem);
+		mlx5_os_umem_dereg(mem_mng->umem);
 		rte_errno = errno;
 		mlx5_free(mem);
 		return -rte_errno;
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index d61e43e55d..749529c410 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -131,13 +131,13 @@ mlx5_txpp_destroy_send_queue(struct mlx5_txpp_wq *wq)
 	if (wq->sq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->sq));
 	if (wq->sq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->sq_umem));
 	if (wq->sq_buf)
 		mlx5_free((void *)(uintptr_t)wq->sq_buf);
 	if (wq->cq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->cq));
 	if (wq->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->cq_umem));
 	if (wq->cq_buf)
 		mlx5_free((void *)(uintptr_t)wq->cq_buf);
 	memset(wq, 0, sizeof(*wq));
@@ -268,7 +268,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -318,7 +318,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -506,7 +506,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -562,7 +562,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 11/32] net/mlx5: fix adding destroy flow action wrapper

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Glue function destroy_flow_action() was wrapped by OS specific operation
mlx5_flow_os_destroy_flow_action(). It was skipped in file mlx5.c.

Fixes: b293fbf9672b ("net/mlx5: add OS specific flow actions operations")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_flow_dv.c | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 84123f8e3d..ea5cf80ac1 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -38,6 +38,7 @@
 #include "mlx5_autoconf.h"
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
+#include "mlx5_flow_os.h"
 #include "rte_pmd_mlx5.h"
 
 /* Device parameter to enable RX completion queue compression. */
@@ -415,7 +416,7 @@ mlx5_flow_aso_age_mng_close(struct mlx5_dev_ctx_shared *sh)
 			for (j = 0; j < MLX5_COUNTERS_PER_POOL; ++j)
 				if (pool->actions[j].dr_action)
 					claim_zero
-						(mlx5_glue->destroy_flow_action
+						(mlx5_flow_os_destroy_flow_action
 						  (pool->actions[j].dr_action));
 			mlx5_free(pool);
 		}
@@ -523,7 +524,7 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh)
 
 				if (cnt->action)
 					claim_zero
-					 (mlx5_glue->destroy_flow_action
+					 (mlx5_flow_os_destroy_flow_action
 					  (cnt->action));
 				if (fallback && MLX5_POOL_GET_CNT
 				    (pool, j)->dcs_when_free)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index c31737652c..78b5a6b338 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -11117,11 +11117,11 @@ flow_dv_sample_remove_cb(struct mlx5_cache_list *list __rte_unused,
 	struct mlx5_priv *priv = dev->data->dev_private;
 
 	if (cache_resource->verbs_action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 				(cache_resource->verbs_action));
 	if (cache_resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB) {
 		if (cache_resource->default_miss)
-			claim_zero(mlx5_glue->destroy_flow_action
+			claim_zero(mlx5_flow_os_destroy_flow_action
 			  (cache_resource->default_miss));
 	}
 	if (cache_resource->normal_path_tbl)
@@ -11174,7 +11174,7 @@ flow_dv_dest_array_remove_cb(struct mlx5_cache_list *list __rte_unused,
 
 	MLX5_ASSERT(cache_resource->action);
 	if (cache_resource->action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 					(cache_resource->action));
 	for (; i < cache_resource->num_of_dest; i++)
 		flow_dv_sample_sub_actions_release(dev,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 12/32] common/mlx5: add definition HAVE_INFINIBAND_VERBS_H

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add a Verbs file presence indication. Under Linux it is required that
file infiniband/verbs.h is installed for building DPDK.  Other
operating systems (e.g. Windows) ignore Verbs completely.  This commit
adds definition HAVE_INFINIBAND_VERBS_H (file mlx5_autoconf.h) to
indicate whether DPDK compiles with Verbs or not.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/meson.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build
index 63b78e4bce..580419e6d9 100644
--- a/drivers/common/mlx5/linux/meson.build
+++ b/drivers/common/mlx5/linux/meson.build
@@ -181,8 +181,10 @@ has_sym_args = [
 	[ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h',
 	'mlx5dv_dr_action_create_dest_array'],
 	[ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ],
-        [ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
-        'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
+	'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_INFINIBAND_VERBS_H', 'infiniband/verbs.h',
+	'INFINIBAND_VERBS_H' ],
 ]
 config = configuration_data()
 foreach arg:has_sym_args
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 13/32] common/mlx5/linux: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_malloc() API has an alignment parameter for system memory
allocations. malloc() is called for non-aligned allocations and
posix_memalign() is called for aligned allocations. When calling
mlx5_free() there is no distinction whether the memory was originally
allocated with or without alignment. Freeing a memory may be handled
differently by operating systems. Therefore this commit wraps these APIs
with OS specific calls: mlx5_os_malloc(), mlx5_os_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 38 ++++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_malloc.c          | 14 +++++------
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index f8b215cc29..bd44ecb7d6 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -6,6 +6,7 @@
 #define RTE_PMD_MLX5_COMMON_OS_H_
 
 #include <stdio.h>
+#include <malloc.h>
 
 #include <rte_pci.h>
 #include <rte_debug.h>
@@ -16,6 +17,7 @@
 
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
+#include "mlx5_malloc.h"
 
 /**
  * Get device name. Given an ibv_device pointer - return a
@@ -224,4 +226,40 @@ mlx5_os_umem_dereg(void *pumem)
 {
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
+
+/**
+ * Memory allocation optionally with alignment.
+ *
+ * @param[in] align
+ *    Alignment size (may be zero)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	void *buf;
+
+	if (posix_memalign(&buf, align, size))
+		return NULL;
+	return buf;
+}
+
+/**
+ * This API de-allocates a memory that originally could have been
+ * allocated aligned or non-aligned. In Linux it is a wrapper
+ * around free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	free(addr);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_malloc.c b/drivers/common/mlx5/mlx5_malloc.c
index 5a3267f730..9d30cedbaa 100644
--- a/drivers/common/mlx5/mlx5_malloc.c
+++ b/drivers/common/mlx5/mlx5_malloc.c
@@ -9,6 +9,7 @@
 #include <string.h>
 
 #include "mlx5_common_utils.h"
+#include "mlx5_common_os.h"
 #include "mlx5_malloc.h"
 
 struct mlx5_sys_mem {
@@ -148,14 +149,11 @@ static void *
 mlx5_alloc_align(size_t size, unsigned int align, unsigned int zero)
 {
 	void *buf;
-	int ret;
-
-	ret = posix_memalign(&buf, align, size);
-	if (ret) {
-		DRV_LOG(ERR,
-			"Couldn't allocate buf size=%zu align=%u. Err=%d\n",
-			size, align, ret);
 
+	buf = mlx5_os_malloc(align, size);
+	if (!buf) {
+		DRV_LOG(ERR, "Couldn't allocate buf size=%zu align=%u.",
+			size, align);
 		return NULL;
 	}
 	if (zero)
@@ -264,7 +262,7 @@ mlx5_free(void *addr)
 		__atomic_add_fetch(&mlx5_sys_mem.free_sys, 1,
 				   __ATOMIC_RELAXED);
 #endif
-		free(addr);
+		mlx5_os_free(addr);
 	} else {
 #ifdef RTE_LIBRTE_MLX5_DEBUG
 		__atomic_add_fetch(&mlx5_sys_mem.free_rte, 1,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 14/32] common/mlx5/windows: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows equivalent of the Linux implementation.  The
APIs included in this commit: mlx5_os_malloc(), mlx5_os_free(). For
memory allocations (with or without alignment) we always call
_aligned_malloc(). Even if zero alignment was requested in the first
place - we always select a minimal alignment value. In this way when the
memory is free - it is always safe to call _aligned_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 51 ++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
new file mode 100644
index 0000000000..cce7c88c6a
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_COMMON_OS_H_
+#define RTE_PMD_MLX5_COMMON_OS_H_
+
+#include <stdio.h>
+
+#include "mlx5_autoconf.h"
+#include "mlx5_glue.h"
+#include "mlx5_malloc.h"
+
+/**
+ * This API allocates aligned or non-aligned memory.  The free can be on either
+ * aligned or nonaligned memory.  To be protected - even though there may be no
+ * alignment - in Windows this API will unconditioanlly call _aligned_malloc()
+ * with at least a minimal alignment size.
+ *
+ * @param[in] align
+ *    The alignment value, which must be an integer power of 2 (or 0 for
+ *    non-alignment)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	if (align < MLX5_MALLOC_ALIGNMENT)
+		align = MLX5_MALLOC_ALIGNMENT;
+	return _aligned_malloc(size, align);
+}
+
+/**
+ * This API de-allocates a memory that originally could have been allocated
+ * aligned or non-aligned. In Windows since the allocation was with
+ * _aligned_malloc() - it is safe to always call _aligned_free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	_aligned_free(addr);
+}
+#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 15/32] common/mlx5/linux: wrap event channel APIs with OS calls

[Tal Shnaiderman]

Wrap the API to create/destroy event channel and to subscribe an event
with OS calls. In Linux those calls are implemented by glue functions
while in Windows they are not supported.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 22 ++++++++++++++++++++++
 drivers/net/mlx5/mlx5_devx.c               |  8 ++++----
 drivers/net/mlx5/mlx5_txpp.c               |  6 +++---
 drivers/vdpa/mlx5/mlx5_vdpa_event.c        |  7 ++++---
 4 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index bd44ecb7d6..63f070d9c4 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -227,6 +227,28 @@ mlx5_os_umem_dereg(void *pumem)
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
 
+static inline void *
+mlx5_os_devx_create_event_channel(void *ctx, int flags)
+{
+	return mlx5_glue->devx_create_event_channel(ctx, flags);
+}
+
+static inline void
+mlx5_os_devx_destroy_event_channel(void *eventc)
+{
+	mlx5_glue->devx_destroy_event_channel(eventc);
+}
+
+static inline int
+mlx5_os_devx_subscribe_devx_event(void *eventc,
+				  void *obj,
+				  uint16_t events_sz, uint16_t events_num[],
+				  uint64_t cookie)
+{
+	return mlx5_glue->devx_subscribe_devx_event(eventc, obj, events_sz,
+						    events_num, cookie);
+}
+
 /**
  * Memory allocation optionally with alignment.
  *
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 235fd5798d..84a5c55ee0 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -216,7 +216,7 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_obj *rxq_obj)
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->devx_cq));
 		if (rxq_obj->devx_channel)
-			mlx5_glue->devx_destroy_event_channel
+			mlx5_os_devx_destroy_event_channel
 							(rxq_obj->devx_channel);
 		mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
 		mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
@@ -533,7 +533,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cqe_n = log_cqe_n;
 	rxq_data->cqn = cq_obj->id;
 	if (rxq_ctrl->obj->devx_channel) {
-		ret = mlx5_glue->devx_subscribe_devx_event
+		ret = mlx5_os_devx_subscribe_devx_event
 						(rxq_ctrl->obj->devx_channel,
 						 cq_obj->obj,
 						 sizeof(event_nums),
@@ -644,7 +644,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		int devx_ev_flag =
 			  MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA;
 
-		tmpl->devx_channel = mlx5_glue->devx_create_event_channel
+		tmpl->devx_channel = mlx5_os_devx_create_event_channel
 								(priv->sh->ctx,
 								 devx_ev_flag);
 		if (!tmpl->devx_channel) {
@@ -686,7 +686,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	if (tmpl->devx_cq)
 		claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq));
 	if (tmpl->devx_channel)
-		mlx5_glue->devx_destroy_event_channel(tmpl->devx_channel);
+		mlx5_os_devx_destroy_event_channel(tmpl->devx_channel);
 	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
 	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	rte_errno = ret; /* Restore rte_errno. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 749529c410..726bdc6ae1 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -37,7 +37,7 @@ static void
 mlx5_txpp_destroy_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	if (sh->txpp.echan) {
-		mlx5_glue->devx_destroy_event_channel(sh->txpp.echan);
+		mlx5_os_devx_destroy_event_channel(sh->txpp.echan);
 		sh->txpp.echan = NULL;
 	}
 }
@@ -47,7 +47,7 @@ static int
 mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	MLX5_ASSERT(!sh->txpp.echan);
-	sh->txpp.echan = mlx5_glue->devx_create_event_channel(sh->ctx,
+	sh->txpp.echan = mlx5_os_devx_create_event_channel(sh->ctx,
 			MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!sh->txpp.echan) {
 		rte_errno = errno;
@@ -937,7 +937,7 @@ mlx5_txpp_start_service(struct mlx5_dev_ctx_shared *sh)
 		return -rte_errno;
 	}
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(sh->txpp.echan,
+	ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan,
 						   sh->txpp.rearm_queue.cq->obj,
 						   sizeof(event_nums),
 						   event_nums, 0);
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
index 3aeaeb893f..fd6150928b 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
@@ -15,6 +15,7 @@
 #include <rte_alarm.h>
 
 #include <mlx5_common.h>
+#include <mlx5_common_os.h>
 #include <mlx5_glue.h>
 
 #include "mlx5_vdpa_utils.h"
@@ -43,7 +44,7 @@ mlx5_vdpa_event_qp_global_release(struct mlx5_vdpa_priv *priv)
 		       sizeof(out.buf)) >=
 		       (ssize_t)sizeof(out.event_resp.cookie))
 			;
-		mlx5_glue->devx_destroy_event_channel(priv->eventc);
+		mlx5_os_devx_destroy_event_channel(priv->eventc);
 		priv->eventc = NULL;
 	}
 #endif
@@ -63,7 +64,7 @@ mlx5_vdpa_event_qp_global_prepare(struct mlx5_vdpa_priv *priv)
 		DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno);
 		return -1;
 	}
-	priv->eventc = mlx5_glue->devx_create_event_channel(priv->ctx,
+	priv->eventc = mlx5_os_devx_create_event_channel(priv->ctx,
 			   MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!priv->eventc) {
 		rte_errno = errno;
@@ -176,7 +177,7 @@ mlx5_vdpa_cq_create(struct mlx5_vdpa_priv *priv, uint16_t log_desc_n,
 	cq->cq_ci = 0;
 	rte_spinlock_init(&cq->sl);
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
+	ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
 						   sizeof(event_nums),
 						   event_nums,
 						   (uint64_t)(uintptr_t)cq);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 16/32] common/mlx5: add Windows exports file

[Tal Shnaiderman]

File drivers/common/mlx5/rte_common_mlx5_exports.def contains mlx5
Windows exported symbols under common/mlx5 directory (DLL file
name librte_common_mlx5*.dll). It is the equivalent of Linux map
file rte_common_mlx5_version.map but the list of symbols may be
different between the two operating systems.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def | 62 +++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
new file mode 100644
index 0000000000..23984b7abd
--- /dev/null
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -0,0 +1,62 @@
+EXPORTS
+	haswell_broadwell_cpu
+
+	mlx5_common_init
+
+	mlx5_create_mr_ext
+
+	mlx5_devx_cmd_create_cq
+	mlx5_devx_cmd_create_flex_parser
+	mlx5_devx_cmd_create_qp
+	mlx5_devx_cmd_create_rq
+	mlx5_devx_cmd_create_rqt
+	mlx5_devx_cmd_create_sq
+	mlx5_devx_cmd_create_tir
+	mlx5_devx_cmd_create_td
+	mlx5_devx_cmd_create_tis
+	mlx5_devx_cmd_create_virtq
+	mlx5_devx_cmd_destroy
+	mlx5_devx_cmd_flow_counter_alloc
+	mlx5_devx_cmd_flow_counter_query
+	mlx5_devx_cmd_flow_dump
+	mlx5_devx_cmd_mkey_create
+	mlx5_devx_cmd_modify_qp_state
+	mlx5_devx_cmd_modify_rq
+	mlx5_devx_cmd_modify_rqt
+	mlx5_devx_cmd_modify_sq
+	mlx5_devx_cmd_modify_tir
+	mlx5_devx_cmd_modify_virtq
+	mlx5_devx_cmd_qp_query_tis_td
+	mlx5_devx_cmd_query_hca_attr
+	mlx5_devx_cmd_query_parse_samples
+	mlx5_devx_cmd_query_virtq
+	mlx5_devx_cmd_register_read
+	mlx5_devx_get_out_command_status
+	mlx5_devx_cmd_create_flow_hit_aso_obj
+
+	mlx5_get_dbr
+
+	mlx5_malloc_mem_select
+	mlx5_mr_btree_init
+	mlx5_mr_btree_free
+	mlx5_mr_btree_dump
+	mlx5_mr_addr2mr_bh
+	mlx5_mr_release_cache
+	mlx5_mr_dump_cache
+	mlx5_mr_rebuild_cache
+	mlx5_mr_insert_cache
+	mlx5_mr_lookup_cache
+	mlx5_mr_lookup_list
+	mlx5_mr_create_primary
+	mlx5_mr_flush_local_cache
+	mlx5_mp_req_queue_state_modify
+	mlx5_mr_free
+
+	mlx5_pci_driver_register
+
+	mlx5_release_dbr
+
+	mlx5_malloc
+	mlx5_realloc
+	mlx5_free
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 17/32] common/mlx5: extend DevX query hca attributes command

[Tal Shnaiderman]

Extend DevX API mlx5_devx_cmd_query_hca_attr() to report on max number
of available objects including: CQ, QP, PD, SRQ.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 11 +++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++++++++-
 drivers/common/mlx5/mlx5_prm.h       |  1 +
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 9c1d1883ea..36d3a421aa 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -720,6 +720,14 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->flow_hit_aso = !!(MLX5_GET64(cmd_hca_cap, hcattr,
 					   general_obj_types) &
 				MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
+	attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
+	attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
+	attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
+	attr->log_max_qp_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp_sz);
+	attr->log_max_mrw_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_mrw_sz);
+	attr->log_max_pd = MLX5_GET(cmd_hca_cap, hcattr, log_max_pd);
+	attr->log_max_srq = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq);
+	attr->log_max_srq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq_sz);
 	if (attr->qos.sup) {
 		MLX5_SET(query_hca_cap_in, in, op_mod,
 			 MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
@@ -834,6 +842,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->tunnel_stateless_gtp = MLX5_GET
 					(per_protocol_networking_offload_caps,
 					 hcattr, tunnel_stateless_gtp);
+	attr->rss_ind_tbl_cap = MLX5_GET
+					(per_protocol_networking_offload_caps,
+					 hcattr, rss_ind_tbl_cap);
 	if (attr->wqe_inline_mode != MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
 		return 0;
 	if (attr->eth_virt) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 726e9f5192..5056d86ed9 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -117,6 +117,15 @@ struct mlx5_hca_attr {
 	uint32_t log_max_ft_sampler_num:8;
 	struct mlx5_hca_qos_attr qos;
 	struct mlx5_hca_vdpa_attr vdpa;
+	int log_max_qp_sz;
+	int log_max_cq_sz;
+	int log_max_qp;
+	int log_max_cq;
+	uint32_t log_max_pd;
+	uint32_t log_max_mrw_sz;
+	uint32_t log_max_srq;
+	uint32_t log_max_srq_sz;
+	uint32_t rss_ind_tbl_cap;
 };
 
 struct mlx5_devx_wq_attr {
@@ -495,7 +504,6 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_virtio_q_counters(void *ctx);
 __rte_internal
 int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 				  struct mlx5_devx_virtio_q_couners_attr *attr);
-
 __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 58d180486e..585e9d1c05 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -665,6 +665,7 @@ typedef uint8_t u8;
 #define MLX5_GET64(typ, p, fld) rte_be_to_cpu_64(*((rte_be64_t *)(p) + \
 						   __mlx5_64_off(typ, fld)))
 #define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
+#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
 
 struct mlx5_ifc_fte_match_set_misc_bits {
 	u8 gre_c_present[0x1];
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 18/32] common/mlx5: add DevX alloc PD command

[Tal Shnaiderman]

Add a new DevX API mlx5_devx_cmd_alloc_pd() that creates a new protection
domain (PD).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c            | 35 ++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  2 ++
 drivers/common/mlx5/mlx5_prm.h                  | 36 +++++++++++++++++++++++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  1 +
 drivers/common/mlx5/version.map                 |  1 +
 5 files changed, 75 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 36d3a421aa..12f51a940c 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2055,3 +2055,38 @@ mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx, uint32_t pd)
 	flow_hit_aso_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
 	return flow_hit_aso_obj;
 }
+
+/*
+ * Create PD using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_alloc_pd(void *ctx)
+{
+	struct mlx5_devx_obj *ppd =
+		mlx5_malloc(MLX5_MEM_ZERO, sizeof(*ppd), 0, SOCKET_ID_ANY);
+	u32 in[MLX5_ST_SZ_DW(alloc_pd_in)] = {0};
+	u32 out[MLX5_ST_SZ_DW(alloc_pd_out)] = {0};
+
+	if (!ppd) {
+		DRV_LOG(ERR, "Failed to allocate PD data.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(alloc_pd_in, in, opcode, MLX5_CMD_OP_ALLOC_PD);
+	ppd->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+				out, sizeof(out));
+	if (!ppd->obj) {
+		mlx5_free(ppd);
+		DRV_LOG(ERR, "Failed to allocate PD Obj using DevX.");
+		rte_errno = errno;
+		return NULL;
+	}
+	ppd->id = MLX5_GET(alloc_pd_out, out, pd);
+	return ppd;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 5056d86ed9..8277fdbc39 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -508,4 +508,6 @@ __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
 
+__rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_alloc_pd(void *ctx);
 #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 585e9d1c05..4ddf865743 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -844,6 +844,8 @@ enum {
 	MLX5_CMD_OP_SUSPEND_QP = 0x50F,
 	MLX5_CMD_OP_RESUME_QP = 0x510,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
+	MLX5_CMD_OP_ALLOC_PD = 0x800,
+	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ACCESS_REGISTER = 0x805,
 	MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -2772,6 +2774,40 @@ struct mlx5_ifc_init2init_qp_in_bits {
 	u8 reserved_at_800[0x80];
 };
 
+struct mlx5_ifc_dealloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x8];
+	u8 pd[0x18];
+	u8 reserved_3[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x8];
+	u8 pd[0x18];
+	u8 reserved_2[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x40];
+};
+
 #ifdef PEDANTIC
 #pragma GCC diagnostic ignored "-Wpedantic"
 #endif
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 23984b7abd..15aafc3809 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -5,6 +5,7 @@ EXPORTS
 
 	mlx5_create_mr_ext
 
+	mlx5_devx_cmd_alloc_pd
 	mlx5_devx_cmd_create_cq
 	mlx5_devx_cmd_create_flex_parser
 	mlx5_devx_cmd_create_qp
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 17dd11f635..fb3952bbc6 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -10,6 +10,7 @@ INTERNAL {
 
 	mlx5_dev_to_pci_addr;
 
+	mlx5_devx_cmd_alloc_pd;
 	mlx5_devx_cmd_create_cq;
 	mlx5_devx_cmd_create_flex_parser;
 	mlx5_devx_cmd_create_qp;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 19/32] common/mlx5/windows: add glue functions APIs

[Tal Shnaiderman]

Windows glue functions are added to file mlx5/windows/mlx5_glue.c.
The following APIs are supported:
get_device_list, free_device_list, open_device, close_device,
query_device, query_hca_iseg, devx_obj_create, devx_obj_destroy,
devx_obj_query, devx_obj_modify, devx_general_cmd, devx_umem_reg,
devx_umem_dereg, devx_alloc_uar, devx_free_uar, devx_fs_rule_add,
devx_fs_rule_del, devx_query_eqn
New added files:
mlx5_win_defs.h - this file imports missing definitions from Linux
rdma-core library and Linux OS.
mlx5_win_ext.h - this file contains structs that enable a unified
Linux/Windows API. Each struct has an equivalent (but different) Linux
struct. By calling with 'void *' pointers - the Linux/Windows API is
identical.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |   1 +
 drivers/common/mlx5/windows/mlx5_glue.c         | 304 ++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 +++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  34 +++
 5 files changed, 422 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 15aafc3809..2f6aab133b 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -36,6 +36,7 @@ EXPORTS
 	mlx5_devx_cmd_create_flow_hit_aso_obj
 
 	mlx5_get_dbr
+	mlx5_glue
 
 	mlx5_malloc_mem_select
 	mlx5_mr_btree_init
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
new file mode 100644
index 0000000000..7f8a00aaa7
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -0,0 +1,304 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+
+#include "mlx5_glue.h"
+#include "mlx5_common_utils.h"
+#include "mlx5_win_ext.h"
+
+/*
+ * The returned value of this API is an array of pointers to mlx5
+ * devices under Windows. The interesting parameters of a device:
+ * Device PCI parameters: domain, bus, device id, function.
+ * Device name.
+ */
+static void *
+mlx5_glue_devx_get_device_list(int *num_devices)
+{
+	struct devx_device_bdf *devx_bdf_devs = NULL;
+	size_t n_devx_devx = 0;
+	int32_t ret = devx_get_device_list(&n_devx_devx, &devx_bdf_devs);
+
+	if (ret) {
+		errno = ret;
+		*num_devices = 0;
+		return NULL;
+	}
+	*num_devices = (int)n_devx_devx;
+	return devx_bdf_devs;
+}
+
+static void
+mlx5_glue_devx_free_device_list(void *list)
+{
+	if (!list) {
+		errno = EINVAL;
+		return;
+	}
+	devx_free_device_list(list);
+}
+
+static int
+mlx5_glue_devx_close_device(void *ctx)
+{
+	mlx5_context_st *mlx5_ctx;
+	int rc;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (mlx5_context_st *)ctx;
+	rc = devx_close_device(mlx5_ctx->devx_ctx);
+	free(mlx5_ctx);
+	return rc;
+}
+
+static void *
+mlx5_glue_devx_open_device(void *device)
+{
+	struct mlx5_context *mlx5_ctx;
+
+	if (!device) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = malloc((sizeof(struct mlx5_context)));
+	if (!mlx5_ctx) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(mlx5_ctx, 0, sizeof(*mlx5_ctx));
+	mlx5_ctx->devx_ctx = devx_open_device(device);
+	if (DEVX_IS_ERR(mlx5_ctx->devx_ctx)) {
+		errno = -DEVX_PTR_ERR(mlx5_ctx->devx_ctx);
+		free(mlx5_ctx);
+		return NULL;
+	}
+	return mlx5_ctx;
+}
+
+static int
+mlx5_glue_devx_query_device(void *device_bdf, void *dev_inf)
+{
+	struct devx_device_bdf *dev_bdf;
+	struct devx_device *mlx5_dev;
+
+	if (!device_bdf)
+		return -EINVAL;
+	dev_bdf = (struct devx_device_bdf *)device_bdf;
+	mlx5_dev = (struct devx_device *)dev_inf;
+	int err = devx_query_device(dev_bdf, mlx5_dev);
+	if (err)
+		return -E_FAIL;
+	return 0;
+}
+
+static void *
+mlx5_glue_devx_query_hca_iseg_mapping(void *ctx, uint32_t *cb_iseg)
+{
+	struct mlx5_context *mlx5_ctx;
+	void *pv_iseg;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	err = devx_query_hca_iseg_mapping(mlx5_ctx->devx_ctx,
+						cb_iseg, &pv_iseg);
+	if (err) {
+		errno = err;
+		return NULL;
+	}
+	return pv_iseg;
+}
+
+static void *
+mlx5_glue_devx_obj_create(void *ctx,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	devx_obj = malloc((sizeof(*devx_obj)));
+	if (!devx_obj) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(devx_obj, 0, sizeof(*devx_obj));
+	devx_obj->devx_ctx = GET_DEVX_CTX(ctx);
+	devx_obj->obj = devx_obj_create(devx_obj->devx_ctx,
+					in, inlen, out, outlen);
+	if (DEVX_IS_ERR(devx_obj->obj)) {
+		errno = -DEVX_PTR_ERR(devx_obj->obj);
+		free(devx_obj);
+		return NULL;
+	}
+	return devx_obj;
+}
+
+static int
+mlx5_glue_devx_obj_destroy(void *obj)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!obj)
+		return -EINVAL;
+	devx_obj = obj;
+	int rc = devx_obj_destroy(devx_obj->obj);
+	free(devx_obj);
+	return rc;
+}
+
+static int
+mlx5_glue_devx_general_cmd(void *ctx,
+			   void *in, size_t inlen,
+			   void *out, size_t outlen)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_cmd(GET_DEVX_CTX(ctx), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_query(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_modify(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_umem_dereg(void *pumem)
+{
+	struct devx_obj_handle *umem;
+
+	if (!pumem)
+		return -EINVAL;
+	umem = pumem;
+	return devx_umem_unreg(umem);
+}
+
+static void *
+mlx5_glue_devx_umem_reg(void *ctx, void *addr, size_t size,
+				  uint32_t access, uint32_t *id)
+{
+	struct devx_obj_handle *umem_hdl;
+	int w_access = DEVX_UMEM_ACCESS_READ;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	if (access)
+		w_access |= DEVX_UMEM_ACCESS_WRITE;
+
+	umem_hdl = devx_umem_reg(GET_DEVX_CTX(ctx), addr,
+					size, w_access, id);
+	if (DEVX_IS_ERR(umem_hdl)) {
+		errno = -DEVX_PTR_ERR(umem_hdl);
+		return NULL;
+	}
+	return umem_hdl;
+}
+
+static void *
+mlx5_glue_devx_alloc_uar(void *ctx,
+		uint32_t flags)
+{
+	devx_uar_handle *uar;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	uar = devx_alloc_uar(GET_DEVX_CTX(ctx), flags);
+	if (DEVX_IS_ERR(uar)) {
+		errno = -DEVX_PTR_ERR(uar);
+		return NULL;
+	}
+	return uar;
+}
+
+static int
+mlx5_glue_devx_query_eqn(void *ctx,
+		uint32_t cpus, uint32_t *eqn)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_query_eqn(GET_DEVX_CTX(ctx), cpus, eqn);
+}
+
+static void
+mlx5_glue_devx_free_uar(void *uar)
+{
+	devx_free_uar((devx_uar_handle *)uar);
+}
+
+static void*
+mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
+
+{
+	struct devx_obj_handle *rule_hdl = NULL;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	rule_hdl = devx_fs_rule_add(GET_DEVX_CTX(ctx), in, inlen);
+	if (DEVX_IS_ERR(rule_hdl)) {
+		errno = -DEVX_PTR_ERR(rule_hdl);
+		return NULL;
+	}
+	return rule_hdl;
+}
+
+static int
+mlx5_glue_devx_fs_rule_del(void *flow)
+{
+	return devx_fs_rule_del(flow);
+}
+
+alignas(RTE_CACHE_LINE_SIZE)
+const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+	.version = MLX5_GLUE_VERSION,
+	.get_device_list = mlx5_glue_devx_get_device_list,
+	.free_device_list = mlx5_glue_devx_free_device_list,
+	.open_device = mlx5_glue_devx_open_device,
+	.close_device = mlx5_glue_devx_close_device,
+	.query_device = mlx5_glue_devx_query_device,
+	.query_hca_iseg = mlx5_glue_devx_query_hca_iseg_mapping,
+	.devx_obj_create = mlx5_glue_devx_obj_create,
+	.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
+	.devx_obj_query = mlx5_glue_devx_obj_query,
+	.devx_obj_modify = mlx5_glue_devx_obj_modify,
+	.devx_general_cmd = mlx5_glue_devx_general_cmd,
+	.devx_umem_reg = mlx5_glue_devx_umem_reg,
+	.devx_umem_dereg = mlx5_glue_devx_umem_dereg,
+	.devx_alloc_uar = mlx5_glue_devx_alloc_uar,
+	.devx_free_uar = mlx5_glue_devx_free_uar,
+	.devx_fs_rule_add = mlx5_glue_devx_fs_rule_add,
+	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
+	.devx_query_eqn = mlx5_glue_devx_query_eqn,
+};
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
new file mode 100644
index 0000000000..f2261ec7ac
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_GLUE_H_
+#define MLX5_GLUE_H_
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include <rte_byteorder.h>
+#include <mlx5_win_defs.h>
+
+#ifndef MLX5_GLUE_VERSION
+#define MLX5_GLUE_VERSION ""
+#endif
+
+/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
+struct mlx5_glue {
+	const char *version;
+	void *(*devx_obj_create)(void *ctx,
+				 void *in, size_t inlen,
+				 void *out, size_t outlen);
+	int (*devx_obj_destroy)(void *obj);
+	int (*devx_obj_query)(void *obj,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen);
+	int (*devx_obj_modify)(void *obj,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_general_cmd)(void *ctx,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_umem_dereg)(void *umem);
+	void *(*devx_umem_reg)(void *ctx,
+			void *addr, size_t size,
+			uint32_t access, uint32_t *id);
+	void *(*devx_alloc_uar)(void *ctx,
+			uint32_t flags);
+	void (*devx_free_uar)(void *uar);
+	void *(*get_device_list)(int *num_devices);
+	void (*free_device_list)(void *list);
+	void *(*open_device)(void *device);
+	int (*close_device)(void *ctx);
+	int (*query_device)(void *device_bdf, void *dev_inf);
+	void* (*query_hca_iseg)(void *ctx, uint32_t *cb_iseg);
+	int (*devx_obj_query_async)(void *obj,
+				    const void *in, size_t inlen,
+				    size_t outlen, uint64_t wr_id,
+				    void *cmd_comp);
+	void *(*devx_fs_rule_add)(void *ctx, void *in, uint32_t inlen);
+	int (*devx_fs_rule_del)(void *flow);
+	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
+};
+
+extern const struct mlx5_glue *mlx5_glue;
+
+#endif /* MLX5_GLUE_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
new file mode 100644
index 0000000000..72a3131f5e
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_DEFS_H__
+#define __MLX5_WIN_DEFS_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum {
+	MLX5_CQE_OWNER_MASK	= 1,
+	MLX5_CQE_REQ		= 0,
+	MLX5_CQE_RESP_WR_IMM	= 1,
+	MLX5_CQE_RESP_SEND	= 2,
+	MLX5_CQE_RESP_SEND_IMM	= 3,
+	MLX5_CQE_RESP_SEND_INV	= 4,
+	MLX5_CQE_RESIZE_CQ	= 5,
+	MLX5_CQE_NO_PACKET	= 6,
+	MLX5_CQE_REQ_ERR	= 13,
+	MLX5_CQE_RESP_ERR	= 14,
+	MLX5_CQE_INVALID	= 15,
+};
+#endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
new file mode 100644
index 0000000000..0e74910e9d
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_ETX_H__
+#define __MLX5_WIN_ETX_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "mlx5devx.h"
+
+typedef struct mlx5_context {
+	devx_device_ctx        *devx_ctx;
+	struct devx_device mlx5_dev;
+
+} mlx5_context_st;
+
+typedef struct {
+	devx_device_ctx *devx_ctx;
+	struct devx_obj_handle *obj;
+} mlx5_devx_obj_st;
+
+struct mlx5_devx_umem {
+	void                   *addr;
+	struct devx_obj_handle *umem_hdl;
+	uint32_t                umem_id;
+};
+
+#define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
+#define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
+
+#endif /* __MLX5_WIN_ETX_H__ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 20/32] mlx5/windows: add mlx5 meson file

[Tal Shnaiderman]

File drivers/common/mlx5/windows/meson.build is added to enable mlx5
source files compilation under common windows directory.
A Devx SDK tool must be installed to export two external H files:
mlx5devx.h and mlx5_ifc_devx.h. The installation is based on environment
variable DEVX_INC_PATH. In addition a DLL lib file is installed based on
environment variable DEVX_LIB_PATH. The meson file is using the
environment variables for compilation and linkage.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 42 +++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/meson.build

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2b2aa3b097
--- /dev/null
+++ b/drivers/common/mlx5/windows/meson.build
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2019 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+
+allow_experimental_apis = true
+sources += files(
+	'mlx5_glue.c',
+)
+
+res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
+res_inc = run_command(python3, '-c', 'import os; print(os.environ["DEVX_INC_PATH"])')
+
+if (res_lib.returncode() != 0 or res_inc.returncode() != 0)
+	build = false
+	reason = 'DevX environment variables are not set, DEVX_LIB_PATH and DEVX_INC_PATH vars must be exported'
+	subdir_done()
+endif
+
+devx_lib_dir = res_lib.stdout().strip()
+devx_inc_dir = res_inc.stdout().strip()
+
+ext_deps += cc.find_library('mlx5devx', dirs: devx_lib_dir, required: true)
+includes += include_directories(devx_inc_dir)
+cflags_options = [
+	'-std=c11',
+	'-Wno-strict-prototypes',
+	'-D_BSD_SOURCE',
+	'-D_DEFAULT_SOURCE',
+	'-D_XOPEN_SOURCE=600'
+]
+foreach option:cflags_options
+	if cc.has_argument(option)
+		cflags += option
+	endif
+endforeach
+if get_option('buildtype').contains('debug')
+	cflags += [ '-pedantic', '-DPEDANTIC' ]
+else
+	cflags += [ '-UPEDANTIC' ]
+endif
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 21/32] mlx5/windows: add initialization routine for external lib

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add function mlx5_glue_constructor() as an initialization routine for
run-time dependency on external lib.  Currently the routine has an empty
body. It is used for compatibility with Linux.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build      |  1 +
 drivers/common/mlx5/windows/mlx5_common_os.c | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 2b2aa3b097..9e74ebdea3 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -6,6 +6,7 @@ includes += include_directories('.')
 allow_experimental_apis = true
 sources += files(
 	'mlx5_glue.c',
+	'mlx5_common_os.c',
 )
 
 res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
new file mode 100644
index 0000000000..5707fb60c3
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <unistd.h>
+#include <string.h>
+#include <stdio.h>
+
+#include <rte_errno.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "mlx5_devx_cmds.h"
+#include "mlx5_common_utils.h"
+#include <mlx5_common.h>
+
+/**
+ * Initialization routine for run-time dependency on external lib
+ */
+void
+mlx5_glue_constructor(void)
+{
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 22/32] mlx5/windows: generate file mlx5_autoconf.h

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

File mlx5_autoconf.h is generated under Windows to maintain
compatibility with the Linux build system.  This file is included in
Linux/Windows shared source files therefore it is required. Currently
the file is created empty.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 9e74ebdea3..5ac5724b40 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -41,3 +41,6 @@ else
 	cflags += [ '-UPEDANTIC' ]
 endif
 
+# Generate an empty mlx5_autoconf.h file for compatibility with Linux
+config = configuration_data()
+configure_file(output : 'mlx5_autoconf.h', configuration : config)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 23/32] common/mlx5/windows: extend PRM match_param_bits struct

[Tal Shnaiderman]

Add reserved size to PRM 'struct mlx5_ifc_fte_match_param_bits' for non
Linux OS.  Windows drivers require this extension since their expected
size should match the actual struct size. Linux drivers do not require
this extension and already use calculations based on the shorter size.
Use a static assert to verify that the PRM fte_match_param struct size
remains correct.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h          | 7 +++++++
 drivers/common/mlx5/windows/mlx5_glue.c | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 4ddf865743..27a4e4f0f1 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -815,6 +815,13 @@ struct mlx5_ifc_fte_match_param_bits {
 	struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
 	struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
 	struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
+/*
+ * Add reserved bit to match the struct size with the size defined in PRM.
+ * This extension is not required in Linux.
+ */
+#ifndef HAVE_INFINIBAND_VERBS_H
+	u8 reserved_0[0x400];
+#endif
 };
 
 enum {
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 7f8a00aaa7..3896cf0932 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -256,6 +256,9 @@ mlx5_glue_devx_free_uar(void *uar)
 	devx_free_uar((devx_uar_handle *)uar);
 }
 
+static_assert(MLX5_ST_SZ_BYTES(fte_match_param) == 0x200,
+	"PRM size of fte_match_param is broken! cannot compile Windows!");
+
 static void*
 mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 24/32] common/mlx5/windows: add getter functions

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add file mlx5/windows/mlx5_common_os.h the equivalent of Linux file
mlx5/linux/mlx5_common_os.h. It contains getters functions
mlx5_os_get_dev_device_name, mlx5_os_get_ctx_device_name,
mlx5_os_get_ctx_device_path, mlx5_os_get_umem_id,
mlx5_os_get_devx_channel_fd.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 91 ++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index cce7c88c6a..2abdb2c3ae 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -48,4 +48,95 @@ mlx5_os_free(void *addr)
 {
 	_aligned_free(addr);
 }
+
+/**
+ * Get fd. Given a pointer to DevX channel object of type
+ * 'struct mlx5dv_devx_event_channel*' - return its fd.
+ * Under Windows it is a stub.
+ *
+ * @param[in] channel
+ *    Pointer to channel object.
+ *
+ * @return
+ *    0
+ */
+static inline int
+mlx5_os_get_devx_channel_fd(void *channel)
+{
+	if (!channel)
+		return 0;
+	return 0;
+}
+
+/**
+ * Get device name. Given a device pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] dev
+ *   Pointer to device.
+ *
+ * @return
+ *   Pointer to device name if dev is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_dev_device_name(void *dev)
+{
+	if (!dev)
+		return NULL;
+	return ((struct devx_device *)dev)->name;
+}
+
+/**
+ * Get device name. Given a context pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device name if ctx is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_ctx_device_name(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.name;
+}
+
+/**
+ * Get a device path name. Given acontext pointer - return a
+ * pointer to the corresponding device path name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device path name if ctx is valid, NULL otherwise.
+ */
+
+static inline const char *
+mlx5_os_get_ctx_device_path(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.dev_pnp_id;
+}
+
+/**
+ * Get umem id. Given a pointer to umem object of type return its id.
+ *
+ * @param[in] umem
+ *    Pointer to umem object.
+ *
+ * @return
+ *    The umem id if umem is valid, 0 otherwise.
+ */
+static inline uint32_t
+mlx5_os_get_umem_id(void *umem)
+{
+	if (!umem)
+		return 0;
+	return ((struct mlx5_devx_umem *)umem)->umem_id;
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 25/32] common/mlx5/windows: add OS alloc/dealloc pd

[Tal Shnaiderman]

Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd(). They
are equivalent to the Linux implementation in (1).

(1)
("net/mlx5: wrap glue alloc/dealloc PD with OS calls")

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
 drivers/common/mlx5/windows/mlx5_common_os.c    | 48 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 2f6aab133b..5db1eaa48c 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -61,4 +61,5 @@ EXPORTS
 	mlx5_malloc
 	mlx5_realloc
 	mlx5_free
-
+	mlx5_os_alloc_pd
+	mlx5_os_dealloc_pd
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 5707fb60c3..85537b6739 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -22,3 +22,51 @@ void
 mlx5_glue_constructor(void)
 {
 }
+
+/**
+ * Allocate PD. Given a devx context object
+ * return an mlx5-pd object.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *    The mlx5_pd if pd is valid, NULL and errno otherwise.
+ */
+void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
+		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
+	if (!ppd)
+		return NULL;
+
+	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
+	if (!obj) {
+		mlx5_free(ppd);
+		return NULL;
+	}
+	ppd->obj = obj;
+	ppd->pdn = obj->id;
+	ppd->devx_ctx = ctx;
+	return ppd;
+}
+
+/**
+ * Release PD. Releases a given mlx5_pd object
+ *
+ * @param[in] pd
+ *   Pointer to mlx5_pd.
+ *
+ * @return
+ *    Zero if pd is released successfully, negative number otherwise.
+ */
+int
+mlx5_os_dealloc_pd(void *pd)
+{
+	if (!pd)
+		return -EINVAL;
+	mlx5_devx_cmd_destroy(((struct mlx5_pd *)pd)->obj);
+	mlx5_free(pd);
+	return 0;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index 2abdb2c3ae..f47351ec41 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -139,4 +139,7 @@ mlx5_os_get_umem_id(void *umem)
 		return 0;
 	return ((struct mlx5_devx_umem *)umem)->umem_id;
 }
+
+void *mlx5_os_alloc_pd(void *ctx);
+int mlx5_os_dealloc_pd(void *pd);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index 0e74910e9d..8e697b30c1 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -28,6 +28,12 @@ struct mlx5_devx_umem {
 	uint32_t                umem_id;
 };
 
+struct mlx5_pd {
+	void                   *obj;
+	uint32_t                pdn;
+	devx_device_ctx        *devx_ctx;
+};
+
 #define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
 #define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 26/32] common/mlx5/windows: add OS umem reg/dereg API

[Tal Shnaiderman]

Implement Windows API mlx5_os_umem_reg() and mlx5_os_umem_dereg(). They
are equivalent to the Linux implementation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 60 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  2 +
 3 files changed, 64 insertions(+)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 5db1eaa48c..1eabe29a58 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,3 +63,5 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_umem_reg
+	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 85537b6739..cbb09906bd 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -70,3 +70,63 @@ mlx5_os_dealloc_pd(void *pd)
 	mlx5_free(pd);
 	return 0;
 }
+
+/**
+ * Register umem.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ * @param[in] addr
+ *   Pointer to memory start address.
+ * @param[in] size
+ *   Size of the memory to register.
+ * @param[out] access
+ *   UMEM access type
+ *
+ * @return
+ *   umem on successful registration, NULL and errno otherwise
+ */
+void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	struct mlx5_devx_umem *umem;
+
+	umem = mlx5_malloc(MLX5_MEM_ZERO,
+		(sizeof(*umem)), 0, SOCKET_ID_ANY);
+	if (!umem) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	umem->umem_hdl = mlx5_glue->devx_umem_reg(ctx, addr, size, access,
+		&umem->umem_id);
+	if (!umem->umem_hdl) {
+		mlx5_free(umem);
+		return NULL;
+	}
+	umem->addr = addr;
+	return umem;
+}
+
+/**
+ * Deregister umem.
+ *
+ * @param[in] pumem
+ *   Pointer to umem.
+ *
+ * @return
+ *   0 on successful release, negative number otherwise
+ */
+int
+mlx5_os_umem_dereg(void *pumem)
+{
+	struct mlx5_devx_umem *umem;
+	int err = 0;
+
+	if (!pumem)
+		return err;
+	umem = pumem;
+	if (umem->umem_hdl)
+		err = mlx5_glue->devx_umem_dereg(umem->umem_hdl);
+	mlx5_free(umem);
+	return err;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index f47351ec41..decb5acd45 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -142,4 +142,6 @@ mlx5_os_get_umem_id(void *umem)
 
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
+void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
+int mlx5_os_umem_dereg(void *pumem);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 27/32] net/mlx5: update MR prototypes for DevX

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well.  Rename 'struct mlx5_verbs_ops' as 'struct mlx5_mr_ops' and
move it to shared file mlx5.h.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c    | 4 ++--
 drivers/net/mlx5/linux/mlx5_verbs.c | 2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h | 8 +-------
 drivers/net/mlx5/mlx5.h             | 6 ++++++
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a6c1eba640..4f68c74267 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -2478,8 +2478,8 @@ void
 mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
 		      mlx5_dereg_mr_t *dereg_mr_cb)
 {
-	*reg_mr_cb = mlx5_verbs_ops.reg_mr;
-	*dereg_mr_cb = mlx5_verbs_ops.dereg_mr;
+	*reg_mr_cb = mlx5_mr_verbs_ops.reg_mr;
+	*dereg_mr_cb = mlx5_mr_verbs_ops.dereg_mr;
 }
 
 /**
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c
index 9161fa3b7d..b52ae2e6c1 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.c
+++ b/drivers/net/mlx5/linux/mlx5_verbs.c
@@ -62,7 +62,7 @@ mlx5_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
 }
 
 /* verbs operations. */
-const struct mlx5_verbs_ops mlx5_verbs_ops = {
+const struct mlx5_mr_ops mlx5_mr_verbs_ops = {
 	.reg_mr = mlx5_reg_mr,
 	.dereg_mr = mlx5_dereg_mr,
 };
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.h b/drivers/net/mlx5/linux/mlx5_verbs.h
index 0670f6c47e..76a79bf4f4 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.h
+++ b/drivers/net/mlx5/linux/mlx5_verbs.h
@@ -7,16 +7,10 @@
 
 #include "mlx5.h"
 
-struct mlx5_verbs_ops {
-	mlx5_reg_mr_t reg_mr;
-	mlx5_dereg_mr_t dereg_mr;
-};
-
 int mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
 
 /* Verbs ops struct */
-extern const struct mlx5_verbs_ops mlx5_verbs_ops;
+extern const struct mlx5_mr_ops mlx5_mr_verbs_ops;
 extern struct mlx5_obj_ops ibv_obj_ops;
-
 #endif /* RTE_PMD_MLX5_VERBS_H_ */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 2186677810..a9db597f2b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -917,6 +917,12 @@ struct mlx5_obj_ops {
 
 #define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields)
 
+/* MR operations structure. */
+struct mlx5_mr_ops {
+	mlx5_reg_mr_t reg_mr;
+	mlx5_dereg_mr_t dereg_mr;
+};
+
 struct mlx5_priv {
 	struct rte_eth_dev_data *dev_data;  /* Pointer to device data. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared device context. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 28/32] common/mlx5/windows: add OS reg/dereg MR

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commits implements Windows API for MR registration and
deregistration. It is based on DevX.  Is support the relaxed ordering
flow in Windows by checking the capabilities and machine type.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.h            |  1 +
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 78 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  7 +++
 4 files changed, 88 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index da0a0f0c79..5cc3f097c2 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -28,6 +28,7 @@ struct mlx5_pmd_mr {
 	void		     *addr;
 	size_t		     len;
 	void		     *obj;  /* verbs mr object or devx umem object. */
+	struct mlx5_devx_obj *mkey; /* devx mkey object. */
 };
 
 /**
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 1eabe29a58..65ae47ae6a 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,5 +63,7 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_dereg_mr
+	mlx5_os_reg_mr
 	mlx5_os_umem_reg
 	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index cbb09906bd..84ed561dc0 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -14,6 +14,8 @@
 #include "mlx5_devx_cmds.h"
 #include "mlx5_common_utils.h"
 #include <mlx5_common.h>
+#include "mlx5_common_os.h"
+#include "mlx5_malloc.h"
 
 /**
  * Initialization routine for run-time dependency on external lib
@@ -130,3 +132,79 @@ mlx5_os_umem_dereg(void *pumem)
 	mlx5_free(umem);
 	return err;
 }
+
+/**
+ * Register mr. Given protection doamin pointer, pointer to addr and length
+ * register the memory region.
+ *
+ * @param[in] pd
+ *   Pointer to protection domain context (type mlx5_pd).
+ * @param[in] addr
+ *   Pointer to memory start address (type devx_device_ctx).
+ * @param[in] length
+ *   Lengtoh of the memory to register.
+ * @param[out] pmd_mr
+ *   pmd_mr struct set with lkey, address, length, pointer to mr object, mkey
+ *
+ * @return
+ *   0 on successful registration, -1 otherwise
+ */
+int
+mlx5_os_reg_mr(void *pd,
+	       void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr)
+{
+	struct mlx5_devx_mkey_attr mkey_attr;
+	struct mlx5_pd *mlx5_pd = (struct mlx5_pd *)pd;
+	struct mlx5_hca_attr attr;
+
+	if (!pd || !addr) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+	if (mlx5_devx_cmd_query_hca_attr(mlx5_pd->devx_ctx, &attr))
+		return -1;
+	pmd_mr->addr = addr;
+	pmd_mr->len = length;
+	pmd_mr->obj = mlx5_os_umem_reg(mlx5_pd->devx_ctx, pmd_mr->addr,
+				       pmd_mr->len, IBV_ACCESS_LOCAL_WRITE);
+	if (!pmd_mr->obj)
+		return -1;
+	mkey_attr.addr = (uintptr_t)addr;
+	mkey_attr.size = length;
+	mkey_attr.umem_id = ((struct mlx5_devx_umem *)(pmd_mr->obj))->umem_id;
+	mkey_attr.pd = mlx5_pd->pdn;
+	mkey_attr.log_entity_size = 0;
+	mkey_attr.pg_access = 0;
+	mkey_attr.klm_array = NULL;
+	mkey_attr.klm_num = 0;
+	mkey_attr.relaxed_ordering_read = 0;
+	mkey_attr.relaxed_ordering_write = 0;
+	if (!haswell_broadwell_cpu) {
+		mkey_attr.relaxed_ordering_write = attr.relaxed_ordering_write;
+		mkey_attr.relaxed_ordering_read = attr.relaxed_ordering_read;
+	}
+	pmd_mr->mkey = mlx5_devx_cmd_mkey_create(mlx5_pd->devx_ctx, &mkey_attr);
+	if (!pmd_mr->mkey) {
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+		return -1;
+	}
+	pmd_mr->lkey = pmd_mr->mkey->id;
+	return 0;
+}
+
+/**
+ * De-register mr.
+ *
+ * @param[in] pmd_mr
+ *  Pointer to PMD mr object
+ */
+void
+mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
+{
+	if (pmd_mr && pmd_mr->mkey)
+		claim_zero(mlx5_glue->devx_obj_destroy(pmd_mr->mkey->obj));
+	if (pmd_mr && pmd_mr->obj)
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index decb5acd45..ba166412cc 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -7,9 +7,13 @@
 
 #include <stdio.h>
 
+#include <rte_errno.h>
+
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
 #include "mlx5_malloc.h"
+#include "mlx5_common_mr.h"
+#include "mlx5_win_ext.h"
 
 /**
  * This API allocates aligned or non-aligned memory.  The free can be on either
@@ -144,4 +148,7 @@ void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
 int mlx5_os_umem_dereg(void *pumem);
+int mlx5_os_reg_mr(void *pd,
+		   void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr);
+void mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 29/32] net/mlx5/windows: implement device attribute getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows implementation of mlx5_os_get_dev_attr() API.
It follows the commit in [1]. A new file named mlx5_os.c is added under
windows directory as its Linux counterpart file: linux/mlx5_os.c.

[1].
commit e85f623e13ea ("net/mlx5: remove attributes dependency on Verbs")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h       | 43 +++++++++++++++++++++
 drivers/net/mlx5/windows/meson.build |  8 ++++
 drivers/net/mlx5/windows/mlx5_os.c   | 75 ++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 27a4e4f0f1..d62d2558eb 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -2217,6 +2217,49 @@ struct mlx5_ifc_cqc_bits {
 	u8 dbr_addr[0x40];
 };
 
+struct mlx5_ifc_health_buffer_bits {
+	u8         reserved_0[0x100];
+	u8         assert_existptr[0x20];
+	u8         assert_callra[0x20];
+	u8         reserved_1[0x40];
+	u8         fw_version[0x20];
+	u8         hw_id[0x20];
+	u8         reserved_2[0x20];
+	u8         irisc_index[0x8];
+	u8         synd[0x8];
+	u8         ext_synd[0x10];
+};
+
+struct mlx5_ifc_initial_seg_bits {
+	u8         fw_rev_minor[0x10];
+	u8         fw_rev_major[0x10];
+	u8         cmd_interface_rev[0x10];
+	u8         fw_rev_subminor[0x10];
+	u8         reserved_0[0x40];
+	u8         cmdq_phy_addr_63_32[0x20];
+	u8         cmdq_phy_addr_31_12[0x14];
+	u8         reserved_1[0x2];
+	u8         nic_interface[0x2];
+	u8         log_cmdq_size[0x4];
+	u8         log_cmdq_stride[0x4];
+	u8         command_doorbell_vector[0x20];
+	u8         reserved_2[0xf00];
+	u8         initializing[0x1];
+	u8         nic_interface_supported[0x7];
+	u8         reserved_4[0x18];
+	struct mlx5_ifc_health_buffer_bits health_buffer;
+	u8         no_dram_nic_offset[0x20];
+	u8         reserved_5[0x6de0];
+	u8         internal_timer_h[0x20];
+	u8         internal_timer_l[0x20];
+	u8         reserved_6[0x20];
+	u8         reserved_7[0x1f];
+	u8         clear_int[0x1];
+	u8         health_syndrome[0x8];
+	u8         health_counter[0x18];
+	u8         reserved_8[0x17fc0];
+};
+
 struct mlx5_ifc_create_cq_out_bits {
 	u8 status[0x8];
 	u8 reserved_at_8[0x18];
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2ea0792a40
--- /dev/null
+++ b/drivers/net/mlx5/windows/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2020 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+sources += files(
+	'mlx5_os.c',
+)
+
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
new file mode 100644
index 0000000000..e0646670a3
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <rte_windows.h>
+
+#include <mlx5_glue.h>
+#include <mlx5_devx_cmds.h>
+#include <mlx5_common.h>
+
+#include "mlx5_defs.h"
+#include "mlx5.h"
+#include "mlx5_autoconf.h"
+
+/**
+ * Get mlx5 device attributes.
+ *
+ * @param ctx
+ *   Pointer to device context.
+ *
+ * @param device_attr
+ *   Pointer to mlx5 device attributes.
+ *
+ * @return
+ *   0 on success, non zero error number otherwise
+ */
+int
+mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
+{
+	struct mlx5_context *mlx5_ctx;
+	struct mlx5_hca_attr hca_attr;
+	void *pv_iseg = NULL;
+	u32 cb_iseg = 0;
+	int err = 0;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	memset(device_attr, 0, sizeof(*device_attr));
+	err = mlx5_devx_cmd_query_hca_attr(mlx5_ctx, &hca_attr);
+	if (err) {
+		DRV_LOG(ERR, "Failed to get device hca_cap");
+		return err;
+	}
+	device_attr->max_cq = 1 << hca_attr.log_max_cq;
+	device_attr->max_qp = 1 << hca_attr.log_max_qp;
+	device_attr->max_qp_wr = 1 << hca_attr.log_max_qp_sz;
+	device_attr->max_cqe = 1 << hca_attr.log_max_cq_sz;
+	device_attr->max_mr = 1 << hca_attr.log_max_mrw_sz;
+	device_attr->max_pd = 1 << hca_attr.log_max_pd;
+	device_attr->max_srq = 1 << hca_attr.log_max_srq;
+	device_attr->max_srq_wr = 1 << hca_attr.log_max_srq_sz;
+	if (hca_attr.rss_ind_tbl_cap) {
+		device_attr->max_rwq_indirection_table_size =
+			1 << hca_attr.rss_ind_tbl_cap;
+	}
+	pv_iseg = mlx5_glue->query_hca_iseg(mlx5_ctx, &cb_iseg);
+	if (pv_iseg == NULL) {
+		DRV_LOG(ERR, "Failed to get device hca_iseg");
+		return errno;
+	}
+	if (!err) {
+		snprintf(device_attr->fw_ver, 64, "%x.%x.%04x",
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_major),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_minor),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_subminor));
+	}
+	return err;
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 30/32] net/mlx5/windows: add mlx5_os.c stubs

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_os_set_nonblock_channel_fd
mlx5_os_dev_shared_handler_install
mlx5_os_dev_shared_handler_uninstall
mlx5_os_read_dev_stat
mlx5_os_mac_addr_flush
mlx5_os_mac_addr_remove
mlx5_os_vf_mac_addr_modify
mlx5_os_set_promisc
mlx5_os_set_allmulti

Set struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops with NULL
pointers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 187 +++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index e0646670a3..bef10aba30 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -13,10 +13,18 @@
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
 #include <mlx5_common.h>
+#include <mlx5_common_mp.h>
+#include <mlx5_common_mr.h>
+#include <mlx5_malloc.h>
 
 #include "mlx5_defs.h"
 #include "mlx5.h"
+#include "mlx5_common_os.h"
+#include "mlx5_utils.h"
+#include "mlx5_rxtx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_mr.h"
+#include "mlx5_flow.h"
 
 /**
  * Get mlx5 device attributes.
@@ -73,3 +81,182 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	}
 	return err;
 }
+
+/**
+ * Set the completion channel file descriptor interrupt as non-blocking.
+ * Currently it has no support under Windows.
+ *
+ * @param[in] rxq_obj
+ *   Pointer to RQ channel object, which includes the channel fd
+ *
+ * @param[out] fd
+ *   The file descriptor (representing the intetrrupt) used in this channel.
+ *
+ * @return
+ *   0 on successfully setting the fd to non-blocking, non-zero otherwise.
+ */
+int
+mlx5_os_set_nonblock_channel_fd(int fd)
+{
+	(void)fd;
+	DRV_LOG(WARNING, "mlx5_os_set_nonblock_channel_fd is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param sh
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "mlx5_os_dev_shared_handler_install is not support");
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "mlx5_os_dev_shared_handler_uninstall is not support");
+}
+
+/**
+ * Read statistics by a named counter.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ * @param[in] ctr_name
+ *   Pointer to the name of the statistic counter to read
+ * @param[out] stat
+ *   Pointer to read statistic value.
+ * @return
+ *   0 on success and stat is valud, 1 if failed to read the value
+ *   rte_errno is set.
+ *
+ */
+int
+mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name,
+		      uint64_t *stat)
+{
+	RTE_SET_USED(priv);
+	RTE_SET_USED(ctr_name);
+	RTE_SET_USED(stat);
+	DRV_LOG(WARNING, "mlx5_os_read_dev_stat is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * Flush device MAC addresses
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ */
+void
+mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
+{
+	(void)dev;
+	DRV_LOG(WARNING, "mlx5_os_mac_addr_flush is not support");
+}
+
+/**
+ * Remove a MAC address from device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param index
+ *   MAC address index.
+ */
+void
+mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+	(void)dev;
+	(void)(index);
+	DRV_LOG(WARNING, "mlx5_os_mac_addr_remove is not support");
+}
+
+/**
+ * Modify a VF MAC address
+ * Currently it has no support under Windows.
+ *
+ * @param priv
+ *   Pointer to device private data.
+ * @param mac_addr
+ *   MAC address to modify into.
+ * @param iface_idx
+ *   Net device interface index
+ * @param vf_index
+ *   VF index
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_vf_mac_addr_modify(struct mlx5_priv *priv,
+			   unsigned int iface_idx,
+			   struct rte_ether_addr *mac_addr,
+			   int vf_index)
+{
+	(void)priv;
+	(void)iface_idx;
+	(void)mac_addr;
+	(void)vf_index;
+	DRV_LOG(WARNING, "mlx5_os_vf_mac_addr_modify is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * Set device promiscuous mode
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - promiscuous is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_promisc(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "mlx5_os_set_promisc is not support");
+	return -ENOTSUP;
+}
+
+/**
+ * Set device allmulti mode
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - all multicase is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "mlx5_os_set_allmulti is not support");
+	return -ENOTSUP;
+}
+
+const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 31/32] net/mlx5/windows: implement mlx5 mac addr add

[Tal Shnaiderman]

Get the list of MAC addresses and verify if the input mac parameter
already exists. If not - return -ENOTSUP (as Windows does not support
adding new MAC addresses). If the MAC address exists (EEXIST) return 0
(the equivalent of Linux implementation of this API).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index bef10aba30..ccf0bc3be9 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -188,6 +188,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
 	DRV_LOG(WARNING, "mlx5_os_mac_addr_remove is not support");
 }
 
+/**
+ * Adds a MAC address to the device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param mac_addr
+ *   MAC address to register.
+ * @param index
+ *   MAC address index.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
+		     uint32_t index)
+{
+	(void)index;
+	struct rte_ether_addr lmac;
+
+	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
+		DRV_LOG(ERR,
+			"port %u cannot get MAC address, is mlx5_en"
+			" loaded? (errno: %s)",
+			dev->data->port_id, strerror(rte_errno));
+		return rte_errno;
+	}
+	if (!rte_is_same_ether_addr(&lmac, mac)) {
+		DRV_LOG(ERR,
+			"adding new mac address to device is unsupported");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
 /**
  * Modify a VF MAC address
  * Currently it has no support under Windows.
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v3 32/32] net/mlx5: refactor eth dev ops for Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

There are two types of eth_dev_ops used under Windows: primary and
isolate mode. Their function calls initialization is added to the OS
specific file mlx5_os.c. Secondary process eth_dev_ops is nullified.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 108 +++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index ccf0bc3be9..43665bff7f 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -296,3 +296,111 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
+
+const struct eth_dev_ops mlx5_os_dev_ops = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.read_clock = mlx5_read_clock,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.reta_update = mlx5_dev_rss_reta_update,
+	.reta_query = mlx5_dev_rss_reta_query,
+	.rss_hash_update = mlx5_rss_hash_update,
+	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
+
+/* Available operations from secondary process. */
+const struct eth_dev_ops mlx5_os_dev_sec_ops = {0};
+
+/* Available operations in flow isolated mode. */
+const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 00/32] mlx5 Windows support - part #5

[Tal Shnaiderman]

This patch series is part of preparing mlx5 PMD to compile and run under Windows OS. Part #5

v1: initial version
v2: Fixes after CR and rebase, also patchset was split and part of the commits sent in v1 will be sent later on.
v3: Remove common compilation as it depends on upcoming patches [DmitryK].
v4: Fix syntax error in warning message.

Ophir Munk (20):
  net/mlx5/linux: extend device attributes getter
  net/mlx5: remove Linux files from Windows compilation
  net/mlx5: fix freeing packet pacing
  net/mlx5: replace Linux sleep with rte sleep
  net/mlx5: define mprq functions as static inline
  net/mlx5: do not define static_assert in Windows
  net/mlx5: move static_assert calls to global scope
  net/mlx5: wrap glue alloc/dealloc PD with OS calls
  net/mlx5: fix adding destroy flow action wrapper
  common/mlx5: add definition HAVE_INFINIBAND_VERBS_H
  common/mlx5/linux: handle memory allocations with alignment
  common/mlx5/windows: handle memory allocations with alignment
  mlx5/windows: add initialization routine for external lib
  mlx5/windows: generate file mlx5_autoconf.h
  common/mlx5/windows: add getter functions
  net/mlx5: update MR prototypes for DevX
  common/mlx5/windows: add OS reg/dereg MR
  net/mlx5/windows: implement device attribute getter
  net/mlx5/windows: add mlx5_os.c stubs
  net/mlx5: refactor eth dev ops for Windows

Tal Shnaiderman (12):
  net/mlx5: fix folding constant array error
  net/mlx5: wrap glue reg/dereg UMEM with OS calls
  common/mlx5/linux: wrap event channel APIs with OS calls
  common/mlx5: add Windows exports file
  common/mlx5: extend DevX query hca attributes command
  common/mlx5: add DevX alloc PD command
  common/mlx5/windows: add glue functions APIs
  mlx5/windows: add mlx5 meson file
  common/mlx5/windows: extend PRM match_param_bits struct
  common/mlx5/windows: add OS alloc/dealloc pd
  common/mlx5/windows: add OS umem reg/dereg API
  net/mlx5/windows: implement mlx5 mac addr add

 drivers/common/mlx5/linux/meson.build           |   6 +-
 drivers/common/mlx5/linux/mlx5_common_os.h      |  83 +++++
 drivers/common/mlx5/mlx5_common.c               |   4 +-
 drivers/common/mlx5/mlx5_common_mr.h            |   1 +
 drivers/common/mlx5/mlx5_devx_cmds.c            |  46 +++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  12 +-
 drivers/common/mlx5/mlx5_malloc.c               |  14 +-
 drivers/common/mlx5/mlx5_prm.h                  |  87 +++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  69 ++++
 drivers/common/mlx5/version.map                 |   1 +
 drivers/common/mlx5/windows/meson.build         |  46 +++
 drivers/common/mlx5/windows/mlx5_common_os.c    | 210 ++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    | 154 +++++++++
 drivers/common/mlx5/windows/mlx5_glue.c         | 307 ++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 ++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  40 +++
 drivers/net/mlx5/linux/mlx5_os.c                |   9 +-
 drivers/net/mlx5/linux/mlx5_verbs.c             |   2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h             |   8 +-
 drivers/net/mlx5/meson.build                    |  16 +-
 drivers/net/mlx5/mlx5.c                         |  18 +-
 drivers/net/mlx5/mlx5.h                         |  11 +
 drivers/net/mlx5/mlx5_defs.h                    |   7 +-
 drivers/net/mlx5/mlx5_devx.c                    |  24 +-
 drivers/net/mlx5/mlx5_flow.c                    |  17 +-
 drivers/net/mlx5/mlx5_flow_dv.c                 |   6 +-
 drivers/net/mlx5/mlx5_rxq.c                     |  71 -----
 drivers/net/mlx5/mlx5_rxtx.c                    |  98 +++---
 drivers/net/mlx5/mlx5_rxtx.h                    |  73 ++++-
 drivers/net/mlx5/mlx5_trigger.c                 |   3 +-
 drivers/net/mlx5/mlx5_txpp.c                    |  34 +-
 drivers/net/mlx5/windows/meson.build            |   8 +
 drivers/net/mlx5/windows/mlx5_os.c              | 406 ++++++++++++++++++++++++
 drivers/vdpa/mlx5/mlx5_vdpa_event.c             |   7 +-
 35 files changed, 1778 insertions(+), 203 deletions(-)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def
 create mode 100644 drivers/common/mlx5/windows/meson.build
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 01/32] net/mlx5: fix folding constant array error

[Tal Shnaiderman]

Before this commit the PMD used:
   const int elt_n = 8
   const int *stack[elt_n];

In Windows clang compiler complains:
net/mlx5/mlx5_flow.c:215:19: error: variable length array folded
to constant array as an extension [-Werror,-Wgnu-folding-constant]

Fix it by using a constant macro definition instead of a variable:
   #define MLX5_RSS_EXP_ELT_N 8
   const int *stack[MLX5_RSS_EXP_ELT_N];

Fixes: c7870bfe09dc ("ethdev: move RSS expansion code to mlx5 driver")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 82e24d7067..bf86aaaa39 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -212,6 +212,8 @@ mlx5_flow_expand_rss_item_complete(const struct rte_flow_item *item)
 	return ret;
 }
 
+#define MLX5_RSS_EXP_ELT_N 8
+
 /**
  * Expand RSS flows into several possible flows according to the RSS hash
  * fields requested and the driver capabilities.
@@ -242,13 +244,12 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		     const struct mlx5_flow_expand_node graph[],
 		     int graph_root_index)
 {
-	const int elt_n = 8;
 	const struct rte_flow_item *item;
 	const struct mlx5_flow_expand_node *node = &graph[graph_root_index];
 	const int *next_node;
-	const int *stack[elt_n];
+	const int *stack[MLX5_RSS_EXP_ELT_N];
 	int stack_pos = 0;
-	struct rte_flow_item flow_items[elt_n];
+	struct rte_flow_item flow_items[MLX5_RSS_EXP_ELT_N];
 	unsigned int i;
 	size_t lsize;
 	size_t user_pattern_size = 0;
@@ -261,10 +262,10 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 
 	memset(&missed_item, 0, sizeof(missed_item));
 	lsize = offsetof(struct mlx5_flow_expand_rss, entry) +
-		elt_n * sizeof(buf->entry[0]);
+		MLX5_RSS_EXP_ELT_N * sizeof(buf->entry[0]);
 	if (lsize <= size) {
 		buf->entry[0].priority = 0;
-		buf->entry[0].pattern = (void *)&buf->entry[elt_n];
+		buf->entry[0].pattern = (void *)&buf->entry[MLX5_RSS_EXP_ELT_N];
 		buf->entries = 0;
 		addr = buf->entry[0].pattern;
 	}
@@ -367,7 +368,7 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		/* Go deeper. */
 		if (node->next) {
 			next_node = node->next;
-			if (stack_pos++ == elt_n) {
+			if (stack_pos++ == MLX5_RSS_EXP_ELT_N) {
 				rte_errno = E2BIG;
 				return -rte_errno;
 			}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 02/32] net/mlx5/linux: extend device attributes getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 5 +++++
 drivers/net/mlx5/mlx5.h          | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 917d6be7b8..a6c1eba640 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -116,7 +116,12 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	device_attr->max_qp_wr = attr_ex.orig_attr.max_qp_wr;
 	device_attr->max_sge = attr_ex.orig_attr.max_sge;
 	device_attr->max_cq = attr_ex.orig_attr.max_cq;
+	device_attr->max_cqe = attr_ex.orig_attr.max_cqe;
+	device_attr->max_mr = attr_ex.orig_attr.max_mr;
+	device_attr->max_pd = attr_ex.orig_attr.max_pd;
 	device_attr->max_qp = attr_ex.orig_attr.max_qp;
+	device_attr->max_srq = attr_ex.orig_attr.max_srq;
+	device_attr->max_srq_wr = attr_ex.orig_attr.max_srq_wr;
 	device_attr->raw_packet_caps = attr_ex.raw_packet_caps;
 	device_attr->max_rwq_indirection_table_size =
 		attr_ex.rss_caps.max_rwq_indirection_table_size;
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 121d726405..2186677810 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -83,6 +83,11 @@ struct mlx5_dev_attr {
 	int		max_sge;
 	int		max_cq;
 	int		max_qp;
+	int		max_cqe;
+	uint32_t	max_pd;
+	uint32_t	max_mr;
+	uint32_t	max_srq;
+	uint32_t	max_srq_wr;
 	uint32_t	raw_packet_caps;
 	uint32_t	max_rwq_indirection_table_size;
 	uint32_t	max_tso;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 03/32] net/mlx5: remove Linux files from Windows compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit removes Linux files flow_verbs.c and mlx5_rxtx_vec.c
from Windows compilation.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/meson.build | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index e7495a764a..f2fafbdd05 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -15,7 +15,6 @@ sources = files(
 	'mlx5_flow.c',
 	'mlx5_flow_meter.c',
 	'mlx5_flow_dv.c',
-	'mlx5_flow_verbs.c',
         'mlx5_flow_age.c',
 	'mlx5_mac.c',
 	'mlx5_mr.c',
@@ -31,11 +30,18 @@ sources = files(
 	'mlx5_utils.c',
 	'mlx5_devx.c',
 )
-if (dpdk_conf.has('RTE_ARCH_X86_64')
-	or dpdk_conf.has('RTE_ARCH_ARM64')
-	or dpdk_conf.has('RTE_ARCH_PPC_64'))
-	sources += files('mlx5_rxtx_vec.c')
+
+if is_linux
+	sources += files(
+		'mlx5_flow_verbs.c',
+	)
+	if (dpdk_conf.has('RTE_ARCH_X86_64')
+		or dpdk_conf.has('RTE_ARCH_ARM64')
+		or dpdk_conf.has('RTE_ARCH_PPC_64'))
+		sources += files('mlx5_rxtx_vec.c')
+	endif
 endif
+
 cflags_options = [
 	'-std=c11',
 	'-Wno-strict-prototypes',
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 04/32] net/mlx5: fix freeing packet pacing

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Packet pacing is allocated under condition #ifdef HAVE_MLX5DV_PP_ALLOC.
In a similar way - free packet pacing index under the same condition.
This update is required to successfully compile under operating systems
which do not support packet pacing.

Fixes: aef1e20ebeb2 ("net/mlx5: allocate packet pacing context")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_txpp.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 2438bf1f1d..21675ab17a 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -57,11 +57,16 @@ mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 static void
 mlx5_txpp_free_pp_index(struct mlx5_dev_ctx_shared *sh)
 {
+#ifdef HAVE_MLX5DV_PP_ALLOC
 	if (sh->txpp.pp) {
 		mlx5_glue->dv_free_pp(sh->txpp.pp);
 		sh->txpp.pp = NULL;
 		sh->txpp.pp_id = 0;
 	}
+#else
+	RTE_SET_USED(sh);
+	DRV_LOG(ERR, "Freeing pacing index is not supported.");
+#endif
 }
 
 /* Allocate Packet Pacing index from kernel via mlx5dv call. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 05/32] net/mlx5: replace Linux sleep with rte sleep

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Replace Linux API usleep() and nanosleep() with rte_delay_us_sleep().
The replacement occurs in shared files compiled under different
operating systems.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_trigger.c | 3 ++-
 drivers/net/mlx5/mlx5_txpp.c    | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 52a8a252d4..8e9df6abcf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -21,6 +21,7 @@
 #include <rte_spinlock.h>
 #include <rte_string_fns.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
@@ -1330,7 +1331,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_flex_parser_ecpri_release(dev);
 	if (priv->rxqs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
@@ -1338,7 +1339,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->txqs_n); ++i)
 			mlx5_txq_release(dev, i);
 		priv->txqs_n = 0;
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index bd029154f8..646f29b923 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -9,6 +9,7 @@
 #include <rte_ethdev_driver.h>
 #include <rte_interrupts.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_malloc.h>
 
@@ -1171,7 +1172,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	rte_wmb();
 	/* Disable datapath on secondary process. */
 	mlx5_mp_os_req_stop_rxtx(dev);
-	usleep(1000 * priv->rxqs_n);
+	rte_delay_us_sleep(1000 * priv->rxqs_n);
 	DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id);
 	mlx5_flow_stop_default(dev);
 	/* Control flows for default traffic can be removed firstly. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 21675ab17a..a1ec294942 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -769,15 +769,11 @@ mlx5_txpp_init_timestamp(struct mlx5_dev_ctx_shared *sh)
 	sh->txpp.ts_p = 0;
 	sh->txpp.ts_n = 0;
 	for (wait = 0; wait < MLX5_TXPP_WAIT_INIT_TS; wait++) {
-		struct timespec onems;
-
 		mlx5_txpp_update_timestamp(sh);
 		if (wq->sq_ci)
 			return;
 		/* Wait one millisecond and try again. */
-		onems.tv_sec = 0;
-		onems.tv_nsec = NS_PER_S / MS_PER_S;
-		nanosleep(&onems, 0);
+		rte_delay_us_sleep(US_PER_S / MS_PER_S);
 	}
 	DRV_LOG(ERR, "Unable to initialize timestamp.");
 	sh->txpp.sync_lost = 1;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 06/32] net/mlx5: define mprq functions as static inline

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Functions mlx5_check_mprq_support(), mlx5_rxq_mprq_enabled(),
mlx5_mprq_enabled() are moved from source file mlx5_rxq.c to header file
mlx5_rxtx.h and their type is updated to 'static __rte_always_inline'.
Previously the functions were declared as 'inline' in the source file
which was reported as 'unresolved external symbol' error by some Windows
linkers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxq.c  | 71 ------------------------------------------
 drivers/net/mlx5/mlx5_rxtx.h | 73 ++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 70 insertions(+), 74 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index da7a8b3cd7..c89e55b1c0 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -48,77 +48,6 @@ static_assert(MLX5_RSS_HASH_KEY_LEN ==
 	      (unsigned int)sizeof(rss_hash_default_key),
 	      "wrong RSS default key size.");
 
-/**
- * Check whether Multi-Packet RQ can be enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   1 if supported, negative errno value if not.
- */
-inline int
-mlx5_check_mprq_support(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	if (priv->config.mprq.enabled &&
-	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
-		return 1;
-	return -ENOTSUP;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the Rx queue.
- *
- *  @param rxq
- *     Pointer to receive queue structure.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
-{
-	return rxq->strd_num_n > 0;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_mprq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	uint32_t i;
-	uint16_t n = 0;
-	uint16_t n_ibv = 0;
-
-	if (mlx5_check_mprq_support(dev) < 0)
-		return 0;
-	/* All the configured queues should be enabled. */
-	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
-
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
-			continue;
-		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
-			++n;
-	}
-	/* Multi-Packet RQ can't be partially configured. */
-	MLX5_ASSERT(n == 0 || n == n_ibv);
-	return n == n_ibv;
-}
-
 /**
  * Calculate the number of CQEs in CQ for the Rx queue.
  *
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 7989a50403..1e9345af61 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -311,9 +311,6 @@ struct mlx5_txq_ctrl {
 
 extern uint8_t rss_hash_default_key[];
 
-int mlx5_check_mprq_support(struct rte_eth_dev *dev);
-int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq);
-int mlx5_mprq_enabled(struct rte_eth_dev *dev);
 unsigned int mlx5_rxq_cqe_num(struct mlx5_rxq_data *rxq_data);
 int mlx5_mprq_free_mp(struct rte_eth_dev *dev);
 int mlx5_mprq_alloc_mp(struct rte_eth_dev *dev);
@@ -920,4 +917,74 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Multi-Packet RQ can be enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   1 if supported, negative errno value if not.
+ */
+static __rte_always_inline int
+mlx5_check_mprq_support(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	if (priv->config.mprq.enabled &&
+	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
+		return 1;
+	return -ENOTSUP;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the Rx queue.
+ *
+ *  @param rxq
+ *     Pointer to receive queue structure.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
+{
+	return rxq->strd_num_n > 0;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_mprq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint32_t i;
+	uint16_t n = 0;
+	uint16_t n_ibv = 0;
+
+	if (mlx5_check_mprq_support(dev) < 0)
+		return 0;
+	/* All the configured queues should be enabled. */
+	for (i = 0; i < priv->rxqs_n; ++i) {
+		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
+			(rxq, struct mlx5_rxq_ctrl, rxq);
+
+		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+			continue;
+		n_ibv++;
+		if (mlx5_rxq_mprq_enabled(rxq))
+			++n;
+	}
+	/* Multi-Packet RQ can't be partially configured. */
+	MLX5_ASSERT(n == 0 || n == n_ibv);
+	return n == n_ibv;
+}
 #endif /* RTE_PMD_MLX5_RXTX_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 07/32] net/mlx5: do not define static_assert in Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

In Linux 'static_assert' is defined in file mlx5_defs.h:

 #ifndef HAVE_STATIC_ASSERT
    #define static_assert _Static_assert
 #endif

The same definition can originate from Linux file /usr/include/assert.h.

In Windows static_assert is used while _Static_assert is unknown.
Therefore update the definition condition to be:

 #if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
    #define static_assert _Static_assert
 #endif

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_defs.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index aa55db3750..85a0979653 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -199,8 +199,11 @@
 /* Maximum number of shared actions supported by rte_flow */
 #define MLX5_MAX_SHARED_ACTIONS 2
 
-/* Definition of static_assert found in /usr/include/assert.h */
-#ifndef HAVE_STATIC_ASSERT
+/*
+ * Linux definition of static_assert is found in /usr/include/assert.h.
+ * Windows does not require a redefinition.
+ */
+#if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
 #define static_assert _Static_assert
 #endif
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 08/32] net/mlx5: move static_assert calls to global scope

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Some Windows compilers consider static_assert() as calls to another
function rather than a compiler directive which allows checking type
information at compile time.  This only occurs if the static_assert call
appears inside another function scope. To solve it move the
static_assert calls to global scope in the files where they are used.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxtx.c | 98 ++++++++++++++++++++++----------------------
 drivers/net/mlx5/mlx5_txpp.c |  5 ++-
 2 files changed, 53 insertions(+), 50 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index d12d746c2f..65a1f997e9 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -79,6 +79,56 @@ static uint16_t mlx5_tx_burst_##func(void *txq, \
 
 #define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx},
 
+/* static asserts */
+static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
+static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
+		(2 * RTE_ETHER_ADDR_LEN),
+		"invalid Data Segment data size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert((sizeof(struct rte_vlan_hdr) +
+			sizeof(struct rte_ether_hdr)) ==
+		MLX5_ESEG_MIN_INLINE_SIZE,
+		"invalid min inline data size");
+static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
+		MLX5_DSEG_MAX, "invalid WQE max size");
+static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Control Segment size");
+static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Ethernet Segment size");
+static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Data Segment size");
+static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
+		"invalid WQE size");
+
 static __rte_always_inline uint32_t
 rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe,
 				   volatile struct mlx5_mini_cqe8 *mcqe);
@@ -2070,8 +2120,6 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq,
 	bool ring_doorbell = false;
 	int ret;
 
-	static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
-	static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
 	do {
 		volatile struct mlx5_cqe *cqe;
 
@@ -2381,15 +2429,6 @@ mlx5_tx_eseg_dmin(struct mlx5_txq_data *__rte_restrict txq __rte_unused,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = RTE_BE16(MLX5_ESEG_MIN_INLINE_SIZE);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2474,15 +2513,6 @@ mlx5_tx_eseg_data(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = rte_cpu_to_be_16(inlen);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2697,15 +2727,6 @@ mlx5_tx_eseg_mdat(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	MLX5_ASSERT(inlen >= MLX5_ESEG_MIN_INLINE_SIZE);
 	pdst = (uint8_t *)&es->inline_data;
 	if (MLX5_TXOFF_CONFIG(VLAN) && vlan) {
@@ -2952,9 +2973,6 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *__rte_restrict txq,
 	uint8_t *pdst;
 
 	MLX5_ASSERT(len > MLX5_ESEG_MIN_INLINE_SIZE);
-	static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
-				 (2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Data Segment data size");
 	if (!MLX5_TXOFF_CONFIG(MPW)) {
 		/* Store the descriptor byte counter for eMPW sessions. */
 		dseg->bcount = rte_cpu_to_be_32
@@ -4070,7 +4088,6 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4247,7 +4264,6 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4561,10 +4577,6 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq,
 			    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
 				vlan = sizeof(struct rte_vlan_hdr);
 				inlen += vlan;
-				static_assert((sizeof(struct rte_vlan_hdr) +
-					       sizeof(struct rte_ether_hdr)) ==
-					       MLX5_ESEG_MIN_INLINE_SIZE,
-					       "invalid min inline data size");
 			}
 			/*
 			 * If inlining is enabled at configuration time
@@ -5567,16 +5579,6 @@ mlx5_select_tx_function(struct rte_eth_dev *dev)
 	uint64_t tx_offloads = dev->data->dev_conf.txmode.offloads;
 	unsigned int diff = 0, olx = 0, i, m;
 
-	static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
-		      MLX5_DSEG_MAX, "invalid WQE max size");
-	static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Control Segment size");
-	static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Ethernet Segment size");
-	static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Data Segment size");
-	static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
-		      "invalid WQE size");
 	MLX5_ASSERT(priv);
 	if (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) {
 		/* We should support Multi-Segment Packets. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index a1ec294942..d61e43e55d 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -18,6 +18,9 @@
 #include "mlx5_rxtx.h"
 #include "mlx5_common_os.h"
 
+static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
+		"Wrong timestamp CQE part size");
+
 static const char * const mlx5_txpp_stat_names[] = {
 	"tx_pp_missed_interrupt_errors", /* Missed service interrupt. */
 	"tx_pp_rearm_queue_errors", /* Rearm Queue errors. */
@@ -741,8 +744,6 @@ mlx5_txpp_update_timestamp(struct mlx5_dev_ctx_shared *sh)
 	uint64_t ts;
 	uint16_t ci;
 
-	static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
-		      "Wrong timestamp CQE part size");
 	mlx5_atomic_read_cqe((rte_int128_t *)&cqe->timestamp, &to.u128);
 	if (to.cts.op_own >> 4) {
 		DRV_LOG(DEBUG, "Clock Queue error sync lost.");
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 09/32] net/mlx5: wrap glue alloc/dealloc PD with OS calls

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Wrap glue calls alloc_pd() and dealloc_pd() with generic OS calls.  In
Linux - protection domain allocations are implemented by Verbs glue API
while in Windows it is by DevX API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 11 +++++++++++
 drivers/net/mlx5/mlx5.c                    |  6 +++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index 3420937859..c9f8d7cbc1 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -201,4 +201,15 @@ mlx5_os_get_devx_uar_page_id(void *uar)
 #endif
 }
 
+static inline void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	return mlx5_glue->alloc_pd(ctx);
+}
+
+static inline int
+mlx5_os_dealloc_pd(void *pd)
+{
+	return mlx5_glue->dealloc_pd(pd);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 8e9df6abcf..7d1b4fbdc0 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -932,7 +932,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sh->port[i].ih_port_id = RTE_MAX_ETHPORTS;
 		sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS;
 	}
-	sh->pd = mlx5_glue->alloc_pd(sh->ctx);
+	sh->pd = mlx5_os_alloc_pd(sh->ctx);
 	if (sh->pd == NULL) {
 		DRV_LOG(ERR, "PD allocation failure");
 		err = ENOMEM;
@@ -1032,7 +1032,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	if (sh->tx_uar)
 		mlx5_glue->devx_free_uar(sh->tx_uar);
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->ctx)
 		claim_zero(mlx5_glue->close_device(sh->ctx));
 	mlx5_free(sh);
@@ -1100,7 +1100,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		sh->tx_uar = NULL;
 	}
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->tis)
 		claim_zero(mlx5_devx_cmd_destroy(sh->tis));
 	if (sh->td)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 10/32] net/mlx5: wrap glue reg/dereg UMEM with OS calls

[Tal Shnaiderman]

Wrap glue calls for UMEM registration and deregistration with generic OS
calls since each OS (Linux or Windows) has a different glue API
parameters.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 12 ++++++++++++
 drivers/common/mlx5/mlx5_common.c          |  4 ++--
 drivers/net/mlx5/mlx5.c                    |  2 +-
 drivers/net/mlx5/mlx5_devx.c               | 16 ++++++++--------
 drivers/net/mlx5/mlx5_flow.c               |  4 ++--
 drivers/net/mlx5/mlx5_txpp.c               | 12 ++++++------
 6 files changed, 31 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index c9f8d7cbc1..f8b215cc29 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -212,4 +212,16 @@ mlx5_os_dealloc_pd(void *pd)
 {
 	return mlx5_glue->dealloc_pd(pd);
 }
+
+static inline void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	return mlx5_glue->devx_umem_reg(ctx, addr, size, access);
+}
+
+static inline int
+mlx5_os_umem_dereg(void *pumem)
+{
+	return mlx5_glue->devx_umem_dereg(pumem);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c
index 044513223c..a00ffcb833 100644
--- a/drivers/common/mlx5/mlx5_common.c
+++ b/drivers/common/mlx5/mlx5_common.c
@@ -148,7 +148,7 @@ mlx5_alloc_dbr_page(void *ctx)
 		return NULL;
 	}
 	/* Register allocated memory. */
-	page->umem = mlx5_glue->devx_umem_reg(ctx, page->dbrs,
+	page->umem = mlx5_os_umem_reg(ctx, page->dbrs,
 					      MLX5_DBR_PAGE_SIZE, 0);
 	if (!page->umem) {
 		DRV_LOG(ERR, "cannot umem reg dbr page");
@@ -232,7 +232,7 @@ mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id,
 		/* Page not used, free it and remove from list. */
 		LIST_REMOVE(page, next);
 		if (page->umem)
-			ret = -mlx5_glue->devx_umem_dereg(page->umem);
+			ret = -mlx5_os_umem_dereg(page->umem);
 		mlx5_free(page);
 	} else {
 		/* Mark in bitmap that this door-bell is not in use. */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 7d1b4fbdc0..84123f8e3d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -482,7 +482,7 @@ mlx5_flow_destroy_counter_stat_mem_mng(struct mlx5_counter_stats_mem_mng *mng)
 
 	LIST_REMOVE(mng, next);
 	claim_zero(mlx5_devx_cmd_destroy(mng->dm));
-	claim_zero(mlx5_glue->devx_umem_dereg(mng->umem));
+	claim_zero(mlx5_os_umem_dereg(mng->umem));
 	mlx5_free(mem);
 }
 
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index de9b204075..235fd5798d 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -155,7 +155,7 @@ mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->rq_dbrec_page;
 
 	if (rxq_ctrl->wq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->wq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->wq_umem);
 		rxq_ctrl->wq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.wqes) {
@@ -182,7 +182,7 @@ mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->cq_dbrec_page;
 
 	if (rxq_ctrl->cq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->cq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->cq_umem);
 		rxq_ctrl->cq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.cqes) {
@@ -375,7 +375,7 @@ mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	if (!buf)
 		return NULL;
 	rxq_data->wqes = buf;
-	rxq_ctrl->wq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	rxq_ctrl->wq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						     buf, wq_size, 0);
 	if (!rxq_ctrl->wq_umem)
 		goto error;
@@ -497,7 +497,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	rxq_data->cqes = (volatile struct mlx5_cqe (*)[])(uintptr_t)buf;
-	rxq_ctrl->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx, buf,
+	rxq_ctrl->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, buf,
 						     cq_size,
 						     IBV_ACCESS_LOCAL_WRITE);
 	if (!rxq_ctrl->cq_umem) {
@@ -1127,7 +1127,7 @@ mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 		txq_obj->sq_devx = NULL;
 	}
 	if (txq_obj->sq_umem) {
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->sq_umem));
 		txq_obj->sq_umem = NULL;
 	}
 	if (txq_obj->sq_buf) {
@@ -1155,7 +1155,7 @@ mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
 	if (txq_obj->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->cq_umem));
 	if (txq_obj->cq_buf)
 		mlx5_free(txq_obj->cq_buf);
 	if (txq_obj->cq_dbrec_page)
@@ -1243,7 +1243,7 @@ mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		return 0;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	txq_obj->cq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						(void *)txq_obj->cq_buf,
 						cqe_n * sizeof(struct mlx5_cqe),
 						IBV_ACCESS_LOCAL_WRITE);
@@ -1342,7 +1342,7 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->sq_umem = mlx5_glue->devx_umem_reg
+	txq_obj->sq_umem = mlx5_os_umem_reg
 					(priv->sh->ctx,
 					 (void *)txq_obj->sq_buf,
 					 wqe_n * sizeof(struct mlx5_wqe),
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bf86aaaa39..66491bbf78 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -6446,7 +6446,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	}
 	mem_mng = (struct mlx5_counter_stats_mem_mng *)(mem + size) - 1;
 	size = sizeof(*raw_data) * MLX5_COUNTERS_PER_POOL * raws_n;
-	mem_mng->umem = mlx5_glue->devx_umem_reg(sh->ctx, mem, size,
+	mem_mng->umem = mlx5_os_umem_reg(sh->ctx, mem, size,
 						 IBV_ACCESS_LOCAL_WRITE);
 	if (!mem_mng->umem) {
 		rte_errno = errno;
@@ -6465,7 +6465,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	mkey_attr.relaxed_ordering_read = sh->cmng.relaxed_ordering_read;
 	mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr);
 	if (!mem_mng->dm) {
-		mlx5_glue->devx_umem_dereg(mem_mng->umem);
+		mlx5_os_umem_dereg(mem_mng->umem);
 		rte_errno = errno;
 		mlx5_free(mem);
 		return -rte_errno;
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index d61e43e55d..749529c410 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -131,13 +131,13 @@ mlx5_txpp_destroy_send_queue(struct mlx5_txpp_wq *wq)
 	if (wq->sq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->sq));
 	if (wq->sq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->sq_umem));
 	if (wq->sq_buf)
 		mlx5_free((void *)(uintptr_t)wq->sq_buf);
 	if (wq->cq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->cq));
 	if (wq->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->cq_umem));
 	if (wq->cq_buf)
 		mlx5_free((void *)(uintptr_t)wq->cq_buf);
 	memset(wq, 0, sizeof(*wq));
@@ -268,7 +268,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -318,7 +318,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -506,7 +506,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -562,7 +562,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 11/32] net/mlx5: fix adding destroy flow action wrapper

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Glue function destroy_flow_action() was wrapped by OS specific operation
mlx5_flow_os_destroy_flow_action(). It was skipped in file mlx5.c.

Fixes: b293fbf9672b ("net/mlx5: add OS specific flow actions operations")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_flow_dv.c | 6 +++---
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 84123f8e3d..ea5cf80ac1 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -38,6 +38,7 @@
 #include "mlx5_autoconf.h"
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
+#include "mlx5_flow_os.h"
 #include "rte_pmd_mlx5.h"
 
 /* Device parameter to enable RX completion queue compression. */
@@ -415,7 +416,7 @@ mlx5_flow_aso_age_mng_close(struct mlx5_dev_ctx_shared *sh)
 			for (j = 0; j < MLX5_COUNTERS_PER_POOL; ++j)
 				if (pool->actions[j].dr_action)
 					claim_zero
-						(mlx5_glue->destroy_flow_action
+						(mlx5_flow_os_destroy_flow_action
 						  (pool->actions[j].dr_action));
 			mlx5_free(pool);
 		}
@@ -523,7 +524,7 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh)
 
 				if (cnt->action)
 					claim_zero
-					 (mlx5_glue->destroy_flow_action
+					 (mlx5_flow_os_destroy_flow_action
 					  (cnt->action));
 				if (fallback && MLX5_POOL_GET_CNT
 				    (pool, j)->dcs_when_free)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index c31737652c..78b5a6b338 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -11117,11 +11117,11 @@ flow_dv_sample_remove_cb(struct mlx5_cache_list *list __rte_unused,
 	struct mlx5_priv *priv = dev->data->dev_private;
 
 	if (cache_resource->verbs_action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 				(cache_resource->verbs_action));
 	if (cache_resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB) {
 		if (cache_resource->default_miss)
-			claim_zero(mlx5_glue->destroy_flow_action
+			claim_zero(mlx5_flow_os_destroy_flow_action
 			  (cache_resource->default_miss));
 	}
 	if (cache_resource->normal_path_tbl)
@@ -11174,7 +11174,7 @@ flow_dv_dest_array_remove_cb(struct mlx5_cache_list *list __rte_unused,
 
 	MLX5_ASSERT(cache_resource->action);
 	if (cache_resource->action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 					(cache_resource->action));
 	for (; i < cache_resource->num_of_dest; i++)
 		flow_dv_sample_sub_actions_release(dev,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 12/32] common/mlx5: add definition HAVE_INFINIBAND_VERBS_H

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add a Verbs file presence indication. Under Linux it is required that
file infiniband/verbs.h is installed for building DPDK.  Other
operating systems (e.g. Windows) ignore Verbs completely.  This commit
adds definition HAVE_INFINIBAND_VERBS_H (file mlx5_autoconf.h) to
indicate whether DPDK compiles with Verbs or not.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/meson.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build
index 63b78e4bce..580419e6d9 100644
--- a/drivers/common/mlx5/linux/meson.build
+++ b/drivers/common/mlx5/linux/meson.build
@@ -181,8 +181,10 @@ has_sym_args = [
 	[ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h',
 	'mlx5dv_dr_action_create_dest_array'],
 	[ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ],
-        [ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
-        'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
+	'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_INFINIBAND_VERBS_H', 'infiniband/verbs.h',
+	'INFINIBAND_VERBS_H' ],
 ]
 config = configuration_data()
 foreach arg:has_sym_args
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 13/32] common/mlx5/linux: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_malloc() API has an alignment parameter for system memory
allocations. malloc() is called for non-aligned allocations and
posix_memalign() is called for aligned allocations. When calling
mlx5_free() there is no distinction whether the memory was originally
allocated with or without alignment. Freeing a memory may be handled
differently by operating systems. Therefore this commit wraps these APIs
with OS specific calls: mlx5_os_malloc(), mlx5_os_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 38 ++++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_malloc.c          | 14 +++++------
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index f8b215cc29..bd44ecb7d6 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -6,6 +6,7 @@
 #define RTE_PMD_MLX5_COMMON_OS_H_
 
 #include <stdio.h>
+#include <malloc.h>
 
 #include <rte_pci.h>
 #include <rte_debug.h>
@@ -16,6 +17,7 @@
 
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
+#include "mlx5_malloc.h"
 
 /**
  * Get device name. Given an ibv_device pointer - return a
@@ -224,4 +226,40 @@ mlx5_os_umem_dereg(void *pumem)
 {
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
+
+/**
+ * Memory allocation optionally with alignment.
+ *
+ * @param[in] align
+ *    Alignment size (may be zero)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	void *buf;
+
+	if (posix_memalign(&buf, align, size))
+		return NULL;
+	return buf;
+}
+
+/**
+ * This API de-allocates a memory that originally could have been
+ * allocated aligned or non-aligned. In Linux it is a wrapper
+ * around free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	free(addr);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_malloc.c b/drivers/common/mlx5/mlx5_malloc.c
index 5a3267f730..9d30cedbaa 100644
--- a/drivers/common/mlx5/mlx5_malloc.c
+++ b/drivers/common/mlx5/mlx5_malloc.c
@@ -9,6 +9,7 @@
 #include <string.h>
 
 #include "mlx5_common_utils.h"
+#include "mlx5_common_os.h"
 #include "mlx5_malloc.h"
 
 struct mlx5_sys_mem {
@@ -148,14 +149,11 @@ static void *
 mlx5_alloc_align(size_t size, unsigned int align, unsigned int zero)
 {
 	void *buf;
-	int ret;
-
-	ret = posix_memalign(&buf, align, size);
-	if (ret) {
-		DRV_LOG(ERR,
-			"Couldn't allocate buf size=%zu align=%u. Err=%d\n",
-			size, align, ret);
 
+	buf = mlx5_os_malloc(align, size);
+	if (!buf) {
+		DRV_LOG(ERR, "Couldn't allocate buf size=%zu align=%u.",
+			size, align);
 		return NULL;
 	}
 	if (zero)
@@ -264,7 +262,7 @@ mlx5_free(void *addr)
 		__atomic_add_fetch(&mlx5_sys_mem.free_sys, 1,
 				   __ATOMIC_RELAXED);
 #endif
-		free(addr);
+		mlx5_os_free(addr);
 	} else {
 #ifdef RTE_LIBRTE_MLX5_DEBUG
 		__atomic_add_fetch(&mlx5_sys_mem.free_rte, 1,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 14/32] common/mlx5/windows: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows equivalent of the Linux implementation.  The
APIs included in this commit: mlx5_os_malloc(), mlx5_os_free(). For
memory allocations (with or without alignment) we always call
_aligned_malloc(). Even if zero alignment was requested in the first
place - we always select a minimal alignment value. In this way when the
memory is free - it is always safe to call _aligned_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 51 ++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
new file mode 100644
index 0000000000..cce7c88c6a
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_COMMON_OS_H_
+#define RTE_PMD_MLX5_COMMON_OS_H_
+
+#include <stdio.h>
+
+#include "mlx5_autoconf.h"
+#include "mlx5_glue.h"
+#include "mlx5_malloc.h"
+
+/**
+ * This API allocates aligned or non-aligned memory.  The free can be on either
+ * aligned or nonaligned memory.  To be protected - even though there may be no
+ * alignment - in Windows this API will unconditioanlly call _aligned_malloc()
+ * with at least a minimal alignment size.
+ *
+ * @param[in] align
+ *    The alignment value, which must be an integer power of 2 (or 0 for
+ *    non-alignment)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	if (align < MLX5_MALLOC_ALIGNMENT)
+		align = MLX5_MALLOC_ALIGNMENT;
+	return _aligned_malloc(size, align);
+}
+
+/**
+ * This API de-allocates a memory that originally could have been allocated
+ * aligned or non-aligned. In Windows since the allocation was with
+ * _aligned_malloc() - it is safe to always call _aligned_free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	_aligned_free(addr);
+}
+#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 15/32] common/mlx5/linux: wrap event channel APIs with OS calls

[Tal Shnaiderman]

Wrap the API to create/destroy event channel and to subscribe an event
with OS calls. In Linux those calls are implemented by glue functions
while in Windows they are not supported.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 22 ++++++++++++++++++++++
 drivers/net/mlx5/mlx5_devx.c               |  8 ++++----
 drivers/net/mlx5/mlx5_txpp.c               |  6 +++---
 drivers/vdpa/mlx5/mlx5_vdpa_event.c        |  7 ++++---
 4 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index bd44ecb7d6..63f070d9c4 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -227,6 +227,28 @@ mlx5_os_umem_dereg(void *pumem)
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
 
+static inline void *
+mlx5_os_devx_create_event_channel(void *ctx, int flags)
+{
+	return mlx5_glue->devx_create_event_channel(ctx, flags);
+}
+
+static inline void
+mlx5_os_devx_destroy_event_channel(void *eventc)
+{
+	mlx5_glue->devx_destroy_event_channel(eventc);
+}
+
+static inline int
+mlx5_os_devx_subscribe_devx_event(void *eventc,
+				  void *obj,
+				  uint16_t events_sz, uint16_t events_num[],
+				  uint64_t cookie)
+{
+	return mlx5_glue->devx_subscribe_devx_event(eventc, obj, events_sz,
+						    events_num, cookie);
+}
+
 /**
  * Memory allocation optionally with alignment.
  *
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 235fd5798d..84a5c55ee0 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -216,7 +216,7 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_obj *rxq_obj)
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->devx_cq));
 		if (rxq_obj->devx_channel)
-			mlx5_glue->devx_destroy_event_channel
+			mlx5_os_devx_destroy_event_channel
 							(rxq_obj->devx_channel);
 		mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
 		mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
@@ -533,7 +533,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cqe_n = log_cqe_n;
 	rxq_data->cqn = cq_obj->id;
 	if (rxq_ctrl->obj->devx_channel) {
-		ret = mlx5_glue->devx_subscribe_devx_event
+		ret = mlx5_os_devx_subscribe_devx_event
 						(rxq_ctrl->obj->devx_channel,
 						 cq_obj->obj,
 						 sizeof(event_nums),
@@ -644,7 +644,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		int devx_ev_flag =
 			  MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA;
 
-		tmpl->devx_channel = mlx5_glue->devx_create_event_channel
+		tmpl->devx_channel = mlx5_os_devx_create_event_channel
 								(priv->sh->ctx,
 								 devx_ev_flag);
 		if (!tmpl->devx_channel) {
@@ -686,7 +686,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	if (tmpl->devx_cq)
 		claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq));
 	if (tmpl->devx_channel)
-		mlx5_glue->devx_destroy_event_channel(tmpl->devx_channel);
+		mlx5_os_devx_destroy_event_channel(tmpl->devx_channel);
 	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
 	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	rte_errno = ret; /* Restore rte_errno. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 749529c410..726bdc6ae1 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -37,7 +37,7 @@ static void
 mlx5_txpp_destroy_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	if (sh->txpp.echan) {
-		mlx5_glue->devx_destroy_event_channel(sh->txpp.echan);
+		mlx5_os_devx_destroy_event_channel(sh->txpp.echan);
 		sh->txpp.echan = NULL;
 	}
 }
@@ -47,7 +47,7 @@ static int
 mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	MLX5_ASSERT(!sh->txpp.echan);
-	sh->txpp.echan = mlx5_glue->devx_create_event_channel(sh->ctx,
+	sh->txpp.echan = mlx5_os_devx_create_event_channel(sh->ctx,
 			MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!sh->txpp.echan) {
 		rte_errno = errno;
@@ -937,7 +937,7 @@ mlx5_txpp_start_service(struct mlx5_dev_ctx_shared *sh)
 		return -rte_errno;
 	}
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(sh->txpp.echan,
+	ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan,
 						   sh->txpp.rearm_queue.cq->obj,
 						   sizeof(event_nums),
 						   event_nums, 0);
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
index 3aeaeb893f..fd6150928b 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
@@ -15,6 +15,7 @@
 #include <rte_alarm.h>
 
 #include <mlx5_common.h>
+#include <mlx5_common_os.h>
 #include <mlx5_glue.h>
 
 #include "mlx5_vdpa_utils.h"
@@ -43,7 +44,7 @@ mlx5_vdpa_event_qp_global_release(struct mlx5_vdpa_priv *priv)
 		       sizeof(out.buf)) >=
 		       (ssize_t)sizeof(out.event_resp.cookie))
 			;
-		mlx5_glue->devx_destroy_event_channel(priv->eventc);
+		mlx5_os_devx_destroy_event_channel(priv->eventc);
 		priv->eventc = NULL;
 	}
 #endif
@@ -63,7 +64,7 @@ mlx5_vdpa_event_qp_global_prepare(struct mlx5_vdpa_priv *priv)
 		DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno);
 		return -1;
 	}
-	priv->eventc = mlx5_glue->devx_create_event_channel(priv->ctx,
+	priv->eventc = mlx5_os_devx_create_event_channel(priv->ctx,
 			   MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!priv->eventc) {
 		rte_errno = errno;
@@ -176,7 +177,7 @@ mlx5_vdpa_cq_create(struct mlx5_vdpa_priv *priv, uint16_t log_desc_n,
 	cq->cq_ci = 0;
 	rte_spinlock_init(&cq->sl);
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
+	ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
 						   sizeof(event_nums),
 						   event_nums,
 						   (uint64_t)(uintptr_t)cq);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 16/32] common/mlx5: add Windows exports file

[Tal Shnaiderman]

File drivers/common/mlx5/rte_common_mlx5_exports.def contains mlx5
Windows exported symbols under common/mlx5 directory (DLL file
name librte_common_mlx5*.dll). It is the equivalent of Linux map
file rte_common_mlx5_version.map but the list of symbols may be
different between the two operating systems.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def | 62 +++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
new file mode 100644
index 0000000000..23984b7abd
--- /dev/null
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -0,0 +1,62 @@
+EXPORTS
+	haswell_broadwell_cpu
+
+	mlx5_common_init
+
+	mlx5_create_mr_ext
+
+	mlx5_devx_cmd_create_cq
+	mlx5_devx_cmd_create_flex_parser
+	mlx5_devx_cmd_create_qp
+	mlx5_devx_cmd_create_rq
+	mlx5_devx_cmd_create_rqt
+	mlx5_devx_cmd_create_sq
+	mlx5_devx_cmd_create_tir
+	mlx5_devx_cmd_create_td
+	mlx5_devx_cmd_create_tis
+	mlx5_devx_cmd_create_virtq
+	mlx5_devx_cmd_destroy
+	mlx5_devx_cmd_flow_counter_alloc
+	mlx5_devx_cmd_flow_counter_query
+	mlx5_devx_cmd_flow_dump
+	mlx5_devx_cmd_mkey_create
+	mlx5_devx_cmd_modify_qp_state
+	mlx5_devx_cmd_modify_rq
+	mlx5_devx_cmd_modify_rqt
+	mlx5_devx_cmd_modify_sq
+	mlx5_devx_cmd_modify_tir
+	mlx5_devx_cmd_modify_virtq
+	mlx5_devx_cmd_qp_query_tis_td
+	mlx5_devx_cmd_query_hca_attr
+	mlx5_devx_cmd_query_parse_samples
+	mlx5_devx_cmd_query_virtq
+	mlx5_devx_cmd_register_read
+	mlx5_devx_get_out_command_status
+	mlx5_devx_cmd_create_flow_hit_aso_obj
+
+	mlx5_get_dbr
+
+	mlx5_malloc_mem_select
+	mlx5_mr_btree_init
+	mlx5_mr_btree_free
+	mlx5_mr_btree_dump
+	mlx5_mr_addr2mr_bh
+	mlx5_mr_release_cache
+	mlx5_mr_dump_cache
+	mlx5_mr_rebuild_cache
+	mlx5_mr_insert_cache
+	mlx5_mr_lookup_cache
+	mlx5_mr_lookup_list
+	mlx5_mr_create_primary
+	mlx5_mr_flush_local_cache
+	mlx5_mp_req_queue_state_modify
+	mlx5_mr_free
+
+	mlx5_pci_driver_register
+
+	mlx5_release_dbr
+
+	mlx5_malloc
+	mlx5_realloc
+	mlx5_free
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 17/32] common/mlx5: extend DevX query hca attributes command

[Tal Shnaiderman]

Extend DevX API mlx5_devx_cmd_query_hca_attr() to report on max number
of available objects including: CQ, QP, PD, SRQ.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 11 +++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++++++++-
 drivers/common/mlx5/mlx5_prm.h       |  1 +
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 9c1d1883ea..36d3a421aa 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -720,6 +720,14 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->flow_hit_aso = !!(MLX5_GET64(cmd_hca_cap, hcattr,
 					   general_obj_types) &
 				MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
+	attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
+	attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
+	attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
+	attr->log_max_qp_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp_sz);
+	attr->log_max_mrw_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_mrw_sz);
+	attr->log_max_pd = MLX5_GET(cmd_hca_cap, hcattr, log_max_pd);
+	attr->log_max_srq = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq);
+	attr->log_max_srq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq_sz);
 	if (attr->qos.sup) {
 		MLX5_SET(query_hca_cap_in, in, op_mod,
 			 MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
@@ -834,6 +842,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->tunnel_stateless_gtp = MLX5_GET
 					(per_protocol_networking_offload_caps,
 					 hcattr, tunnel_stateless_gtp);
+	attr->rss_ind_tbl_cap = MLX5_GET
+					(per_protocol_networking_offload_caps,
+					 hcattr, rss_ind_tbl_cap);
 	if (attr->wqe_inline_mode != MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
 		return 0;
 	if (attr->eth_virt) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 726e9f5192..5056d86ed9 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -117,6 +117,15 @@ struct mlx5_hca_attr {
 	uint32_t log_max_ft_sampler_num:8;
 	struct mlx5_hca_qos_attr qos;
 	struct mlx5_hca_vdpa_attr vdpa;
+	int log_max_qp_sz;
+	int log_max_cq_sz;
+	int log_max_qp;
+	int log_max_cq;
+	uint32_t log_max_pd;
+	uint32_t log_max_mrw_sz;
+	uint32_t log_max_srq;
+	uint32_t log_max_srq_sz;
+	uint32_t rss_ind_tbl_cap;
 };
 
 struct mlx5_devx_wq_attr {
@@ -495,7 +504,6 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_virtio_q_counters(void *ctx);
 __rte_internal
 int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 				  struct mlx5_devx_virtio_q_couners_attr *attr);
-
 __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 58d180486e..585e9d1c05 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -665,6 +665,7 @@ typedef uint8_t u8;
 #define MLX5_GET64(typ, p, fld) rte_be_to_cpu_64(*((rte_be64_t *)(p) + \
 						   __mlx5_64_off(typ, fld)))
 #define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
+#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
 
 struct mlx5_ifc_fte_match_set_misc_bits {
 	u8 gre_c_present[0x1];
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 18/32] common/mlx5: add DevX alloc PD command

[Tal Shnaiderman]

Add a new DevX API mlx5_devx_cmd_alloc_pd() that creates a new protection
domain (PD).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c            | 35 ++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  2 ++
 drivers/common/mlx5/mlx5_prm.h                  | 36 +++++++++++++++++++++++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  1 +
 drivers/common/mlx5/version.map                 |  1 +
 5 files changed, 75 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 36d3a421aa..12f51a940c 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2055,3 +2055,38 @@ mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx, uint32_t pd)
 	flow_hit_aso_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
 	return flow_hit_aso_obj;
 }
+
+/*
+ * Create PD using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_alloc_pd(void *ctx)
+{
+	struct mlx5_devx_obj *ppd =
+		mlx5_malloc(MLX5_MEM_ZERO, sizeof(*ppd), 0, SOCKET_ID_ANY);
+	u32 in[MLX5_ST_SZ_DW(alloc_pd_in)] = {0};
+	u32 out[MLX5_ST_SZ_DW(alloc_pd_out)] = {0};
+
+	if (!ppd) {
+		DRV_LOG(ERR, "Failed to allocate PD data.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(alloc_pd_in, in, opcode, MLX5_CMD_OP_ALLOC_PD);
+	ppd->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+				out, sizeof(out));
+	if (!ppd->obj) {
+		mlx5_free(ppd);
+		DRV_LOG(ERR, "Failed to allocate PD Obj using DevX.");
+		rte_errno = errno;
+		return NULL;
+	}
+	ppd->id = MLX5_GET(alloc_pd_out, out, pd);
+	return ppd;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 5056d86ed9..8277fdbc39 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -508,4 +508,6 @@ __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
 
+__rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_alloc_pd(void *ctx);
 #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 585e9d1c05..4ddf865743 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -844,6 +844,8 @@ enum {
 	MLX5_CMD_OP_SUSPEND_QP = 0x50F,
 	MLX5_CMD_OP_RESUME_QP = 0x510,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
+	MLX5_CMD_OP_ALLOC_PD = 0x800,
+	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ACCESS_REGISTER = 0x805,
 	MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -2772,6 +2774,40 @@ struct mlx5_ifc_init2init_qp_in_bits {
 	u8 reserved_at_800[0x80];
 };
 
+struct mlx5_ifc_dealloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x8];
+	u8 pd[0x18];
+	u8 reserved_3[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x8];
+	u8 pd[0x18];
+	u8 reserved_2[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x40];
+};
+
 #ifdef PEDANTIC
 #pragma GCC diagnostic ignored "-Wpedantic"
 #endif
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 23984b7abd..15aafc3809 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -5,6 +5,7 @@ EXPORTS
 
 	mlx5_create_mr_ext
 
+	mlx5_devx_cmd_alloc_pd
 	mlx5_devx_cmd_create_cq
 	mlx5_devx_cmd_create_flex_parser
 	mlx5_devx_cmd_create_qp
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 17dd11f635..fb3952bbc6 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -10,6 +10,7 @@ INTERNAL {
 
 	mlx5_dev_to_pci_addr;
 
+	mlx5_devx_cmd_alloc_pd;
 	mlx5_devx_cmd_create_cq;
 	mlx5_devx_cmd_create_flex_parser;
 	mlx5_devx_cmd_create_qp;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 19/32] common/mlx5/windows: add glue functions APIs

[Tal Shnaiderman]

Windows glue functions are added to file mlx5/windows/mlx5_glue.c.
The following APIs are supported:
get_device_list, free_device_list, open_device, close_device,
query_device, query_hca_iseg, devx_obj_create, devx_obj_destroy,
devx_obj_query, devx_obj_modify, devx_general_cmd, devx_umem_reg,
devx_umem_dereg, devx_alloc_uar, devx_free_uar, devx_fs_rule_add,
devx_fs_rule_del, devx_query_eqn
New added files:
mlx5_win_defs.h - this file imports missing definitions from Linux
rdma-core library and Linux OS.
mlx5_win_ext.h - this file contains structs that enable a unified
Linux/Windows API. Each struct has an equivalent (but different) Linux
struct. By calling with 'void *' pointers - the Linux/Windows API is
identical.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |   1 +
 drivers/common/mlx5/windows/mlx5_glue.c         | 304 ++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 +++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  34 +++
 5 files changed, 422 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 15aafc3809..2f6aab133b 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -36,6 +36,7 @@ EXPORTS
 	mlx5_devx_cmd_create_flow_hit_aso_obj
 
 	mlx5_get_dbr
+	mlx5_glue
 
 	mlx5_malloc_mem_select
 	mlx5_mr_btree_init
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
new file mode 100644
index 0000000000..7f8a00aaa7
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -0,0 +1,304 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+
+#include "mlx5_glue.h"
+#include "mlx5_common_utils.h"
+#include "mlx5_win_ext.h"
+
+/*
+ * The returned value of this API is an array of pointers to mlx5
+ * devices under Windows. The interesting parameters of a device:
+ * Device PCI parameters: domain, bus, device id, function.
+ * Device name.
+ */
+static void *
+mlx5_glue_devx_get_device_list(int *num_devices)
+{
+	struct devx_device_bdf *devx_bdf_devs = NULL;
+	size_t n_devx_devx = 0;
+	int32_t ret = devx_get_device_list(&n_devx_devx, &devx_bdf_devs);
+
+	if (ret) {
+		errno = ret;
+		*num_devices = 0;
+		return NULL;
+	}
+	*num_devices = (int)n_devx_devx;
+	return devx_bdf_devs;
+}
+
+static void
+mlx5_glue_devx_free_device_list(void *list)
+{
+	if (!list) {
+		errno = EINVAL;
+		return;
+	}
+	devx_free_device_list(list);
+}
+
+static int
+mlx5_glue_devx_close_device(void *ctx)
+{
+	mlx5_context_st *mlx5_ctx;
+	int rc;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (mlx5_context_st *)ctx;
+	rc = devx_close_device(mlx5_ctx->devx_ctx);
+	free(mlx5_ctx);
+	return rc;
+}
+
+static void *
+mlx5_glue_devx_open_device(void *device)
+{
+	struct mlx5_context *mlx5_ctx;
+
+	if (!device) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = malloc((sizeof(struct mlx5_context)));
+	if (!mlx5_ctx) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(mlx5_ctx, 0, sizeof(*mlx5_ctx));
+	mlx5_ctx->devx_ctx = devx_open_device(device);
+	if (DEVX_IS_ERR(mlx5_ctx->devx_ctx)) {
+		errno = -DEVX_PTR_ERR(mlx5_ctx->devx_ctx);
+		free(mlx5_ctx);
+		return NULL;
+	}
+	return mlx5_ctx;
+}
+
+static int
+mlx5_glue_devx_query_device(void *device_bdf, void *dev_inf)
+{
+	struct devx_device_bdf *dev_bdf;
+	struct devx_device *mlx5_dev;
+
+	if (!device_bdf)
+		return -EINVAL;
+	dev_bdf = (struct devx_device_bdf *)device_bdf;
+	mlx5_dev = (struct devx_device *)dev_inf;
+	int err = devx_query_device(dev_bdf, mlx5_dev);
+	if (err)
+		return -E_FAIL;
+	return 0;
+}
+
+static void *
+mlx5_glue_devx_query_hca_iseg_mapping(void *ctx, uint32_t *cb_iseg)
+{
+	struct mlx5_context *mlx5_ctx;
+	void *pv_iseg;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	err = devx_query_hca_iseg_mapping(mlx5_ctx->devx_ctx,
+						cb_iseg, &pv_iseg);
+	if (err) {
+		errno = err;
+		return NULL;
+	}
+	return pv_iseg;
+}
+
+static void *
+mlx5_glue_devx_obj_create(void *ctx,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	devx_obj = malloc((sizeof(*devx_obj)));
+	if (!devx_obj) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(devx_obj, 0, sizeof(*devx_obj));
+	devx_obj->devx_ctx = GET_DEVX_CTX(ctx);
+	devx_obj->obj = devx_obj_create(devx_obj->devx_ctx,
+					in, inlen, out, outlen);
+	if (DEVX_IS_ERR(devx_obj->obj)) {
+		errno = -DEVX_PTR_ERR(devx_obj->obj);
+		free(devx_obj);
+		return NULL;
+	}
+	return devx_obj;
+}
+
+static int
+mlx5_glue_devx_obj_destroy(void *obj)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!obj)
+		return -EINVAL;
+	devx_obj = obj;
+	int rc = devx_obj_destroy(devx_obj->obj);
+	free(devx_obj);
+	return rc;
+}
+
+static int
+mlx5_glue_devx_general_cmd(void *ctx,
+			   void *in, size_t inlen,
+			   void *out, size_t outlen)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_cmd(GET_DEVX_CTX(ctx), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_query(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_modify(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_umem_dereg(void *pumem)
+{
+	struct devx_obj_handle *umem;
+
+	if (!pumem)
+		return -EINVAL;
+	umem = pumem;
+	return devx_umem_unreg(umem);
+}
+
+static void *
+mlx5_glue_devx_umem_reg(void *ctx, void *addr, size_t size,
+				  uint32_t access, uint32_t *id)
+{
+	struct devx_obj_handle *umem_hdl;
+	int w_access = DEVX_UMEM_ACCESS_READ;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	if (access)
+		w_access |= DEVX_UMEM_ACCESS_WRITE;
+
+	umem_hdl = devx_umem_reg(GET_DEVX_CTX(ctx), addr,
+					size, w_access, id);
+	if (DEVX_IS_ERR(umem_hdl)) {
+		errno = -DEVX_PTR_ERR(umem_hdl);
+		return NULL;
+	}
+	return umem_hdl;
+}
+
+static void *
+mlx5_glue_devx_alloc_uar(void *ctx,
+		uint32_t flags)
+{
+	devx_uar_handle *uar;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	uar = devx_alloc_uar(GET_DEVX_CTX(ctx), flags);
+	if (DEVX_IS_ERR(uar)) {
+		errno = -DEVX_PTR_ERR(uar);
+		return NULL;
+	}
+	return uar;
+}
+
+static int
+mlx5_glue_devx_query_eqn(void *ctx,
+		uint32_t cpus, uint32_t *eqn)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_query_eqn(GET_DEVX_CTX(ctx), cpus, eqn);
+}
+
+static void
+mlx5_glue_devx_free_uar(void *uar)
+{
+	devx_free_uar((devx_uar_handle *)uar);
+}
+
+static void*
+mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
+
+{
+	struct devx_obj_handle *rule_hdl = NULL;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	rule_hdl = devx_fs_rule_add(GET_DEVX_CTX(ctx), in, inlen);
+	if (DEVX_IS_ERR(rule_hdl)) {
+		errno = -DEVX_PTR_ERR(rule_hdl);
+		return NULL;
+	}
+	return rule_hdl;
+}
+
+static int
+mlx5_glue_devx_fs_rule_del(void *flow)
+{
+	return devx_fs_rule_del(flow);
+}
+
+alignas(RTE_CACHE_LINE_SIZE)
+const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+	.version = MLX5_GLUE_VERSION,
+	.get_device_list = mlx5_glue_devx_get_device_list,
+	.free_device_list = mlx5_glue_devx_free_device_list,
+	.open_device = mlx5_glue_devx_open_device,
+	.close_device = mlx5_glue_devx_close_device,
+	.query_device = mlx5_glue_devx_query_device,
+	.query_hca_iseg = mlx5_glue_devx_query_hca_iseg_mapping,
+	.devx_obj_create = mlx5_glue_devx_obj_create,
+	.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
+	.devx_obj_query = mlx5_glue_devx_obj_query,
+	.devx_obj_modify = mlx5_glue_devx_obj_modify,
+	.devx_general_cmd = mlx5_glue_devx_general_cmd,
+	.devx_umem_reg = mlx5_glue_devx_umem_reg,
+	.devx_umem_dereg = mlx5_glue_devx_umem_dereg,
+	.devx_alloc_uar = mlx5_glue_devx_alloc_uar,
+	.devx_free_uar = mlx5_glue_devx_free_uar,
+	.devx_fs_rule_add = mlx5_glue_devx_fs_rule_add,
+	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
+	.devx_query_eqn = mlx5_glue_devx_query_eqn,
+};
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
new file mode 100644
index 0000000000..f2261ec7ac
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_GLUE_H_
+#define MLX5_GLUE_H_
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include <rte_byteorder.h>
+#include <mlx5_win_defs.h>
+
+#ifndef MLX5_GLUE_VERSION
+#define MLX5_GLUE_VERSION ""
+#endif
+
+/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
+struct mlx5_glue {
+	const char *version;
+	void *(*devx_obj_create)(void *ctx,
+				 void *in, size_t inlen,
+				 void *out, size_t outlen);
+	int (*devx_obj_destroy)(void *obj);
+	int (*devx_obj_query)(void *obj,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen);
+	int (*devx_obj_modify)(void *obj,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_general_cmd)(void *ctx,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_umem_dereg)(void *umem);
+	void *(*devx_umem_reg)(void *ctx,
+			void *addr, size_t size,
+			uint32_t access, uint32_t *id);
+	void *(*devx_alloc_uar)(void *ctx,
+			uint32_t flags);
+	void (*devx_free_uar)(void *uar);
+	void *(*get_device_list)(int *num_devices);
+	void (*free_device_list)(void *list);
+	void *(*open_device)(void *device);
+	int (*close_device)(void *ctx);
+	int (*query_device)(void *device_bdf, void *dev_inf);
+	void* (*query_hca_iseg)(void *ctx, uint32_t *cb_iseg);
+	int (*devx_obj_query_async)(void *obj,
+				    const void *in, size_t inlen,
+				    size_t outlen, uint64_t wr_id,
+				    void *cmd_comp);
+	void *(*devx_fs_rule_add)(void *ctx, void *in, uint32_t inlen);
+	int (*devx_fs_rule_del)(void *flow);
+	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
+};
+
+extern const struct mlx5_glue *mlx5_glue;
+
+#endif /* MLX5_GLUE_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
new file mode 100644
index 0000000000..72a3131f5e
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_DEFS_H__
+#define __MLX5_WIN_DEFS_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum {
+	MLX5_CQE_OWNER_MASK	= 1,
+	MLX5_CQE_REQ		= 0,
+	MLX5_CQE_RESP_WR_IMM	= 1,
+	MLX5_CQE_RESP_SEND	= 2,
+	MLX5_CQE_RESP_SEND_IMM	= 3,
+	MLX5_CQE_RESP_SEND_INV	= 4,
+	MLX5_CQE_RESIZE_CQ	= 5,
+	MLX5_CQE_NO_PACKET	= 6,
+	MLX5_CQE_REQ_ERR	= 13,
+	MLX5_CQE_RESP_ERR	= 14,
+	MLX5_CQE_INVALID	= 15,
+};
+#endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
new file mode 100644
index 0000000000..0e74910e9d
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_ETX_H__
+#define __MLX5_WIN_ETX_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "mlx5devx.h"
+
+typedef struct mlx5_context {
+	devx_device_ctx        *devx_ctx;
+	struct devx_device mlx5_dev;
+
+} mlx5_context_st;
+
+typedef struct {
+	devx_device_ctx *devx_ctx;
+	struct devx_obj_handle *obj;
+} mlx5_devx_obj_st;
+
+struct mlx5_devx_umem {
+	void                   *addr;
+	struct devx_obj_handle *umem_hdl;
+	uint32_t                umem_id;
+};
+
+#define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
+#define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
+
+#endif /* __MLX5_WIN_ETX_H__ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 20/32] mlx5/windows: add mlx5 meson file

[Tal Shnaiderman]

File drivers/common/mlx5/windows/meson.build is added to enable mlx5
source files compilation under common windows directory.
A Devx SDK tool must be installed to export two external H files:
mlx5devx.h and mlx5_ifc_devx.h. The installation is based on environment
variable DEVX_INC_PATH. In addition a DLL lib file is installed based on
environment variable DEVX_LIB_PATH. The meson file is using the
environment variables for compilation and linkage.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 42 +++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/meson.build

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2b2aa3b097
--- /dev/null
+++ b/drivers/common/mlx5/windows/meson.build
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2019 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+
+allow_experimental_apis = true
+sources += files(
+	'mlx5_glue.c',
+)
+
+res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
+res_inc = run_command(python3, '-c', 'import os; print(os.environ["DEVX_INC_PATH"])')
+
+if (res_lib.returncode() != 0 or res_inc.returncode() != 0)
+	build = false
+	reason = 'DevX environment variables are not set, DEVX_LIB_PATH and DEVX_INC_PATH vars must be exported'
+	subdir_done()
+endif
+
+devx_lib_dir = res_lib.stdout().strip()
+devx_inc_dir = res_inc.stdout().strip()
+
+ext_deps += cc.find_library('mlx5devx', dirs: devx_lib_dir, required: true)
+includes += include_directories(devx_inc_dir)
+cflags_options = [
+	'-std=c11',
+	'-Wno-strict-prototypes',
+	'-D_BSD_SOURCE',
+	'-D_DEFAULT_SOURCE',
+	'-D_XOPEN_SOURCE=600'
+]
+foreach option:cflags_options
+	if cc.has_argument(option)
+		cflags += option
+	endif
+endforeach
+if get_option('buildtype').contains('debug')
+	cflags += [ '-pedantic', '-DPEDANTIC' ]
+else
+	cflags += [ '-UPEDANTIC' ]
+endif
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 21/32] mlx5/windows: add initialization routine for external lib

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add function mlx5_glue_constructor() as an initialization routine for
run-time dependency on external lib.  Currently the routine has an empty
body. It is used for compatibility with Linux.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build      |  1 +
 drivers/common/mlx5/windows/mlx5_common_os.c | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 2b2aa3b097..9e74ebdea3 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -6,6 +6,7 @@ includes += include_directories('.')
 allow_experimental_apis = true
 sources += files(
 	'mlx5_glue.c',
+	'mlx5_common_os.c',
 )
 
 res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
new file mode 100644
index 0000000000..5707fb60c3
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <unistd.h>
+#include <string.h>
+#include <stdio.h>
+
+#include <rte_errno.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "mlx5_devx_cmds.h"
+#include "mlx5_common_utils.h"
+#include <mlx5_common.h>
+
+/**
+ * Initialization routine for run-time dependency on external lib
+ */
+void
+mlx5_glue_constructor(void)
+{
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 22/32] mlx5/windows: generate file mlx5_autoconf.h

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

File mlx5_autoconf.h is generated under Windows to maintain
compatibility with the Linux build system.  This file is included in
Linux/Windows shared source files therefore it is required. Currently
the file is created empty.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 9e74ebdea3..5ac5724b40 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -41,3 +41,6 @@ else
 	cflags += [ '-UPEDANTIC' ]
 endif
 
+# Generate an empty mlx5_autoconf.h file for compatibility with Linux
+config = configuration_data()
+configure_file(output : 'mlx5_autoconf.h', configuration : config)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 23/32] common/mlx5/windows: extend PRM match_param_bits struct

[Tal Shnaiderman]

Add reserved size to PRM 'struct mlx5_ifc_fte_match_param_bits' for non
Linux OS.  Windows drivers require this extension since their expected
size should match the actual struct size. Linux drivers do not require
this extension and already use calculations based on the shorter size.
Use a static assert to verify that the PRM fte_match_param struct size
remains correct.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h          | 7 +++++++
 drivers/common/mlx5/windows/mlx5_glue.c | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 4ddf865743..27a4e4f0f1 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -815,6 +815,13 @@ struct mlx5_ifc_fte_match_param_bits {
 	struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
 	struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
 	struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
+/*
+ * Add reserved bit to match the struct size with the size defined in PRM.
+ * This extension is not required in Linux.
+ */
+#ifndef HAVE_INFINIBAND_VERBS_H
+	u8 reserved_0[0x400];
+#endif
 };
 
 enum {
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 7f8a00aaa7..3896cf0932 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -256,6 +256,9 @@ mlx5_glue_devx_free_uar(void *uar)
 	devx_free_uar((devx_uar_handle *)uar);
 }
 
+static_assert(MLX5_ST_SZ_BYTES(fte_match_param) == 0x200,
+	"PRM size of fte_match_param is broken! cannot compile Windows!");
+
 static void*
 mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 24/32] common/mlx5/windows: add getter functions

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add file mlx5/windows/mlx5_common_os.h the equivalent of Linux file
mlx5/linux/mlx5_common_os.h. It contains getters functions
mlx5_os_get_dev_device_name, mlx5_os_get_ctx_device_name,
mlx5_os_get_ctx_device_path, mlx5_os_get_umem_id,
mlx5_os_get_devx_channel_fd.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 91 ++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index cce7c88c6a..2abdb2c3ae 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -48,4 +48,95 @@ mlx5_os_free(void *addr)
 {
 	_aligned_free(addr);
 }
+
+/**
+ * Get fd. Given a pointer to DevX channel object of type
+ * 'struct mlx5dv_devx_event_channel*' - return its fd.
+ * Under Windows it is a stub.
+ *
+ * @param[in] channel
+ *    Pointer to channel object.
+ *
+ * @return
+ *    0
+ */
+static inline int
+mlx5_os_get_devx_channel_fd(void *channel)
+{
+	if (!channel)
+		return 0;
+	return 0;
+}
+
+/**
+ * Get device name. Given a device pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] dev
+ *   Pointer to device.
+ *
+ * @return
+ *   Pointer to device name if dev is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_dev_device_name(void *dev)
+{
+	if (!dev)
+		return NULL;
+	return ((struct devx_device *)dev)->name;
+}
+
+/**
+ * Get device name. Given a context pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device name if ctx is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_ctx_device_name(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.name;
+}
+
+/**
+ * Get a device path name. Given acontext pointer - return a
+ * pointer to the corresponding device path name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device path name if ctx is valid, NULL otherwise.
+ */
+
+static inline const char *
+mlx5_os_get_ctx_device_path(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.dev_pnp_id;
+}
+
+/**
+ * Get umem id. Given a pointer to umem object of type return its id.
+ *
+ * @param[in] umem
+ *    Pointer to umem object.
+ *
+ * @return
+ *    The umem id if umem is valid, 0 otherwise.
+ */
+static inline uint32_t
+mlx5_os_get_umem_id(void *umem)
+{
+	if (!umem)
+		return 0;
+	return ((struct mlx5_devx_umem *)umem)->umem_id;
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 25/32] common/mlx5/windows: add OS alloc/dealloc pd

[Tal Shnaiderman]

Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd(). They
are equivalent to the Linux implementation in [1].

[1]
("net/mlx5: wrap glue alloc/dealloc PD with OS calls")

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
 drivers/common/mlx5/windows/mlx5_common_os.c    | 48 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 2f6aab133b..5db1eaa48c 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -61,4 +61,5 @@ EXPORTS
 	mlx5_malloc
 	mlx5_realloc
 	mlx5_free
-
+	mlx5_os_alloc_pd
+	mlx5_os_dealloc_pd
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 5707fb60c3..85537b6739 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -22,3 +22,51 @@ void
 mlx5_glue_constructor(void)
 {
 }
+
+/**
+ * Allocate PD. Given a devx context object
+ * return an mlx5-pd object.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *    The mlx5_pd if pd is valid, NULL and errno otherwise.
+ */
+void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
+		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
+	if (!ppd)
+		return NULL;
+
+	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
+	if (!obj) {
+		mlx5_free(ppd);
+		return NULL;
+	}
+	ppd->obj = obj;
+	ppd->pdn = obj->id;
+	ppd->devx_ctx = ctx;
+	return ppd;
+}
+
+/**
+ * Release PD. Releases a given mlx5_pd object
+ *
+ * @param[in] pd
+ *   Pointer to mlx5_pd.
+ *
+ * @return
+ *    Zero if pd is released successfully, negative number otherwise.
+ */
+int
+mlx5_os_dealloc_pd(void *pd)
+{
+	if (!pd)
+		return -EINVAL;
+	mlx5_devx_cmd_destroy(((struct mlx5_pd *)pd)->obj);
+	mlx5_free(pd);
+	return 0;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index 2abdb2c3ae..f47351ec41 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -139,4 +139,7 @@ mlx5_os_get_umem_id(void *umem)
 		return 0;
 	return ((struct mlx5_devx_umem *)umem)->umem_id;
 }
+
+void *mlx5_os_alloc_pd(void *ctx);
+int mlx5_os_dealloc_pd(void *pd);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index 0e74910e9d..8e697b30c1 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -28,6 +28,12 @@ struct mlx5_devx_umem {
 	uint32_t                umem_id;
 };
 
+struct mlx5_pd {
+	void                   *obj;
+	uint32_t                pdn;
+	devx_device_ctx        *devx_ctx;
+};
+
 #define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
 #define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 26/32] common/mlx5/windows: add OS umem reg/dereg API

[Tal Shnaiderman]

Implement Windows API mlx5_os_umem_reg() and mlx5_os_umem_dereg(). They
are equivalent to the Linux implementation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 60 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  2 +
 3 files changed, 64 insertions(+)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 5db1eaa48c..1eabe29a58 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,3 +63,5 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_umem_reg
+	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 85537b6739..cbb09906bd 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -70,3 +70,63 @@ mlx5_os_dealloc_pd(void *pd)
 	mlx5_free(pd);
 	return 0;
 }
+
+/**
+ * Register umem.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ * @param[in] addr
+ *   Pointer to memory start address.
+ * @param[in] size
+ *   Size of the memory to register.
+ * @param[out] access
+ *   UMEM access type
+ *
+ * @return
+ *   umem on successful registration, NULL and errno otherwise
+ */
+void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	struct mlx5_devx_umem *umem;
+
+	umem = mlx5_malloc(MLX5_MEM_ZERO,
+		(sizeof(*umem)), 0, SOCKET_ID_ANY);
+	if (!umem) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	umem->umem_hdl = mlx5_glue->devx_umem_reg(ctx, addr, size, access,
+		&umem->umem_id);
+	if (!umem->umem_hdl) {
+		mlx5_free(umem);
+		return NULL;
+	}
+	umem->addr = addr;
+	return umem;
+}
+
+/**
+ * Deregister umem.
+ *
+ * @param[in] pumem
+ *   Pointer to umem.
+ *
+ * @return
+ *   0 on successful release, negative number otherwise
+ */
+int
+mlx5_os_umem_dereg(void *pumem)
+{
+	struct mlx5_devx_umem *umem;
+	int err = 0;
+
+	if (!pumem)
+		return err;
+	umem = pumem;
+	if (umem->umem_hdl)
+		err = mlx5_glue->devx_umem_dereg(umem->umem_hdl);
+	mlx5_free(umem);
+	return err;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index f47351ec41..decb5acd45 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -142,4 +142,6 @@ mlx5_os_get_umem_id(void *umem)
 
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
+void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
+int mlx5_os_umem_dereg(void *pumem);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 27/32] net/mlx5: update MR prototypes for DevX

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well.  Rename 'struct mlx5_verbs_ops' as 'struct mlx5_mr_ops' and
move it to shared file mlx5.h.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c    | 4 ++--
 drivers/net/mlx5/linux/mlx5_verbs.c | 2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h | 8 +-------
 drivers/net/mlx5/mlx5.h             | 6 ++++++
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a6c1eba640..4f68c74267 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -2478,8 +2478,8 @@ void
 mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
 		      mlx5_dereg_mr_t *dereg_mr_cb)
 {
-	*reg_mr_cb = mlx5_verbs_ops.reg_mr;
-	*dereg_mr_cb = mlx5_verbs_ops.dereg_mr;
+	*reg_mr_cb = mlx5_mr_verbs_ops.reg_mr;
+	*dereg_mr_cb = mlx5_mr_verbs_ops.dereg_mr;
 }
 
 /**
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c
index 9161fa3b7d..b52ae2e6c1 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.c
+++ b/drivers/net/mlx5/linux/mlx5_verbs.c
@@ -62,7 +62,7 @@ mlx5_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
 }
 
 /* verbs operations. */
-const struct mlx5_verbs_ops mlx5_verbs_ops = {
+const struct mlx5_mr_ops mlx5_mr_verbs_ops = {
 	.reg_mr = mlx5_reg_mr,
 	.dereg_mr = mlx5_dereg_mr,
 };
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.h b/drivers/net/mlx5/linux/mlx5_verbs.h
index 0670f6c47e..76a79bf4f4 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.h
+++ b/drivers/net/mlx5/linux/mlx5_verbs.h
@@ -7,16 +7,10 @@
 
 #include "mlx5.h"
 
-struct mlx5_verbs_ops {
-	mlx5_reg_mr_t reg_mr;
-	mlx5_dereg_mr_t dereg_mr;
-};
-
 int mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
 
 /* Verbs ops struct */
-extern const struct mlx5_verbs_ops mlx5_verbs_ops;
+extern const struct mlx5_mr_ops mlx5_mr_verbs_ops;
 extern struct mlx5_obj_ops ibv_obj_ops;
-
 #endif /* RTE_PMD_MLX5_VERBS_H_ */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 2186677810..a9db597f2b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -917,6 +917,12 @@ struct mlx5_obj_ops {
 
 #define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields)
 
+/* MR operations structure. */
+struct mlx5_mr_ops {
+	mlx5_reg_mr_t reg_mr;
+	mlx5_dereg_mr_t dereg_mr;
+};
+
 struct mlx5_priv {
 	struct rte_eth_dev_data *dev_data;  /* Pointer to device data. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared device context. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 28/32] common/mlx5/windows: add OS reg/dereg MR

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commits implements Windows API for MR registration and
deregistration. It is based on DevX.  Is support the relaxed ordering
flow in Windows by checking the capabilities and machine type.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.h            |  1 +
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 78 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  7 +++
 4 files changed, 88 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index da0a0f0c79..5cc3f097c2 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -28,6 +28,7 @@ struct mlx5_pmd_mr {
 	void		     *addr;
 	size_t		     len;
 	void		     *obj;  /* verbs mr object or devx umem object. */
+	struct mlx5_devx_obj *mkey; /* devx mkey object. */
 };
 
 /**
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 1eabe29a58..65ae47ae6a 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,5 +63,7 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_dereg_mr
+	mlx5_os_reg_mr
 	mlx5_os_umem_reg
 	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index cbb09906bd..84ed561dc0 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -14,6 +14,8 @@
 #include "mlx5_devx_cmds.h"
 #include "mlx5_common_utils.h"
 #include <mlx5_common.h>
+#include "mlx5_common_os.h"
+#include "mlx5_malloc.h"
 
 /**
  * Initialization routine for run-time dependency on external lib
@@ -130,3 +132,79 @@ mlx5_os_umem_dereg(void *pumem)
 	mlx5_free(umem);
 	return err;
 }
+
+/**
+ * Register mr. Given protection doamin pointer, pointer to addr and length
+ * register the memory region.
+ *
+ * @param[in] pd
+ *   Pointer to protection domain context (type mlx5_pd).
+ * @param[in] addr
+ *   Pointer to memory start address (type devx_device_ctx).
+ * @param[in] length
+ *   Lengtoh of the memory to register.
+ * @param[out] pmd_mr
+ *   pmd_mr struct set with lkey, address, length, pointer to mr object, mkey
+ *
+ * @return
+ *   0 on successful registration, -1 otherwise
+ */
+int
+mlx5_os_reg_mr(void *pd,
+	       void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr)
+{
+	struct mlx5_devx_mkey_attr mkey_attr;
+	struct mlx5_pd *mlx5_pd = (struct mlx5_pd *)pd;
+	struct mlx5_hca_attr attr;
+
+	if (!pd || !addr) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+	if (mlx5_devx_cmd_query_hca_attr(mlx5_pd->devx_ctx, &attr))
+		return -1;
+	pmd_mr->addr = addr;
+	pmd_mr->len = length;
+	pmd_mr->obj = mlx5_os_umem_reg(mlx5_pd->devx_ctx, pmd_mr->addr,
+				       pmd_mr->len, IBV_ACCESS_LOCAL_WRITE);
+	if (!pmd_mr->obj)
+		return -1;
+	mkey_attr.addr = (uintptr_t)addr;
+	mkey_attr.size = length;
+	mkey_attr.umem_id = ((struct mlx5_devx_umem *)(pmd_mr->obj))->umem_id;
+	mkey_attr.pd = mlx5_pd->pdn;
+	mkey_attr.log_entity_size = 0;
+	mkey_attr.pg_access = 0;
+	mkey_attr.klm_array = NULL;
+	mkey_attr.klm_num = 0;
+	mkey_attr.relaxed_ordering_read = 0;
+	mkey_attr.relaxed_ordering_write = 0;
+	if (!haswell_broadwell_cpu) {
+		mkey_attr.relaxed_ordering_write = attr.relaxed_ordering_write;
+		mkey_attr.relaxed_ordering_read = attr.relaxed_ordering_read;
+	}
+	pmd_mr->mkey = mlx5_devx_cmd_mkey_create(mlx5_pd->devx_ctx, &mkey_attr);
+	if (!pmd_mr->mkey) {
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+		return -1;
+	}
+	pmd_mr->lkey = pmd_mr->mkey->id;
+	return 0;
+}
+
+/**
+ * De-register mr.
+ *
+ * @param[in] pmd_mr
+ *  Pointer to PMD mr object
+ */
+void
+mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
+{
+	if (pmd_mr && pmd_mr->mkey)
+		claim_zero(mlx5_glue->devx_obj_destroy(pmd_mr->mkey->obj));
+	if (pmd_mr && pmd_mr->obj)
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index decb5acd45..ba166412cc 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -7,9 +7,13 @@
 
 #include <stdio.h>
 
+#include <rte_errno.h>
+
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
 #include "mlx5_malloc.h"
+#include "mlx5_common_mr.h"
+#include "mlx5_win_ext.h"
 
 /**
  * This API allocates aligned or non-aligned memory.  The free can be on either
@@ -144,4 +148,7 @@ void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
 int mlx5_os_umem_dereg(void *pumem);
+int mlx5_os_reg_mr(void *pd,
+		   void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr);
+void mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 29/32] net/mlx5/windows: implement device attribute getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows implementation of mlx5_os_get_dev_attr() API.
It follows the commit in [1]. A new file named mlx5_os.c is added under
windows directory as its Linux counterpart file: linux/mlx5_os.c.

[1].
commit e85f623e13ea ("net/mlx5: remove attributes dependency on Verbs")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h       | 43 +++++++++++++++++++++
 drivers/net/mlx5/windows/meson.build |  8 ++++
 drivers/net/mlx5/windows/mlx5_os.c   | 75 ++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 27a4e4f0f1..d62d2558eb 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -2217,6 +2217,49 @@ struct mlx5_ifc_cqc_bits {
 	u8 dbr_addr[0x40];
 };
 
+struct mlx5_ifc_health_buffer_bits {
+	u8         reserved_0[0x100];
+	u8         assert_existptr[0x20];
+	u8         assert_callra[0x20];
+	u8         reserved_1[0x40];
+	u8         fw_version[0x20];
+	u8         hw_id[0x20];
+	u8         reserved_2[0x20];
+	u8         irisc_index[0x8];
+	u8         synd[0x8];
+	u8         ext_synd[0x10];
+};
+
+struct mlx5_ifc_initial_seg_bits {
+	u8         fw_rev_minor[0x10];
+	u8         fw_rev_major[0x10];
+	u8         cmd_interface_rev[0x10];
+	u8         fw_rev_subminor[0x10];
+	u8         reserved_0[0x40];
+	u8         cmdq_phy_addr_63_32[0x20];
+	u8         cmdq_phy_addr_31_12[0x14];
+	u8         reserved_1[0x2];
+	u8         nic_interface[0x2];
+	u8         log_cmdq_size[0x4];
+	u8         log_cmdq_stride[0x4];
+	u8         command_doorbell_vector[0x20];
+	u8         reserved_2[0xf00];
+	u8         initializing[0x1];
+	u8         nic_interface_supported[0x7];
+	u8         reserved_4[0x18];
+	struct mlx5_ifc_health_buffer_bits health_buffer;
+	u8         no_dram_nic_offset[0x20];
+	u8         reserved_5[0x6de0];
+	u8         internal_timer_h[0x20];
+	u8         internal_timer_l[0x20];
+	u8         reserved_6[0x20];
+	u8         reserved_7[0x1f];
+	u8         clear_int[0x1];
+	u8         health_syndrome[0x8];
+	u8         health_counter[0x18];
+	u8         reserved_8[0x17fc0];
+};
+
 struct mlx5_ifc_create_cq_out_bits {
 	u8 status[0x8];
 	u8 reserved_at_8[0x18];
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2ea0792a40
--- /dev/null
+++ b/drivers/net/mlx5/windows/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2020 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+sources += files(
+	'mlx5_os.c',
+)
+
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
new file mode 100644
index 0000000000..e0646670a3
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <rte_windows.h>
+
+#include <mlx5_glue.h>
+#include <mlx5_devx_cmds.h>
+#include <mlx5_common.h>
+
+#include "mlx5_defs.h"
+#include "mlx5.h"
+#include "mlx5_autoconf.h"
+
+/**
+ * Get mlx5 device attributes.
+ *
+ * @param ctx
+ *   Pointer to device context.
+ *
+ * @param device_attr
+ *   Pointer to mlx5 device attributes.
+ *
+ * @return
+ *   0 on success, non zero error number otherwise
+ */
+int
+mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
+{
+	struct mlx5_context *mlx5_ctx;
+	struct mlx5_hca_attr hca_attr;
+	void *pv_iseg = NULL;
+	u32 cb_iseg = 0;
+	int err = 0;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	memset(device_attr, 0, sizeof(*device_attr));
+	err = mlx5_devx_cmd_query_hca_attr(mlx5_ctx, &hca_attr);
+	if (err) {
+		DRV_LOG(ERR, "Failed to get device hca_cap");
+		return err;
+	}
+	device_attr->max_cq = 1 << hca_attr.log_max_cq;
+	device_attr->max_qp = 1 << hca_attr.log_max_qp;
+	device_attr->max_qp_wr = 1 << hca_attr.log_max_qp_sz;
+	device_attr->max_cqe = 1 << hca_attr.log_max_cq_sz;
+	device_attr->max_mr = 1 << hca_attr.log_max_mrw_sz;
+	device_attr->max_pd = 1 << hca_attr.log_max_pd;
+	device_attr->max_srq = 1 << hca_attr.log_max_srq;
+	device_attr->max_srq_wr = 1 << hca_attr.log_max_srq_sz;
+	if (hca_attr.rss_ind_tbl_cap) {
+		device_attr->max_rwq_indirection_table_size =
+			1 << hca_attr.rss_ind_tbl_cap;
+	}
+	pv_iseg = mlx5_glue->query_hca_iseg(mlx5_ctx, &cb_iseg);
+	if (pv_iseg == NULL) {
+		DRV_LOG(ERR, "Failed to get device hca_iseg");
+		return errno;
+	}
+	if (!err) {
+		snprintf(device_attr->fw_ver, 64, "%x.%x.%04x",
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_major),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_minor),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_subminor));
+	}
+	return err;
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 30/32] net/mlx5/windows: add mlx5_os.c stubs

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_os_set_nonblock_channel_fd
mlx5_os_dev_shared_handler_install
mlx5_os_dev_shared_handler_uninstall
mlx5_os_read_dev_stat
mlx5_os_mac_addr_flush
mlx5_os_mac_addr_remove
mlx5_os_vf_mac_addr_modify
mlx5_os_set_promisc
mlx5_os_set_allmulti

Set struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops with NULL
pointers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 187 +++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index e0646670a3..4a7c745a71 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -13,10 +13,18 @@
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
 #include <mlx5_common.h>
+#include <mlx5_common_mp.h>
+#include <mlx5_common_mr.h>
+#include <mlx5_malloc.h>
 
 #include "mlx5_defs.h"
 #include "mlx5.h"
+#include "mlx5_common_os.h"
+#include "mlx5_utils.h"
+#include "mlx5_rxtx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_mr.h"
+#include "mlx5_flow.h"
 
 /**
  * Get mlx5 device attributes.
@@ -73,3 +81,182 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	}
 	return err;
 }
+
+/**
+ * Set the completion channel file descriptor interrupt as non-blocking.
+ * Currently it has no support under Windows.
+ *
+ * @param[in] rxq_obj
+ *   Pointer to RQ channel object, which includes the channel fd
+ *
+ * @param[out] fd
+ *   The file descriptor (representing the intetrrupt) used in this channel.
+ *
+ * @return
+ *   0 on successfully setting the fd to non-blocking, non-zero otherwise.
+ */
+int
+mlx5_os_set_nonblock_channel_fd(int fd)
+{
+	(void)fd;
+	DRV_LOG(WARNING, "mlx5_os_set_nonblock_channel_fd is not supported");
+	return -ENOTSUP;
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param sh
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "mlx5_os_dev_shared_handler_install is not supported");
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "mlx5_os_dev_shared_handler_uninstall is not supported");
+}
+
+/**
+ * Read statistics by a named counter.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ * @param[in] ctr_name
+ *   Pointer to the name of the statistic counter to read
+ * @param[out] stat
+ *   Pointer to read statistic value.
+ * @return
+ *   0 on success and stat is valud, 1 if failed to read the value
+ *   rte_errno is set.
+ *
+ */
+int
+mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name,
+		      uint64_t *stat)
+{
+	RTE_SET_USED(priv);
+	RTE_SET_USED(ctr_name);
+	RTE_SET_USED(stat);
+	DRV_LOG(WARNING, "mlx5_os_read_dev_stat is not supported");
+	return -ENOTSUP;
+}
+
+/**
+ * Flush device MAC addresses
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ */
+void
+mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
+{
+	(void)dev;
+	DRV_LOG(WARNING, "mlx5_os_mac_addr_flush is not supported");
+}
+
+/**
+ * Remove a MAC address from device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param index
+ *   MAC address index.
+ */
+void
+mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+	(void)dev;
+	(void)(index);
+	DRV_LOG(WARNING, "mlx5_os_mac_addr_remove is not supported");
+}
+
+/**
+ * Modify a VF MAC address
+ * Currently it has no support under Windows.
+ *
+ * @param priv
+ *   Pointer to device private data.
+ * @param mac_addr
+ *   MAC address to modify into.
+ * @param iface_idx
+ *   Net device interface index
+ * @param vf_index
+ *   VF index
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_vf_mac_addr_modify(struct mlx5_priv *priv,
+			   unsigned int iface_idx,
+			   struct rte_ether_addr *mac_addr,
+			   int vf_index)
+{
+	(void)priv;
+	(void)iface_idx;
+	(void)mac_addr;
+	(void)vf_index;
+	DRV_LOG(WARNING, "mlx5_os_vf_mac_addr_modify is not supported");
+	return -ENOTSUP;
+}
+
+/**
+ * Set device promiscuous mode
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - promiscuous is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_promisc(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "mlx5_os_set_promisc is not supported");
+	return -ENOTSUP;
+}
+
+/**
+ * Set device allmulti mode
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - all multicase is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "mlx5_os_set_allmulti is not supported");
+	return -ENOTSUP;
+}
+
+const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 31/32] net/mlx5/windows: implement mlx5 mac addr add

[Tal Shnaiderman]

Get the list of MAC addresses and verify if the input mac parameter
already exists. If not - return -ENOTSUP (as Windows does not support
adding new MAC addresses). If the MAC address exists (EEXIST) return 0
(the equivalent of Linux implementation of this API).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 4a7c745a71..52ddd00399 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -188,6 +188,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
 	DRV_LOG(WARNING, "mlx5_os_mac_addr_remove is not supported");
 }
 
+/**
+ * Adds a MAC address to the device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param mac_addr
+ *   MAC address to register.
+ * @param index
+ *   MAC address index.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
+		     uint32_t index)
+{
+	(void)index;
+	struct rte_ether_addr lmac;
+
+	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
+		DRV_LOG(ERR,
+			"port %u cannot get MAC address, is mlx5_en"
+			" loaded? (errno: %s)",
+			dev->data->port_id, strerror(rte_errno));
+		return rte_errno;
+	}
+	if (!rte_is_same_ether_addr(&lmac, mac)) {
+		DRV_LOG(ERR,
+			"adding new mac address to device is unsupported");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
 /**
  * Modify a VF MAC address
  * Currently it has no support under Windows.
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v4 32/32] net/mlx5: refactor eth dev ops for Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

There are two types of eth_dev_ops used under Windows: primary and
isolate mode. Their function calls initialization is added to the OS
specific file mlx5_os.c. Secondary process eth_dev_ops is nullified.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 108 +++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 52ddd00399..e207d186aa 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -296,3 +296,111 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
+
+const struct eth_dev_ops mlx5_os_dev_ops = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.read_clock = mlx5_read_clock,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.reta_update = mlx5_dev_rss_reta_update,
+	.reta_query = mlx5_dev_rss_reta_query,
+	.rss_hash_update = mlx5_rss_hash_update,
+	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
+
+/* Available operations from secondary process. */
+const struct eth_dev_ops mlx5_os_dev_sec_ops = {0};
+
+/* Available operations in flow isolated mode. */
+const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 00/32] mlx5 Windows support - part #5

[Tal Shnaiderman]

This patch series is part of preparing mlx5 PMD to compile and run under Windows OS. Part #5

v1: initial version
v2: Fixes after CR and rebase, also patchset was split and part of the commits sent in v1 will be sent later on.
v3: Remove common compilation as it depends on upcoming patches [DmitryK].
v4: Fix syntax error in warning message.
v5: Fix style issues.

Ophir Munk (20):
  net/mlx5/linux: extend device attributes getter
  net/mlx5: remove Linux files from Windows compilation
  net/mlx5: fix freeing packet pacing
  net/mlx5: replace Linux sleep with rte sleep
  net/mlx5: define mprq functions as static inline
  net/mlx5: do not define static_assert in Windows
  net/mlx5: move static_assert calls to global scope
  net/mlx5: wrap glue alloc/dealloc PD with OS calls
  net/mlx5: fix adding destroy flow action wrapper
  common/mlx5: add definition HAVE_INFINIBAND_VERBS_H
  common/mlx5/linux: handle memory allocations with alignment
  common/mlx5/windows: handle memory allocations with alignment
  mlx5/windows: add initialization routine for external lib
  mlx5/windows: generate file mlx5_autoconf.h
  common/mlx5/windows: add getter functions
  net/mlx5: update MR prototypes for DevX
  common/mlx5/windows: add OS reg/dereg MR
  net/mlx5/windows: implement device attribute getter
  net/mlx5/windows: add mlx5_os.c stubs
  net/mlx5: refactor eth dev ops for Windows

Tal Shnaiderman (12):
  net/mlx5: fix folding constant array error
  net/mlx5: wrap glue reg/dereg UMEM with OS calls
  common/mlx5/linux: wrap event channel APIs with OS calls
  common/mlx5: add Windows exports file
  common/mlx5: extend DevX query hca attributes command
  common/mlx5: add DevX alloc PD command
  common/mlx5/windows: add glue functions APIs
  mlx5/windows: add mlx5 meson file
  common/mlx5/windows: extend PRM match_param_bits struct
  common/mlx5/windows: add OS alloc/dealloc pd
  common/mlx5/windows: add OS umem reg/dereg API
  net/mlx5/windows: implement mlx5 mac addr add

 drivers/common/mlx5/linux/meson.build           |   6 +-
 drivers/common/mlx5/linux/mlx5_common_os.h      |  83 +++++
 drivers/common/mlx5/mlx5_common.c               |   4 +-
 drivers/common/mlx5/mlx5_common_mr.h            |   1 +
 drivers/common/mlx5/mlx5_devx_cmds.c            |  46 +++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  12 +-
 drivers/common/mlx5/mlx5_malloc.c               |  14 +-
 drivers/common/mlx5/mlx5_prm.h                  |  87 +++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  69 ++++
 drivers/common/mlx5/version.map                 |   1 +
 drivers/common/mlx5/windows/meson.build         |  45 +++
 drivers/common/mlx5/windows/mlx5_common_os.c    | 210 ++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    | 154 +++++++++
 drivers/common/mlx5/windows/mlx5_glue.c         | 307 ++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 ++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  40 +++
 drivers/net/mlx5/linux/mlx5_os.c                |   9 +-
 drivers/net/mlx5/linux/mlx5_verbs.c             |   2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h             |   8 +-
 drivers/net/mlx5/meson.build                    |  16 +-
 drivers/net/mlx5/mlx5.c                         |  20 +-
 drivers/net/mlx5/mlx5.h                         |  11 +
 drivers/net/mlx5/mlx5_defs.h                    |   7 +-
 drivers/net/mlx5/mlx5_devx.c                    |  24 +-
 drivers/net/mlx5/mlx5_flow.c                    |  17 +-
 drivers/net/mlx5/mlx5_flow_dv.c                 |   6 +-
 drivers/net/mlx5/mlx5_rxq.c                     |  71 -----
 drivers/net/mlx5/mlx5_rxtx.c                    |  98 +++---
 drivers/net/mlx5/mlx5_rxtx.h                    |  73 ++++-
 drivers/net/mlx5/mlx5_trigger.c                 |   3 +-
 drivers/net/mlx5/mlx5_txpp.c                    |  34 +-
 drivers/net/mlx5/windows/meson.build            |   8 +
 drivers/net/mlx5/windows/mlx5_os.c              | 406 ++++++++++++++++++++++++
 drivers/vdpa/mlx5/mlx5_vdpa_event.c             |   7 +-
 35 files changed, 1778 insertions(+), 204 deletions(-)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def
 create mode 100644 drivers/common/mlx5/windows/meson.build
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 01/32] net/mlx5: fix folding constant array error

[Tal Shnaiderman]

Before this commit the PMD used:
   const int elt_n = 8
   const int *stack[elt_n];

In Windows clang compiler complains:
net/mlx5/mlx5_flow.c:215:19: error: variable length array folded
to constant array as an extension [-Werror,-Wgnu-folding-constant]

Fix it by using a constant macro definition instead of a variable:
   #define MLX5_RSS_EXP_ELT_N 8
   const int *stack[MLX5_RSS_EXP_ELT_N];

Fixes: c7870bfe09dc ("ethdev: move RSS expansion code to mlx5 driver")
Cc: stable@dpdk.org

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_flow.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index 82e24d7067..bf86aaaa39 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -212,6 +212,8 @@ mlx5_flow_expand_rss_item_complete(const struct rte_flow_item *item)
 	return ret;
 }
 
+#define MLX5_RSS_EXP_ELT_N 8
+
 /**
  * Expand RSS flows into several possible flows according to the RSS hash
  * fields requested and the driver capabilities.
@@ -242,13 +244,12 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		     const struct mlx5_flow_expand_node graph[],
 		     int graph_root_index)
 {
-	const int elt_n = 8;
 	const struct rte_flow_item *item;
 	const struct mlx5_flow_expand_node *node = &graph[graph_root_index];
 	const int *next_node;
-	const int *stack[elt_n];
+	const int *stack[MLX5_RSS_EXP_ELT_N];
 	int stack_pos = 0;
-	struct rte_flow_item flow_items[elt_n];
+	struct rte_flow_item flow_items[MLX5_RSS_EXP_ELT_N];
 	unsigned int i;
 	size_t lsize;
 	size_t user_pattern_size = 0;
@@ -261,10 +262,10 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 
 	memset(&missed_item, 0, sizeof(missed_item));
 	lsize = offsetof(struct mlx5_flow_expand_rss, entry) +
-		elt_n * sizeof(buf->entry[0]);
+		MLX5_RSS_EXP_ELT_N * sizeof(buf->entry[0]);
 	if (lsize <= size) {
 		buf->entry[0].priority = 0;
-		buf->entry[0].pattern = (void *)&buf->entry[elt_n];
+		buf->entry[0].pattern = (void *)&buf->entry[MLX5_RSS_EXP_ELT_N];
 		buf->entries = 0;
 		addr = buf->entry[0].pattern;
 	}
@@ -367,7 +368,7 @@ mlx5_flow_expand_rss(struct mlx5_flow_expand_rss *buf, size_t size,
 		/* Go deeper. */
 		if (node->next) {
 			next_node = node->next;
-			if (stack_pos++ == elt_n) {
+			if (stack_pos++ == MLX5_RSS_EXP_ELT_N) {
 				rte_errno = E2BIG;
 				return -rte_errno;
 			}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 02/32] net/mlx5/linux: extend device attributes getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit adds device attributes parameters to be reported by
mlx5_os_get_dev_attr(): max_cqe, max_mr, max_pd, max_srq, max_srq_wr

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c | 5 +++++
 drivers/net/mlx5/mlx5.h          | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 917d6be7b8..a6c1eba640 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -116,7 +116,12 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	device_attr->max_qp_wr = attr_ex.orig_attr.max_qp_wr;
 	device_attr->max_sge = attr_ex.orig_attr.max_sge;
 	device_attr->max_cq = attr_ex.orig_attr.max_cq;
+	device_attr->max_cqe = attr_ex.orig_attr.max_cqe;
+	device_attr->max_mr = attr_ex.orig_attr.max_mr;
+	device_attr->max_pd = attr_ex.orig_attr.max_pd;
 	device_attr->max_qp = attr_ex.orig_attr.max_qp;
+	device_attr->max_srq = attr_ex.orig_attr.max_srq;
+	device_attr->max_srq_wr = attr_ex.orig_attr.max_srq_wr;
 	device_attr->raw_packet_caps = attr_ex.raw_packet_caps;
 	device_attr->max_rwq_indirection_table_size =
 		attr_ex.rss_caps.max_rwq_indirection_table_size;
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 121d726405..2186677810 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -83,6 +83,11 @@ struct mlx5_dev_attr {
 	int		max_sge;
 	int		max_cq;
 	int		max_qp;
+	int		max_cqe;
+	uint32_t	max_pd;
+	uint32_t	max_mr;
+	uint32_t	max_srq;
+	uint32_t	max_srq_wr;
 	uint32_t	raw_packet_caps;
 	uint32_t	max_rwq_indirection_table_size;
 	uint32_t	max_tso;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 03/32] net/mlx5: remove Linux files from Windows compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit removes Linux files flow_verbs.c and mlx5_rxtx_vec.c
from Windows compilation.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/meson.build | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/net/mlx5/meson.build b/drivers/net/mlx5/meson.build
index e7495a764a..f2fafbdd05 100644
--- a/drivers/net/mlx5/meson.build
+++ b/drivers/net/mlx5/meson.build
@@ -15,7 +15,6 @@ sources = files(
 	'mlx5_flow.c',
 	'mlx5_flow_meter.c',
 	'mlx5_flow_dv.c',
-	'mlx5_flow_verbs.c',
         'mlx5_flow_age.c',
 	'mlx5_mac.c',
 	'mlx5_mr.c',
@@ -31,11 +30,18 @@ sources = files(
 	'mlx5_utils.c',
 	'mlx5_devx.c',
 )
-if (dpdk_conf.has('RTE_ARCH_X86_64')
-	or dpdk_conf.has('RTE_ARCH_ARM64')
-	or dpdk_conf.has('RTE_ARCH_PPC_64'))
-	sources += files('mlx5_rxtx_vec.c')
+
+if is_linux
+	sources += files(
+		'mlx5_flow_verbs.c',
+	)
+	if (dpdk_conf.has('RTE_ARCH_X86_64')
+		or dpdk_conf.has('RTE_ARCH_ARM64')
+		or dpdk_conf.has('RTE_ARCH_PPC_64'))
+		sources += files('mlx5_rxtx_vec.c')
+	endif
 endif
+
 cflags_options = [
 	'-std=c11',
 	'-Wno-strict-prototypes',
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 04/32] net/mlx5: fix freeing packet pacing

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Packet pacing is allocated under condition #ifdef HAVE_MLX5DV_PP_ALLOC.
In a similar way - free packet pacing index under the same condition.
This update is required to successfully compile under operating systems
which do not support packet pacing.

Fixes: aef1e20ebeb2 ("net/mlx5: allocate packet pacing context")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_txpp.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 2438bf1f1d..21675ab17a 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -57,11 +57,16 @@ mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 static void
 mlx5_txpp_free_pp_index(struct mlx5_dev_ctx_shared *sh)
 {
+#ifdef HAVE_MLX5DV_PP_ALLOC
 	if (sh->txpp.pp) {
 		mlx5_glue->dv_free_pp(sh->txpp.pp);
 		sh->txpp.pp = NULL;
 		sh->txpp.pp_id = 0;
 	}
+#else
+	RTE_SET_USED(sh);
+	DRV_LOG(ERR, "Freeing pacing index is not supported.");
+#endif
 }
 
 /* Allocate Packet Pacing index from kernel via mlx5dv call. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 05/32] net/mlx5: replace Linux sleep with rte sleep

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Replace Linux API usleep() and nanosleep() with rte_delay_us_sleep().
The replacement occurs in shared files compiled under different
operating systems.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 5 +++--
 drivers/net/mlx5/mlx5_trigger.c | 3 ++-
 drivers/net/mlx5/mlx5_txpp.c    | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 52a8a252d4..8e9df6abcf 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -21,6 +21,7 @@
 #include <rte_spinlock.h>
 #include <rte_string_fns.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
@@ -1330,7 +1331,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_flex_parser_ecpri_release(dev);
 	if (priv->rxqs != NULL) {
 		/* XXX race condition if mlx5_rx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->rxqs_n); ++i)
 			mlx5_rxq_release(dev, i);
 		priv->rxqs_n = 0;
@@ -1338,7 +1339,7 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	}
 	if (priv->txqs != NULL) {
 		/* XXX race condition if mlx5_tx_burst() is still running. */
-		usleep(1000);
+		rte_delay_us_sleep(1000);
 		for (i = 0; (i != priv->txqs_n); ++i)
 			mlx5_txq_release(dev, i);
 		priv->txqs_n = 0;
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index bd029154f8..646f29b923 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -9,6 +9,7 @@
 #include <rte_ethdev_driver.h>
 #include <rte_interrupts.h>
 #include <rte_alarm.h>
+#include <rte_cycles.h>
 
 #include <mlx5_malloc.h>
 
@@ -1171,7 +1172,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	rte_wmb();
 	/* Disable datapath on secondary process. */
 	mlx5_mp_os_req_stop_rxtx(dev);
-	usleep(1000 * priv->rxqs_n);
+	rte_delay_us_sleep(1000 * priv->rxqs_n);
 	DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id);
 	mlx5_flow_stop_default(dev);
 	/* Control flows for default traffic can be removed firstly. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 21675ab17a..a1ec294942 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -769,15 +769,11 @@ mlx5_txpp_init_timestamp(struct mlx5_dev_ctx_shared *sh)
 	sh->txpp.ts_p = 0;
 	sh->txpp.ts_n = 0;
 	for (wait = 0; wait < MLX5_TXPP_WAIT_INIT_TS; wait++) {
-		struct timespec onems;
-
 		mlx5_txpp_update_timestamp(sh);
 		if (wq->sq_ci)
 			return;
 		/* Wait one millisecond and try again. */
-		onems.tv_sec = 0;
-		onems.tv_nsec = NS_PER_S / MS_PER_S;
-		nanosleep(&onems, 0);
+		rte_delay_us_sleep(US_PER_S / MS_PER_S);
 	}
 	DRV_LOG(ERR, "Unable to initialize timestamp.");
 	sh->txpp.sync_lost = 1;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 06/32] net/mlx5: define mprq functions as static inline

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Functions mlx5_check_mprq_support(), mlx5_rxq_mprq_enabled(),
mlx5_mprq_enabled() are moved from source file mlx5_rxq.c to header file
mlx5_rxtx.h and their type is updated to 'static __rte_always_inline'.
Previously the functions were declared as 'inline' in the source file
which was reported as 'unresolved external symbol' error by some Windows
linkers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxq.c  | 71 ------------------------------------------
 drivers/net/mlx5/mlx5_rxtx.h | 73 ++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 70 insertions(+), 74 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index da7a8b3cd7..c89e55b1c0 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -48,77 +48,6 @@ static_assert(MLX5_RSS_HASH_KEY_LEN ==
 	      (unsigned int)sizeof(rss_hash_default_key),
 	      "wrong RSS default key size.");
 
-/**
- * Check whether Multi-Packet RQ can be enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   1 if supported, negative errno value if not.
- */
-inline int
-mlx5_check_mprq_support(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-
-	if (priv->config.mprq.enabled &&
-	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
-		return 1;
-	return -ENOTSUP;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the Rx queue.
- *
- *  @param rxq
- *     Pointer to receive queue structure.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
-{
-	return rxq->strd_num_n > 0;
-}
-
-/**
- * Check whether Multi-Packet RQ is enabled for the device.
- *
- * @param dev
- *   Pointer to Ethernet device.
- *
- * @return
- *   0 if disabled, otherwise enabled.
- */
-inline int
-mlx5_mprq_enabled(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	uint32_t i;
-	uint16_t n = 0;
-	uint16_t n_ibv = 0;
-
-	if (mlx5_check_mprq_support(dev) < 0)
-		return 0;
-	/* All the configured queues should be enabled. */
-	for (i = 0; i < priv->rxqs_n; ++i) {
-		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
-		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
-			(rxq, struct mlx5_rxq_ctrl, rxq);
-
-		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
-			continue;
-		n_ibv++;
-		if (mlx5_rxq_mprq_enabled(rxq))
-			++n;
-	}
-	/* Multi-Packet RQ can't be partially configured. */
-	MLX5_ASSERT(n == 0 || n == n_ibv);
-	return n == n_ibv;
-}
-
 /**
  * Calculate the number of CQEs in CQ for the Rx queue.
  *
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 7989a50403..1e9345af61 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -311,9 +311,6 @@ struct mlx5_txq_ctrl {
 
 extern uint8_t rss_hash_default_key[];
 
-int mlx5_check_mprq_support(struct rte_eth_dev *dev);
-int mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq);
-int mlx5_mprq_enabled(struct rte_eth_dev *dev);
 unsigned int mlx5_rxq_cqe_num(struct mlx5_rxq_data *rxq_data);
 int mlx5_mprq_free_mp(struct rte_eth_dev *dev);
 int mlx5_mprq_alloc_mp(struct rte_eth_dev *dev);
@@ -920,4 +917,74 @@ mprq_buf_to_pkt(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, uint32_t len,
 	return MLX5_RXQ_CODE_EXIT;
 }
 
+/**
+ * Check whether Multi-Packet RQ can be enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   1 if supported, negative errno value if not.
+ */
+static __rte_always_inline int
+mlx5_check_mprq_support(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+
+	if (priv->config.mprq.enabled &&
+	    priv->rxqs_n >= priv->config.mprq.min_rxqs_num)
+		return 1;
+	return -ENOTSUP;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the Rx queue.
+ *
+ *  @param rxq
+ *     Pointer to receive queue structure.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_rxq_mprq_enabled(struct mlx5_rxq_data *rxq)
+{
+	return rxq->strd_num_n > 0;
+}
+
+/**
+ * Check whether Multi-Packet RQ is enabled for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device.
+ *
+ * @return
+ *   0 if disabled, otherwise enabled.
+ */
+static __rte_always_inline int
+mlx5_mprq_enabled(struct rte_eth_dev *dev)
+{
+	struct mlx5_priv *priv = dev->data->dev_private;
+	uint32_t i;
+	uint16_t n = 0;
+	uint16_t n_ibv = 0;
+
+	if (mlx5_check_mprq_support(dev) < 0)
+		return 0;
+	/* All the configured queues should be enabled. */
+	for (i = 0; i < priv->rxqs_n; ++i) {
+		struct mlx5_rxq_data *rxq = (*priv->rxqs)[i];
+		struct mlx5_rxq_ctrl *rxq_ctrl = container_of
+			(rxq, struct mlx5_rxq_ctrl, rxq);
+
+		if (rxq == NULL || rxq_ctrl->type != MLX5_RXQ_TYPE_STANDARD)
+			continue;
+		n_ibv++;
+		if (mlx5_rxq_mprq_enabled(rxq))
+			++n;
+	}
+	/* Multi-Packet RQ can't be partially configured. */
+	MLX5_ASSERT(n == 0 || n == n_ibv);
+	return n == n_ibv;
+}
 #endif /* RTE_PMD_MLX5_RXTX_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 07/32] net/mlx5: do not define static_assert in Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

In Linux 'static_assert' is defined in file mlx5_defs.h:

 #ifndef HAVE_STATIC_ASSERT
    #define static_assert _Static_assert
 #endif

The same definition can originate from Linux file /usr/include/assert.h.

In Windows static_assert is used while _Static_assert is unknown.
Therefore update the definition condition to be:

 #if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
    #define static_assert _Static_assert
 #endif

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_defs.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
index aa55db3750..85a0979653 100644
--- a/drivers/net/mlx5/mlx5_defs.h
+++ b/drivers/net/mlx5/mlx5_defs.h
@@ -199,8 +199,11 @@
 /* Maximum number of shared actions supported by rte_flow */
 #define MLX5_MAX_SHARED_ACTIONS 2
 
-/* Definition of static_assert found in /usr/include/assert.h */
-#ifndef HAVE_STATIC_ASSERT
+/*
+ * Linux definition of static_assert is found in /usr/include/assert.h.
+ * Windows does not require a redefinition.
+ */
+#if !defined(HAVE_STATIC_ASSERT) && !defined(RTE_EXEC_ENV_WINDOWS)
 #define static_assert _Static_assert
 #endif
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 08/32] net/mlx5: move static_assert calls to global scope

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Some Windows compilers consider static_assert() as calls to another
function rather than a compiler directive which allows checking type
information at compile time.  This only occurs if the static_assert call
appears inside another function scope. To solve it move the
static_assert calls to global scope in the files where they are used.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5_rxtx.c | 98 ++++++++++++++++++++++----------------------
 drivers/net/mlx5/mlx5_txpp.c |  5 ++-
 2 files changed, 53 insertions(+), 50 deletions(-)

diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index d12d746c2f..65a1f997e9 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -79,6 +79,56 @@ static uint16_t mlx5_tx_burst_##func(void *txq, \
 
 #define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx},
 
+/* static asserts */
+static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
+static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(rte_v128u32_t)),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
+		(sizeof(uint16_t) +
+		 sizeof(struct rte_vlan_hdr) +
+		 2 * RTE_ETHER_ADDR_LEN),
+		"invalid Ethernet Segment data size");
+static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
+		(2 * RTE_ETHER_ADDR_LEN),
+		"invalid Data Segment data size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
+static_assert((sizeof(struct rte_vlan_hdr) +
+			sizeof(struct rte_ether_hdr)) ==
+		MLX5_ESEG_MIN_INLINE_SIZE,
+		"invalid min inline data size");
+static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
+		MLX5_DSEG_MAX, "invalid WQE max size");
+static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Control Segment size");
+static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Ethernet Segment size");
+static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
+		"invalid WQE Data Segment size");
+static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
+		"invalid WQE size");
+
 static __rte_always_inline uint32_t
 rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe,
 				   volatile struct mlx5_mini_cqe8 *mcqe);
@@ -2070,8 +2120,6 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq,
 	bool ring_doorbell = false;
 	int ret;
 
-	static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative value");
-	static_assert(MLX5_CQE_STATUS_SW_OWN < 0, "Must be negative value");
 	do {
 		volatile struct mlx5_cqe *cqe;
 
@@ -2381,15 +2429,6 @@ mlx5_tx_eseg_dmin(struct mlx5_txq_data *__rte_restrict txq __rte_unused,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = RTE_BE16(MLX5_ESEG_MIN_INLINE_SIZE);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2474,15 +2513,6 @@ mlx5_tx_eseg_data(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	psrc = rte_pktmbuf_mtod(loc->mbuf, uint8_t *);
 	es->inline_hdr_sz = rte_cpu_to_be_16(inlen);
 	es->inline_data = *(unaligned_uint16_t *)psrc;
@@ -2697,15 +2727,6 @@ mlx5_tx_eseg_mdat(struct mlx5_txq_data *__rte_restrict txq,
 	es->metadata = MLX5_TXOFF_CONFIG(METADATA) ?
 		       loc->mbuf->ol_flags & PKT_TX_DYNF_METADATA ?
 		       *RTE_FLOW_DYNF_METADATA(loc->mbuf) : 0 : 0;
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(rte_v128u32_t)),
-		      "invalid Ethernet Segment data size");
-	static_assert(MLX5_ESEG_MIN_INLINE_SIZE ==
-				(sizeof(uint16_t) +
-				 sizeof(struct rte_vlan_hdr) +
-				 2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Ethernet Segment data size");
 	MLX5_ASSERT(inlen >= MLX5_ESEG_MIN_INLINE_SIZE);
 	pdst = (uint8_t *)&es->inline_data;
 	if (MLX5_TXOFF_CONFIG(VLAN) && vlan) {
@@ -2952,9 +2973,6 @@ mlx5_tx_dseg_vlan(struct mlx5_txq_data *__rte_restrict txq,
 	uint8_t *pdst;
 
 	MLX5_ASSERT(len > MLX5_ESEG_MIN_INLINE_SIZE);
-	static_assert(MLX5_DSEG_MIN_INLINE_SIZE ==
-				 (2 * RTE_ETHER_ADDR_LEN),
-		      "invalid Data Segment data size");
 	if (!MLX5_TXOFF_CONFIG(MPW)) {
 		/* Store the descriptor byte counter for eMPW sessions. */
 		dseg->bcount = rte_cpu_to_be_32
@@ -4070,7 +4088,6 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4247,7 +4264,6 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq,
 	MLX5_ASSERT(MLX5_TXOFF_CONFIG(EMPW));
 	MLX5_ASSERT(loc->elts_free && loc->wqe_free);
 	MLX5_ASSERT(pkts_n > loc->pkts_sent);
-	static_assert(MLX5_EMPW_MIN_PACKETS >= 2, "invalid min size");
 	pkts += loc->pkts_sent + 1;
 	pkts_n -= loc->pkts_sent;
 	for (;;) {
@@ -4561,10 +4577,6 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq,
 			    loc->mbuf->ol_flags & PKT_TX_VLAN_PKT) {
 				vlan = sizeof(struct rte_vlan_hdr);
 				inlen += vlan;
-				static_assert((sizeof(struct rte_vlan_hdr) +
-					       sizeof(struct rte_ether_hdr)) ==
-					       MLX5_ESEG_MIN_INLINE_SIZE,
-					       "invalid min inline data size");
 			}
 			/*
 			 * If inlining is enabled at configuration time
@@ -5567,16 +5579,6 @@ mlx5_select_tx_function(struct rte_eth_dev *dev)
 	uint64_t tx_offloads = dev->data->dev_conf.txmode.offloads;
 	unsigned int diff = 0, olx = 0, i, m;
 
-	static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=
-		      MLX5_DSEG_MAX, "invalid WQE max size");
-	static_assert(MLX5_WQE_CSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Control Segment size");
-	static_assert(MLX5_WQE_ESEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Ethernet Segment size");
-	static_assert(MLX5_WQE_DSEG_SIZE == MLX5_WSEG_SIZE,
-		      "invalid WQE Data Segment size");
-	static_assert(MLX5_WQE_SIZE == 4 * MLX5_WSEG_SIZE,
-		      "invalid WQE size");
 	MLX5_ASSERT(priv);
 	if (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) {
 		/* We should support Multi-Segment Packets. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index a1ec294942..d61e43e55d 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -18,6 +18,9 @@
 #include "mlx5_rxtx.h"
 #include "mlx5_common_os.h"
 
+static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
+		"Wrong timestamp CQE part size");
+
 static const char * const mlx5_txpp_stat_names[] = {
 	"tx_pp_missed_interrupt_errors", /* Missed service interrupt. */
 	"tx_pp_rearm_queue_errors", /* Rearm Queue errors. */
@@ -741,8 +744,6 @@ mlx5_txpp_update_timestamp(struct mlx5_dev_ctx_shared *sh)
 	uint64_t ts;
 	uint16_t ci;
 
-	static_assert(sizeof(struct mlx5_cqe_ts) == sizeof(rte_int128_t),
-		      "Wrong timestamp CQE part size");
 	mlx5_atomic_read_cqe((rte_int128_t *)&cqe->timestamp, &to.u128);
 	if (to.cts.op_own >> 4) {
 		DRV_LOG(DEBUG, "Clock Queue error sync lost.");
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 09/32] net/mlx5: wrap glue alloc/dealloc PD with OS calls

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Wrap glue calls alloc_pd() and dealloc_pd() with generic OS calls.  In
Linux - protection domain allocations are implemented by Verbs glue API
while in Windows it is by DevX API.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 11 +++++++++++
 drivers/net/mlx5/mlx5.c                    |  6 +++---
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index 3420937859..c9f8d7cbc1 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -201,4 +201,15 @@ mlx5_os_get_devx_uar_page_id(void *uar)
 #endif
 }
 
+static inline void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	return mlx5_glue->alloc_pd(ctx);
+}
+
+static inline int
+mlx5_os_dealloc_pd(void *pd)
+{
+	return mlx5_glue->dealloc_pd(pd);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 8e9df6abcf..7d1b4fbdc0 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -932,7 +932,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 		sh->port[i].ih_port_id = RTE_MAX_ETHPORTS;
 		sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS;
 	}
-	sh->pd = mlx5_glue->alloc_pd(sh->ctx);
+	sh->pd = mlx5_os_alloc_pd(sh->ctx);
 	if (sh->pd == NULL) {
 		DRV_LOG(ERR, "PD allocation failure");
 		err = ENOMEM;
@@ -1032,7 +1032,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	if (sh->tx_uar)
 		mlx5_glue->devx_free_uar(sh->tx_uar);
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->ctx)
 		claim_zero(mlx5_glue->close_device(sh->ctx));
 	mlx5_free(sh);
@@ -1100,7 +1100,7 @@ mlx5_free_shared_dev_ctx(struct mlx5_dev_ctx_shared *sh)
 		sh->tx_uar = NULL;
 	}
 	if (sh->pd)
-		claim_zero(mlx5_glue->dealloc_pd(sh->pd));
+		claim_zero(mlx5_os_dealloc_pd(sh->pd));
 	if (sh->tis)
 		claim_zero(mlx5_devx_cmd_destroy(sh->tis));
 	if (sh->td)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 10/32] net/mlx5: wrap glue reg/dereg UMEM with OS calls

[Tal Shnaiderman]

Wrap glue calls for UMEM registration and deregistration with generic OS
calls since each OS (Linux or Windows) has a different glue API
parameters.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 12 ++++++++++++
 drivers/common/mlx5/mlx5_common.c          |  4 ++--
 drivers/net/mlx5/mlx5.c                    |  2 +-
 drivers/net/mlx5/mlx5_devx.c               | 16 ++++++++--------
 drivers/net/mlx5/mlx5_flow.c               |  4 ++--
 drivers/net/mlx5/mlx5_txpp.c               | 12 ++++++------
 6 files changed, 31 insertions(+), 19 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index c9f8d7cbc1..f8b215cc29 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -212,4 +212,16 @@ mlx5_os_dealloc_pd(void *pd)
 {
 	return mlx5_glue->dealloc_pd(pd);
 }
+
+static inline void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	return mlx5_glue->devx_umem_reg(ctx, addr, size, access);
+}
+
+static inline int
+mlx5_os_umem_dereg(void *pumem)
+{
+	return mlx5_glue->devx_umem_dereg(pumem);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c
index 044513223c..a00ffcb833 100644
--- a/drivers/common/mlx5/mlx5_common.c
+++ b/drivers/common/mlx5/mlx5_common.c
@@ -148,7 +148,7 @@ mlx5_alloc_dbr_page(void *ctx)
 		return NULL;
 	}
 	/* Register allocated memory. */
-	page->umem = mlx5_glue->devx_umem_reg(ctx, page->dbrs,
+	page->umem = mlx5_os_umem_reg(ctx, page->dbrs,
 					      MLX5_DBR_PAGE_SIZE, 0);
 	if (!page->umem) {
 		DRV_LOG(ERR, "cannot umem reg dbr page");
@@ -232,7 +232,7 @@ mlx5_release_dbr(struct mlx5_dbr_page_list *head, uint32_t umem_id,
 		/* Page not used, free it and remove from list. */
 		LIST_REMOVE(page, next);
 		if (page->umem)
-			ret = -mlx5_glue->devx_umem_dereg(page->umem);
+			ret = -mlx5_os_umem_dereg(page->umem);
 		mlx5_free(page);
 	} else {
 		/* Mark in bitmap that this door-bell is not in use. */
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 7d1b4fbdc0..84123f8e3d 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -482,7 +482,7 @@ mlx5_flow_destroy_counter_stat_mem_mng(struct mlx5_counter_stats_mem_mng *mng)
 
 	LIST_REMOVE(mng, next);
 	claim_zero(mlx5_devx_cmd_destroy(mng->dm));
-	claim_zero(mlx5_glue->devx_umem_dereg(mng->umem));
+	claim_zero(mlx5_os_umem_dereg(mng->umem));
 	mlx5_free(mem);
 }
 
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index de9b204075..235fd5798d 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -155,7 +155,7 @@ mlx5_rxq_release_devx_rq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->rq_dbrec_page;
 
 	if (rxq_ctrl->wq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->wq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->wq_umem);
 		rxq_ctrl->wq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.wqes) {
@@ -182,7 +182,7 @@ mlx5_rxq_release_devx_cq_resources(struct mlx5_rxq_ctrl *rxq_ctrl)
 	struct mlx5_devx_dbr_page *dbr_page = rxq_ctrl->cq_dbrec_page;
 
 	if (rxq_ctrl->cq_umem) {
-		mlx5_glue->devx_umem_dereg(rxq_ctrl->cq_umem);
+		mlx5_os_umem_dereg(rxq_ctrl->cq_umem);
 		rxq_ctrl->cq_umem = NULL;
 	}
 	if (rxq_ctrl->rxq.cqes) {
@@ -375,7 +375,7 @@ mlx5_rxq_create_devx_rq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	if (!buf)
 		return NULL;
 	rxq_data->wqes = buf;
-	rxq_ctrl->wq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	rxq_ctrl->wq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						     buf, wq_size, 0);
 	if (!rxq_ctrl->wq_umem)
 		goto error;
@@ -497,7 +497,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	rxq_data->cqes = (volatile struct mlx5_cqe (*)[])(uintptr_t)buf;
-	rxq_ctrl->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx, buf,
+	rxq_ctrl->cq_umem = mlx5_os_umem_reg(priv->sh->ctx, buf,
 						     cq_size,
 						     IBV_ACCESS_LOCAL_WRITE);
 	if (!rxq_ctrl->cq_umem) {
@@ -1127,7 +1127,7 @@ mlx5_txq_release_devx_sq_resources(struct mlx5_txq_obj *txq_obj)
 		txq_obj->sq_devx = NULL;
 	}
 	if (txq_obj->sq_umem) {
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->sq_umem));
 		txq_obj->sq_umem = NULL;
 	}
 	if (txq_obj->sq_buf) {
@@ -1155,7 +1155,7 @@ mlx5_txq_release_devx_cq_resources(struct mlx5_txq_obj *txq_obj)
 	if (txq_obj->cq_devx)
 		claim_zero(mlx5_devx_cmd_destroy(txq_obj->cq_devx));
 	if (txq_obj->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(txq_obj->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(txq_obj->cq_umem));
 	if (txq_obj->cq_buf)
 		mlx5_free(txq_obj->cq_buf);
 	if (txq_obj->cq_dbrec_page)
@@ -1243,7 +1243,7 @@ mlx5_txq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		return 0;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->cq_umem = mlx5_glue->devx_umem_reg(priv->sh->ctx,
+	txq_obj->cq_umem = mlx5_os_umem_reg(priv->sh->ctx,
 						(void *)txq_obj->cq_buf,
 						cqe_n * sizeof(struct mlx5_cqe),
 						IBV_ACCESS_LOCAL_WRITE);
@@ -1342,7 +1342,7 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	txq_obj->sq_umem = mlx5_glue->devx_umem_reg
+	txq_obj->sq_umem = mlx5_os_umem_reg
 					(priv->sh->ctx,
 					 (void *)txq_obj->sq_buf,
 					 wqe_n * sizeof(struct mlx5_wqe),
diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c
index bf86aaaa39..66491bbf78 100644
--- a/drivers/net/mlx5/mlx5_flow.c
+++ b/drivers/net/mlx5/mlx5_flow.c
@@ -6446,7 +6446,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	}
 	mem_mng = (struct mlx5_counter_stats_mem_mng *)(mem + size) - 1;
 	size = sizeof(*raw_data) * MLX5_COUNTERS_PER_POOL * raws_n;
-	mem_mng->umem = mlx5_glue->devx_umem_reg(sh->ctx, mem, size,
+	mem_mng->umem = mlx5_os_umem_reg(sh->ctx, mem, size,
 						 IBV_ACCESS_LOCAL_WRITE);
 	if (!mem_mng->umem) {
 		rte_errno = errno;
@@ -6465,7 +6465,7 @@ mlx5_flow_create_counter_stat_mem_mng(struct mlx5_dev_ctx_shared *sh)
 	mkey_attr.relaxed_ordering_read = sh->cmng.relaxed_ordering_read;
 	mem_mng->dm = mlx5_devx_cmd_mkey_create(sh->ctx, &mkey_attr);
 	if (!mem_mng->dm) {
-		mlx5_glue->devx_umem_dereg(mem_mng->umem);
+		mlx5_os_umem_dereg(mem_mng->umem);
 		rte_errno = errno;
 		mlx5_free(mem);
 		return -rte_errno;
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index d61e43e55d..749529c410 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -131,13 +131,13 @@ mlx5_txpp_destroy_send_queue(struct mlx5_txpp_wq *wq)
 	if (wq->sq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->sq));
 	if (wq->sq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->sq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->sq_umem));
 	if (wq->sq_buf)
 		mlx5_free((void *)(uintptr_t)wq->sq_buf);
 	if (wq->cq)
 		claim_zero(mlx5_devx_cmd_destroy(wq->cq));
 	if (wq->cq_umem)
-		claim_zero(mlx5_glue->devx_umem_dereg(wq->cq_umem));
+		claim_zero(mlx5_os_umem_dereg(wq->cq_umem));
 	if (wq->cq_buf)
 		mlx5_free((void *)(uintptr_t)wq->cq_buf);
 	memset(wq, 0, sizeof(*wq));
@@ -268,7 +268,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -318,7 +318,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -506,7 +506,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		return -ENOMEM;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->cq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->cq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->cq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
@@ -562,7 +562,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh)
 		goto error;
 	}
 	/* Register allocated buffer in user space with DevX. */
-	wq->sq_umem = mlx5_glue->devx_umem_reg(sh->ctx,
+	wq->sq_umem = mlx5_os_umem_reg(sh->ctx,
 					       (void *)(uintptr_t)wq->sq_buf,
 					       umem_size,
 					       IBV_ACCESS_LOCAL_WRITE);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 11/32] net/mlx5: fix adding destroy flow action wrapper

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Glue function destroy_flow_action() was wrapped by OS specific operation
mlx5_flow_os_destroy_flow_action(). It was skipped in file mlx5.c.

Fixes: b293fbf9672b ("net/mlx5: add OS specific flow actions operations")
Cc: stable@dpdk.org

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/mlx5.c         | 7 ++++---
 drivers/net/mlx5/mlx5_flow_dv.c | 6 +++---
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 84123f8e3d..60301d3244 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -38,6 +38,7 @@
 #include "mlx5_autoconf.h"
 #include "mlx5_mr.h"
 #include "mlx5_flow.h"
+#include "mlx5_flow_os.h"
 #include "rte_pmd_mlx5.h"
 
 /* Device parameter to enable RX completion queue compression. */
@@ -415,8 +416,8 @@ mlx5_flow_aso_age_mng_close(struct mlx5_dev_ctx_shared *sh)
 			for (j = 0; j < MLX5_COUNTERS_PER_POOL; ++j)
 				if (pool->actions[j].dr_action)
 					claim_zero
-						(mlx5_glue->destroy_flow_action
-						  (pool->actions[j].dr_action));
+					    (mlx5_flow_os_destroy_flow_action
+					      (pool->actions[j].dr_action));
 			mlx5_free(pool);
 		}
 		mlx5_free(sh->aso_age_mng->pools);
@@ -523,7 +524,7 @@ mlx5_flow_counters_mng_close(struct mlx5_dev_ctx_shared *sh)
 
 				if (cnt->action)
 					claim_zero
-					 (mlx5_glue->destroy_flow_action
+					 (mlx5_flow_os_destroy_flow_action
 					  (cnt->action));
 				if (fallback && MLX5_POOL_GET_CNT
 				    (pool, j)->dcs_when_free)
diff --git a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c
index c31737652c..78b5a6b338 100644
--- a/drivers/net/mlx5/mlx5_flow_dv.c
+++ b/drivers/net/mlx5/mlx5_flow_dv.c
@@ -11117,11 +11117,11 @@ flow_dv_sample_remove_cb(struct mlx5_cache_list *list __rte_unused,
 	struct mlx5_priv *priv = dev->data->dev_private;
 
 	if (cache_resource->verbs_action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 				(cache_resource->verbs_action));
 	if (cache_resource->ft_type == MLX5DV_FLOW_TABLE_TYPE_FDB) {
 		if (cache_resource->default_miss)
-			claim_zero(mlx5_glue->destroy_flow_action
+			claim_zero(mlx5_flow_os_destroy_flow_action
 			  (cache_resource->default_miss));
 	}
 	if (cache_resource->normal_path_tbl)
@@ -11174,7 +11174,7 @@ flow_dv_dest_array_remove_cb(struct mlx5_cache_list *list __rte_unused,
 
 	MLX5_ASSERT(cache_resource->action);
 	if (cache_resource->action)
-		claim_zero(mlx5_glue->destroy_flow_action
+		claim_zero(mlx5_flow_os_destroy_flow_action
 					(cache_resource->action));
 	for (; i < cache_resource->num_of_dest; i++)
 		flow_dv_sample_sub_actions_release(dev,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 12/32] common/mlx5: add definition HAVE_INFINIBAND_VERBS_H

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add a Verbs file presence indication. Under Linux it is required that
file infiniband/verbs.h is installed for building DPDK.  Other
operating systems (e.g. Windows) ignore Verbs completely.  This commit
adds definition HAVE_INFINIBAND_VERBS_H (file mlx5_autoconf.h) to
indicate whether DPDK compiles with Verbs or not.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/meson.build | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build
index 63b78e4bce..580419e6d9 100644
--- a/drivers/common/mlx5/linux/meson.build
+++ b/drivers/common/mlx5/linux/meson.build
@@ -181,8 +181,10 @@ has_sym_args = [
 	[ 'HAVE_MLX5_DR_CREATE_ACTION_DEST_ARRAY', 'infiniband/mlx5dv.h',
 	'mlx5dv_dr_action_create_dest_array'],
 	[ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ],
-        [ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
-        'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_MLX5_DR_CREATE_ACTION_ASO', 'infiniband/mlx5dv.h',
+	'mlx5dv_dr_action_create_aso' ],
+	[ 'HAVE_INFINIBAND_VERBS_H', 'infiniband/verbs.h',
+	'INFINIBAND_VERBS_H' ],
 ]
 config = configuration_data()
 foreach arg:has_sym_args
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 13/32] common/mlx5/linux: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_malloc() API has an alignment parameter for system memory
allocations. malloc() is called for non-aligned allocations and
posix_memalign() is called for aligned allocations. When calling
mlx5_free() there is no distinction whether the memory was originally
allocated with or without alignment. Freeing a memory may be handled
differently by operating systems. Therefore this commit wraps these APIs
with OS specific calls: mlx5_os_malloc(), mlx5_os_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 38 ++++++++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_malloc.c          | 14 +++++------
 2 files changed, 44 insertions(+), 8 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index f8b215cc29..bd44ecb7d6 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -6,6 +6,7 @@
 #define RTE_PMD_MLX5_COMMON_OS_H_
 
 #include <stdio.h>
+#include <malloc.h>
 
 #include <rte_pci.h>
 #include <rte_debug.h>
@@ -16,6 +17,7 @@
 
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
+#include "mlx5_malloc.h"
 
 /**
  * Get device name. Given an ibv_device pointer - return a
@@ -224,4 +226,40 @@ mlx5_os_umem_dereg(void *pumem)
 {
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
+
+/**
+ * Memory allocation optionally with alignment.
+ *
+ * @param[in] align
+ *    Alignment size (may be zero)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	void *buf;
+
+	if (posix_memalign(&buf, align, size))
+		return NULL;
+	return buf;
+}
+
+/**
+ * This API de-allocates a memory that originally could have been
+ * allocated aligned or non-aligned. In Linux it is a wrapper
+ * around free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	free(addr);
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_malloc.c b/drivers/common/mlx5/mlx5_malloc.c
index 5a3267f730..9d30cedbaa 100644
--- a/drivers/common/mlx5/mlx5_malloc.c
+++ b/drivers/common/mlx5/mlx5_malloc.c
@@ -9,6 +9,7 @@
 #include <string.h>
 
 #include "mlx5_common_utils.h"
+#include "mlx5_common_os.h"
 #include "mlx5_malloc.h"
 
 struct mlx5_sys_mem {
@@ -148,14 +149,11 @@ static void *
 mlx5_alloc_align(size_t size, unsigned int align, unsigned int zero)
 {
 	void *buf;
-	int ret;
-
-	ret = posix_memalign(&buf, align, size);
-	if (ret) {
-		DRV_LOG(ERR,
-			"Couldn't allocate buf size=%zu align=%u. Err=%d\n",
-			size, align, ret);
 
+	buf = mlx5_os_malloc(align, size);
+	if (!buf) {
+		DRV_LOG(ERR, "Couldn't allocate buf size=%zu align=%u.",
+			size, align);
 		return NULL;
 	}
 	if (zero)
@@ -264,7 +262,7 @@ mlx5_free(void *addr)
 		__atomic_add_fetch(&mlx5_sys_mem.free_sys, 1,
 				   __ATOMIC_RELAXED);
 #endif
-		free(addr);
+		mlx5_os_free(addr);
 	} else {
 #ifdef RTE_LIBRTE_MLX5_DEBUG
 		__atomic_add_fetch(&mlx5_sys_mem.free_rte, 1,
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 14/32] common/mlx5/windows: handle memory allocations with alignment

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows equivalent of the Linux implementation.  The
APIs included in this commit: mlx5_os_malloc(), mlx5_os_free(). For
memory allocations (with or without alignment) we always call
_aligned_malloc(). Even if zero alignment was requested in the first
place - we always select a minimal alignment value. In this way when the
memory is free - it is always safe to call _aligned_free().

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 51 ++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
new file mode 100644
index 0000000000..cce7c88c6a
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef RTE_PMD_MLX5_COMMON_OS_H_
+#define RTE_PMD_MLX5_COMMON_OS_H_
+
+#include <stdio.h>
+
+#include "mlx5_autoconf.h"
+#include "mlx5_glue.h"
+#include "mlx5_malloc.h"
+
+/**
+ * This API allocates aligned or non-aligned memory.  The free can be on either
+ * aligned or nonaligned memory.  To be protected - even though there may be no
+ * alignment - in Windows this API will unconditioanlly call _aligned_malloc()
+ * with at least a minimal alignment size.
+ *
+ * @param[in] align
+ *    The alignment value, which must be an integer power of 2 (or 0 for
+ *    non-alignment)
+ * @param[in] size
+ *    Size in bytes to allocate
+ *
+ * @return
+ *    Valid pointer to allocated memory, NULL in case of failure
+ */
+static inline void *
+mlx5_os_malloc(size_t align, size_t size)
+{
+	if (align < MLX5_MALLOC_ALIGNMENT)
+		align = MLX5_MALLOC_ALIGNMENT;
+	return _aligned_malloc(size, align);
+}
+
+/**
+ * This API de-allocates a memory that originally could have been allocated
+ * aligned or non-aligned. In Windows since the allocation was with
+ * _aligned_malloc() - it is safe to always call _aligned_free().
+ *
+ * @param[in] addr
+ *    Pointer to address to free
+ *
+ */
+static inline void
+mlx5_os_free(void *addr)
+{
+	_aligned_free(addr);
+}
+#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 15/32] common/mlx5/linux: wrap event channel APIs with OS calls

[Tal Shnaiderman]

Wrap the API to create/destroy event channel and to subscribe an event
with OS calls. In Linux those calls are implemented by glue functions
while in Windows they are not supported.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/linux/mlx5_common_os.h | 22 ++++++++++++++++++++++
 drivers/net/mlx5/mlx5_devx.c               |  8 ++++----
 drivers/net/mlx5/mlx5_txpp.c               |  6 +++---
 drivers/vdpa/mlx5/mlx5_vdpa_event.c        |  7 ++++---
 4 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h
index bd44ecb7d6..63f070d9c4 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -227,6 +227,28 @@ mlx5_os_umem_dereg(void *pumem)
 	return mlx5_glue->devx_umem_dereg(pumem);
 }
 
+static inline void *
+mlx5_os_devx_create_event_channel(void *ctx, int flags)
+{
+	return mlx5_glue->devx_create_event_channel(ctx, flags);
+}
+
+static inline void
+mlx5_os_devx_destroy_event_channel(void *eventc)
+{
+	mlx5_glue->devx_destroy_event_channel(eventc);
+}
+
+static inline int
+mlx5_os_devx_subscribe_devx_event(void *eventc,
+				  void *obj,
+				  uint16_t events_sz, uint16_t events_num[],
+				  uint64_t cookie)
+{
+	return mlx5_glue->devx_subscribe_devx_event(eventc, obj, events_sz,
+						    events_num, cookie);
+}
+
 /**
  * Memory allocation optionally with alignment.
  *
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 235fd5798d..84a5c55ee0 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -216,7 +216,7 @@ mlx5_rxq_devx_obj_release(struct mlx5_rxq_obj *rxq_obj)
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->rq));
 		claim_zero(mlx5_devx_cmd_destroy(rxq_obj->devx_cq));
 		if (rxq_obj->devx_channel)
-			mlx5_glue->devx_destroy_event_channel
+			mlx5_os_devx_destroy_event_channel
 							(rxq_obj->devx_channel);
 		mlx5_rxq_release_devx_rq_resources(rxq_obj->rxq_ctrl);
 		mlx5_rxq_release_devx_cq_resources(rxq_obj->rxq_ctrl);
@@ -533,7 +533,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx)
 	rxq_data->cqe_n = log_cqe_n;
 	rxq_data->cqn = cq_obj->id;
 	if (rxq_ctrl->obj->devx_channel) {
-		ret = mlx5_glue->devx_subscribe_devx_event
+		ret = mlx5_os_devx_subscribe_devx_event
 						(rxq_ctrl->obj->devx_channel,
 						 cq_obj->obj,
 						 sizeof(event_nums),
@@ -644,7 +644,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 		int devx_ev_flag =
 			  MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA;
 
-		tmpl->devx_channel = mlx5_glue->devx_create_event_channel
+		tmpl->devx_channel = mlx5_os_devx_create_event_channel
 								(priv->sh->ctx,
 								 devx_ev_flag);
 		if (!tmpl->devx_channel) {
@@ -686,7 +686,7 @@ mlx5_rxq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx)
 	if (tmpl->devx_cq)
 		claim_zero(mlx5_devx_cmd_destroy(tmpl->devx_cq));
 	if (tmpl->devx_channel)
-		mlx5_glue->devx_destroy_event_channel(tmpl->devx_channel);
+		mlx5_os_devx_destroy_event_channel(tmpl->devx_channel);
 	mlx5_rxq_release_devx_rq_resources(rxq_ctrl);
 	mlx5_rxq_release_devx_cq_resources(rxq_ctrl);
 	rte_errno = ret; /* Restore rte_errno. */
diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 749529c410..726bdc6ae1 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -37,7 +37,7 @@ static void
 mlx5_txpp_destroy_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	if (sh->txpp.echan) {
-		mlx5_glue->devx_destroy_event_channel(sh->txpp.echan);
+		mlx5_os_devx_destroy_event_channel(sh->txpp.echan);
 		sh->txpp.echan = NULL;
 	}
 }
@@ -47,7 +47,7 @@ static int
 mlx5_txpp_create_event_channel(struct mlx5_dev_ctx_shared *sh)
 {
 	MLX5_ASSERT(!sh->txpp.echan);
-	sh->txpp.echan = mlx5_glue->devx_create_event_channel(sh->ctx,
+	sh->txpp.echan = mlx5_os_devx_create_event_channel(sh->ctx,
 			MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!sh->txpp.echan) {
 		rte_errno = errno;
@@ -937,7 +937,7 @@ mlx5_txpp_start_service(struct mlx5_dev_ctx_shared *sh)
 		return -rte_errno;
 	}
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(sh->txpp.echan,
+	ret = mlx5_os_devx_subscribe_devx_event(sh->txpp.echan,
 						   sh->txpp.rearm_queue.cq->obj,
 						   sizeof(event_nums),
 						   event_nums, 0);
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
index 3aeaeb893f..fd6150928b 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
@@ -15,6 +15,7 @@
 #include <rte_alarm.h>
 
 #include <mlx5_common.h>
+#include <mlx5_common_os.h>
 #include <mlx5_glue.h>
 
 #include "mlx5_vdpa_utils.h"
@@ -43,7 +44,7 @@ mlx5_vdpa_event_qp_global_release(struct mlx5_vdpa_priv *priv)
 		       sizeof(out.buf)) >=
 		       (ssize_t)sizeof(out.event_resp.cookie))
 			;
-		mlx5_glue->devx_destroy_event_channel(priv->eventc);
+		mlx5_os_devx_destroy_event_channel(priv->eventc);
 		priv->eventc = NULL;
 	}
 #endif
@@ -63,7 +64,7 @@ mlx5_vdpa_event_qp_global_prepare(struct mlx5_vdpa_priv *priv)
 		DRV_LOG(ERR, "Failed to query EQ number %d.", rte_errno);
 		return -1;
 	}
-	priv->eventc = mlx5_glue->devx_create_event_channel(priv->ctx,
+	priv->eventc = mlx5_os_devx_create_event_channel(priv->ctx,
 			   MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA);
 	if (!priv->eventc) {
 		rte_errno = errno;
@@ -176,7 +177,7 @@ mlx5_vdpa_cq_create(struct mlx5_vdpa_priv *priv, uint16_t log_desc_n,
 	cq->cq_ci = 0;
 	rte_spinlock_init(&cq->sl);
 	/* Subscribe CQ event to the event channel controlled by the driver. */
-	ret = mlx5_glue->devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
+	ret = mlx5_os_devx_subscribe_devx_event(priv->eventc, cq->cq->obj,
 						   sizeof(event_nums),
 						   event_nums,
 						   (uint64_t)(uintptr_t)cq);
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 16/32] common/mlx5: add Windows exports file

[Tal Shnaiderman]

File drivers/common/mlx5/rte_common_mlx5_exports.def contains mlx5
Windows exported symbols under common/mlx5 directory (DLL file
name librte_common_mlx5*.dll). It is the equivalent of Linux map
file rte_common_mlx5_version.map but the list of symbols may be
different between the two operating systems.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def | 62 +++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
 create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
new file mode 100644
index 0000000000..23984b7abd
--- /dev/null
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -0,0 +1,62 @@
+EXPORTS
+	haswell_broadwell_cpu
+
+	mlx5_common_init
+
+	mlx5_create_mr_ext
+
+	mlx5_devx_cmd_create_cq
+	mlx5_devx_cmd_create_flex_parser
+	mlx5_devx_cmd_create_qp
+	mlx5_devx_cmd_create_rq
+	mlx5_devx_cmd_create_rqt
+	mlx5_devx_cmd_create_sq
+	mlx5_devx_cmd_create_tir
+	mlx5_devx_cmd_create_td
+	mlx5_devx_cmd_create_tis
+	mlx5_devx_cmd_create_virtq
+	mlx5_devx_cmd_destroy
+	mlx5_devx_cmd_flow_counter_alloc
+	mlx5_devx_cmd_flow_counter_query
+	mlx5_devx_cmd_flow_dump
+	mlx5_devx_cmd_mkey_create
+	mlx5_devx_cmd_modify_qp_state
+	mlx5_devx_cmd_modify_rq
+	mlx5_devx_cmd_modify_rqt
+	mlx5_devx_cmd_modify_sq
+	mlx5_devx_cmd_modify_tir
+	mlx5_devx_cmd_modify_virtq
+	mlx5_devx_cmd_qp_query_tis_td
+	mlx5_devx_cmd_query_hca_attr
+	mlx5_devx_cmd_query_parse_samples
+	mlx5_devx_cmd_query_virtq
+	mlx5_devx_cmd_register_read
+	mlx5_devx_get_out_command_status
+	mlx5_devx_cmd_create_flow_hit_aso_obj
+
+	mlx5_get_dbr
+
+	mlx5_malloc_mem_select
+	mlx5_mr_btree_init
+	mlx5_mr_btree_free
+	mlx5_mr_btree_dump
+	mlx5_mr_addr2mr_bh
+	mlx5_mr_release_cache
+	mlx5_mr_dump_cache
+	mlx5_mr_rebuild_cache
+	mlx5_mr_insert_cache
+	mlx5_mr_lookup_cache
+	mlx5_mr_lookup_list
+	mlx5_mr_create_primary
+	mlx5_mr_flush_local_cache
+	mlx5_mp_req_queue_state_modify
+	mlx5_mr_free
+
+	mlx5_pci_driver_register
+
+	mlx5_release_dbr
+
+	mlx5_malloc
+	mlx5_realloc
+	mlx5_free
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 17/32] common/mlx5: extend DevX query hca attributes command

[Tal Shnaiderman]

Extend DevX API mlx5_devx_cmd_query_hca_attr() to report on max number
of available objects including: CQ, QP, PD, SRQ.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c | 11 +++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h | 10 +++++++++-
 drivers/common/mlx5/mlx5_prm.h       |  1 +
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 9c1d1883ea..36d3a421aa 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -720,6 +720,14 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->flow_hit_aso = !!(MLX5_GET64(cmd_hca_cap, hcattr,
 					   general_obj_types) &
 				MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
+	attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
+	attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
+	attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
+	attr->log_max_qp_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp_sz);
+	attr->log_max_mrw_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_mrw_sz);
+	attr->log_max_pd = MLX5_GET(cmd_hca_cap, hcattr, log_max_pd);
+	attr->log_max_srq = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq);
+	attr->log_max_srq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_srq_sz);
 	if (attr->qos.sup) {
 		MLX5_SET(query_hca_cap_in, in, op_mod,
 			 MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
@@ -834,6 +842,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
 	attr->tunnel_stateless_gtp = MLX5_GET
 					(per_protocol_networking_offload_caps,
 					 hcattr, tunnel_stateless_gtp);
+	attr->rss_ind_tbl_cap = MLX5_GET
+					(per_protocol_networking_offload_caps,
+					 hcattr, rss_ind_tbl_cap);
 	if (attr->wqe_inline_mode != MLX5_CAP_INLINE_MODE_VPORT_CONTEXT)
 		return 0;
 	if (attr->eth_virt) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 726e9f5192..5056d86ed9 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -117,6 +117,15 @@ struct mlx5_hca_attr {
 	uint32_t log_max_ft_sampler_num:8;
 	struct mlx5_hca_qos_attr qos;
 	struct mlx5_hca_vdpa_attr vdpa;
+	int log_max_qp_sz;
+	int log_max_cq_sz;
+	int log_max_qp;
+	int log_max_cq;
+	uint32_t log_max_pd;
+	uint32_t log_max_mrw_sz;
+	uint32_t log_max_srq;
+	uint32_t log_max_srq_sz;
+	uint32_t rss_ind_tbl_cap;
 };
 
 struct mlx5_devx_wq_attr {
@@ -495,7 +504,6 @@ struct mlx5_devx_obj *mlx5_devx_cmd_create_virtio_q_counters(void *ctx);
 __rte_internal
 int mlx5_devx_cmd_query_virtio_q_counters(struct mlx5_devx_obj *couners_obj,
 				  struct mlx5_devx_virtio_q_couners_attr *attr);
-
 __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 58d180486e..585e9d1c05 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -665,6 +665,7 @@ typedef uint8_t u8;
 #define MLX5_GET64(typ, p, fld) rte_be_to_cpu_64(*((rte_be64_t *)(p) + \
 						   __mlx5_64_off(typ, fld)))
 #define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
+#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
 
 struct mlx5_ifc_fte_match_set_misc_bits {
 	u8 gre_c_present[0x1];
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 18/32] common/mlx5: add DevX alloc PD command

[Tal Shnaiderman]

Add a new DevX API mlx5_devx_cmd_alloc_pd() that creates a new protection
domain (PD).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_devx_cmds.c            | 35 ++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_devx_cmds.h            |  2 ++
 drivers/common/mlx5/mlx5_prm.h                  | 36 +++++++++++++++++++++++++
 drivers/common/mlx5/rte_common_mlx5_exports.def |  1 +
 drivers/common/mlx5/version.map                 |  1 +
 5 files changed, 75 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 36d3a421aa..12f51a940c 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2055,3 +2055,38 @@ mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx, uint32_t pd)
 	flow_hit_aso_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
 	return flow_hit_aso_obj;
 }
+
+/*
+ * Create PD using DevX API.
+ *
+ * @param[in] ctx
+ *   Context returned from mlx5 open_device() glue function.
+ *
+ * @return
+ *   The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_alloc_pd(void *ctx)
+{
+	struct mlx5_devx_obj *ppd =
+		mlx5_malloc(MLX5_MEM_ZERO, sizeof(*ppd), 0, SOCKET_ID_ANY);
+	u32 in[MLX5_ST_SZ_DW(alloc_pd_in)] = {0};
+	u32 out[MLX5_ST_SZ_DW(alloc_pd_out)] = {0};
+
+	if (!ppd) {
+		DRV_LOG(ERR, "Failed to allocate PD data.");
+		rte_errno = ENOMEM;
+		return NULL;
+	}
+	MLX5_SET(alloc_pd_in, in, opcode, MLX5_CMD_OP_ALLOC_PD);
+	ppd->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+				out, sizeof(out));
+	if (!ppd->obj) {
+		mlx5_free(ppd);
+		DRV_LOG(ERR, "Failed to allocate PD Obj using DevX.");
+		rte_errno = errno;
+		return NULL;
+	}
+	ppd->id = MLX5_GET(alloc_pd_out, out, pd);
+	return ppd;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 5056d86ed9..8277fdbc39 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -508,4 +508,6 @@ __rte_internal
 struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_hit_aso_obj(void *ctx,
 							    uint32_t pd);
 
+__rte_internal
+struct mlx5_devx_obj *mlx5_devx_cmd_alloc_pd(void *ctx);
 #endif /* RTE_PMD_MLX5_DEVX_CMDS_H_ */
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 585e9d1c05..4ddf865743 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -844,6 +844,8 @@ enum {
 	MLX5_CMD_OP_SUSPEND_QP = 0x50F,
 	MLX5_CMD_OP_RESUME_QP = 0x510,
 	MLX5_CMD_OP_QUERY_NIC_VPORT_CONTEXT = 0x754,
+	MLX5_CMD_OP_ALLOC_PD = 0x800,
+	MLX5_CMD_OP_DEALLOC_PD = 0x801,
 	MLX5_CMD_OP_ACCESS_REGISTER = 0x805,
 	MLX5_CMD_OP_ALLOC_TRANSPORT_DOMAIN = 0x816,
 	MLX5_CMD_OP_CREATE_TIR = 0x900,
@@ -2772,6 +2774,40 @@ struct mlx5_ifc_init2init_qp_in_bits {
 	u8 reserved_at_800[0x80];
 };
 
+struct mlx5_ifc_dealloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x40];
+};
+
+struct mlx5_ifc_dealloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x8];
+	u8 pd[0x18];
+	u8 reserved_3[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_out_bits {
+	u8 status[0x8];
+	u8 reserved_0[0x18];
+	u8 syndrome[0x20];
+	u8 reserved_1[0x8];
+	u8 pd[0x18];
+	u8 reserved_2[0x20];
+};
+
+struct mlx5_ifc_alloc_pd_in_bits {
+	u8 opcode[0x10];
+	u8 reserved_0[0x10];
+	u8 reserved_1[0x10];
+	u8 op_mod[0x10];
+	u8 reserved_2[0x40];
+};
+
 #ifdef PEDANTIC
 #pragma GCC diagnostic ignored "-Wpedantic"
 #endif
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 23984b7abd..15aafc3809 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -5,6 +5,7 @@ EXPORTS
 
 	mlx5_create_mr_ext
 
+	mlx5_devx_cmd_alloc_pd
 	mlx5_devx_cmd_create_cq
 	mlx5_devx_cmd_create_flex_parser
 	mlx5_devx_cmd_create_qp
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 17dd11f635..fb3952bbc6 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -10,6 +10,7 @@ INTERNAL {
 
 	mlx5_dev_to_pci_addr;
 
+	mlx5_devx_cmd_alloc_pd;
 	mlx5_devx_cmd_create_cq;
 	mlx5_devx_cmd_create_flex_parser;
 	mlx5_devx_cmd_create_qp;
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 19/32] common/mlx5/windows: add glue functions APIs

[Tal Shnaiderman]

Windows glue functions are added to file mlx5/windows/mlx5_glue.c.
The following APIs are supported:
get_device_list, free_device_list, open_device, close_device,
query_device, query_hca_iseg, devx_obj_create, devx_obj_destroy,
devx_obj_query, devx_obj_modify, devx_general_cmd, devx_umem_reg,
devx_umem_dereg, devx_alloc_uar, devx_free_uar, devx_fs_rule_add,
devx_fs_rule_del, devx_query_eqn
New added files:
mlx5_win_defs.h - this file imports missing definitions from Linux
rdma-core library and Linux OS.
mlx5_win_ext.h - this file contains structs that enable a unified
Linux/Windows API. Each struct has an equivalent (but different) Linux
struct. By calling with 'void *' pointers - the Linux/Windows API is
identical.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |   1 +
 drivers/common/mlx5/windows/mlx5_glue.c         | 304 ++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_glue.h         |  58 +++++
 drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  34 +++
 5 files changed, 422 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
 create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
 create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 15aafc3809..2f6aab133b 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -36,6 +36,7 @@ EXPORTS
 	mlx5_devx_cmd_create_flow_hit_aso_obj
 
 	mlx5_get_dbr
+	mlx5_glue
 
 	mlx5_malloc_mem_select
 	mlx5_mr_btree_init
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
new file mode 100644
index 0000000000..7f8a00aaa7
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -0,0 +1,304 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <unistd.h>
+
+#include <rte_malloc.h>
+
+#include "mlx5_glue.h"
+#include "mlx5_common_utils.h"
+#include "mlx5_win_ext.h"
+
+/*
+ * The returned value of this API is an array of pointers to mlx5
+ * devices under Windows. The interesting parameters of a device:
+ * Device PCI parameters: domain, bus, device id, function.
+ * Device name.
+ */
+static void *
+mlx5_glue_devx_get_device_list(int *num_devices)
+{
+	struct devx_device_bdf *devx_bdf_devs = NULL;
+	size_t n_devx_devx = 0;
+	int32_t ret = devx_get_device_list(&n_devx_devx, &devx_bdf_devs);
+
+	if (ret) {
+		errno = ret;
+		*num_devices = 0;
+		return NULL;
+	}
+	*num_devices = (int)n_devx_devx;
+	return devx_bdf_devs;
+}
+
+static void
+mlx5_glue_devx_free_device_list(void *list)
+{
+	if (!list) {
+		errno = EINVAL;
+		return;
+	}
+	devx_free_device_list(list);
+}
+
+static int
+mlx5_glue_devx_close_device(void *ctx)
+{
+	mlx5_context_st *mlx5_ctx;
+	int rc;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (mlx5_context_st *)ctx;
+	rc = devx_close_device(mlx5_ctx->devx_ctx);
+	free(mlx5_ctx);
+	return rc;
+}
+
+static void *
+mlx5_glue_devx_open_device(void *device)
+{
+	struct mlx5_context *mlx5_ctx;
+
+	if (!device) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = malloc((sizeof(struct mlx5_context)));
+	if (!mlx5_ctx) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(mlx5_ctx, 0, sizeof(*mlx5_ctx));
+	mlx5_ctx->devx_ctx = devx_open_device(device);
+	if (DEVX_IS_ERR(mlx5_ctx->devx_ctx)) {
+		errno = -DEVX_PTR_ERR(mlx5_ctx->devx_ctx);
+		free(mlx5_ctx);
+		return NULL;
+	}
+	return mlx5_ctx;
+}
+
+static int
+mlx5_glue_devx_query_device(void *device_bdf, void *dev_inf)
+{
+	struct devx_device_bdf *dev_bdf;
+	struct devx_device *mlx5_dev;
+
+	if (!device_bdf)
+		return -EINVAL;
+	dev_bdf = (struct devx_device_bdf *)device_bdf;
+	mlx5_dev = (struct devx_device *)dev_inf;
+	int err = devx_query_device(dev_bdf, mlx5_dev);
+	if (err)
+		return -E_FAIL;
+	return 0;
+}
+
+static void *
+mlx5_glue_devx_query_hca_iseg_mapping(void *ctx, uint32_t *cb_iseg)
+{
+	struct mlx5_context *mlx5_ctx;
+	void *pv_iseg;
+	int err;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	err = devx_query_hca_iseg_mapping(mlx5_ctx->devx_ctx,
+						cb_iseg, &pv_iseg);
+	if (err) {
+		errno = err;
+		return NULL;
+	}
+	return pv_iseg;
+}
+
+static void *
+mlx5_glue_devx_obj_create(void *ctx,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	devx_obj = malloc((sizeof(*devx_obj)));
+	if (!devx_obj) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	memset(devx_obj, 0, sizeof(*devx_obj));
+	devx_obj->devx_ctx = GET_DEVX_CTX(ctx);
+	devx_obj->obj = devx_obj_create(devx_obj->devx_ctx,
+					in, inlen, out, outlen);
+	if (DEVX_IS_ERR(devx_obj->obj)) {
+		errno = -DEVX_PTR_ERR(devx_obj->obj);
+		free(devx_obj);
+		return NULL;
+	}
+	return devx_obj;
+}
+
+static int
+mlx5_glue_devx_obj_destroy(void *obj)
+{
+	mlx5_devx_obj_st *devx_obj;
+
+	if (!obj)
+		return -EINVAL;
+	devx_obj = obj;
+	int rc = devx_obj_destroy(devx_obj->obj);
+	free(devx_obj);
+	return rc;
+}
+
+static int
+mlx5_glue_devx_general_cmd(void *ctx,
+			   void *in, size_t inlen,
+			   void *out, size_t outlen)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_cmd(GET_DEVX_CTX(ctx), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_query(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_obj_modify(void *obj,
+			    void *in, size_t inlen,
+			    void *out, size_t outlen)
+{
+	return devx_cmd(GET_OBJ_CTX(obj), in, inlen, out, outlen);
+}
+
+static int
+mlx5_glue_devx_umem_dereg(void *pumem)
+{
+	struct devx_obj_handle *umem;
+
+	if (!pumem)
+		return -EINVAL;
+	umem = pumem;
+	return devx_umem_unreg(umem);
+}
+
+static void *
+mlx5_glue_devx_umem_reg(void *ctx, void *addr, size_t size,
+				  uint32_t access, uint32_t *id)
+{
+	struct devx_obj_handle *umem_hdl;
+	int w_access = DEVX_UMEM_ACCESS_READ;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	if (access)
+		w_access |= DEVX_UMEM_ACCESS_WRITE;
+
+	umem_hdl = devx_umem_reg(GET_DEVX_CTX(ctx), addr,
+					size, w_access, id);
+	if (DEVX_IS_ERR(umem_hdl)) {
+		errno = -DEVX_PTR_ERR(umem_hdl);
+		return NULL;
+	}
+	return umem_hdl;
+}
+
+static void *
+mlx5_glue_devx_alloc_uar(void *ctx,
+		uint32_t flags)
+{
+	devx_uar_handle *uar;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	uar = devx_alloc_uar(GET_DEVX_CTX(ctx), flags);
+	if (DEVX_IS_ERR(uar)) {
+		errno = -DEVX_PTR_ERR(uar);
+		return NULL;
+	}
+	return uar;
+}
+
+static int
+mlx5_glue_devx_query_eqn(void *ctx,
+		uint32_t cpus, uint32_t *eqn)
+{
+	if (!ctx)
+		return -EINVAL;
+	return devx_query_eqn(GET_DEVX_CTX(ctx), cpus, eqn);
+}
+
+static void
+mlx5_glue_devx_free_uar(void *uar)
+{
+	devx_free_uar((devx_uar_handle *)uar);
+}
+
+static void*
+mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
+
+{
+	struct devx_obj_handle *rule_hdl = NULL;
+
+	if (!ctx) {
+		errno = EINVAL;
+		return NULL;
+	}
+	rule_hdl = devx_fs_rule_add(GET_DEVX_CTX(ctx), in, inlen);
+	if (DEVX_IS_ERR(rule_hdl)) {
+		errno = -DEVX_PTR_ERR(rule_hdl);
+		return NULL;
+	}
+	return rule_hdl;
+}
+
+static int
+mlx5_glue_devx_fs_rule_del(void *flow)
+{
+	return devx_fs_rule_del(flow);
+}
+
+alignas(RTE_CACHE_LINE_SIZE)
+const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue){
+	.version = MLX5_GLUE_VERSION,
+	.get_device_list = mlx5_glue_devx_get_device_list,
+	.free_device_list = mlx5_glue_devx_free_device_list,
+	.open_device = mlx5_glue_devx_open_device,
+	.close_device = mlx5_glue_devx_close_device,
+	.query_device = mlx5_glue_devx_query_device,
+	.query_hca_iseg = mlx5_glue_devx_query_hca_iseg_mapping,
+	.devx_obj_create = mlx5_glue_devx_obj_create,
+	.devx_obj_destroy = mlx5_glue_devx_obj_destroy,
+	.devx_obj_query = mlx5_glue_devx_obj_query,
+	.devx_obj_modify = mlx5_glue_devx_obj_modify,
+	.devx_general_cmd = mlx5_glue_devx_general_cmd,
+	.devx_umem_reg = mlx5_glue_devx_umem_reg,
+	.devx_umem_dereg = mlx5_glue_devx_umem_dereg,
+	.devx_alloc_uar = mlx5_glue_devx_alloc_uar,
+	.devx_free_uar = mlx5_glue_devx_free_uar,
+	.devx_fs_rule_add = mlx5_glue_devx_fs_rule_add,
+	.devx_fs_rule_del = mlx5_glue_devx_fs_rule_del,
+	.devx_query_eqn = mlx5_glue_devx_query_eqn,
+};
diff --git a/drivers/common/mlx5/windows/mlx5_glue.h b/drivers/common/mlx5/windows/mlx5_glue.h
new file mode 100644
index 0000000000..f2261ec7ac
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_glue.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_GLUE_H_
+#define MLX5_GLUE_H_
+
+#include <stddef.h>
+#include <stdint.h>
+
+#include <rte_byteorder.h>
+#include <mlx5_win_defs.h>
+
+#ifndef MLX5_GLUE_VERSION
+#define MLX5_GLUE_VERSION ""
+#endif
+
+/* LIB_GLUE_VERSION must be updated every time this structure is modified. */
+struct mlx5_glue {
+	const char *version;
+	void *(*devx_obj_create)(void *ctx,
+				 void *in, size_t inlen,
+				 void *out, size_t outlen);
+	int (*devx_obj_destroy)(void *obj);
+	int (*devx_obj_query)(void *obj,
+			      void *in, size_t inlen,
+			      void *out, size_t outlen);
+	int (*devx_obj_modify)(void *obj,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_general_cmd)(void *ctx,
+			       void *in, size_t inlen,
+			       void *out, size_t outlen);
+	int (*devx_umem_dereg)(void *umem);
+	void *(*devx_umem_reg)(void *ctx,
+			void *addr, size_t size,
+			uint32_t access, uint32_t *id);
+	void *(*devx_alloc_uar)(void *ctx,
+			uint32_t flags);
+	void (*devx_free_uar)(void *uar);
+	void *(*get_device_list)(int *num_devices);
+	void (*free_device_list)(void *list);
+	void *(*open_device)(void *device);
+	int (*close_device)(void *ctx);
+	int (*query_device)(void *device_bdf, void *dev_inf);
+	void* (*query_hca_iseg)(void *ctx, uint32_t *cb_iseg);
+	int (*devx_obj_query_async)(void *obj,
+				    const void *in, size_t inlen,
+				    size_t outlen, uint64_t wr_id,
+				    void *cmd_comp);
+	void *(*devx_fs_rule_add)(void *ctx, void *in, uint32_t inlen);
+	int (*devx_fs_rule_del)(void *flow);
+	int (*devx_query_eqn)(void *context, uint32_t cpus, uint32_t *eqn);
+};
+
+extern const struct mlx5_glue *mlx5_glue;
+
+#endif /* MLX5_GLUE_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_defs.h b/drivers/common/mlx5/windows/mlx5_win_defs.h
new file mode 100644
index 0000000000..72a3131f5e
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_defs.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_DEFS_H__
+#define __MLX5_WIN_DEFS_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+enum {
+	MLX5_CQE_OWNER_MASK	= 1,
+	MLX5_CQE_REQ		= 0,
+	MLX5_CQE_RESP_WR_IMM	= 1,
+	MLX5_CQE_RESP_SEND	= 2,
+	MLX5_CQE_RESP_SEND_IMM	= 3,
+	MLX5_CQE_RESP_SEND_INV	= 4,
+	MLX5_CQE_RESIZE_CQ	= 5,
+	MLX5_CQE_NO_PACKET	= 6,
+	MLX5_CQE_REQ_ERR	= 13,
+	MLX5_CQE_RESP_ERR	= 14,
+	MLX5_CQE_INVALID	= 15,
+};
+#endif /* __MLX5_WIN_DEFS_H__ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
new file mode 100644
index 0000000000..0e74910e9d
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) Mellanox Technologies, Ltd. 2001-2020.
+ *
+ */
+#ifndef __MLX5_WIN_ETX_H__
+#define __MLX5_WIN_ETX_H__
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "mlx5devx.h"
+
+typedef struct mlx5_context {
+	devx_device_ctx        *devx_ctx;
+	struct devx_device mlx5_dev;
+
+} mlx5_context_st;
+
+typedef struct {
+	devx_device_ctx *devx_ctx;
+	struct devx_obj_handle *obj;
+} mlx5_devx_obj_st;
+
+struct mlx5_devx_umem {
+	void                   *addr;
+	struct devx_obj_handle *umem_hdl;
+	uint32_t                umem_id;
+};
+
+#define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
+#define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
+
+#endif /* __MLX5_WIN_ETX_H__ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 20/32] mlx5/windows: add mlx5 meson file

[Tal Shnaiderman]

File drivers/common/mlx5/windows/meson.build is added to enable mlx5
source files compilation under common windows directory.
A Devx SDK tool must be installed to export two external H files:
mlx5devx.h and mlx5_ifc_devx.h. The installation is based on environment
variable DEVX_INC_PATH. In addition a DLL lib file is installed based on
environment variable DEVX_LIB_PATH. The meson file is using the
environment variables for compilation and linkage.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 41 +++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/meson.build

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
new file mode 100644
index 0000000000..4f9825ea19
--- /dev/null
+++ b/drivers/common/mlx5/windows/meson.build
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2019 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+
+sources += files(
+	'mlx5_glue.c',
+)
+
+res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
+res_inc = run_command(python3, '-c', 'import os; print(os.environ["DEVX_INC_PATH"])')
+
+if (res_lib.returncode() != 0 or res_inc.returncode() != 0)
+	build = false
+	reason = 'DevX environment variables are not set, DEVX_LIB_PATH and DEVX_INC_PATH vars must be exported'
+	subdir_done()
+endif
+
+devx_lib_dir = res_lib.stdout().strip()
+devx_inc_dir = res_inc.stdout().strip()
+
+ext_deps += cc.find_library('mlx5devx', dirs: devx_lib_dir, required: true)
+includes += include_directories(devx_inc_dir)
+cflags_options = [
+	'-std=c11',
+	'-Wno-strict-prototypes',
+	'-D_BSD_SOURCE',
+	'-D_DEFAULT_SOURCE',
+	'-D_XOPEN_SOURCE=600'
+]
+foreach option:cflags_options
+	if cc.has_argument(option)
+		cflags += option
+	endif
+endforeach
+if get_option('buildtype').contains('debug')
+	cflags += [ '-pedantic', '-DPEDANTIC' ]
+else
+	cflags += [ '-UPEDANTIC' ]
+endif
+
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 21/32] mlx5/windows: add initialization routine for external lib

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add function mlx5_glue_constructor() as an initialization routine for
run-time dependency on external lib.  Currently the routine has an empty
body. It is used for compatibility with Linux.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build      |  1 +
 drivers/common/mlx5/windows/mlx5_common_os.c | 24 ++++++++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index 4f9825ea19..e5a5be0708 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -5,6 +5,7 @@ includes += include_directories('.')
 
 sources += files(
 	'mlx5_glue.c',
+	'mlx5_common_os.c',
 )
 
 res_lib = run_command(python3, '-c', 'import os; print(os.environ["DEVX_LIB_PATH"])')
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
new file mode 100644
index 0000000000..5707fb60c3
--- /dev/null
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <unistd.h>
+#include <string.h>
+#include <stdio.h>
+
+#include <rte_errno.h>
+#include <rte_mempool.h>
+#include <rte_malloc.h>
+#include <rte_errno.h>
+
+#include "mlx5_devx_cmds.h"
+#include "mlx5_common_utils.h"
+#include <mlx5_common.h>
+
+/**
+ * Initialization routine for run-time dependency on external lib
+ */
+void
+mlx5_glue_constructor(void)
+{
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 22/32] mlx5/windows: generate file mlx5_autoconf.h

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

File mlx5_autoconf.h is generated under Windows to maintain
compatibility with the Linux build system.  This file is included in
Linux/Windows shared source files therefore it is required. Currently
the file is created empty.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/meson.build | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/common/mlx5/windows/meson.build b/drivers/common/mlx5/windows/meson.build
index e5a5be0708..16349c34f1 100644
--- a/drivers/common/mlx5/windows/meson.build
+++ b/drivers/common/mlx5/windows/meson.build
@@ -40,3 +40,6 @@ else
 	cflags += [ '-UPEDANTIC' ]
 endif
 
+# Generate an empty mlx5_autoconf.h file for compatibility with Linux
+config = configuration_data()
+configure_file(output : 'mlx5_autoconf.h', configuration : config)
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 23/32] common/mlx5/windows: extend PRM match_param_bits struct

[Tal Shnaiderman]

Add reserved size to PRM 'struct mlx5_ifc_fte_match_param_bits' for non
Linux OS.  Windows drivers require this extension since their expected
size should match the actual struct size. Linux drivers do not require
this extension and already use calculations based on the shorter size.
Use a static assert to verify that the PRM fte_match_param struct size
remains correct.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h          | 7 +++++++
 drivers/common/mlx5/windows/mlx5_glue.c | 3 +++
 2 files changed, 10 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 4ddf865743..27a4e4f0f1 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -815,6 +815,13 @@ struct mlx5_ifc_fte_match_param_bits {
 	struct mlx5_ifc_fte_match_set_misc2_bits misc_parameters_2;
 	struct mlx5_ifc_fte_match_set_misc3_bits misc_parameters_3;
 	struct mlx5_ifc_fte_match_set_misc4_bits misc_parameters_4;
+/*
+ * Add reserved bit to match the struct size with the size defined in PRM.
+ * This extension is not required in Linux.
+ */
+#ifndef HAVE_INFINIBAND_VERBS_H
+	u8 reserved_0[0x400];
+#endif
 };
 
 enum {
diff --git a/drivers/common/mlx5/windows/mlx5_glue.c b/drivers/common/mlx5/windows/mlx5_glue.c
index 7f8a00aaa7..3896cf0932 100644
--- a/drivers/common/mlx5/windows/mlx5_glue.c
+++ b/drivers/common/mlx5/windows/mlx5_glue.c
@@ -256,6 +256,9 @@ mlx5_glue_devx_free_uar(void *uar)
 	devx_free_uar((devx_uar_handle *)uar);
 }
 
+static_assert(MLX5_ST_SZ_BYTES(fte_match_param) == 0x200,
+	"PRM size of fte_match_param is broken! cannot compile Windows!");
+
 static void*
 mlx5_glue_devx_fs_rule_add(void *ctx, void *in, uint32_t inlen)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 24/32] common/mlx5/windows: add getter functions

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Add file mlx5/windows/mlx5_common_os.h the equivalent of Linux file
mlx5/linux/mlx5_common_os.h. It contains getters functions
mlx5_os_get_dev_device_name, mlx5_os_get_ctx_device_name,
mlx5_os_get_ctx_device_path, mlx5_os_get_umem_id,
mlx5_os_get_devx_channel_fd.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/windows/mlx5_common_os.h | 91 ++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index cce7c88c6a..2abdb2c3ae 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -48,4 +48,95 @@ mlx5_os_free(void *addr)
 {
 	_aligned_free(addr);
 }
+
+/**
+ * Get fd. Given a pointer to DevX channel object of type
+ * 'struct mlx5dv_devx_event_channel*' - return its fd.
+ * Under Windows it is a stub.
+ *
+ * @param[in] channel
+ *    Pointer to channel object.
+ *
+ * @return
+ *    0
+ */
+static inline int
+mlx5_os_get_devx_channel_fd(void *channel)
+{
+	if (!channel)
+		return 0;
+	return 0;
+}
+
+/**
+ * Get device name. Given a device pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] dev
+ *   Pointer to device.
+ *
+ * @return
+ *   Pointer to device name if dev is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_dev_device_name(void *dev)
+{
+	if (!dev)
+		return NULL;
+	return ((struct devx_device *)dev)->name;
+}
+
+/**
+ * Get device name. Given a context pointer - return a
+ * pointer to the corresponding device name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device name if ctx is valid, NULL otherwise.
+ */
+static inline const char *
+mlx5_os_get_ctx_device_name(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.name;
+}
+
+/**
+ * Get a device path name. Given acontext pointer - return a
+ * pointer to the corresponding device path name.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *   Pointer to device path name if ctx is valid, NULL otherwise.
+ */
+
+static inline const char *
+mlx5_os_get_ctx_device_path(void *ctx)
+{
+	if (!ctx)
+		return NULL;
+	return ((mlx5_context_st *)ctx)->mlx5_dev.dev_pnp_id;
+}
+
+/**
+ * Get umem id. Given a pointer to umem object of type return its id.
+ *
+ * @param[in] umem
+ *    Pointer to umem object.
+ *
+ * @return
+ *    The umem id if umem is valid, 0 otherwise.
+ */
+static inline uint32_t
+mlx5_os_get_umem_id(void *umem)
+{
+	if (!umem)
+		return 0;
+	return ((struct mlx5_devx_umem *)umem)->umem_id;
+}
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 25/32] common/mlx5/windows: add OS alloc/dealloc pd

[Tal Shnaiderman]

Implement Windows API mlx5_os_alloc_pd() and mlx5_os_dealloc_pd(). They
are equivalent to the Linux implementation in [1].

[1]
("net/mlx5: wrap glue alloc/dealloc PD with OS calls")

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  3 +-
 drivers/common/mlx5/windows/mlx5_common_os.c    | 48 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  3 ++
 drivers/common/mlx5/windows/mlx5_win_ext.h      |  6 ++++
 4 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 2f6aab133b..5db1eaa48c 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -61,4 +61,5 @@ EXPORTS
 	mlx5_malloc
 	mlx5_realloc
 	mlx5_free
-
+	mlx5_os_alloc_pd
+	mlx5_os_dealloc_pd
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 5707fb60c3..85537b6739 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -22,3 +22,51 @@ void
 mlx5_glue_constructor(void)
 {
 }
+
+/**
+ * Allocate PD. Given a devx context object
+ * return an mlx5-pd object.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ *
+ * @return
+ *    The mlx5_pd if pd is valid, NULL and errno otherwise.
+ */
+void *
+mlx5_os_alloc_pd(void *ctx)
+{
+	struct mlx5_pd *ppd =  mlx5_malloc(MLX5_MEM_ZERO,
+		sizeof(struct mlx5_pd), 0, SOCKET_ID_ANY);
+	if (!ppd)
+		return NULL;
+
+	struct mlx5_devx_obj *obj = mlx5_devx_cmd_alloc_pd(ctx);
+	if (!obj) {
+		mlx5_free(ppd);
+		return NULL;
+	}
+	ppd->obj = obj;
+	ppd->pdn = obj->id;
+	ppd->devx_ctx = ctx;
+	return ppd;
+}
+
+/**
+ * Release PD. Releases a given mlx5_pd object
+ *
+ * @param[in] pd
+ *   Pointer to mlx5_pd.
+ *
+ * @return
+ *    Zero if pd is released successfully, negative number otherwise.
+ */
+int
+mlx5_os_dealloc_pd(void *pd)
+{
+	if (!pd)
+		return -EINVAL;
+	mlx5_devx_cmd_destroy(((struct mlx5_pd *)pd)->obj);
+	mlx5_free(pd);
+	return 0;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index 2abdb2c3ae..f47351ec41 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -139,4 +139,7 @@ mlx5_os_get_umem_id(void *umem)
 		return 0;
 	return ((struct mlx5_devx_umem *)umem)->umem_id;
 }
+
+void *mlx5_os_alloc_pd(void *ctx);
+int mlx5_os_dealloc_pd(void *pd);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/windows/mlx5_win_ext.h b/drivers/common/mlx5/windows/mlx5_win_ext.h
index 0e74910e9d..8e697b30c1 100644
--- a/drivers/common/mlx5/windows/mlx5_win_ext.h
+++ b/drivers/common/mlx5/windows/mlx5_win_ext.h
@@ -28,6 +28,12 @@ struct mlx5_devx_umem {
 	uint32_t                umem_id;
 };
 
+struct mlx5_pd {
+	void                   *obj;
+	uint32_t                pdn;
+	devx_device_ctx        *devx_ctx;
+};
+
 #define GET_DEVX_CTX(ctx) (((mlx5_context_st *)ctx)->devx_ctx)
 #define GET_OBJ_CTX(obj)  (((mlx5_devx_obj_st *)obj)->devx_ctx)
 
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 26/32] common/mlx5/windows: add OS umem reg/dereg API

[Tal Shnaiderman]

Implement Windows API mlx5_os_umem_reg() and mlx5_os_umem_dereg(). They
are equivalent to the Linux implementation.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 60 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  2 +
 3 files changed, 64 insertions(+)

diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 5db1eaa48c..1eabe29a58 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,3 +63,5 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_umem_reg
+	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index 85537b6739..cbb09906bd 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -70,3 +70,63 @@ mlx5_os_dealloc_pd(void *pd)
 	mlx5_free(pd);
 	return 0;
 }
+
+/**
+ * Register umem.
+ *
+ * @param[in] ctx
+ *   Pointer to context.
+ * @param[in] addr
+ *   Pointer to memory start address.
+ * @param[in] size
+ *   Size of the memory to register.
+ * @param[out] access
+ *   UMEM access type
+ *
+ * @return
+ *   umem on successful registration, NULL and errno otherwise
+ */
+void *
+mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access)
+{
+	struct mlx5_devx_umem *umem;
+
+	umem = mlx5_malloc(MLX5_MEM_ZERO,
+		(sizeof(*umem)), 0, SOCKET_ID_ANY);
+	if (!umem) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	umem->umem_hdl = mlx5_glue->devx_umem_reg(ctx, addr, size, access,
+		&umem->umem_id);
+	if (!umem->umem_hdl) {
+		mlx5_free(umem);
+		return NULL;
+	}
+	umem->addr = addr;
+	return umem;
+}
+
+/**
+ * Deregister umem.
+ *
+ * @param[in] pumem
+ *   Pointer to umem.
+ *
+ * @return
+ *   0 on successful release, negative number otherwise
+ */
+int
+mlx5_os_umem_dereg(void *pumem)
+{
+	struct mlx5_devx_umem *umem;
+	int err = 0;
+
+	if (!pumem)
+		return err;
+	umem = pumem;
+	if (umem->umem_hdl)
+		err = mlx5_glue->devx_umem_dereg(umem->umem_hdl);
+	mlx5_free(umem);
+	return err;
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index f47351ec41..decb5acd45 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -142,4 +142,6 @@ mlx5_os_get_umem_id(void *umem)
 
 void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
+void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
+int mlx5_os_umem_dereg(void *pumem);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 27/32] net/mlx5: update MR prototypes for DevX

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Currently MR operations are Verbs based. This commit updates MR
operations prototypes such that DevX MR operations callbacks can be used
as well.  Rename 'struct mlx5_verbs_ops' as 'struct mlx5_mr_ops' and
move it to shared file mlx5.h.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/linux/mlx5_os.c    | 4 ++--
 drivers/net/mlx5/linux/mlx5_verbs.c | 2 +-
 drivers/net/mlx5/linux/mlx5_verbs.h | 8 +-------
 drivers/net/mlx5/mlx5.h             | 6 ++++++
 4 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index a6c1eba640..4f68c74267 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -2478,8 +2478,8 @@ void
 mlx5_os_set_reg_mr_cb(mlx5_reg_mr_t *reg_mr_cb,
 		      mlx5_dereg_mr_t *dereg_mr_cb)
 {
-	*reg_mr_cb = mlx5_verbs_ops.reg_mr;
-	*dereg_mr_cb = mlx5_verbs_ops.dereg_mr;
+	*reg_mr_cb = mlx5_mr_verbs_ops.reg_mr;
+	*dereg_mr_cb = mlx5_mr_verbs_ops.dereg_mr;
 }
 
 /**
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c
index 9161fa3b7d..b52ae2e6c1 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.c
+++ b/drivers/net/mlx5/linux/mlx5_verbs.c
@@ -62,7 +62,7 @@ mlx5_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
 }
 
 /* verbs operations. */
-const struct mlx5_verbs_ops mlx5_verbs_ops = {
+const struct mlx5_mr_ops mlx5_mr_verbs_ops = {
 	.reg_mr = mlx5_reg_mr,
 	.dereg_mr = mlx5_dereg_mr,
 };
diff --git a/drivers/net/mlx5/linux/mlx5_verbs.h b/drivers/net/mlx5/linux/mlx5_verbs.h
index 0670f6c47e..76a79bf4f4 100644
--- a/drivers/net/mlx5/linux/mlx5_verbs.h
+++ b/drivers/net/mlx5/linux/mlx5_verbs.h
@@ -7,16 +7,10 @@
 
 #include "mlx5.h"
 
-struct mlx5_verbs_ops {
-	mlx5_reg_mr_t reg_mr;
-	mlx5_dereg_mr_t dereg_mr;
-};
-
 int mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx);
 void mlx5_txq_ibv_obj_release(struct mlx5_txq_obj *txq_obj);
 
 /* Verbs ops struct */
-extern const struct mlx5_verbs_ops mlx5_verbs_ops;
+extern const struct mlx5_mr_ops mlx5_mr_verbs_ops;
 extern struct mlx5_obj_ops ibv_obj_ops;
-
 #endif /* RTE_PMD_MLX5_VERBS_H_ */
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index 2186677810..a9db597f2b 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -917,6 +917,12 @@ struct mlx5_obj_ops {
 
 #define MLX5_RSS_HASH_FIELDS_LEN RTE_DIM(mlx5_rss_hash_fields)
 
+/* MR operations structure. */
+struct mlx5_mr_ops {
+	mlx5_reg_mr_t reg_mr;
+	mlx5_dereg_mr_t dereg_mr;
+};
+
 struct mlx5_priv {
 	struct rte_eth_dev_data *dev_data;  /* Pointer to device data. */
 	struct mlx5_dev_ctx_shared *sh; /* Shared device context. */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 28/32] common/mlx5/windows: add OS reg/dereg MR

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commits implements Windows API for MR registration and
deregistration. It is based on DevX.  Is support the relaxed ordering
flow in Windows by checking the capabilities and machine type.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_common_mr.h            |  1 +
 drivers/common/mlx5/rte_common_mlx5_exports.def |  2 +
 drivers/common/mlx5/windows/mlx5_common_os.c    | 78 +++++++++++++++++++++++++
 drivers/common/mlx5/windows/mlx5_common_os.h    |  7 +++
 4 files changed, 88 insertions(+)

diff --git a/drivers/common/mlx5/mlx5_common_mr.h b/drivers/common/mlx5/mlx5_common_mr.h
index da0a0f0c79..5cc3f097c2 100644
--- a/drivers/common/mlx5/mlx5_common_mr.h
+++ b/drivers/common/mlx5/mlx5_common_mr.h
@@ -28,6 +28,7 @@ struct mlx5_pmd_mr {
 	void		     *addr;
 	size_t		     len;
 	void		     *obj;  /* verbs mr object or devx umem object. */
+	struct mlx5_devx_obj *mkey; /* devx mkey object. */
 };
 
 /**
diff --git a/drivers/common/mlx5/rte_common_mlx5_exports.def b/drivers/common/mlx5/rte_common_mlx5_exports.def
index 1eabe29a58..65ae47ae6a 100644
--- a/drivers/common/mlx5/rte_common_mlx5_exports.def
+++ b/drivers/common/mlx5/rte_common_mlx5_exports.def
@@ -63,5 +63,7 @@ EXPORTS
 	mlx5_free
 	mlx5_os_alloc_pd
 	mlx5_os_dealloc_pd
+	mlx5_os_dereg_mr
+	mlx5_os_reg_mr
 	mlx5_os_umem_reg
 	mlx5_os_umem_dereg
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c b/drivers/common/mlx5/windows/mlx5_common_os.c
index cbb09906bd..84ed561dc0 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.c
+++ b/drivers/common/mlx5/windows/mlx5_common_os.c
@@ -14,6 +14,8 @@
 #include "mlx5_devx_cmds.h"
 #include "mlx5_common_utils.h"
 #include <mlx5_common.h>
+#include "mlx5_common_os.h"
+#include "mlx5_malloc.h"
 
 /**
  * Initialization routine for run-time dependency on external lib
@@ -130,3 +132,79 @@ mlx5_os_umem_dereg(void *pumem)
 	mlx5_free(umem);
 	return err;
 }
+
+/**
+ * Register mr. Given protection doamin pointer, pointer to addr and length
+ * register the memory region.
+ *
+ * @param[in] pd
+ *   Pointer to protection domain context (type mlx5_pd).
+ * @param[in] addr
+ *   Pointer to memory start address (type devx_device_ctx).
+ * @param[in] length
+ *   Lengtoh of the memory to register.
+ * @param[out] pmd_mr
+ *   pmd_mr struct set with lkey, address, length, pointer to mr object, mkey
+ *
+ * @return
+ *   0 on successful registration, -1 otherwise
+ */
+int
+mlx5_os_reg_mr(void *pd,
+	       void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr)
+{
+	struct mlx5_devx_mkey_attr mkey_attr;
+	struct mlx5_pd *mlx5_pd = (struct mlx5_pd *)pd;
+	struct mlx5_hca_attr attr;
+
+	if (!pd || !addr) {
+		rte_errno = EINVAL;
+		return -1;
+	}
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+	if (mlx5_devx_cmd_query_hca_attr(mlx5_pd->devx_ctx, &attr))
+		return -1;
+	pmd_mr->addr = addr;
+	pmd_mr->len = length;
+	pmd_mr->obj = mlx5_os_umem_reg(mlx5_pd->devx_ctx, pmd_mr->addr,
+				       pmd_mr->len, IBV_ACCESS_LOCAL_WRITE);
+	if (!pmd_mr->obj)
+		return -1;
+	mkey_attr.addr = (uintptr_t)addr;
+	mkey_attr.size = length;
+	mkey_attr.umem_id = ((struct mlx5_devx_umem *)(pmd_mr->obj))->umem_id;
+	mkey_attr.pd = mlx5_pd->pdn;
+	mkey_attr.log_entity_size = 0;
+	mkey_attr.pg_access = 0;
+	mkey_attr.klm_array = NULL;
+	mkey_attr.klm_num = 0;
+	mkey_attr.relaxed_ordering_read = 0;
+	mkey_attr.relaxed_ordering_write = 0;
+	if (!haswell_broadwell_cpu) {
+		mkey_attr.relaxed_ordering_write = attr.relaxed_ordering_write;
+		mkey_attr.relaxed_ordering_read = attr.relaxed_ordering_read;
+	}
+	pmd_mr->mkey = mlx5_devx_cmd_mkey_create(mlx5_pd->devx_ctx, &mkey_attr);
+	if (!pmd_mr->mkey) {
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+		return -1;
+	}
+	pmd_mr->lkey = pmd_mr->mkey->id;
+	return 0;
+}
+
+/**
+ * De-register mr.
+ *
+ * @param[in] pmd_mr
+ *  Pointer to PMD mr object
+ */
+void
+mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr)
+{
+	if (pmd_mr && pmd_mr->mkey)
+		claim_zero(mlx5_glue->devx_obj_destroy(pmd_mr->mkey->obj));
+	if (pmd_mr && pmd_mr->obj)
+		claim_zero(mlx5_os_umem_dereg(pmd_mr->obj));
+	memset(pmd_mr, 0, sizeof(*pmd_mr));
+}
diff --git a/drivers/common/mlx5/windows/mlx5_common_os.h b/drivers/common/mlx5/windows/mlx5_common_os.h
index decb5acd45..ba166412cc 100644
--- a/drivers/common/mlx5/windows/mlx5_common_os.h
+++ b/drivers/common/mlx5/windows/mlx5_common_os.h
@@ -7,9 +7,13 @@
 
 #include <stdio.h>
 
+#include <rte_errno.h>
+
 #include "mlx5_autoconf.h"
 #include "mlx5_glue.h"
 #include "mlx5_malloc.h"
+#include "mlx5_common_mr.h"
+#include "mlx5_win_ext.h"
 
 /**
  * This API allocates aligned or non-aligned memory.  The free can be on either
@@ -144,4 +148,7 @@ void *mlx5_os_alloc_pd(void *ctx);
 int mlx5_os_dealloc_pd(void *pd);
 void *mlx5_os_umem_reg(void *ctx, void *addr, size_t size, uint32_t access);
 int mlx5_os_umem_dereg(void *pumem);
+int mlx5_os_reg_mr(void *pd,
+		   void *addr, size_t length, struct mlx5_pmd_mr *pmd_mr);
+void mlx5_os_dereg_mr(struct mlx5_pmd_mr *pmd_mr);
 #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 29/32] net/mlx5/windows: implement device attribute getter

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

This commit is the Windows implementation of mlx5_os_get_dev_attr() API.
It follows the commit in [1]. A new file named mlx5_os.c is added under
windows directory as its Linux counterpart file: linux/mlx5_os.c.

[1].
commit e85f623e13ea ("net/mlx5: remove attributes dependency on Verbs")

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/common/mlx5/mlx5_prm.h       | 43 +++++++++++++++++++++
 drivers/net/mlx5/windows/meson.build |  8 ++++
 drivers/net/mlx5/windows/mlx5_os.c   | 75 ++++++++++++++++++++++++++++++++++++
 3 files changed, 126 insertions(+)
 create mode 100644 drivers/net/mlx5/windows/meson.build
 create mode 100644 drivers/net/mlx5/windows/mlx5_os.c

diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 27a4e4f0f1..d62d2558eb 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -2217,6 +2217,49 @@ struct mlx5_ifc_cqc_bits {
 	u8 dbr_addr[0x40];
 };
 
+struct mlx5_ifc_health_buffer_bits {
+	u8         reserved_0[0x100];
+	u8         assert_existptr[0x20];
+	u8         assert_callra[0x20];
+	u8         reserved_1[0x40];
+	u8         fw_version[0x20];
+	u8         hw_id[0x20];
+	u8         reserved_2[0x20];
+	u8         irisc_index[0x8];
+	u8         synd[0x8];
+	u8         ext_synd[0x10];
+};
+
+struct mlx5_ifc_initial_seg_bits {
+	u8         fw_rev_minor[0x10];
+	u8         fw_rev_major[0x10];
+	u8         cmd_interface_rev[0x10];
+	u8         fw_rev_subminor[0x10];
+	u8         reserved_0[0x40];
+	u8         cmdq_phy_addr_63_32[0x20];
+	u8         cmdq_phy_addr_31_12[0x14];
+	u8         reserved_1[0x2];
+	u8         nic_interface[0x2];
+	u8         log_cmdq_size[0x4];
+	u8         log_cmdq_stride[0x4];
+	u8         command_doorbell_vector[0x20];
+	u8         reserved_2[0xf00];
+	u8         initializing[0x1];
+	u8         nic_interface_supported[0x7];
+	u8         reserved_4[0x18];
+	struct mlx5_ifc_health_buffer_bits health_buffer;
+	u8         no_dram_nic_offset[0x20];
+	u8         reserved_5[0x6de0];
+	u8         internal_timer_h[0x20];
+	u8         internal_timer_l[0x20];
+	u8         reserved_6[0x20];
+	u8         reserved_7[0x1f];
+	u8         clear_int[0x1];
+	u8         health_syndrome[0x8];
+	u8         health_counter[0x18];
+	u8         reserved_8[0x17fc0];
+};
+
 struct mlx5_ifc_create_cq_out_bits {
 	u8 status[0x8];
 	u8 reserved_at_8[0x18];
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
new file mode 100644
index 0000000000..2ea0792a40
--- /dev/null
+++ b/drivers/net/mlx5/windows/meson.build
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2020 Mellanox Technologies, Ltd
+
+includes += include_directories('.')
+sources += files(
+	'mlx5_os.c',
+)
+
diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
new file mode 100644
index 0000000000..e0646670a3
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <stdalign.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include <rte_windows.h>
+
+#include <mlx5_glue.h>
+#include <mlx5_devx_cmds.h>
+#include <mlx5_common.h>
+
+#include "mlx5_defs.h"
+#include "mlx5.h"
+#include "mlx5_autoconf.h"
+
+/**
+ * Get mlx5 device attributes.
+ *
+ * @param ctx
+ *   Pointer to device context.
+ *
+ * @param device_attr
+ *   Pointer to mlx5 device attributes.
+ *
+ * @return
+ *   0 on success, non zero error number otherwise
+ */
+int
+mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
+{
+	struct mlx5_context *mlx5_ctx;
+	struct mlx5_hca_attr hca_attr;
+	void *pv_iseg = NULL;
+	u32 cb_iseg = 0;
+	int err = 0;
+
+	if (!ctx)
+		return -EINVAL;
+	mlx5_ctx = (struct mlx5_context *)ctx;
+	memset(device_attr, 0, sizeof(*device_attr));
+	err = mlx5_devx_cmd_query_hca_attr(mlx5_ctx, &hca_attr);
+	if (err) {
+		DRV_LOG(ERR, "Failed to get device hca_cap");
+		return err;
+	}
+	device_attr->max_cq = 1 << hca_attr.log_max_cq;
+	device_attr->max_qp = 1 << hca_attr.log_max_qp;
+	device_attr->max_qp_wr = 1 << hca_attr.log_max_qp_sz;
+	device_attr->max_cqe = 1 << hca_attr.log_max_cq_sz;
+	device_attr->max_mr = 1 << hca_attr.log_max_mrw_sz;
+	device_attr->max_pd = 1 << hca_attr.log_max_pd;
+	device_attr->max_srq = 1 << hca_attr.log_max_srq;
+	device_attr->max_srq_wr = 1 << hca_attr.log_max_srq_sz;
+	if (hca_attr.rss_ind_tbl_cap) {
+		device_attr->max_rwq_indirection_table_size =
+			1 << hca_attr.rss_ind_tbl_cap;
+	}
+	pv_iseg = mlx5_glue->query_hca_iseg(mlx5_ctx, &cb_iseg);
+	if (pv_iseg == NULL) {
+		DRV_LOG(ERR, "Failed to get device hca_iseg");
+		return errno;
+	}
+	if (!err) {
+		snprintf(device_attr->fw_ver, 64, "%x.%x.%04x",
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_major),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_minor),
+			MLX5_GET(initial_seg, pv_iseg, fw_rev_subminor));
+	}
+	return err;
+}
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 30/32] net/mlx5/windows: add mlx5_os.c stubs

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

mlx5_os_set_nonblock_channel_fd
mlx5_os_dev_shared_handler_install
mlx5_os_dev_shared_handler_uninstall
mlx5_os_read_dev_stat
mlx5_os_mac_addr_flush
mlx5_os_mac_addr_remove
mlx5_os_vf_mac_addr_modify
mlx5_os_set_promisc
mlx5_os_set_allmulti

Set struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops with NULL
pointers.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 187 +++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index e0646670a3..a8bef70fa2 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -13,10 +13,18 @@
 #include <mlx5_glue.h>
 #include <mlx5_devx_cmds.h>
 #include <mlx5_common.h>
+#include <mlx5_common_mp.h>
+#include <mlx5_common_mr.h>
+#include <mlx5_malloc.h>
 
 #include "mlx5_defs.h"
 #include "mlx5.h"
+#include "mlx5_common_os.h"
+#include "mlx5_utils.h"
+#include "mlx5_rxtx.h"
 #include "mlx5_autoconf.h"
+#include "mlx5_mr.h"
+#include "mlx5_flow.h"
 
 /**
  * Get mlx5 device attributes.
@@ -73,3 +81,182 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr)
 	}
 	return err;
 }
+
+/**
+ * Set the completion channel file descriptor interrupt as non-blocking.
+ * Currently it has no support under Windows.
+ *
+ * @param[in] rxq_obj
+ *   Pointer to RQ channel object, which includes the channel fd
+ *
+ * @param[out] fd
+ *   The file descriptor (representing the intetrrupt) used in this channel.
+ *
+ * @return
+ *   0 on successfully setting the fd to non-blocking, non-zero otherwise.
+ */
+int
+mlx5_os_set_nonblock_channel_fd(int fd)
+{
+	(void)fd;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+	return -ENOTSUP;
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param sh
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+}
+
+/**
+ * This function should share events between multiple ports of single IB
+ * device.  Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to mlx5_dev_ctx_shared object.
+ */
+void
+mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
+{
+	(void)sh;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+}
+
+/**
+ * Read statistics by a named counter.
+ *
+ * @param[in] priv
+ *   Pointer to the private device data structure.
+ * @param[in] ctr_name
+ *   Pointer to the name of the statistic counter to read
+ * @param[out] stat
+ *   Pointer to read statistic value.
+ * @return
+ *   0 on success and stat is valud, 1 if failed to read the value
+ *   rte_errno is set.
+ *
+ */
+int
+mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name,
+		      uint64_t *stat)
+{
+	RTE_SET_USED(priv);
+	RTE_SET_USED(ctr_name);
+	RTE_SET_USED(stat);
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+	return -ENOTSUP;
+}
+
+/**
+ * Flush device MAC addresses
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ */
+void
+mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
+{
+	(void)dev;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+}
+
+/**
+ * Remove a MAC address from device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param index
+ *   MAC address index.
+ */
+void
+mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
+{
+	(void)dev;
+	(void)(index);
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+}
+
+/**
+ * Modify a VF MAC address
+ * Currently it has no support under Windows.
+ *
+ * @param priv
+ *   Pointer to device private data.
+ * @param mac_addr
+ *   MAC address to modify into.
+ * @param iface_idx
+ *   Net device interface index
+ * @param vf_index
+ *   VF index
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_vf_mac_addr_modify(struct mlx5_priv *priv,
+			   unsigned int iface_idx,
+			   struct rte_ether_addr *mac_addr,
+			   int vf_index)
+{
+	(void)priv;
+	(void)iface_idx;
+	(void)mac_addr;
+	(void)vf_index;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+	return -ENOTSUP;
+}
+
+/**
+ * Set device promiscuous mode
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - promiscuous is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_promisc(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+	return -ENOTSUP;
+}
+
+/**
+ * Set device allmulti mode
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param enable
+ *   0 - all multicase is disabled, otherwise - enabled
+ *
+ * @return
+ *   0 on success, a negative error value otherwise
+ */
+int
+mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
+{
+	(void)dev;
+	(void)enable;
+	DRV_LOG(WARNING, "%s: is not supported", __func__);
+	return -ENOTSUP;
+}
+
+const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 31/32] net/mlx5/windows: implement mlx5 mac addr add

[Tal Shnaiderman]

Get the list of MAC addresses and verify if the input mac parameter
already exists. If not - return -ENOTSUP (as Windows does not support
adding new MAC addresses). If the MAC address exists (EEXIST) return 0
(the equivalent of Linux implementation of this API).

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index a8bef70fa2..0e5433e05b 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -188,6 +188,42 @@ mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
 	DRV_LOG(WARNING, "%s: is not supported", __func__);
 }
 
+/**
+ * Adds a MAC address to the device
+ * Currently it has no support under Windows.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ * @param mac_addr
+ *   MAC address to register.
+ * @param index
+ *   MAC address index.
+ *
+ * @return
+ *   0 on success, a negative errno value otherwise
+ */
+int
+mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
+		     uint32_t index)
+{
+	(void)index;
+	struct rte_ether_addr lmac;
+
+	if (mlx5_get_mac(dev, &lmac.addr_bytes)) {
+		DRV_LOG(ERR,
+			"port %u cannot get MAC address, is mlx5_en"
+			" loaded? (errno: %s)",
+			dev->data->port_id, strerror(rte_errno));
+		return rte_errno;
+	}
+	if (!rte_is_same_ether_addr(&lmac, mac)) {
+		DRV_LOG(ERR,
+			"adding new mac address to device is unsupported");
+		return -ENOTSUP;
+	}
+	return 0;
+}
+
 /**
  * Modify a VF MAC address
  * Currently it has no support under Windows.
-- 
2.16.1.windows.4

[dpdk-dev] [PATCH v5 32/32] net/mlx5: refactor eth dev ops for Windows

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

There are two types of eth_dev_ops used under Windows: primary and
isolate mode. Their function calls initialization is added to the OS
specific file mlx5_os.c. Secondary process eth_dev_ops is nullified.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
 drivers/net/mlx5/windows/mlx5_os.c | 108 +++++++++++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)

diff --git a/drivers/net/mlx5/windows/mlx5_os.c b/drivers/net/mlx5/windows/mlx5_os.c
index 0e5433e05b..7bcce9dae0 100644
--- a/drivers/net/mlx5/windows/mlx5_os.c
+++ b/drivers/net/mlx5/windows/mlx5_os.c
@@ -296,3 +296,111 @@ mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
 }
 
 const struct mlx5_flow_driver_ops mlx5_flow_verbs_drv_ops = {0};
+
+const struct eth_dev_ops mlx5_os_dev_ops = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.read_clock = mlx5_read_clock,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.reta_update = mlx5_dev_rss_reta_update,
+	.reta_query = mlx5_dev_rss_reta_query,
+	.rss_hash_update = mlx5_rss_hash_update,
+	.rss_hash_conf_get = mlx5_rss_hash_conf_get,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.udp_tunnel_port_add  = mlx5_udp_tunnel_port_add,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
+
+/* Available operations from secondary process. */
+const struct eth_dev_ops mlx5_os_dev_sec_ops = {0};
+
+/* Available operations in flow isolated mode. */
+const struct eth_dev_ops mlx5_os_dev_ops_isolate = {
+	.dev_configure = mlx5_dev_configure,
+	.dev_start = mlx5_dev_start,
+	.dev_stop = mlx5_dev_stop,
+	.dev_close = mlx5_dev_close,
+	.mtu_set = mlx5_dev_set_mtu,
+	.link_update = mlx5_link_update,
+	.stats_get = mlx5_stats_get,
+	.stats_reset = mlx5_stats_reset,
+	.dev_infos_get = mlx5_dev_infos_get,
+	.dev_set_link_down = mlx5_set_link_down,
+	.dev_set_link_up = mlx5_set_link_up,
+	.promiscuous_enable = mlx5_promiscuous_enable,
+	.promiscuous_disable = mlx5_promiscuous_disable,
+	.allmulticast_enable = mlx5_allmulticast_enable,
+	.allmulticast_disable = mlx5_allmulticast_disable,
+	.xstats_get = mlx5_xstats_get,
+	.xstats_reset = mlx5_xstats_reset,
+	.xstats_get_names = mlx5_xstats_get_names,
+	.fw_version_get = mlx5_fw_version_get,
+	.dev_supported_ptypes_get = mlx5_dev_supported_ptypes_get,
+	.vlan_filter_set = mlx5_vlan_filter_set,
+	.rx_queue_setup = mlx5_rx_queue_setup,
+	.rx_hairpin_queue_setup = mlx5_rx_hairpin_queue_setup,
+	.tx_queue_setup = mlx5_tx_queue_setup,
+	.tx_hairpin_queue_setup = mlx5_tx_hairpin_queue_setup,
+	.rx_queue_release = mlx5_rx_queue_release,
+	.tx_queue_release = mlx5_tx_queue_release,
+	.flow_ctrl_get = mlx5_dev_get_flow_ctrl,
+	.flow_ctrl_set = mlx5_dev_set_flow_ctrl,
+	.mac_addr_remove = mlx5_mac_addr_remove,
+	.mac_addr_add = mlx5_mac_addr_add,
+	.mac_addr_set = mlx5_mac_addr_set,
+	.set_mc_addr_list = mlx5_set_mc_addr_list,
+	.vlan_strip_queue_set = mlx5_vlan_strip_queue_set,
+	.vlan_offload_set = mlx5_vlan_offload_set,
+	.filter_ctrl = mlx5_dev_filter_ctrl,
+	.rxq_info_get = mlx5_rxq_info_get,
+	.txq_info_get = mlx5_txq_info_get,
+	.rx_burst_mode_get = mlx5_rx_burst_mode_get,
+	.tx_burst_mode_get = mlx5_tx_burst_mode_get,
+	.rx_queue_intr_enable = mlx5_rx_intr_enable,
+	.rx_queue_intr_disable = mlx5_rx_intr_disable,
+	.is_removed = mlx5_is_removed,
+	.get_module_info = mlx5_get_module_info,
+	.get_module_eeprom = mlx5_get_module_eeprom,
+	.hairpin_cap_get = mlx5_hairpin_cap_get,
+	.mtr_ops_get = mlx5_flow_meter_ops_get,
+};
-- 
2.16.1.windows.4

Re: [dpdk-dev] [PATCH v5 00/32] mlx5 Windows support - part #5

[Raslan Darawsheh]

Hi,
> -----Original Message-----
> From: Tal Shnaiderman <talshn@nvidia.com>
> Sent: Monday, December 28, 2020 11:54 AM
> To: dev@dpdk.org
> Cc: NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; Matan Azrad
> <matan@nvidia.com>; Raslan Darawsheh <rasland@nvidia.com>; Ophir
> Munk <ophirmu@nvidia.com>
> Subject: [PATCH v5 00/32] mlx5 Windows support - part #5
> 
> This patch series is part of preparing mlx5 PMD to compile and run under
> Windows OS. Part #5
> 
> v1: initial version
> v2: Fixes after CR and rebase, also patchset was split and part of the commits
> sent in v1 will be sent later on.
> v3: Remove common compilation as it depends on upcoming patches
> [DmitryK].
> v4: Fix syntax error in warning message.
> v5: Fix style issues.
> 
> Ophir Munk (20):
>   net/mlx5/linux: extend device attributes getter
>   net/mlx5: remove Linux files from Windows compilation
>   net/mlx5: fix freeing packet pacing
>   net/mlx5: replace Linux sleep with rte sleep
>   net/mlx5: define mprq functions as static inline
>   net/mlx5: do not define static_assert in Windows
>   net/mlx5: move static_assert calls to global scope
>   net/mlx5: wrap glue alloc/dealloc PD with OS calls
>   net/mlx5: fix adding destroy flow action wrapper
>   common/mlx5: add definition HAVE_INFINIBAND_VERBS_H
>   common/mlx5/linux: handle memory allocations with alignment
>   common/mlx5/windows: handle memory allocations with alignment
>   mlx5/windows: add initialization routine for external lib
>   mlx5/windows: generate file mlx5_autoconf.h
>   common/mlx5/windows: add getter functions
>   net/mlx5: update MR prototypes for DevX
>   common/mlx5/windows: add OS reg/dereg MR
>   net/mlx5/windows: implement device attribute getter
>   net/mlx5/windows: add mlx5_os.c stubs
>   net/mlx5: refactor eth dev ops for Windows
> 
> Tal Shnaiderman (12):
>   net/mlx5: fix folding constant array error
>   net/mlx5: wrap glue reg/dereg UMEM with OS calls
>   common/mlx5/linux: wrap event channel APIs with OS calls
>   common/mlx5: add Windows exports file
>   common/mlx5: extend DevX query hca attributes command
>   common/mlx5: add DevX alloc PD command
>   common/mlx5/windows: add glue functions APIs
>   mlx5/windows: add mlx5 meson file
>   common/mlx5/windows: extend PRM match_param_bits struct
>   common/mlx5/windows: add OS alloc/dealloc pd
>   common/mlx5/windows: add OS umem reg/dereg API
>   net/mlx5/windows: implement mlx5 mac addr add
> 
>  drivers/common/mlx5/linux/meson.build           |   6 +-
>  drivers/common/mlx5/linux/mlx5_common_os.h      |  83 +++++
>  drivers/common/mlx5/mlx5_common.c               |   4 +-
>  drivers/common/mlx5/mlx5_common_mr.h            |   1 +
>  drivers/common/mlx5/mlx5_devx_cmds.c            |  46 +++
>  drivers/common/mlx5/mlx5_devx_cmds.h            |  12 +-
>  drivers/common/mlx5/mlx5_malloc.c               |  14 +-
>  drivers/common/mlx5/mlx5_prm.h                  |  87 +++++
>  drivers/common/mlx5/rte_common_mlx5_exports.def |  69 ++++
>  drivers/common/mlx5/version.map                 |   1 +
>  drivers/common/mlx5/windows/meson.build         |  45 +++
>  drivers/common/mlx5/windows/mlx5_common_os.c    | 210
> ++++++++++++
>  drivers/common/mlx5/windows/mlx5_common_os.h    | 154 +++++++++
>  drivers/common/mlx5/windows/mlx5_glue.c         | 307
> ++++++++++++++++++
>  drivers/common/mlx5/windows/mlx5_glue.h         |  58 ++++
>  drivers/common/mlx5/windows/mlx5_win_defs.h     |  25 ++
>  drivers/common/mlx5/windows/mlx5_win_ext.h      |  40 +++
>  drivers/net/mlx5/linux/mlx5_os.c                |   9 +-
>  drivers/net/mlx5/linux/mlx5_verbs.c             |   2 +-
>  drivers/net/mlx5/linux/mlx5_verbs.h             |   8 +-
>  drivers/net/mlx5/meson.build                    |  16 +-
>  drivers/net/mlx5/mlx5.c                         |  20 +-
>  drivers/net/mlx5/mlx5.h                         |  11 +
>  drivers/net/mlx5/mlx5_defs.h                    |   7 +-
>  drivers/net/mlx5/mlx5_devx.c                    |  24 +-
>  drivers/net/mlx5/mlx5_flow.c                    |  17 +-
>  drivers/net/mlx5/mlx5_flow_dv.c                 |   6 +-
>  drivers/net/mlx5/mlx5_rxq.c                     |  71 -----
>  drivers/net/mlx5/mlx5_rxtx.c                    |  98 +++---
>  drivers/net/mlx5/mlx5_rxtx.h                    |  73 ++++-
>  drivers/net/mlx5/mlx5_trigger.c                 |   3 +-
>  drivers/net/mlx5/mlx5_txpp.c                    |  34 +-
>  drivers/net/mlx5/windows/meson.build            |   8 +
>  drivers/net/mlx5/windows/mlx5_os.c              | 406
> ++++++++++++++++++++++++
>  drivers/vdpa/mlx5/mlx5_vdpa_event.c             |   7 +-
>  35 files changed, 1778 insertions(+), 204 deletions(-)
>  create mode 100644 drivers/common/mlx5/rte_common_mlx5_exports.def
>  create mode 100644 drivers/common/mlx5/windows/meson.build
>  create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.c
>  create mode 100644 drivers/common/mlx5/windows/mlx5_common_os.h
>  create mode 100644 drivers/common/mlx5/windows/mlx5_glue.c
>  create mode 100644 drivers/common/mlx5/windows/mlx5_glue.h
>  create mode 100644 drivers/common/mlx5/windows/mlx5_win_defs.h
>  create mode 100644 drivers/common/mlx5/windows/mlx5_win_ext.h
>  create mode 100644 drivers/net/mlx5/windows/meson.build
>  create mode 100644 drivers/net/mlx5/windows/mlx5_os.c
> 
> --
> 2.16.1.windows.4

Series applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

[dpdk-dev] [PATCH v3] drivers/common: enable Windows common mlx5 compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Prior to this commit meson did not progress beyond directory
drivers/common for Windows compilations.  This commit enables drivers
compilation under this directory.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
v3: change commit place with dependencies.
Depends-on: patch-82392 ("net/mlx5: fix flow sample definitions")
Depends-on: patch-85737 ("mlx5: split multi-threaded flows per OS")
Depends-on: patch-85621 ("build: i40e PMD on Windows")
---
 drivers/common/meson.build      | 8 +++-----
 drivers/common/mlx5/meson.build | 4 ++--
 2 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/drivers/common/meson.build b/drivers/common/meson.build
index d78882f05e..be97424548 100644
--- a/drivers/common/meson.build
+++ b/drivers/common/meson.build
@@ -1,10 +1,8 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Cavium, Inc
 
-if is_windows
-	subdir_done()
+if not is_windows
+	std_deps = ['eal']
+	drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
 endif
-
-std_deps = ['eal']
-drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
 config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON'
diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build
index 3dacc6f683..3047b455cf 100644
--- a/drivers/common/mlx5/meson.build
+++ b/drivers/common/mlx5/meson.build
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2019 Mellanox Technologies, Ltd
 
-if not is_linux
+if not (is_linux or (is_windows and is_ms_linker))
 	build = false
-	reason = 'only supported on Linux'
+	reason = 'only supported on Linux and Windows build with clang'
 	subdir_done()
 endif
 
-- 
2.16.1.windows.4

Re: [dpdk-dev] [PATCH v3] drivers/common: enable Windows common mlx5 compilation

[Thomas Monjalon]

03/01/2021 09:00, Tal Shnaiderman:
> From: Ophir Munk <ophirmu@nvidia.com>
> 
> Prior to this commit meson did not progress beyond directory
> drivers/common for Windows compilations.  This commit enables drivers
> compilation under this directory.
> 
> Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> Acked-by: Matan Azrad <matan@nvidia.com>
> ---
> --- a/drivers/common/meson.build
> +++ b/drivers/common/meson.build
> @@ -1,10 +1,8 @@
>  # SPDX-License-Identifier: BSD-3-Clause
>  # Copyright(c) 2018 Cavium, Inc
>  
> -if is_windows
> -	subdir_done()
> +if not is_windows
> +	std_deps = ['eal']
> +	drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']
>  endif
> -
> -std_deps = ['eal']
> -drivers = ['cpt', 'dpaax', 'iavf', 'mvep', 'octeontx', 'octeontx2', 'sfc_efx']

The change in drivers/common/meson.build is useless.
common/mlx5 is called directly from drivers/meson.build

And when one of these drivers will support Windows,
the Windows check will have to move in drivers file.

[dpdk-dev] [PATCH v4] drivers/common: enable Windows common mlx5 compilation

[Tal Shnaiderman]

From: Ophir Munk <ophirmu@nvidia.com>

Prior to this commit meson did not progress beyond directory
drivers/common for Windows compilations.  This commit enables drivers
compilation under this directory.

Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
v3: change commit place with dependencies.
v4: remove unneeded change in common/meson.build [Thomas]
Depends-on: patch-86125 ("mlx5: fix __mlx5_bit_off macro warning for Windows")
---
 drivers/common/mlx5/meson.build | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build
index 3dacc6f683..3047b455cf 100644
--- a/drivers/common/mlx5/meson.build
+++ b/drivers/common/mlx5/meson.build
@@ -1,9 +1,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright 2019 Mellanox Technologies, Ltd
 
-if not is_linux
+if not (is_linux or (is_windows and is_ms_linker))
 	build = false
-	reason = 'only supported on Linux'
+	reason = 'only supported on Linux and Windows build with clang'
 	subdir_done()
 endif
 
-- 
2.16.1.windows.4

Re: [dpdk-dev] [PATCH v4] drivers/common: enable Windows common mlx5 compilation

[Thomas Monjalon]

12/01/2021 13:58, Tal Shnaiderman:
> From: Ophir Munk <ophirmu@nvidia.com>
> 
> Prior to this commit meson did not progress beyond directory
> drivers/common for Windows compilations.  This commit enables drivers
> compilation under this directory.

This commit is not changing compilation of drivers/common/*
Changing the commit log to:
"Enable mlx5 PMD on Windows with clang compilation."

> Signed-off-by: Ophir Munk <ophirmu@nvidia.com>
> Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
> Acked-by: Matan Azrad <matan@nvidia.com>

Applied to next-net-mlx, thanks