From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B6A3A0093; Wed, 27 May 2020 18:14:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AA3311D654; Wed, 27 May 2020 18:14:46 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70075.outbound.protection.outlook.com [40.107.7.75]) by dpdk.org (Postfix) with ESMTP id 56C371D545 for ; Wed, 27 May 2020 18:14:43 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BQFEt6tcZXlCLp5M+yYKFVGqY675dZtONggwPyuZG7Tawf/I7a2eBp6FMsIFmG9uDLIJEZr4/8P9m06Dc95mTaiQ4PsvPCjoN6sv2NH6l3g8PQ8O52p1ssuMe2D9Jf2KTLi46R3sLWxErF+23rf2wVlOQdu7j9osNS6UruQUYLb3eSIo9vjBVcxY04q4r8Lx5Ek1TuB26yrRqal2Xa/eU3IFvQE+VJtHmFANxZw9quR72wypXy0IX6Ms/0AXa9YX+MbvinAeZZdLPYYo73K8jauaGc76/bcaDwwq42YKZVVpaIzl5cGjCXZr9simrpKCwXdUNwdSzj3p45pOkE6uQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N+vhtqhEp/fp3NLXNJTqBF+dPkEecWB3dg8N/XMw1HU=; b=BNlQJFkIlHHto14jX2/G8a1kgoCr5O3ntM+5bLLRTSdcV8B3bzFEQF+GVLMUnI2cf0FQiniS06toL6/3313kJcr1o7JZvvOf77CLznAGdF4Q5JldP7+eSTgHCvWliVtiYh6tyw+mXi2/0C0Fcykw4wDFq2seGDt4I9Vf6aEn1HPH5Cf8JnM/Pzg5wLdT5nHv5j4kW7nRjd/xXphlRRms1QM5/RvIA+KDxpScP+F+mSDEOEKZ74cJrMLTITMCNiqUKLh7peOo4+MUtFqpIdBxrFAx0Cv+BkPS9GwUCb0RbaBtCwc2SoYL8nCJu8o2DNOz29DKcLAC5psjrvhcUk+yPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N+vhtqhEp/fp3NLXNJTqBF+dPkEecWB3dg8N/XMw1HU=; b=S1hyryhNLbyU0RnwrG1UTVFcCJjZXVZ+xLvWyY/5nvUjik0XaxVwncp9bndYPYfOp/YuQxXzlJAiIJ9wMDRzefZ2ltaO67o0V9jt1WGeMbi8+4VWCjhEkaJQQknW2bUdRkV4SeL0TO4/RNGvUMT830sCMBTHscRqPCUxZrpt18c= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com; Received: from AM0PR05MB4209.eurprd05.prod.outlook.com (2603:10a6:208:61::22) by AM0PR05MB4497.eurprd05.prod.outlook.com (2603:10a6:208:67::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3021.24; Wed, 27 May 2020 16:14:40 +0000 Received: from AM0PR05MB4209.eurprd05.prod.outlook.com ([fe80::1068:89a9:41d3:b14a]) by AM0PR05MB4209.eurprd05.prod.outlook.com ([fe80::1068:89a9:41d3:b14a%3]) with mapi id 15.20.3021.029; Wed, 27 May 2020 16:14:40 +0000 From: Ophir Munk To: dev@dpdk.org, Matan Azrad Cc: Ophir Munk Date: Wed, 27 May 2020 16:14:23 +0000 Message-Id: <20200527161425.22944-1-ophirmu@mellanox.com> X-Mailer: git-send-email 2.8.4 Content-Type: text/plain X-ClientProxiedBy: AM0PR10CA0108.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:e6::25) To AM0PR05MB4209.eurprd05.prod.outlook.com (2603:10a6:208:61::22) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mellanox.com (37.142.13.130) by AM0PR10CA0108.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:208:e6::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Wed, 27 May 2020 16:14:39 +0000 X-Mailer: git-send-email 2.8.4 X-Originating-IP: [37.142.13.130] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 16b86879-f600-4d1f-f464-08d802590eaf X-MS-TrafficTypeDiagnostic: AM0PR05MB4497: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2399; X-Forefront-PRVS: 04163EF38A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oXhlYZ7ZgsEkQPpaPyMOZdAPd56wBr9iMmJMJcSce46pcy8Z2MG/uwLOTihVlsFJIj1PV87V8gB5woDmocI41GhVK6D9JdWnEl4wLD88r4DVdst350ci5jhWEjuw9r14K6GmlDktwJA2L1aHvr4WVgeH38JXW9ilqIT4P7+3JO0g9yH1mtv5ZbXHD385Zcr/AQLg3xZnj1O9VXgiljV+ohGEiUxnMrUvi1+qI+wzFttLZ5CJ1Fxkz9gmnqgEDy2HTrtbiN+r+jLWFfdH3YeoxQ7410PAbpnI8ZWsXBeOAiD9Rb1/+S8pEzGKkDh/pFla X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB4209.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(376002)(346002)(396003)(366004)(136003)(2906002)(6636002)(7696005)(8886007)(1076003)(26005)(8676002)(5660300002)(37006003)(52116002)(36756003)(186003)(16526019)(30864003)(6666004)(83380400001)(8936002)(316002)(478600001)(66946007)(66556008)(86362001)(2616005)(956004)(66476007)(6862004)(107886003)(4326008)(55016002)(559001)(579004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: OuP+DolcaCuDvqAUlbzqjpVDtzHQUmSrzXaIx+VmM5ubaNs5sZ1Vt8P4k/QS7Ut/bMFhBUcsi6S9roU+ofAlmyaAj16o+fUIpTvqOKoSJBRNqGlKW/2UA7dtBDrv2qccG5ktmGY+OhfS/Z/cWUFN/JtkNvnatfC/XDvTHu7q3qepPg5rdvUrJARdEPOVemS1X7Xe+yoOWF8Lu/5svOkl+0sMhXZ//j1zAtPHa/aV7Ew7NM9gfdAQiJRJAwSgWHUZYovAn539S4/tT1n3GeTTNieKvVUZVV5I1F5ZUj7eEaNJoEn2iQ5zKmkvPwTu/O49XgObDCaOifHrR+TytOaCNEPLP+4HUnef7HZsIb91XAM+weYxXCBEk4bJ/rfAXRtrGkZhaG2XDoRHtnSEYFkajMQizXC35wpL0iFix7TpJjCwpS/CO5xxCl20Riyhd7h1ILXSI07wh82FKUdWzRrVyMW6AylxCynJHmI9dlvF7WU= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 16b86879-f600-4d1f-f464-08d802590eaf X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 May 2020 16:14:39.9239 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: n6wHuWtrToS0Cj/njRjOoGE4pb7ol0uQMkULWj8ZMuNgQKDEA36G2TAAx5scSquZ6YP+skG7EyDS30ibVNTFFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4497 Subject: [dpdk-dev] [PATCH v1 1/3] common/mlx5: move glue files under Linux directory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The glue file mlx5_glue.c is based on Linux specifics APIs. Move it (including file mlx5_glue.h) to common/mlx5/linux directory. Signed-off-by: Ophir Munk --- drivers/common/mlx5/Makefile | 12 +- drivers/common/mlx5/linux/meson.build | 199 +++++ drivers/common/mlx5/linux/mlx5_glue.c | 1294 +++++++++++++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_glue.h | 309 ++++++++ drivers/common/mlx5/meson.build | 200 +---- drivers/common/mlx5/mlx5_glue.c | 1294 --------------------------------- drivers/common/mlx5/mlx5_glue.h | 309 -------- drivers/net/mlx5/Makefile | 1 + drivers/vdpa/mlx5/Makefile | 1 + 9 files changed, 1817 insertions(+), 1802 deletions(-) create mode 100644 drivers/common/mlx5/linux/meson.build create mode 100644 drivers/common/mlx5/linux/mlx5_glue.c create mode 100644 drivers/common/mlx5/linux/mlx5_glue.h delete mode 100644 drivers/common/mlx5/mlx5_glue.c delete mode 100644 drivers/common/mlx5/mlx5_glue.h diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile index 0d8cc1b..19b1466 100644 --- a/drivers/common/mlx5/Makefile +++ b/drivers/common/mlx5/Makefile @@ -12,7 +12,7 @@ LIB_GLUE_VERSION = 20.02.0 # Sources. ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y) ifneq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) -SRCS-y += mlx5_glue.c +SRCS-y += linux/mlx5_glue.c endif SRCS-y += mlx5_devx_cmds.c SRCS-y += mlx5_common.c @@ -29,6 +29,8 @@ CFLAGS += -O3 CFLAGS += -std=c11 -Wall -Wextra CFLAGS += -g CFLAGS += -I. +CFLAGS += -I$(SRCDIR) +CFLAGS += -I$(SRCDIR)/linux CFLAGS += -D_BSD_SOURCE CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 @@ -37,7 +39,7 @@ CFLAGS += -Wno-strict-prototypes ifeq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) CFLAGS += -DMLX5_GLUE='"$(LIB_GLUE)"' CFLAGS += -DMLX5_GLUE_VERSION='"$(LIB_GLUE_VERSION)"' -CFLAGS_mlx5_glue.o += -fPIC +CFLAGS_linux/mlx5_glue.o += -fPIC LDLIBS += -ldl else ifeq ($(CONFIG_RTE_IBVERBS_LINK_STATIC),y) LDLIBS += $(shell $(RTE_SDK)/buildtools/options-ibverbs-static.sh) @@ -346,17 +348,17 @@ GLUE_LDFLAGS := $(call linkerprefix,$(LDFLAGS)) else GLUE_LDFLAGS := $(LDFLAGS) endif -$(LIB_GLUE): mlx5_glue.o +$(LIB_GLUE): linux/mlx5_glue.o $Q $(LD) $(GLUE_LDFLAGS) $(EXTRA_LDFLAGS) \ -Wl,-h,$(LIB_GLUE) \ -shared -o $@ $< -libverbs -lmlx5 -mlx5_glue.o: mlx5_autoconf.h +linux/mlx5_glue.o: mlx5_autoconf.h endif clean_mlx5: FORCE $Q rm -f -- mlx5_autoconf.h mlx5_autoconf.h.new - $Q rm -f -- mlx5_glue.o $(LIB_GLUE_BASE)* + $Q rm -f -- linux/mlx5_glue.o $(LIB_GLUE_BASE)* clean: clean_mlx5 diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build new file mode 100644 index 0000000..a123fd6 --- /dev/null +++ b/drivers/common/mlx5/linux/meson.build @@ -0,0 +1,199 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2020 Mellanox Technologies, Ltd + +includes += include_directories('.') + +static_ibverbs = (get_option('ibverbs_link') == 'static') +dlopen_ibverbs = (get_option('ibverbs_link') == 'dlopen') +LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so' +LIB_GLUE_VERSION = '20.02.0' +LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION +if dlopen_ibverbs + dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1) + cflags += [ + '-DMLX5_GLUE="@0@"'.format(LIB_GLUE), + '-DMLX5_GLUE_VERSION="@0@"'.format(LIB_GLUE_VERSION), + ] +endif + +libnames = [ 'mlx5', 'ibverbs' ] +libs = [] +foreach libname:libnames + lib = dependency('lib' + libname, static:static_ibverbs, required:false) + if not lib.found() and not static_ibverbs + lib = cc.find_library(libname, required:false) + endif + if lib.found() + libs += lib + if not static_ibverbs and not dlopen_ibverbs + ext_deps += lib + endif + else + build = false + reason = 'missing dependency, "' + libname + '"' + subdir_done() + endif +endforeach +if static_ibverbs or dlopen_ibverbs + # Build without adding shared libs to Requires.private + ibv_cflags = run_command(pkgconf, '--cflags', 'libibverbs').stdout() + ext_deps += declare_dependency(compile_args: ibv_cflags.split()) +endif +if static_ibverbs + # Add static deps ldflags to internal apps and Libs.private + ibv_ldflags = run_command(ldflags_ibverbs_static, check:true).stdout() + ext_deps += declare_dependency(link_args:ibv_ldflags.split()) +endif + +if not dlopen_ibverbs + sources += files('mlx5_glue.c') +endif + +# To maintain the compatibility with the make build system +# mlx5_autoconf.h file is still generated. +# input array for meson member search: +# [ "MACRO to define if found", "header for the search", +# "symbol to search", "struct member to search" ] +has_member_args = [ + [ 'HAVE_IBV_MLX5_MOD_SWP', 'infiniband/mlx5dv.h', + 'struct mlx5dv_sw_parsing_caps', 'sw_parsing_offloads' ], + [ 'HAVE_IBV_DEVICE_COUNTERS_SET_V42', 'infiniband/verbs.h', + 'struct ibv_counter_set_init_attr', 'counter_set_id' ], + [ 'HAVE_IBV_DEVICE_COUNTERS_SET_V45', 'infiniband/verbs.h', + 'struct ibv_counters_init_attr', 'comp_mask' ], +] +# input array for meson symbol search: +# [ "MACRO to define if found", "header for the search", +# "symbol to search" ] +has_sym_args = [ + [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', + 'IBV_ACCESS_RELAXED_ORDERING ' ], + [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h', + 'MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX' ], + [ 'HAVE_IBV_DEVICE_TUNNEL_SUPPORT', 'infiniband/mlx5dv.h', + 'MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS' ], + [ 'HAVE_IBV_MLX5_MOD_MPW', 'infiniband/mlx5dv.h', + 'MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED' ], + [ 'HAVE_IBV_MLX5_MOD_CQE_128B_COMP', 'infiniband/mlx5dv.h', + 'MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP' ], + [ 'HAVE_IBV_MLX5_MOD_CQE_128B_PAD', 'infiniband/mlx5dv.h', + 'MLX5DV_CQ_INIT_ATTR_FLAGS_CQE_PAD' ], + [ 'HAVE_IBV_FLOW_DV_SUPPORT', 'infiniband/mlx5dv.h', + 'mlx5dv_create_flow_action_packet_reformat' ], + [ 'HAVE_IBV_DEVICE_MPLS_SUPPORT', 'infiniband/verbs.h', + 'IBV_FLOW_SPEC_MPLS' ], + [ 'HAVE_IBV_WQ_FLAGS_PCI_WRITE_END_PADDING', 'infiniband/verbs.h', + 'IBV_WQ_FLAGS_PCI_WRITE_END_PADDING' ], + [ 'HAVE_IBV_WQ_FLAG_RX_END_PADDING', 'infiniband/verbs.h', + 'IBV_WQ_FLAG_RX_END_PADDING' ], + [ 'HAVE_MLX5DV_DR_DEVX_PORT', 'infiniband/mlx5dv.h', + 'mlx5dv_query_devx_port' ], + [ 'HAVE_IBV_DEVX_OBJ', 'infiniband/mlx5dv.h', + 'mlx5dv_devx_obj_create' ], + [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h', + 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ], + [ 'HAVE_IBV_DEVX_ASYNC', 'infiniband/mlx5dv.h', + 'mlx5dv_devx_obj_query_async' ], + [ 'HAVE_IBV_DEVX_QP', 'infiniband/mlx5dv.h', + 'mlx5dv_devx_qp_query' ], + [ 'HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR', 'infiniband/mlx5dv.h', + 'mlx5dv_dr_action_create_dest_devx_tir' ], + [ 'HAVE_IBV_DEVX_EVENT', 'infiniband/mlx5dv.h', + 'mlx5dv_devx_get_event' ], + [ 'HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER', 'infiniband/mlx5dv.h', + 'mlx5dv_dr_action_create_flow_meter' ], + [ 'HAVE_MLX5DV_MMAP_GET_NC_PAGES_CMD', 'infiniband/mlx5dv.h', + 'MLX5_MMAP_GET_NC_PAGES_CMD' ], + [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h', + 'MLX5DV_DR_DOMAIN_TYPE_NIC_RX' ], + [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h', + 'MLX5DV_DR_DOMAIN_TYPE_FDB' ], + [ 'HAVE_MLX5DV_DR_VLAN', 'infiniband/mlx5dv.h', + 'mlx5dv_dr_action_create_push_vlan' ], + [ 'HAVE_IBV_VAR', 'infiniband/mlx5dv.h', 'mlx5dv_alloc_var' ], + [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h', + 'SUPPORTED_40000baseKR4_Full' ], + [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h', + 'SUPPORTED_40000baseCR4_Full' ], + [ 'HAVE_SUPPORTED_40000baseSR4_Full', 'linux/ethtool.h', + 'SUPPORTED_40000baseSR4_Full' ], + [ 'HAVE_SUPPORTED_40000baseLR4_Full', 'linux/ethtool.h', + 'SUPPORTED_40000baseLR4_Full' ], + [ 'HAVE_SUPPORTED_56000baseKR4_Full', 'linux/ethtool.h', + 'SUPPORTED_56000baseKR4_Full' ], + [ 'HAVE_SUPPORTED_56000baseCR4_Full', 'linux/ethtool.h', + 'SUPPORTED_56000baseCR4_Full' ], + [ 'HAVE_SUPPORTED_56000baseSR4_Full', 'linux/ethtool.h', + 'SUPPORTED_56000baseSR4_Full' ], + [ 'HAVE_SUPPORTED_56000baseLR4_Full', 'linux/ethtool.h', + 'SUPPORTED_56000baseLR4_Full' ], + [ 'HAVE_ETHTOOL_LINK_MODE_25G', 'linux/ethtool.h', + 'ETHTOOL_LINK_MODE_25000baseCR_Full_BIT' ], + [ 'HAVE_ETHTOOL_LINK_MODE_50G', 'linux/ethtool.h', + 'ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT' ], + [ 'HAVE_ETHTOOL_LINK_MODE_100G', 'linux/ethtool.h', + 'ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT' ], + [ 'HAVE_IFLA_NUM_VF', 'linux/if_link.h', + 'IFLA_NUM_VF' ], + [ 'HAVE_IFLA_EXT_MASK', 'linux/if_link.h', + 'IFLA_EXT_MASK' ], + [ 'HAVE_IFLA_PHYS_SWITCH_ID', 'linux/if_link.h', + 'IFLA_PHYS_SWITCH_ID' ], + [ 'HAVE_IFLA_PHYS_PORT_NAME', 'linux/if_link.h', + 'IFLA_PHYS_PORT_NAME' ], + [ 'HAVE_RDMA_NL_NLDEV', 'rdma/rdma_netlink.h', + 'RDMA_NL_NLDEV' ], + [ 'HAVE_RDMA_NLDEV_CMD_GET', 'rdma/rdma_netlink.h', + 'RDMA_NLDEV_CMD_GET' ], + [ 'HAVE_RDMA_NLDEV_CMD_PORT_GET', 'rdma/rdma_netlink.h', + 'RDMA_NLDEV_CMD_PORT_GET' ], + [ 'HAVE_RDMA_NLDEV_ATTR_DEV_INDEX', 'rdma/rdma_netlink.h', + 'RDMA_NLDEV_ATTR_DEV_INDEX' ], + [ 'HAVE_RDMA_NLDEV_ATTR_DEV_NAME', 'rdma/rdma_netlink.h', + 'RDMA_NLDEV_ATTR_DEV_NAME' ], + [ 'HAVE_RDMA_NLDEV_ATTR_PORT_INDEX', 'rdma/rdma_netlink.h', + 'RDMA_NLDEV_ATTR_PORT_INDEX' ], + [ 'HAVE_RDMA_NLDEV_ATTR_NDEV_INDEX', 'rdma/rdma_netlink.h', + 'RDMA_NLDEV_ATTR_NDEV_INDEX' ], + [ 'HAVE_MLX5_DR_FLOW_DUMP', 'infiniband/mlx5dv.h', + 'mlx5dv_dump_dr_domain'], + [ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ], +] +config = configuration_data() +foreach arg:has_sym_args + config.set(arg[0], cc.has_header_symbol(arg[1], arg[2], + dependencies: libs)) +endforeach +foreach arg:has_member_args + file_prefix = '#include <' + arg[1] + '>' + config.set(arg[0], cc.has_member(arg[2], arg[3], + prefix : file_prefix, dependencies: libs)) +endforeach +configure_file(output : 'mlx5_autoconf.h', configuration : config) + +# Build Glue Library +if dlopen_ibverbs + dlopen_name = 'mlx5_glue' + dlopen_lib_name = 'rte_pmd_@0@'.format(dlopen_name) + dlopen_so_version = LIB_GLUE_VERSION + dlopen_sources = files('mlx5_glue.c') + dlopen_install_dir = [ eal_pmd_path + '-glue' ] + dlopen_includes = [global_inc] + dlopen_includes += include_directories( + '../../../../lib/librte_eal/include/generic', + ) + shared_lib = shared_library( + dlopen_lib_name, + dlopen_sources, + include_directories: dlopen_includes, + c_args: cflags, + dependencies: libs, + link_args: [ + '-Wl,-export-dynamic', + '-Wl,-h,@0@'.format(LIB_GLUE), + ], + soversion: dlopen_so_version, + install: true, + install_dir: dlopen_install_dir, + ) +endif diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c new file mode 100644 index 0000000..f270f67 --- /dev/null +++ b/drivers/common/mlx5/linux/mlx5_glue.c @@ -0,0 +1,1294 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2018 6WIND S.A. + * Copyright 2018 Mellanox Technologies, Ltd + */ + +#include +#include +#include +#include +#include +/* + * Not needed by this file; included to work around the lack of off_t + * definition for mlx5dv.h with unpatched rdma-core versions. + */ +#include + +#include "mlx5_glue.h" + +static int +mlx5_glue_fork_init(void) +{ + return ibv_fork_init(); +} + +static struct ibv_pd * +mlx5_glue_alloc_pd(struct ibv_context *context) +{ + return ibv_alloc_pd(context); +} + +static int +mlx5_glue_dealloc_pd(struct ibv_pd *pd) +{ + return ibv_dealloc_pd(pd); +} + +static struct ibv_device ** +mlx5_glue_get_device_list(int *num_devices) +{ + return ibv_get_device_list(num_devices); +} + +static void +mlx5_glue_free_device_list(struct ibv_device **list) +{ + ibv_free_device_list(list); +} + +static struct ibv_context * +mlx5_glue_open_device(struct ibv_device *device) +{ + return ibv_open_device(device); +} + +static int +mlx5_glue_close_device(struct ibv_context *context) +{ + return ibv_close_device(context); +} + +static int +mlx5_glue_query_device(struct ibv_context *context, + struct ibv_device_attr *device_attr) +{ + return ibv_query_device(context, device_attr); +} + +static int +mlx5_glue_query_device_ex(struct ibv_context *context, + const struct ibv_query_device_ex_input *input, + struct ibv_device_attr_ex *attr) +{ + return ibv_query_device_ex(context, input, attr); +} + +static int +mlx5_glue_query_rt_values_ex(struct ibv_context *context, + struct ibv_values_ex *values) +{ + return ibv_query_rt_values_ex(context, values); +} + +static int +mlx5_glue_query_port(struct ibv_context *context, uint8_t port_num, + struct ibv_port_attr *port_attr) +{ + return ibv_query_port(context, port_num, port_attr); +} + +static struct ibv_comp_channel * +mlx5_glue_create_comp_channel(struct ibv_context *context) +{ + return ibv_create_comp_channel(context); +} + +static int +mlx5_glue_destroy_comp_channel(struct ibv_comp_channel *channel) +{ + return ibv_destroy_comp_channel(channel); +} + +static struct ibv_cq * +mlx5_glue_create_cq(struct ibv_context *context, int cqe, void *cq_context, + struct ibv_comp_channel *channel, int comp_vector) +{ + return ibv_create_cq(context, cqe, cq_context, channel, comp_vector); +} + +static int +mlx5_glue_destroy_cq(struct ibv_cq *cq) +{ + return ibv_destroy_cq(cq); +} + +static int +mlx5_glue_get_cq_event(struct ibv_comp_channel *channel, struct ibv_cq **cq, + void **cq_context) +{ + return ibv_get_cq_event(channel, cq, cq_context); +} + +static void +mlx5_glue_ack_cq_events(struct ibv_cq *cq, unsigned int nevents) +{ + ibv_ack_cq_events(cq, nevents); +} + +static struct ibv_rwq_ind_table * +mlx5_glue_create_rwq_ind_table(struct ibv_context *context, + struct ibv_rwq_ind_table_init_attr *init_attr) +{ + return ibv_create_rwq_ind_table(context, init_attr); +} + +static int +mlx5_glue_destroy_rwq_ind_table(struct ibv_rwq_ind_table *rwq_ind_table) +{ + return ibv_destroy_rwq_ind_table(rwq_ind_table); +} + +static struct ibv_wq * +mlx5_glue_create_wq(struct ibv_context *context, + struct ibv_wq_init_attr *wq_init_attr) +{ + return ibv_create_wq(context, wq_init_attr); +} + +static int +mlx5_glue_destroy_wq(struct ibv_wq *wq) +{ + return ibv_destroy_wq(wq); +} +static int +mlx5_glue_modify_wq(struct ibv_wq *wq, struct ibv_wq_attr *wq_attr) +{ + return ibv_modify_wq(wq, wq_attr); +} + +static struct ibv_flow * +mlx5_glue_create_flow(struct ibv_qp *qp, struct ibv_flow_attr *flow) +{ + return ibv_create_flow(qp, flow); +} + +static int +mlx5_glue_destroy_flow(struct ibv_flow *flow_id) +{ + return ibv_destroy_flow(flow_id); +} + +static int +mlx5_glue_destroy_flow_action(void *action) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_action_destroy(action); +#else + struct mlx5dv_flow_action_attr *attr = action; + int res = 0; + switch (attr->type) { + case MLX5DV_FLOW_ACTION_TAG: + break; + default: + res = ibv_destroy_flow_action(attr->action); + break; + } + free(action); + return res; +#endif +#else + (void)action; + return ENOTSUP; +#endif +} + +static struct ibv_qp * +mlx5_glue_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr) +{ + return ibv_create_qp(pd, qp_init_attr); +} + +static struct ibv_qp * +mlx5_glue_create_qp_ex(struct ibv_context *context, + struct ibv_qp_init_attr_ex *qp_init_attr_ex) +{ + return ibv_create_qp_ex(context, qp_init_attr_ex); +} + +static int +mlx5_glue_destroy_qp(struct ibv_qp *qp) +{ + return ibv_destroy_qp(qp); +} + +static int +mlx5_glue_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mask) +{ + return ibv_modify_qp(qp, attr, attr_mask); +} + +static struct ibv_mr * +mlx5_glue_reg_mr(struct ibv_pd *pd, void *addr, size_t length, int access) +{ + return ibv_reg_mr(pd, addr, length, access); +} + +static struct ibv_mr * +mlx5_glue_alloc_null_mr(struct ibv_pd *pd) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return ibv_alloc_null_mr(pd); +#else + (void)pd; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_dereg_mr(struct ibv_mr *mr) +{ + return ibv_dereg_mr(mr); +} + +static struct ibv_counter_set * +mlx5_glue_create_counter_set(struct ibv_context *context, + struct ibv_counter_set_init_attr *init_attr) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 + (void)context; + (void)init_attr; + return NULL; +#else + return ibv_create_counter_set(context, init_attr); +#endif +} + +static int +mlx5_glue_destroy_counter_set(struct ibv_counter_set *cs) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 + (void)cs; + return ENOTSUP; +#else + return ibv_destroy_counter_set(cs); +#endif +} + +static int +mlx5_glue_describe_counter_set(struct ibv_context *context, + uint16_t counter_set_id, + struct ibv_counter_set_description *cs_desc) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 + (void)context; + (void)counter_set_id; + (void)cs_desc; + return ENOTSUP; +#else + return ibv_describe_counter_set(context, counter_set_id, cs_desc); +#endif +} + +static int +mlx5_glue_query_counter_set(struct ibv_query_counter_set_attr *query_attr, + struct ibv_counter_set_data *cs_data) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 + (void)query_attr; + (void)cs_data; + return ENOTSUP; +#else + return ibv_query_counter_set(query_attr, cs_data); +#endif +} + +static struct ibv_counters * +mlx5_glue_create_counters(struct ibv_context *context, + struct ibv_counters_init_attr *init_attr) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 + (void)context; + (void)init_attr; + errno = ENOTSUP; + return NULL; +#else + return ibv_create_counters(context, init_attr); +#endif +} + +static int +mlx5_glue_destroy_counters(struct ibv_counters *counters) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 + (void)counters; + return ENOTSUP; +#else + return ibv_destroy_counters(counters); +#endif +} + +static int +mlx5_glue_attach_counters(struct ibv_counters *counters, + struct ibv_counter_attach_attr *attr, + struct ibv_flow *flow) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 + (void)counters; + (void)attr; + (void)flow; + return ENOTSUP; +#else + return ibv_attach_counters_point_flow(counters, attr, flow); +#endif +} + +static int +mlx5_glue_query_counters(struct ibv_counters *counters, + uint64_t *counters_value, + uint32_t ncounters, + uint32_t flags) +{ +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 + (void)counters; + (void)counters_value; + (void)ncounters; + (void)flags; + return ENOTSUP; +#else + return ibv_read_counters(counters, counters_value, ncounters, flags); +#endif +} + +static void +mlx5_glue_ack_async_event(struct ibv_async_event *event) +{ + ibv_ack_async_event(event); +} + +static int +mlx5_glue_get_async_event(struct ibv_context *context, + struct ibv_async_event *event) +{ + return ibv_get_async_event(context, event); +} + +static const char * +mlx5_glue_port_state_str(enum ibv_port_state port_state) +{ + return ibv_port_state_str(port_state); +} + +static struct ibv_cq * +mlx5_glue_cq_ex_to_cq(struct ibv_cq_ex *cq) +{ + return ibv_cq_ex_to_cq(cq); +} + +static void * +mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_action_create_dest_table(tbl); +#else + (void)tbl; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dr_create_flow_action_dest_port(void *domain, uint32_t port) +{ +#ifdef HAVE_MLX5DV_DR_DEVX_PORT + return mlx5dv_dr_action_create_dest_ib_port(domain, port); +#else +#ifdef HAVE_MLX5DV_DR_ESWITCH + return mlx5dv_dr_action_create_dest_vport(domain, port); +#else + (void)domain; + (void)port; + errno = ENOTSUP; + return NULL; +#endif +#endif +} + +static void * +mlx5_glue_dr_create_flow_action_drop(void) +{ +#ifdef HAVE_MLX5DV_DR_ESWITCH + return mlx5dv_dr_action_create_drop(); +#else + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dr_create_flow_action_push_vlan(struct mlx5dv_dr_domain *domain, + rte_be32_t vlan_tag) +{ +#ifdef HAVE_MLX5DV_DR_VLAN + return mlx5dv_dr_action_create_push_vlan(domain, vlan_tag); +#else + (void)domain; + (void)vlan_tag; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dr_create_flow_action_pop_vlan(void) +{ +#ifdef HAVE_MLX5DV_DR_VLAN + return mlx5dv_dr_action_create_pop_vlan(); +#else + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dr_create_flow_tbl(void *domain, uint32_t level) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_table_create(domain, level); +#else + (void)domain; + (void)level; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_dr_destroy_flow_tbl(void *tbl) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_table_destroy(tbl); +#else + (void)tbl; + errno = ENOTSUP; + return errno; +#endif +} + +static void * +mlx5_glue_dr_create_domain(struct ibv_context *ctx, + enum mlx5dv_dr_domain_type domain) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_domain_create(ctx, domain); +#else + (void)ctx; + (void)domain; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_dr_destroy_domain(void *domain) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_domain_destroy(domain); +#else + (void)domain; + errno = ENOTSUP; + return errno; +#endif +} + +static struct ibv_cq_ex * +mlx5_glue_dv_create_cq(struct ibv_context *context, + struct ibv_cq_init_attr_ex *cq_attr, + struct mlx5dv_cq_init_attr *mlx5_cq_attr) +{ + return mlx5dv_create_cq(context, cq_attr, mlx5_cq_attr); +} + +static struct ibv_wq * +mlx5_glue_dv_create_wq(struct ibv_context *context, + struct ibv_wq_init_attr *wq_attr, + struct mlx5dv_wq_init_attr *mlx5_wq_attr) +{ +#ifndef HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT + (void)context; + (void)wq_attr; + (void)mlx5_wq_attr; + errno = ENOTSUP; + return NULL; +#else + return mlx5dv_create_wq(context, wq_attr, mlx5_wq_attr); +#endif +} + +static int +mlx5_glue_dv_query_device(struct ibv_context *ctx, + struct mlx5dv_context *attrs_out) +{ + return mlx5dv_query_device(ctx, attrs_out); +} + +static int +mlx5_glue_dv_set_context_attr(struct ibv_context *ibv_ctx, + enum mlx5dv_set_ctx_attr_type type, void *attr) +{ + return mlx5dv_set_context_attr(ibv_ctx, type, attr); +} + +static int +mlx5_glue_dv_init_obj(struct mlx5dv_obj *obj, uint64_t obj_type) +{ + return mlx5dv_init_obj(obj, obj_type); +} + +static struct ibv_qp * +mlx5_glue_dv_create_qp(struct ibv_context *context, + struct ibv_qp_init_attr_ex *qp_init_attr_ex, + struct mlx5dv_qp_init_attr *dv_qp_init_attr) +{ +#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT + return mlx5dv_create_qp(context, qp_init_attr_ex, dv_qp_init_attr); +#else + (void)context; + (void)qp_init_attr_ex; + (void)dv_qp_init_attr; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_matcher(struct ibv_context *context, + struct mlx5dv_flow_matcher_attr *matcher_attr, + void *tbl) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + (void)context; + return mlx5dv_dr_matcher_create(tbl, matcher_attr->priority, + matcher_attr->match_criteria_enable, + matcher_attr->match_mask); +#else + (void)tbl; + return mlx5dv_create_flow_matcher(context, matcher_attr); +#endif +#else + (void)context; + (void)matcher_attr; + (void)tbl; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow(void *matcher, + void *match_value, + size_t num_actions, + void *actions[]) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_rule_create(matcher, match_value, num_actions, + (struct mlx5dv_dr_action **)actions); +#else + struct mlx5dv_flow_action_attr actions_attr[8]; + + if (num_actions > 8) + return NULL; + for (size_t i = 0; i < num_actions; i++) + actions_attr[i] = + *((struct mlx5dv_flow_action_attr *)(actions[i])); + return mlx5dv_create_flow(matcher, match_value, + num_actions, actions_attr); +#endif +#else + (void)matcher; + (void)match_value; + (void)num_actions; + (void)actions; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_action_create_flow_counter(counter_obj, offset); +#else + struct mlx5dv_flow_action_attr *action; + + (void)offset; + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX; + action->obj = counter_obj; + return action; +#endif +#else + (void)counter_obj; + (void)offset; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_action_create_dest_ibv_qp(qp); +#else + struct mlx5dv_flow_action_attr *action; + + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP; + action->obj = qp; + return action; +#endif +#else + (void)qp; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_dest_devx_tir(void *tir) +{ +#ifdef HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR + return mlx5dv_dr_action_create_dest_devx_tir(tir); +#else + (void)tir; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_modify_header + (struct ibv_context *ctx, + enum mlx5dv_flow_table_type ft_type, + void *domain, uint64_t flags, + size_t actions_sz, + uint64_t actions[]) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + (void)ctx; + (void)ft_type; + return mlx5dv_dr_action_create_modify_header(domain, flags, actions_sz, + (__be64 *)actions); +#else + struct mlx5dv_flow_action_attr *action; + + (void)domain; + (void)flags; + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; + action->action = mlx5dv_create_flow_action_modify_header + (ctx, actions_sz, actions, ft_type); + return action; +#endif +#else + (void)ctx; + (void)ft_type; + (void)domain; + (void)flags; + (void)actions_sz; + (void)actions; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_packet_reformat + (struct ibv_context *ctx, + enum mlx5dv_flow_action_packet_reformat_type reformat_type, + enum mlx5dv_flow_table_type ft_type, + struct mlx5dv_dr_domain *domain, + uint32_t flags, size_t data_sz, void *data) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + (void)ctx; + (void)ft_type; + return mlx5dv_dr_action_create_packet_reformat(domain, flags, + reformat_type, data_sz, + data); +#else + (void)domain; + (void)flags; + struct mlx5dv_flow_action_attr *action; + + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; + action->action = mlx5dv_create_flow_action_packet_reformat + (ctx, data_sz, data, reformat_type, ft_type); + return action; +#endif +#else + (void)ctx; + (void)reformat_type; + (void)ft_type; + (void)domain; + (void)flags; + (void)data_sz; + (void)data; + errno = ENOTSUP; + return NULL; +#endif +} + +static void * +mlx5_glue_dv_create_flow_action_tag(uint32_t tag) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_action_create_tag(tag); +#else + struct mlx5dv_flow_action_attr *action; + action = malloc(sizeof(*action)); + if (!action) + return NULL; + action->type = MLX5DV_FLOW_ACTION_TAG; + action->tag_value = tag; + return action; +#endif +#endif + (void)tag; + errno = ENOTSUP; + return NULL; +} + +static void * +mlx5_glue_dv_create_flow_action_meter(struct mlx5dv_dr_flow_meter_attr *attr) +{ +#if defined(HAVE_MLX5DV_DR) && defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) + return mlx5dv_dr_action_create_flow_meter(attr); +#else + (void)attr; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_dv_modify_flow_action_meter(void *action, + struct mlx5dv_dr_flow_meter_attr *attr, + uint64_t modify_bits) +{ +#if defined(HAVE_MLX5DV_DR) && defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) + return mlx5dv_dr_action_modify_flow_meter(action, attr, modify_bits); +#else + (void)action; + (void)attr; + (void)modify_bits; + errno = ENOTSUP; + return errno; +#endif +} + +static int +mlx5_glue_dv_destroy_flow(void *flow_id) +{ +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_rule_destroy(flow_id); +#else + return ibv_destroy_flow(flow_id); +#endif +} + +static int +mlx5_glue_dv_destroy_flow_matcher(void *matcher) +{ +#ifdef HAVE_IBV_FLOW_DV_SUPPORT +#ifdef HAVE_MLX5DV_DR + return mlx5dv_dr_matcher_destroy(matcher); +#else + return mlx5dv_destroy_flow_matcher(matcher); +#endif +#else + (void)matcher; + errno = ENOTSUP; + return errno; +#endif +} + +static struct ibv_context * +mlx5_glue_dv_open_device(struct ibv_device *device) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_open_device(device, + &(struct mlx5dv_context_attr){ + .flags = MLX5DV_CONTEXT_FLAGS_DEVX, + }); +#else + (void)device; + errno = ENOTSUP; + return NULL; +#endif +} + +static struct mlx5dv_devx_obj * +mlx5_glue_devx_obj_create(struct ibv_context *ctx, + const void *in, size_t inlen, + void *out, size_t outlen) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_obj_create(ctx, in, inlen, out, outlen); +#else + (void)ctx; + (void)in; + (void)inlen; + (void)out; + (void)outlen; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_devx_obj_destroy(struct mlx5dv_devx_obj *obj) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_obj_destroy(obj); +#else + (void)obj; + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_obj_query(struct mlx5dv_devx_obj *obj, + const void *in, size_t inlen, + void *out, size_t outlen) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_obj_query(obj, in, inlen, out, outlen); +#else + (void)obj; + (void)in; + (void)inlen; + (void)out; + (void)outlen; + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_obj_modify(struct mlx5dv_devx_obj *obj, + const void *in, size_t inlen, + void *out, size_t outlen) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_obj_modify(obj, in, inlen, out, outlen); +#else + (void)obj; + (void)in; + (void)inlen; + (void)out; + (void)outlen; + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_general_cmd(struct ibv_context *ctx, + const void *in, size_t inlen, + void *out, size_t outlen) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_general_cmd(ctx, in, inlen, out, outlen); +#else + (void)ctx; + (void)in; + (void)inlen; + (void)out; + (void)outlen; + return -ENOTSUP; +#endif +} + +static struct mlx5dv_devx_cmd_comp * +mlx5_glue_devx_create_cmd_comp(struct ibv_context *ctx) +{ +#ifdef HAVE_IBV_DEVX_ASYNC + return mlx5dv_devx_create_cmd_comp(ctx); +#else + (void)ctx; + errno = -ENOTSUP; + return NULL; +#endif +} + +static void +mlx5_glue_devx_destroy_cmd_comp(struct mlx5dv_devx_cmd_comp *cmd_comp) +{ +#ifdef HAVE_IBV_DEVX_ASYNC + mlx5dv_devx_destroy_cmd_comp(cmd_comp); +#else + (void)cmd_comp; + errno = -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_obj_query_async(struct mlx5dv_devx_obj *obj, const void *in, + size_t inlen, size_t outlen, uint64_t wr_id, + struct mlx5dv_devx_cmd_comp *cmd_comp) +{ +#ifdef HAVE_IBV_DEVX_ASYNC + return mlx5dv_devx_obj_query_async(obj, in, inlen, outlen, wr_id, + cmd_comp); +#else + (void)obj; + (void)in; + (void)inlen; + (void)outlen; + (void)wr_id; + (void)cmd_comp; + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_get_async_cmd_comp(struct mlx5dv_devx_cmd_comp *cmd_comp, + struct mlx5dv_devx_async_cmd_hdr *cmd_resp, + size_t cmd_resp_len) +{ +#ifdef HAVE_IBV_DEVX_ASYNC + return mlx5dv_devx_get_async_cmd_comp(cmd_comp, cmd_resp, + cmd_resp_len); +#else + (void)cmd_comp; + (void)cmd_resp; + (void)cmd_resp_len; + return -ENOTSUP; +#endif +} + +static struct mlx5dv_devx_umem * +mlx5_glue_devx_umem_reg(struct ibv_context *context, void *addr, size_t size, + uint32_t access) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_umem_reg(context, addr, size, access); +#else + (void)context; + (void)addr; + (void)size; + (void)access; + errno = -ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_devx_umem_dereg(struct mlx5dv_devx_umem *dv_devx_umem) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_umem_dereg(dv_devx_umem); +#else + (void)dv_devx_umem; + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_qp_query(struct ibv_qp *qp, + const void *in, size_t inlen, + void *out, size_t outlen) +{ +#ifdef HAVE_IBV_DEVX_QP + return mlx5dv_devx_qp_query(qp, in, inlen, out, outlen); +#else + (void)qp; + (void)in; + (void)inlen; + (void)out; + (void)outlen; + errno = ENOTSUP; + return errno; +#endif +} + +static int +mlx5_glue_devx_port_query(struct ibv_context *ctx, + uint32_t port_num, + struct mlx5dv_devx_port *mlx5_devx_port) +{ +#ifdef HAVE_MLX5DV_DR_DEVX_PORT + return mlx5dv_query_devx_port(ctx, port_num, mlx5_devx_port); +#else + (void)ctx; + (void)port_num; + (void)mlx5_devx_port; + errno = ENOTSUP; + return errno; +#endif +} + +static int +mlx5_glue_dr_dump_domain(FILE *file, void *domain) +{ +#ifdef HAVE_MLX5_DR_FLOW_DUMP + return mlx5dv_dump_dr_domain(file, domain); +#else + RTE_SET_USED(file); + RTE_SET_USED(domain); + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_query_eqn(struct ibv_context *ctx, uint32_t cpus, + uint32_t *eqn) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_query_eqn(ctx, cpus, eqn); +#else + (void)ctx; + (void)cpus; + (void)eqn; + return -ENOTSUP; +#endif +} + +static struct mlx5dv_devx_event_channel * +mlx5_glue_devx_create_event_channel(struct ibv_context *ctx, int flags) +{ +#ifdef HAVE_IBV_DEVX_EVENT + return mlx5dv_devx_create_event_channel(ctx, flags); +#else + (void)ctx; + (void)flags; + errno = ENOTSUP; + return NULL; +#endif +} + +static void +mlx5_glue_devx_destroy_event_channel(struct mlx5dv_devx_event_channel *eventc) +{ +#ifdef HAVE_IBV_DEVX_EVENT + mlx5dv_devx_destroy_event_channel(eventc); +#else + (void)eventc; +#endif +} + +static int +mlx5_glue_devx_subscribe_devx_event(struct mlx5dv_devx_event_channel *eventc, + struct mlx5dv_devx_obj *obj, + uint16_t events_sz, uint16_t events_num[], + uint64_t cookie) +{ +#ifdef HAVE_IBV_DEVX_EVENT + return mlx5dv_devx_subscribe_devx_event(eventc, obj, events_sz, + events_num, cookie); +#else + (void)eventc; + (void)obj; + (void)events_sz; + (void)events_num; + (void)cookie; + return -ENOTSUP; +#endif +} + +static int +mlx5_glue_devx_subscribe_devx_event_fd(struct mlx5dv_devx_event_channel *eventc, + int fd, struct mlx5dv_devx_obj *obj, + uint16_t event_num) +{ +#ifdef HAVE_IBV_DEVX_EVENT + return mlx5dv_devx_subscribe_devx_event_fd(eventc, fd, obj, event_num); +#else + (void)eventc; + (void)fd; + (void)obj; + (void)event_num; + return -ENOTSUP; +#endif +} + +static ssize_t +mlx5_glue_devx_get_event(struct mlx5dv_devx_event_channel *eventc, + struct mlx5dv_devx_async_event_hdr *event_data, + size_t event_resp_len) +{ +#ifdef HAVE_IBV_DEVX_EVENT + return mlx5dv_devx_get_event(eventc, event_data, event_resp_len); +#else + (void)eventc; + (void)event_data; + (void)event_resp_len; + errno = ENOTSUP; + return -1; +#endif +} + +static struct mlx5dv_devx_uar * +mlx5_glue_devx_alloc_uar(struct ibv_context *context, uint32_t flags) +{ +#ifdef HAVE_IBV_DEVX_OBJ + return mlx5dv_devx_alloc_uar(context, flags); +#else + (void)context; + (void)flags; + errno = ENOTSUP; + return NULL; +#endif +} + +static void +mlx5_glue_devx_free_uar(struct mlx5dv_devx_uar *devx_uar) +{ +#ifdef HAVE_IBV_DEVX_OBJ + mlx5dv_devx_free_uar(devx_uar); +#else + (void)devx_uar; +#endif +} + +static struct mlx5dv_var * +mlx5_glue_dv_alloc_var(struct ibv_context *context, uint32_t flags) +{ +#ifdef HAVE_IBV_VAR + return mlx5dv_alloc_var(context, flags); +#else + (void)context; + (void)flags; + errno = ENOTSUP; + return NULL; +#endif +} + +static void +mlx5_glue_dv_free_var(struct mlx5dv_var *var) +{ +#ifdef HAVE_IBV_VAR + mlx5dv_free_var(var); +#else + (void)var; + errno = ENOTSUP; +#endif +} + +__rte_cache_aligned +const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { + .version = MLX5_GLUE_VERSION, + .fork_init = mlx5_glue_fork_init, + .alloc_pd = mlx5_glue_alloc_pd, + .dealloc_pd = mlx5_glue_dealloc_pd, + .get_device_list = mlx5_glue_get_device_list, + .free_device_list = mlx5_glue_free_device_list, + .open_device = mlx5_glue_open_device, + .close_device = mlx5_glue_close_device, + .query_device = mlx5_glue_query_device, + .query_device_ex = mlx5_glue_query_device_ex, + .query_rt_values_ex = mlx5_glue_query_rt_values_ex, + .query_port = mlx5_glue_query_port, + .create_comp_channel = mlx5_glue_create_comp_channel, + .destroy_comp_channel = mlx5_glue_destroy_comp_channel, + .create_cq = mlx5_glue_create_cq, + .destroy_cq = mlx5_glue_destroy_cq, + .get_cq_event = mlx5_glue_get_cq_event, + .ack_cq_events = mlx5_glue_ack_cq_events, + .create_rwq_ind_table = mlx5_glue_create_rwq_ind_table, + .destroy_rwq_ind_table = mlx5_glue_destroy_rwq_ind_table, + .create_wq = mlx5_glue_create_wq, + .destroy_wq = mlx5_glue_destroy_wq, + .modify_wq = mlx5_glue_modify_wq, + .create_flow = mlx5_glue_create_flow, + .destroy_flow = mlx5_glue_destroy_flow, + .destroy_flow_action = mlx5_glue_destroy_flow_action, + .create_qp = mlx5_glue_create_qp, + .create_qp_ex = mlx5_glue_create_qp_ex, + .destroy_qp = mlx5_glue_destroy_qp, + .modify_qp = mlx5_glue_modify_qp, + .reg_mr = mlx5_glue_reg_mr, + .alloc_null_mr = mlx5_glue_alloc_null_mr, + .dereg_mr = mlx5_glue_dereg_mr, + .create_counter_set = mlx5_glue_create_counter_set, + .destroy_counter_set = mlx5_glue_destroy_counter_set, + .describe_counter_set = mlx5_glue_describe_counter_set, + .query_counter_set = mlx5_glue_query_counter_set, + .create_counters = mlx5_glue_create_counters, + .destroy_counters = mlx5_glue_destroy_counters, + .attach_counters = mlx5_glue_attach_counters, + .query_counters = mlx5_glue_query_counters, + .ack_async_event = mlx5_glue_ack_async_event, + .get_async_event = mlx5_glue_get_async_event, + .port_state_str = mlx5_glue_port_state_str, + .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq, + .dr_create_flow_action_dest_flow_tbl = + mlx5_glue_dr_create_flow_action_dest_flow_tbl, + .dr_create_flow_action_dest_port = + mlx5_glue_dr_create_flow_action_dest_port, + .dr_create_flow_action_drop = + mlx5_glue_dr_create_flow_action_drop, + .dr_create_flow_action_push_vlan = + mlx5_glue_dr_create_flow_action_push_vlan, + .dr_create_flow_action_pop_vlan = + mlx5_glue_dr_create_flow_action_pop_vlan, + .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl, + .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl, + .dr_create_domain = mlx5_glue_dr_create_domain, + .dr_destroy_domain = mlx5_glue_dr_destroy_domain, + .dv_create_cq = mlx5_glue_dv_create_cq, + .dv_create_wq = mlx5_glue_dv_create_wq, + .dv_query_device = mlx5_glue_dv_query_device, + .dv_set_context_attr = mlx5_glue_dv_set_context_attr, + .dv_init_obj = mlx5_glue_dv_init_obj, + .dv_create_qp = mlx5_glue_dv_create_qp, + .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher, + .dv_create_flow = mlx5_glue_dv_create_flow, + .dv_create_flow_action_counter = + mlx5_glue_dv_create_flow_action_counter, + .dv_create_flow_action_dest_ibv_qp = + mlx5_glue_dv_create_flow_action_dest_ibv_qp, + .dv_create_flow_action_dest_devx_tir = + mlx5_glue_dv_create_flow_action_dest_devx_tir, + .dv_create_flow_action_modify_header = + mlx5_glue_dv_create_flow_action_modify_header, + .dv_create_flow_action_packet_reformat = + mlx5_glue_dv_create_flow_action_packet_reformat, + .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag, + .dv_create_flow_action_meter = mlx5_glue_dv_create_flow_action_meter, + .dv_modify_flow_action_meter = mlx5_glue_dv_modify_flow_action_meter, + .dv_destroy_flow = mlx5_glue_dv_destroy_flow, + .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher, + .dv_open_device = mlx5_glue_dv_open_device, + .devx_obj_create = mlx5_glue_devx_obj_create, + .devx_obj_destroy = mlx5_glue_devx_obj_destroy, + .devx_obj_query = mlx5_glue_devx_obj_query, + .devx_obj_modify = mlx5_glue_devx_obj_modify, + .devx_general_cmd = mlx5_glue_devx_general_cmd, + .devx_create_cmd_comp = mlx5_glue_devx_create_cmd_comp, + .devx_destroy_cmd_comp = mlx5_glue_devx_destroy_cmd_comp, + .devx_obj_query_async = mlx5_glue_devx_obj_query_async, + .devx_get_async_cmd_comp = mlx5_glue_devx_get_async_cmd_comp, + .devx_umem_reg = mlx5_glue_devx_umem_reg, + .devx_umem_dereg = mlx5_glue_devx_umem_dereg, + .devx_qp_query = mlx5_glue_devx_qp_query, + .devx_port_query = mlx5_glue_devx_port_query, + .dr_dump_domain = mlx5_glue_dr_dump_domain, + .devx_query_eqn = mlx5_glue_devx_query_eqn, + .devx_create_event_channel = mlx5_glue_devx_create_event_channel, + .devx_destroy_event_channel = mlx5_glue_devx_destroy_event_channel, + .devx_subscribe_devx_event = mlx5_glue_devx_subscribe_devx_event, + .devx_subscribe_devx_event_fd = mlx5_glue_devx_subscribe_devx_event_fd, + .devx_get_event = mlx5_glue_devx_get_event, + .devx_alloc_uar = mlx5_glue_devx_alloc_uar, + .devx_free_uar = mlx5_glue_devx_free_uar, + .dv_alloc_var = mlx5_glue_dv_alloc_var, + .dv_free_var = mlx5_glue_dv_free_var, +}; diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h new file mode 100644 index 0000000..81d6a22 --- /dev/null +++ b/drivers/common/mlx5/linux/mlx5_glue.h @@ -0,0 +1,309 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2018 6WIND S.A. + * Copyright 2018 Mellanox Technologies, Ltd + */ + +#ifndef MLX5_GLUE_H_ +#define MLX5_GLUE_H_ + +#include +#include +/* Verbs headers do not support -pedantic. */ +#ifdef PEDANTIC +#pragma GCC diagnostic ignored "-Wpedantic" +#endif +#include +#include +#ifdef PEDANTIC +#pragma GCC diagnostic error "-Wpedantic" +#endif + +#include + +#include "mlx5_autoconf.h" + +#ifndef MLX5_GLUE_VERSION +#define MLX5_GLUE_VERSION "" +#endif + +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 +struct ibv_counter_set; +struct ibv_counter_set_data; +struct ibv_counter_set_description; +struct ibv_counter_set_init_attr; +struct ibv_query_counter_set_attr; +#endif + +#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 +struct ibv_counters; +struct ibv_counters_init_attr; +struct ibv_counter_attach_attr; +#endif + +#ifndef HAVE_IBV_DEVICE_TUNNEL_SUPPORT +struct mlx5dv_qp_init_attr; +#endif + +#ifndef HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT +struct mlx5dv_wq_init_attr; +#endif + +#ifndef HAVE_IBV_FLOW_DV_SUPPORT +struct mlx5dv_flow_matcher; +struct mlx5dv_flow_matcher_attr; +struct mlx5dv_flow_action_attr; +struct mlx5dv_flow_match_parameters; +struct mlx5dv_dr_flow_meter_attr; +struct ibv_flow_action; +enum mlx5dv_flow_action_packet_reformat_type { packet_reformat_type = 0, }; +enum mlx5dv_flow_table_type { flow_table_type = 0, }; +#endif + +#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS +#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0 +#endif + +#ifndef HAVE_IBV_DEVX_OBJ +struct mlx5dv_devx_obj; +struct mlx5dv_devx_umem { uint32_t umem_id; }; +struct mlx5dv_devx_uar { void *reg_addr; void *base_addr; uint32_t page_id; }; +#endif + +#ifndef HAVE_IBV_DEVX_ASYNC +struct mlx5dv_devx_cmd_comp; +struct mlx5dv_devx_async_cmd_hdr; +#endif + +#ifndef HAVE_MLX5DV_DR +enum mlx5dv_dr_domain_type { unused, }; +struct mlx5dv_dr_domain; +#endif + +#ifndef HAVE_MLX5DV_DR_DEVX_PORT +struct mlx5dv_devx_port; +#endif + +#ifndef HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER +struct mlx5dv_dr_flow_meter_attr; +#endif + +#ifndef HAVE_IBV_DEVX_EVENT +struct mlx5dv_devx_event_channel { int fd; }; +struct mlx5dv_devx_async_event_hdr; +#define MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA 1 +#endif + +#ifndef HAVE_IBV_VAR +struct mlx5dv_var { uint32_t page_id; uint32_t length; off_t mmap_off; + uint64_t comp_mask; }; +#endif + +#ifndef HAVE_IBV_RELAXED_ORDERING +#define IBV_ACCESS_RELAXED_ORDERING 0 +#endif + +/* LIB_GLUE_VERSION must be updated every time this structure is modified. */ +struct mlx5_glue { + const char *version; + int (*fork_init)(void); + struct ibv_pd *(*alloc_pd)(struct ibv_context *context); + int (*dealloc_pd)(struct ibv_pd *pd); + struct ibv_device **(*get_device_list)(int *num_devices); + void (*free_device_list)(struct ibv_device **list); + struct ibv_context *(*open_device)(struct ibv_device *device); + int (*close_device)(struct ibv_context *context); + int (*query_device)(struct ibv_context *context, + struct ibv_device_attr *device_attr); + int (*query_device_ex)(struct ibv_context *context, + const struct ibv_query_device_ex_input *input, + struct ibv_device_attr_ex *attr); + int (*query_rt_values_ex)(struct ibv_context *context, + struct ibv_values_ex *values); + int (*query_port)(struct ibv_context *context, uint8_t port_num, + struct ibv_port_attr *port_attr); + struct ibv_comp_channel *(*create_comp_channel) + (struct ibv_context *context); + int (*destroy_comp_channel)(struct ibv_comp_channel *channel); + struct ibv_cq *(*create_cq)(struct ibv_context *context, int cqe, + void *cq_context, + struct ibv_comp_channel *channel, + int comp_vector); + int (*destroy_cq)(struct ibv_cq *cq); + int (*get_cq_event)(struct ibv_comp_channel *channel, + struct ibv_cq **cq, void **cq_context); + void (*ack_cq_events)(struct ibv_cq *cq, unsigned int nevents); + struct ibv_rwq_ind_table *(*create_rwq_ind_table) + (struct ibv_context *context, + struct ibv_rwq_ind_table_init_attr *init_attr); + int (*destroy_rwq_ind_table)(struct ibv_rwq_ind_table *rwq_ind_table); + struct ibv_wq *(*create_wq)(struct ibv_context *context, + struct ibv_wq_init_attr *wq_init_attr); + int (*destroy_wq)(struct ibv_wq *wq); + int (*modify_wq)(struct ibv_wq *wq, struct ibv_wq_attr *wq_attr); + struct ibv_flow *(*create_flow)(struct ibv_qp *qp, + struct ibv_flow_attr *flow); + int (*destroy_flow)(struct ibv_flow *flow_id); + int (*destroy_flow_action)(void *action); + struct ibv_qp *(*create_qp)(struct ibv_pd *pd, + struct ibv_qp_init_attr *qp_init_attr); + struct ibv_qp *(*create_qp_ex) + (struct ibv_context *context, + struct ibv_qp_init_attr_ex *qp_init_attr_ex); + int (*destroy_qp)(struct ibv_qp *qp); + int (*modify_qp)(struct ibv_qp *qp, struct ibv_qp_attr *attr, + int attr_mask); + struct ibv_mr *(*reg_mr)(struct ibv_pd *pd, void *addr, + size_t length, int access); + struct ibv_mr *(*alloc_null_mr)(struct ibv_pd *pd); + int (*dereg_mr)(struct ibv_mr *mr); + struct ibv_counter_set *(*create_counter_set) + (struct ibv_context *context, + struct ibv_counter_set_init_attr *init_attr); + int (*destroy_counter_set)(struct ibv_counter_set *cs); + int (*describe_counter_set) + (struct ibv_context *context, + uint16_t counter_set_id, + struct ibv_counter_set_description *cs_desc); + int (*query_counter_set)(struct ibv_query_counter_set_attr *query_attr, + struct ibv_counter_set_data *cs_data); + struct ibv_counters *(*create_counters) + (struct ibv_context *context, + struct ibv_counters_init_attr *init_attr); + int (*destroy_counters)(struct ibv_counters *counters); + int (*attach_counters)(struct ibv_counters *counters, + struct ibv_counter_attach_attr *attr, + struct ibv_flow *flow); + int (*query_counters)(struct ibv_counters *counters, + uint64_t *counters_value, + uint32_t ncounters, + uint32_t flags); + void (*ack_async_event)(struct ibv_async_event *event); + int (*get_async_event)(struct ibv_context *context, + struct ibv_async_event *event); + const char *(*port_state_str)(enum ibv_port_state port_state); + struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq); + void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl); + void *(*dr_create_flow_action_dest_port)(void *domain, + uint32_t port); + void *(*dr_create_flow_action_drop)(); + void *(*dr_create_flow_action_push_vlan) + (struct mlx5dv_dr_domain *domain, + rte_be32_t vlan_tag); + void *(*dr_create_flow_action_pop_vlan)(); + void *(*dr_create_flow_tbl)(void *domain, uint32_t level); + int (*dr_destroy_flow_tbl)(void *tbl); + void *(*dr_create_domain)(struct ibv_context *ctx, + enum mlx5dv_dr_domain_type domain); + int (*dr_destroy_domain)(void *domain); + struct ibv_cq_ex *(*dv_create_cq) + (struct ibv_context *context, + struct ibv_cq_init_attr_ex *cq_attr, + struct mlx5dv_cq_init_attr *mlx5_cq_attr); + struct ibv_wq *(*dv_create_wq) + (struct ibv_context *context, + struct ibv_wq_init_attr *wq_attr, + struct mlx5dv_wq_init_attr *mlx5_wq_attr); + int (*dv_query_device)(struct ibv_context *ctx_in, + struct mlx5dv_context *attrs_out); + int (*dv_set_context_attr)(struct ibv_context *ibv_ctx, + enum mlx5dv_set_ctx_attr_type type, + void *attr); + int (*dv_init_obj)(struct mlx5dv_obj *obj, uint64_t obj_type); + struct ibv_qp *(*dv_create_qp) + (struct ibv_context *context, + struct ibv_qp_init_attr_ex *qp_init_attr_ex, + struct mlx5dv_qp_init_attr *dv_qp_init_attr); + void *(*dv_create_flow_matcher) + (struct ibv_context *context, + struct mlx5dv_flow_matcher_attr *matcher_attr, + void *tbl); + void *(*dv_create_flow)(void *matcher, void *match_value, + size_t num_actions, void *actions[]); + void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset); + void *(*dv_create_flow_action_dest_ibv_qp)(void *qp); + void *(*dv_create_flow_action_dest_devx_tir)(void *tir); + void *(*dv_create_flow_action_modify_header) + (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type, + void *domain, uint64_t flags, size_t actions_sz, + uint64_t actions[]); + void *(*dv_create_flow_action_packet_reformat) + (struct ibv_context *ctx, + enum mlx5dv_flow_action_packet_reformat_type reformat_type, + enum mlx5dv_flow_table_type ft_type, + struct mlx5dv_dr_domain *domain, + uint32_t flags, size_t data_sz, void *data); + void *(*dv_create_flow_action_tag)(uint32_t tag); + void *(*dv_create_flow_action_meter) + (struct mlx5dv_dr_flow_meter_attr *attr); + int (*dv_modify_flow_action_meter)(void *action, + struct mlx5dv_dr_flow_meter_attr *attr, uint64_t modify_bits); + int (*dv_destroy_flow)(void *flow); + int (*dv_destroy_flow_matcher)(void *matcher); + struct ibv_context *(*dv_open_device)(struct ibv_device *device); + struct mlx5dv_var *(*dv_alloc_var)(struct ibv_context *context, + uint32_t flags); + void (*dv_free_var)(struct mlx5dv_var *var); + struct mlx5dv_devx_uar *(*devx_alloc_uar)(struct ibv_context *context, + uint32_t flags); + void (*devx_free_uar)(struct mlx5dv_devx_uar *devx_uar); + struct mlx5dv_devx_obj *(*devx_obj_create) + (struct ibv_context *ctx, + const void *in, size_t inlen, + void *out, size_t outlen); + int (*devx_obj_destroy)(struct mlx5dv_devx_obj *obj); + int (*devx_obj_query)(struct mlx5dv_devx_obj *obj, + const void *in, size_t inlen, + void *out, size_t outlen); + int (*devx_obj_modify)(struct mlx5dv_devx_obj *obj, + const void *in, size_t inlen, + void *out, size_t outlen); + int (*devx_general_cmd)(struct ibv_context *context, + const void *in, size_t inlen, + void *out, size_t outlen); + struct mlx5dv_devx_cmd_comp *(*devx_create_cmd_comp) + (struct ibv_context *context); + void (*devx_destroy_cmd_comp)(struct mlx5dv_devx_cmd_comp *cmd_comp); + int (*devx_obj_query_async)(struct mlx5dv_devx_obj *obj, + const void *in, size_t inlen, + size_t outlen, uint64_t wr_id, + struct mlx5dv_devx_cmd_comp *cmd_comp); + int (*devx_get_async_cmd_comp)(struct mlx5dv_devx_cmd_comp *cmd_comp, + struct mlx5dv_devx_async_cmd_hdr *resp, + size_t cmd_resp_len); + struct mlx5dv_devx_umem *(*devx_umem_reg)(struct ibv_context *context, + void *addr, size_t size, + uint32_t access); + int (*devx_umem_dereg)(struct mlx5dv_devx_umem *dv_devx_umem); + int (*devx_qp_query)(struct ibv_qp *qp, + const void *in, size_t inlen, + void *out, size_t outlen); + int (*devx_port_query)(struct ibv_context *ctx, + uint32_t port_num, + struct mlx5dv_devx_port *mlx5_devx_port); + int (*dr_dump_domain)(FILE *file, void *domain); + int (*devx_query_eqn)(struct ibv_context *context, uint32_t cpus, + uint32_t *eqn); + struct mlx5dv_devx_event_channel *(*devx_create_event_channel) + (struct ibv_context *context, int flags); + void (*devx_destroy_event_channel) + (struct mlx5dv_devx_event_channel *event_channel); + int (*devx_subscribe_devx_event) + (struct mlx5dv_devx_event_channel *event_channel, + struct mlx5dv_devx_obj *obj, + uint16_t events_sz, + uint16_t events_num[], + uint64_t cookie); + int (*devx_subscribe_devx_event_fd) + (struct mlx5dv_devx_event_channel *event_channel, + int fd, + struct mlx5dv_devx_obj *obj, + uint16_t event_num); + ssize_t (*devx_get_event) + (struct mlx5dv_devx_event_channel *event_channel, + struct mlx5dv_devx_async_event_hdr *event_data, + size_t event_resp_len); +}; + +extern const struct mlx5_glue *mlx5_glue; + +#endif /* MLX5_GLUE_H_ */ diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index 5a802ba..9806865 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -1,65 +1,23 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright 2019 Mellanox Technologies, Ltd -if not is_linux +if not (is_linux or is_windows) build = false - reason = 'only supported on Linux' + reason = 'only supported on Linux and Windows' subdir_done() endif -static_ibverbs = (get_option('ibverbs_link') == 'static') -dlopen_ibverbs = (get_option('ibverbs_link') == 'dlopen') -LIB_GLUE_BASE = 'librte_pmd_mlx5_glue.so' -LIB_GLUE_VERSION = '20.02.0' -LIB_GLUE = LIB_GLUE_BASE + '.' + LIB_GLUE_VERSION -if dlopen_ibverbs - dpdk_conf.set('RTE_IBVERBS_LINK_DLOPEN', 1) - cflags += [ - '-DMLX5_GLUE="@0@"'.format(LIB_GLUE), - '-DMLX5_GLUE_VERSION="@0@"'.format(LIB_GLUE_VERSION), - ] -endif - -libnames = [ 'mlx5', 'ibverbs' ] -libs = [] -foreach libname:libnames - lib = dependency('lib' + libname, static:static_ibverbs, required:false) - if not lib.found() and not static_ibverbs - lib = cc.find_library(libname, required:false) - endif - if lib.found() - libs += lib - if not static_ibverbs and not dlopen_ibverbs - ext_deps += lib - endif - else - build = false - reason = 'missing dependency, "' + libname + '"' - subdir_done() - endif -endforeach -if static_ibverbs or dlopen_ibverbs - # Build without adding shared libs to Requires.private - ibv_cflags = run_command(pkgconf, '--cflags', 'libibverbs').stdout() - ext_deps += declare_dependency(compile_args: ibv_cflags.split()) -endif -if static_ibverbs - # Add static deps ldflags to internal apps and Libs.private - ibv_ldflags = run_command(ldflags_ibverbs_static, check:true).stdout() - ext_deps += declare_dependency(link_args:ibv_ldflags.split()) -endif +allow_experimental_apis = true deps += ['hash', 'pci', 'net', 'eal', 'kvargs'] -sources = files( +sources += files( 'mlx5_devx_cmds.c', 'mlx5_common.c', 'mlx5_nl.c', 'mlx5_common_mp.c', 'mlx5_common_mr.c', ) -if not dlopen_ibverbs - sources += files('mlx5_glue.c') -endif + cflags_options = [ '-std=c11', '-Wno-strict-prototypes', @@ -77,151 +35,5 @@ if get_option('buildtype').contains('debug') else cflags += [ '-UPEDANTIC' ] endif -# To maintain the compatibility with the make build system -# mlx5_autoconf.h file is still generated. -# input array for meson member search: -# [ "MACRO to define if found", "header for the search", -# "symbol to search", "struct member to search" ] -has_member_args = [ - [ 'HAVE_IBV_MLX5_MOD_SWP', 'infiniband/mlx5dv.h', - 'struct mlx5dv_sw_parsing_caps', 'sw_parsing_offloads' ], - [ 'HAVE_IBV_DEVICE_COUNTERS_SET_V42', 'infiniband/verbs.h', - 'struct ibv_counter_set_init_attr', 'counter_set_id' ], - [ 'HAVE_IBV_DEVICE_COUNTERS_SET_V45', 'infiniband/verbs.h', - 'struct ibv_counters_init_attr', 'comp_mask' ], -] -# input array for meson symbol search: -# [ "MACRO to define if found", "header for the search", -# "symbol to search" ] -has_sym_args = [ - [ 'HAVE_IBV_RELAXED_ORDERING', 'infiniband/verbs.h', - 'IBV_ACCESS_RELAXED_ORDERING ' ], - [ 'HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT', 'infiniband/mlx5dv.h', - 'MLX5DV_CQE_RES_FORMAT_CSUM_STRIDX' ], - [ 'HAVE_IBV_DEVICE_TUNNEL_SUPPORT', 'infiniband/mlx5dv.h', - 'MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS' ], - [ 'HAVE_IBV_MLX5_MOD_MPW', 'infiniband/mlx5dv.h', - 'MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED' ], - [ 'HAVE_IBV_MLX5_MOD_CQE_128B_COMP', 'infiniband/mlx5dv.h', - 'MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP' ], - [ 'HAVE_IBV_MLX5_MOD_CQE_128B_PAD', 'infiniband/mlx5dv.h', - 'MLX5DV_CQ_INIT_ATTR_FLAGS_CQE_PAD' ], - [ 'HAVE_IBV_FLOW_DV_SUPPORT', 'infiniband/mlx5dv.h', - 'mlx5dv_create_flow_action_packet_reformat' ], - [ 'HAVE_IBV_DEVICE_MPLS_SUPPORT', 'infiniband/verbs.h', - 'IBV_FLOW_SPEC_MPLS' ], - [ 'HAVE_IBV_WQ_FLAGS_PCI_WRITE_END_PADDING', 'infiniband/verbs.h', - 'IBV_WQ_FLAGS_PCI_WRITE_END_PADDING' ], - [ 'HAVE_IBV_WQ_FLAG_RX_END_PADDING', 'infiniband/verbs.h', - 'IBV_WQ_FLAG_RX_END_PADDING' ], - [ 'HAVE_MLX5DV_DR_DEVX_PORT', 'infiniband/mlx5dv.h', - 'mlx5dv_query_devx_port' ], - [ 'HAVE_IBV_DEVX_OBJ', 'infiniband/mlx5dv.h', - 'mlx5dv_devx_obj_create' ], - [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h', - 'MLX5DV_FLOW_ACTION_COUNTERS_DEVX' ], - [ 'HAVE_IBV_DEVX_ASYNC', 'infiniband/mlx5dv.h', - 'mlx5dv_devx_obj_query_async' ], - [ 'HAVE_IBV_DEVX_QP', 'infiniband/mlx5dv.h', - 'mlx5dv_devx_qp_query' ], - [ 'HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR', 'infiniband/mlx5dv.h', - 'mlx5dv_dr_action_create_dest_devx_tir' ], - [ 'HAVE_IBV_DEVX_EVENT', 'infiniband/mlx5dv.h', - 'mlx5dv_devx_get_event' ], - [ 'HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER', 'infiniband/mlx5dv.h', - 'mlx5dv_dr_action_create_flow_meter' ], - [ 'HAVE_MLX5DV_MMAP_GET_NC_PAGES_CMD', 'infiniband/mlx5dv.h', - 'MLX5_MMAP_GET_NC_PAGES_CMD' ], - [ 'HAVE_MLX5DV_DR', 'infiniband/mlx5dv.h', - 'MLX5DV_DR_DOMAIN_TYPE_NIC_RX' ], - [ 'HAVE_MLX5DV_DR_ESWITCH', 'infiniband/mlx5dv.h', - 'MLX5DV_DR_DOMAIN_TYPE_FDB' ], - [ 'HAVE_MLX5DV_DR_VLAN', 'infiniband/mlx5dv.h', - 'mlx5dv_dr_action_create_push_vlan' ], - [ 'HAVE_IBV_VAR', 'infiniband/mlx5dv.h', 'mlx5dv_alloc_var' ], - [ 'HAVE_SUPPORTED_40000baseKR4_Full', 'linux/ethtool.h', - 'SUPPORTED_40000baseKR4_Full' ], - [ 'HAVE_SUPPORTED_40000baseCR4_Full', 'linux/ethtool.h', - 'SUPPORTED_40000baseCR4_Full' ], - [ 'HAVE_SUPPORTED_40000baseSR4_Full', 'linux/ethtool.h', - 'SUPPORTED_40000baseSR4_Full' ], - [ 'HAVE_SUPPORTED_40000baseLR4_Full', 'linux/ethtool.h', - 'SUPPORTED_40000baseLR4_Full' ], - [ 'HAVE_SUPPORTED_56000baseKR4_Full', 'linux/ethtool.h', - 'SUPPORTED_56000baseKR4_Full' ], - [ 'HAVE_SUPPORTED_56000baseCR4_Full', 'linux/ethtool.h', - 'SUPPORTED_56000baseCR4_Full' ], - [ 'HAVE_SUPPORTED_56000baseSR4_Full', 'linux/ethtool.h', - 'SUPPORTED_56000baseSR4_Full' ], - [ 'HAVE_SUPPORTED_56000baseLR4_Full', 'linux/ethtool.h', - 'SUPPORTED_56000baseLR4_Full' ], - [ 'HAVE_ETHTOOL_LINK_MODE_25G', 'linux/ethtool.h', - 'ETHTOOL_LINK_MODE_25000baseCR_Full_BIT' ], - [ 'HAVE_ETHTOOL_LINK_MODE_50G', 'linux/ethtool.h', - 'ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT' ], - [ 'HAVE_ETHTOOL_LINK_MODE_100G', 'linux/ethtool.h', - 'ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT' ], - [ 'HAVE_IFLA_NUM_VF', 'linux/if_link.h', - 'IFLA_NUM_VF' ], - [ 'HAVE_IFLA_EXT_MASK', 'linux/if_link.h', - 'IFLA_EXT_MASK' ], - [ 'HAVE_IFLA_PHYS_SWITCH_ID', 'linux/if_link.h', - 'IFLA_PHYS_SWITCH_ID' ], - [ 'HAVE_IFLA_PHYS_PORT_NAME', 'linux/if_link.h', - 'IFLA_PHYS_PORT_NAME' ], - [ 'HAVE_RDMA_NL_NLDEV', 'rdma/rdma_netlink.h', - 'RDMA_NL_NLDEV' ], - [ 'HAVE_RDMA_NLDEV_CMD_GET', 'rdma/rdma_netlink.h', - 'RDMA_NLDEV_CMD_GET' ], - [ 'HAVE_RDMA_NLDEV_CMD_PORT_GET', 'rdma/rdma_netlink.h', - 'RDMA_NLDEV_CMD_PORT_GET' ], - [ 'HAVE_RDMA_NLDEV_ATTR_DEV_INDEX', 'rdma/rdma_netlink.h', - 'RDMA_NLDEV_ATTR_DEV_INDEX' ], - [ 'HAVE_RDMA_NLDEV_ATTR_DEV_NAME', 'rdma/rdma_netlink.h', - 'RDMA_NLDEV_ATTR_DEV_NAME' ], - [ 'HAVE_RDMA_NLDEV_ATTR_PORT_INDEX', 'rdma/rdma_netlink.h', - 'RDMA_NLDEV_ATTR_PORT_INDEX' ], - [ 'HAVE_RDMA_NLDEV_ATTR_NDEV_INDEX', 'rdma/rdma_netlink.h', - 'RDMA_NLDEV_ATTR_NDEV_INDEX' ], - [ 'HAVE_MLX5_DR_FLOW_DUMP', 'infiniband/mlx5dv.h', - 'mlx5dv_dump_dr_domain'], - [ 'HAVE_DEVLINK', 'linux/devlink.h', 'DEVLINK_GENL_NAME' ], -] -config = configuration_data() -foreach arg:has_sym_args - config.set(arg[0], cc.has_header_symbol(arg[1], arg[2], - dependencies: libs)) -endforeach -foreach arg:has_member_args - file_prefix = '#include <' + arg[1] + '>' - config.set(arg[0], cc.has_member(arg[2], arg[3], - prefix : file_prefix, dependencies: libs)) -endforeach -configure_file(output : 'mlx5_autoconf.h', configuration : config) -# Build Glue Library -if dlopen_ibverbs - dlopen_name = 'mlx5_glue' - dlopen_lib_name = 'rte_pmd_@0@'.format(dlopen_name) - dlopen_so_version = LIB_GLUE_VERSION - dlopen_sources = files('mlx5_glue.c') - dlopen_install_dir = [ eal_pmd_path + '-glue' ] - dlopen_includes = [global_inc] - dlopen_includes += include_directories( - '../../../lib/librte_eal/include/generic', - ) - shared_lib = shared_library( - dlopen_lib_name, - dlopen_sources, - include_directories: dlopen_includes, - c_args: cflags, - dependencies: libs, - link_args: [ - '-Wl,-export-dynamic', - '-Wl,-h,@0@'.format(LIB_GLUE), - ], - soversion: dlopen_so_version, - install: true, - install_dir: dlopen_install_dir, - ) -endif +subdir(exec_env) diff --git a/drivers/common/mlx5/mlx5_glue.c b/drivers/common/mlx5/mlx5_glue.c deleted file mode 100644 index f270f67..0000000 --- a/drivers/common/mlx5/mlx5_glue.c +++ /dev/null @@ -1,1294 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2018 6WIND S.A. - * Copyright 2018 Mellanox Technologies, Ltd - */ - -#include -#include -#include -#include -#include -/* - * Not needed by this file; included to work around the lack of off_t - * definition for mlx5dv.h with unpatched rdma-core versions. - */ -#include - -#include "mlx5_glue.h" - -static int -mlx5_glue_fork_init(void) -{ - return ibv_fork_init(); -} - -static struct ibv_pd * -mlx5_glue_alloc_pd(struct ibv_context *context) -{ - return ibv_alloc_pd(context); -} - -static int -mlx5_glue_dealloc_pd(struct ibv_pd *pd) -{ - return ibv_dealloc_pd(pd); -} - -static struct ibv_device ** -mlx5_glue_get_device_list(int *num_devices) -{ - return ibv_get_device_list(num_devices); -} - -static void -mlx5_glue_free_device_list(struct ibv_device **list) -{ - ibv_free_device_list(list); -} - -static struct ibv_context * -mlx5_glue_open_device(struct ibv_device *device) -{ - return ibv_open_device(device); -} - -static int -mlx5_glue_close_device(struct ibv_context *context) -{ - return ibv_close_device(context); -} - -static int -mlx5_glue_query_device(struct ibv_context *context, - struct ibv_device_attr *device_attr) -{ - return ibv_query_device(context, device_attr); -} - -static int -mlx5_glue_query_device_ex(struct ibv_context *context, - const struct ibv_query_device_ex_input *input, - struct ibv_device_attr_ex *attr) -{ - return ibv_query_device_ex(context, input, attr); -} - -static int -mlx5_glue_query_rt_values_ex(struct ibv_context *context, - struct ibv_values_ex *values) -{ - return ibv_query_rt_values_ex(context, values); -} - -static int -mlx5_glue_query_port(struct ibv_context *context, uint8_t port_num, - struct ibv_port_attr *port_attr) -{ - return ibv_query_port(context, port_num, port_attr); -} - -static struct ibv_comp_channel * -mlx5_glue_create_comp_channel(struct ibv_context *context) -{ - return ibv_create_comp_channel(context); -} - -static int -mlx5_glue_destroy_comp_channel(struct ibv_comp_channel *channel) -{ - return ibv_destroy_comp_channel(channel); -} - -static struct ibv_cq * -mlx5_glue_create_cq(struct ibv_context *context, int cqe, void *cq_context, - struct ibv_comp_channel *channel, int comp_vector) -{ - return ibv_create_cq(context, cqe, cq_context, channel, comp_vector); -} - -static int -mlx5_glue_destroy_cq(struct ibv_cq *cq) -{ - return ibv_destroy_cq(cq); -} - -static int -mlx5_glue_get_cq_event(struct ibv_comp_channel *channel, struct ibv_cq **cq, - void **cq_context) -{ - return ibv_get_cq_event(channel, cq, cq_context); -} - -static void -mlx5_glue_ack_cq_events(struct ibv_cq *cq, unsigned int nevents) -{ - ibv_ack_cq_events(cq, nevents); -} - -static struct ibv_rwq_ind_table * -mlx5_glue_create_rwq_ind_table(struct ibv_context *context, - struct ibv_rwq_ind_table_init_attr *init_attr) -{ - return ibv_create_rwq_ind_table(context, init_attr); -} - -static int -mlx5_glue_destroy_rwq_ind_table(struct ibv_rwq_ind_table *rwq_ind_table) -{ - return ibv_destroy_rwq_ind_table(rwq_ind_table); -} - -static struct ibv_wq * -mlx5_glue_create_wq(struct ibv_context *context, - struct ibv_wq_init_attr *wq_init_attr) -{ - return ibv_create_wq(context, wq_init_attr); -} - -static int -mlx5_glue_destroy_wq(struct ibv_wq *wq) -{ - return ibv_destroy_wq(wq); -} -static int -mlx5_glue_modify_wq(struct ibv_wq *wq, struct ibv_wq_attr *wq_attr) -{ - return ibv_modify_wq(wq, wq_attr); -} - -static struct ibv_flow * -mlx5_glue_create_flow(struct ibv_qp *qp, struct ibv_flow_attr *flow) -{ - return ibv_create_flow(qp, flow); -} - -static int -mlx5_glue_destroy_flow(struct ibv_flow *flow_id) -{ - return ibv_destroy_flow(flow_id); -} - -static int -mlx5_glue_destroy_flow_action(void *action) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_action_destroy(action); -#else - struct mlx5dv_flow_action_attr *attr = action; - int res = 0; - switch (attr->type) { - case MLX5DV_FLOW_ACTION_TAG: - break; - default: - res = ibv_destroy_flow_action(attr->action); - break; - } - free(action); - return res; -#endif -#else - (void)action; - return ENOTSUP; -#endif -} - -static struct ibv_qp * -mlx5_glue_create_qp(struct ibv_pd *pd, struct ibv_qp_init_attr *qp_init_attr) -{ - return ibv_create_qp(pd, qp_init_attr); -} - -static struct ibv_qp * -mlx5_glue_create_qp_ex(struct ibv_context *context, - struct ibv_qp_init_attr_ex *qp_init_attr_ex) -{ - return ibv_create_qp_ex(context, qp_init_attr_ex); -} - -static int -mlx5_glue_destroy_qp(struct ibv_qp *qp) -{ - return ibv_destroy_qp(qp); -} - -static int -mlx5_glue_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mask) -{ - return ibv_modify_qp(qp, attr, attr_mask); -} - -static struct ibv_mr * -mlx5_glue_reg_mr(struct ibv_pd *pd, void *addr, size_t length, int access) -{ - return ibv_reg_mr(pd, addr, length, access); -} - -static struct ibv_mr * -mlx5_glue_alloc_null_mr(struct ibv_pd *pd) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return ibv_alloc_null_mr(pd); -#else - (void)pd; - errno = ENOTSUP; - return NULL; -#endif -} - -static int -mlx5_glue_dereg_mr(struct ibv_mr *mr) -{ - return ibv_dereg_mr(mr); -} - -static struct ibv_counter_set * -mlx5_glue_create_counter_set(struct ibv_context *context, - struct ibv_counter_set_init_attr *init_attr) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 - (void)context; - (void)init_attr; - return NULL; -#else - return ibv_create_counter_set(context, init_attr); -#endif -} - -static int -mlx5_glue_destroy_counter_set(struct ibv_counter_set *cs) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 - (void)cs; - return ENOTSUP; -#else - return ibv_destroy_counter_set(cs); -#endif -} - -static int -mlx5_glue_describe_counter_set(struct ibv_context *context, - uint16_t counter_set_id, - struct ibv_counter_set_description *cs_desc) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 - (void)context; - (void)counter_set_id; - (void)cs_desc; - return ENOTSUP; -#else - return ibv_describe_counter_set(context, counter_set_id, cs_desc); -#endif -} - -static int -mlx5_glue_query_counter_set(struct ibv_query_counter_set_attr *query_attr, - struct ibv_counter_set_data *cs_data) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 - (void)query_attr; - (void)cs_data; - return ENOTSUP; -#else - return ibv_query_counter_set(query_attr, cs_data); -#endif -} - -static struct ibv_counters * -mlx5_glue_create_counters(struct ibv_context *context, - struct ibv_counters_init_attr *init_attr) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 - (void)context; - (void)init_attr; - errno = ENOTSUP; - return NULL; -#else - return ibv_create_counters(context, init_attr); -#endif -} - -static int -mlx5_glue_destroy_counters(struct ibv_counters *counters) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 - (void)counters; - return ENOTSUP; -#else - return ibv_destroy_counters(counters); -#endif -} - -static int -mlx5_glue_attach_counters(struct ibv_counters *counters, - struct ibv_counter_attach_attr *attr, - struct ibv_flow *flow) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 - (void)counters; - (void)attr; - (void)flow; - return ENOTSUP; -#else - return ibv_attach_counters_point_flow(counters, attr, flow); -#endif -} - -static int -mlx5_glue_query_counters(struct ibv_counters *counters, - uint64_t *counters_value, - uint32_t ncounters, - uint32_t flags) -{ -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 - (void)counters; - (void)counters_value; - (void)ncounters; - (void)flags; - return ENOTSUP; -#else - return ibv_read_counters(counters, counters_value, ncounters, flags); -#endif -} - -static void -mlx5_glue_ack_async_event(struct ibv_async_event *event) -{ - ibv_ack_async_event(event); -} - -static int -mlx5_glue_get_async_event(struct ibv_context *context, - struct ibv_async_event *event) -{ - return ibv_get_async_event(context, event); -} - -static const char * -mlx5_glue_port_state_str(enum ibv_port_state port_state) -{ - return ibv_port_state_str(port_state); -} - -static struct ibv_cq * -mlx5_glue_cq_ex_to_cq(struct ibv_cq_ex *cq) -{ - return ibv_cq_ex_to_cq(cq); -} - -static void * -mlx5_glue_dr_create_flow_action_dest_flow_tbl(void *tbl) -{ -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_action_create_dest_table(tbl); -#else - (void)tbl; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dr_create_flow_action_dest_port(void *domain, uint32_t port) -{ -#ifdef HAVE_MLX5DV_DR_DEVX_PORT - return mlx5dv_dr_action_create_dest_ib_port(domain, port); -#else -#ifdef HAVE_MLX5DV_DR_ESWITCH - return mlx5dv_dr_action_create_dest_vport(domain, port); -#else - (void)domain; - (void)port; - errno = ENOTSUP; - return NULL; -#endif -#endif -} - -static void * -mlx5_glue_dr_create_flow_action_drop(void) -{ -#ifdef HAVE_MLX5DV_DR_ESWITCH - return mlx5dv_dr_action_create_drop(); -#else - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dr_create_flow_action_push_vlan(struct mlx5dv_dr_domain *domain, - rte_be32_t vlan_tag) -{ -#ifdef HAVE_MLX5DV_DR_VLAN - return mlx5dv_dr_action_create_push_vlan(domain, vlan_tag); -#else - (void)domain; - (void)vlan_tag; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dr_create_flow_action_pop_vlan(void) -{ -#ifdef HAVE_MLX5DV_DR_VLAN - return mlx5dv_dr_action_create_pop_vlan(); -#else - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dr_create_flow_tbl(void *domain, uint32_t level) -{ -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_table_create(domain, level); -#else - (void)domain; - (void)level; - errno = ENOTSUP; - return NULL; -#endif -} - -static int -mlx5_glue_dr_destroy_flow_tbl(void *tbl) -{ -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_table_destroy(tbl); -#else - (void)tbl; - errno = ENOTSUP; - return errno; -#endif -} - -static void * -mlx5_glue_dr_create_domain(struct ibv_context *ctx, - enum mlx5dv_dr_domain_type domain) -{ -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_domain_create(ctx, domain); -#else - (void)ctx; - (void)domain; - errno = ENOTSUP; - return NULL; -#endif -} - -static int -mlx5_glue_dr_destroy_domain(void *domain) -{ -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_domain_destroy(domain); -#else - (void)domain; - errno = ENOTSUP; - return errno; -#endif -} - -static struct ibv_cq_ex * -mlx5_glue_dv_create_cq(struct ibv_context *context, - struct ibv_cq_init_attr_ex *cq_attr, - struct mlx5dv_cq_init_attr *mlx5_cq_attr) -{ - return mlx5dv_create_cq(context, cq_attr, mlx5_cq_attr); -} - -static struct ibv_wq * -mlx5_glue_dv_create_wq(struct ibv_context *context, - struct ibv_wq_init_attr *wq_attr, - struct mlx5dv_wq_init_attr *mlx5_wq_attr) -{ -#ifndef HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT - (void)context; - (void)wq_attr; - (void)mlx5_wq_attr; - errno = ENOTSUP; - return NULL; -#else - return mlx5dv_create_wq(context, wq_attr, mlx5_wq_attr); -#endif -} - -static int -mlx5_glue_dv_query_device(struct ibv_context *ctx, - struct mlx5dv_context *attrs_out) -{ - return mlx5dv_query_device(ctx, attrs_out); -} - -static int -mlx5_glue_dv_set_context_attr(struct ibv_context *ibv_ctx, - enum mlx5dv_set_ctx_attr_type type, void *attr) -{ - return mlx5dv_set_context_attr(ibv_ctx, type, attr); -} - -static int -mlx5_glue_dv_init_obj(struct mlx5dv_obj *obj, uint64_t obj_type) -{ - return mlx5dv_init_obj(obj, obj_type); -} - -static struct ibv_qp * -mlx5_glue_dv_create_qp(struct ibv_context *context, - struct ibv_qp_init_attr_ex *qp_init_attr_ex, - struct mlx5dv_qp_init_attr *dv_qp_init_attr) -{ -#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT - return mlx5dv_create_qp(context, qp_init_attr_ex, dv_qp_init_attr); -#else - (void)context; - (void)qp_init_attr_ex; - (void)dv_qp_init_attr; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_matcher(struct ibv_context *context, - struct mlx5dv_flow_matcher_attr *matcher_attr, - void *tbl) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - (void)context; - return mlx5dv_dr_matcher_create(tbl, matcher_attr->priority, - matcher_attr->match_criteria_enable, - matcher_attr->match_mask); -#else - (void)tbl; - return mlx5dv_create_flow_matcher(context, matcher_attr); -#endif -#else - (void)context; - (void)matcher_attr; - (void)tbl; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow(void *matcher, - void *match_value, - size_t num_actions, - void *actions[]) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_rule_create(matcher, match_value, num_actions, - (struct mlx5dv_dr_action **)actions); -#else - struct mlx5dv_flow_action_attr actions_attr[8]; - - if (num_actions > 8) - return NULL; - for (size_t i = 0; i < num_actions; i++) - actions_attr[i] = - *((struct mlx5dv_flow_action_attr *)(actions[i])); - return mlx5dv_create_flow(matcher, match_value, - num_actions, actions_attr); -#endif -#else - (void)matcher; - (void)match_value; - (void)num_actions; - (void)actions; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_action_counter(void *counter_obj, uint32_t offset) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_action_create_flow_counter(counter_obj, offset); -#else - struct mlx5dv_flow_action_attr *action; - - (void)offset; - action = malloc(sizeof(*action)); - if (!action) - return NULL; - action->type = MLX5DV_FLOW_ACTION_COUNTERS_DEVX; - action->obj = counter_obj; - return action; -#endif -#else - (void)counter_obj; - (void)offset; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_action_dest_ibv_qp(void *qp) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_action_create_dest_ibv_qp(qp); -#else - struct mlx5dv_flow_action_attr *action; - - action = malloc(sizeof(*action)); - if (!action) - return NULL; - action->type = MLX5DV_FLOW_ACTION_DEST_IBV_QP; - action->obj = qp; - return action; -#endif -#else - (void)qp; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_action_dest_devx_tir(void *tir) -{ -#ifdef HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR - return mlx5dv_dr_action_create_dest_devx_tir(tir); -#else - (void)tir; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_action_modify_header - (struct ibv_context *ctx, - enum mlx5dv_flow_table_type ft_type, - void *domain, uint64_t flags, - size_t actions_sz, - uint64_t actions[]) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - (void)ctx; - (void)ft_type; - return mlx5dv_dr_action_create_modify_header(domain, flags, actions_sz, - (__be64 *)actions); -#else - struct mlx5dv_flow_action_attr *action; - - (void)domain; - (void)flags; - action = malloc(sizeof(*action)); - if (!action) - return NULL; - action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - action->action = mlx5dv_create_flow_action_modify_header - (ctx, actions_sz, actions, ft_type); - return action; -#endif -#else - (void)ctx; - (void)ft_type; - (void)domain; - (void)flags; - (void)actions_sz; - (void)actions; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_action_packet_reformat - (struct ibv_context *ctx, - enum mlx5dv_flow_action_packet_reformat_type reformat_type, - enum mlx5dv_flow_table_type ft_type, - struct mlx5dv_dr_domain *domain, - uint32_t flags, size_t data_sz, void *data) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - (void)ctx; - (void)ft_type; - return mlx5dv_dr_action_create_packet_reformat(domain, flags, - reformat_type, data_sz, - data); -#else - (void)domain; - (void)flags; - struct mlx5dv_flow_action_attr *action; - - action = malloc(sizeof(*action)); - if (!action) - return NULL; - action->type = MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION; - action->action = mlx5dv_create_flow_action_packet_reformat - (ctx, data_sz, data, reformat_type, ft_type); - return action; -#endif -#else - (void)ctx; - (void)reformat_type; - (void)ft_type; - (void)domain; - (void)flags; - (void)data_sz; - (void)data; - errno = ENOTSUP; - return NULL; -#endif -} - -static void * -mlx5_glue_dv_create_flow_action_tag(uint32_t tag) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_action_create_tag(tag); -#else - struct mlx5dv_flow_action_attr *action; - action = malloc(sizeof(*action)); - if (!action) - return NULL; - action->type = MLX5DV_FLOW_ACTION_TAG; - action->tag_value = tag; - return action; -#endif -#endif - (void)tag; - errno = ENOTSUP; - return NULL; -} - -static void * -mlx5_glue_dv_create_flow_action_meter(struct mlx5dv_dr_flow_meter_attr *attr) -{ -#if defined(HAVE_MLX5DV_DR) && defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) - return mlx5dv_dr_action_create_flow_meter(attr); -#else - (void)attr; - errno = ENOTSUP; - return NULL; -#endif -} - -static int -mlx5_glue_dv_modify_flow_action_meter(void *action, - struct mlx5dv_dr_flow_meter_attr *attr, - uint64_t modify_bits) -{ -#if defined(HAVE_MLX5DV_DR) && defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) - return mlx5dv_dr_action_modify_flow_meter(action, attr, modify_bits); -#else - (void)action; - (void)attr; - (void)modify_bits; - errno = ENOTSUP; - return errno; -#endif -} - -static int -mlx5_glue_dv_destroy_flow(void *flow_id) -{ -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_rule_destroy(flow_id); -#else - return ibv_destroy_flow(flow_id); -#endif -} - -static int -mlx5_glue_dv_destroy_flow_matcher(void *matcher) -{ -#ifdef HAVE_IBV_FLOW_DV_SUPPORT -#ifdef HAVE_MLX5DV_DR - return mlx5dv_dr_matcher_destroy(matcher); -#else - return mlx5dv_destroy_flow_matcher(matcher); -#endif -#else - (void)matcher; - errno = ENOTSUP; - return errno; -#endif -} - -static struct ibv_context * -mlx5_glue_dv_open_device(struct ibv_device *device) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_open_device(device, - &(struct mlx5dv_context_attr){ - .flags = MLX5DV_CONTEXT_FLAGS_DEVX, - }); -#else - (void)device; - errno = ENOTSUP; - return NULL; -#endif -} - -static struct mlx5dv_devx_obj * -mlx5_glue_devx_obj_create(struct ibv_context *ctx, - const void *in, size_t inlen, - void *out, size_t outlen) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_obj_create(ctx, in, inlen, out, outlen); -#else - (void)ctx; - (void)in; - (void)inlen; - (void)out; - (void)outlen; - errno = ENOTSUP; - return NULL; -#endif -} - -static int -mlx5_glue_devx_obj_destroy(struct mlx5dv_devx_obj *obj) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_obj_destroy(obj); -#else - (void)obj; - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_obj_query(struct mlx5dv_devx_obj *obj, - const void *in, size_t inlen, - void *out, size_t outlen) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_obj_query(obj, in, inlen, out, outlen); -#else - (void)obj; - (void)in; - (void)inlen; - (void)out; - (void)outlen; - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_obj_modify(struct mlx5dv_devx_obj *obj, - const void *in, size_t inlen, - void *out, size_t outlen) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_obj_modify(obj, in, inlen, out, outlen); -#else - (void)obj; - (void)in; - (void)inlen; - (void)out; - (void)outlen; - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_general_cmd(struct ibv_context *ctx, - const void *in, size_t inlen, - void *out, size_t outlen) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_general_cmd(ctx, in, inlen, out, outlen); -#else - (void)ctx; - (void)in; - (void)inlen; - (void)out; - (void)outlen; - return -ENOTSUP; -#endif -} - -static struct mlx5dv_devx_cmd_comp * -mlx5_glue_devx_create_cmd_comp(struct ibv_context *ctx) -{ -#ifdef HAVE_IBV_DEVX_ASYNC - return mlx5dv_devx_create_cmd_comp(ctx); -#else - (void)ctx; - errno = -ENOTSUP; - return NULL; -#endif -} - -static void -mlx5_glue_devx_destroy_cmd_comp(struct mlx5dv_devx_cmd_comp *cmd_comp) -{ -#ifdef HAVE_IBV_DEVX_ASYNC - mlx5dv_devx_destroy_cmd_comp(cmd_comp); -#else - (void)cmd_comp; - errno = -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_obj_query_async(struct mlx5dv_devx_obj *obj, const void *in, - size_t inlen, size_t outlen, uint64_t wr_id, - struct mlx5dv_devx_cmd_comp *cmd_comp) -{ -#ifdef HAVE_IBV_DEVX_ASYNC - return mlx5dv_devx_obj_query_async(obj, in, inlen, outlen, wr_id, - cmd_comp); -#else - (void)obj; - (void)in; - (void)inlen; - (void)outlen; - (void)wr_id; - (void)cmd_comp; - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_get_async_cmd_comp(struct mlx5dv_devx_cmd_comp *cmd_comp, - struct mlx5dv_devx_async_cmd_hdr *cmd_resp, - size_t cmd_resp_len) -{ -#ifdef HAVE_IBV_DEVX_ASYNC - return mlx5dv_devx_get_async_cmd_comp(cmd_comp, cmd_resp, - cmd_resp_len); -#else - (void)cmd_comp; - (void)cmd_resp; - (void)cmd_resp_len; - return -ENOTSUP; -#endif -} - -static struct mlx5dv_devx_umem * -mlx5_glue_devx_umem_reg(struct ibv_context *context, void *addr, size_t size, - uint32_t access) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_umem_reg(context, addr, size, access); -#else - (void)context; - (void)addr; - (void)size; - (void)access; - errno = -ENOTSUP; - return NULL; -#endif -} - -static int -mlx5_glue_devx_umem_dereg(struct mlx5dv_devx_umem *dv_devx_umem) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_umem_dereg(dv_devx_umem); -#else - (void)dv_devx_umem; - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_qp_query(struct ibv_qp *qp, - const void *in, size_t inlen, - void *out, size_t outlen) -{ -#ifdef HAVE_IBV_DEVX_QP - return mlx5dv_devx_qp_query(qp, in, inlen, out, outlen); -#else - (void)qp; - (void)in; - (void)inlen; - (void)out; - (void)outlen; - errno = ENOTSUP; - return errno; -#endif -} - -static int -mlx5_glue_devx_port_query(struct ibv_context *ctx, - uint32_t port_num, - struct mlx5dv_devx_port *mlx5_devx_port) -{ -#ifdef HAVE_MLX5DV_DR_DEVX_PORT - return mlx5dv_query_devx_port(ctx, port_num, mlx5_devx_port); -#else - (void)ctx; - (void)port_num; - (void)mlx5_devx_port; - errno = ENOTSUP; - return errno; -#endif -} - -static int -mlx5_glue_dr_dump_domain(FILE *file, void *domain) -{ -#ifdef HAVE_MLX5_DR_FLOW_DUMP - return mlx5dv_dump_dr_domain(file, domain); -#else - RTE_SET_USED(file); - RTE_SET_USED(domain); - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_query_eqn(struct ibv_context *ctx, uint32_t cpus, - uint32_t *eqn) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_query_eqn(ctx, cpus, eqn); -#else - (void)ctx; - (void)cpus; - (void)eqn; - return -ENOTSUP; -#endif -} - -static struct mlx5dv_devx_event_channel * -mlx5_glue_devx_create_event_channel(struct ibv_context *ctx, int flags) -{ -#ifdef HAVE_IBV_DEVX_EVENT - return mlx5dv_devx_create_event_channel(ctx, flags); -#else - (void)ctx; - (void)flags; - errno = ENOTSUP; - return NULL; -#endif -} - -static void -mlx5_glue_devx_destroy_event_channel(struct mlx5dv_devx_event_channel *eventc) -{ -#ifdef HAVE_IBV_DEVX_EVENT - mlx5dv_devx_destroy_event_channel(eventc); -#else - (void)eventc; -#endif -} - -static int -mlx5_glue_devx_subscribe_devx_event(struct mlx5dv_devx_event_channel *eventc, - struct mlx5dv_devx_obj *obj, - uint16_t events_sz, uint16_t events_num[], - uint64_t cookie) -{ -#ifdef HAVE_IBV_DEVX_EVENT - return mlx5dv_devx_subscribe_devx_event(eventc, obj, events_sz, - events_num, cookie); -#else - (void)eventc; - (void)obj; - (void)events_sz; - (void)events_num; - (void)cookie; - return -ENOTSUP; -#endif -} - -static int -mlx5_glue_devx_subscribe_devx_event_fd(struct mlx5dv_devx_event_channel *eventc, - int fd, struct mlx5dv_devx_obj *obj, - uint16_t event_num) -{ -#ifdef HAVE_IBV_DEVX_EVENT - return mlx5dv_devx_subscribe_devx_event_fd(eventc, fd, obj, event_num); -#else - (void)eventc; - (void)fd; - (void)obj; - (void)event_num; - return -ENOTSUP; -#endif -} - -static ssize_t -mlx5_glue_devx_get_event(struct mlx5dv_devx_event_channel *eventc, - struct mlx5dv_devx_async_event_hdr *event_data, - size_t event_resp_len) -{ -#ifdef HAVE_IBV_DEVX_EVENT - return mlx5dv_devx_get_event(eventc, event_data, event_resp_len); -#else - (void)eventc; - (void)event_data; - (void)event_resp_len; - errno = ENOTSUP; - return -1; -#endif -} - -static struct mlx5dv_devx_uar * -mlx5_glue_devx_alloc_uar(struct ibv_context *context, uint32_t flags) -{ -#ifdef HAVE_IBV_DEVX_OBJ - return mlx5dv_devx_alloc_uar(context, flags); -#else - (void)context; - (void)flags; - errno = ENOTSUP; - return NULL; -#endif -} - -static void -mlx5_glue_devx_free_uar(struct mlx5dv_devx_uar *devx_uar) -{ -#ifdef HAVE_IBV_DEVX_OBJ - mlx5dv_devx_free_uar(devx_uar); -#else - (void)devx_uar; -#endif -} - -static struct mlx5dv_var * -mlx5_glue_dv_alloc_var(struct ibv_context *context, uint32_t flags) -{ -#ifdef HAVE_IBV_VAR - return mlx5dv_alloc_var(context, flags); -#else - (void)context; - (void)flags; - errno = ENOTSUP; - return NULL; -#endif -} - -static void -mlx5_glue_dv_free_var(struct mlx5dv_var *var) -{ -#ifdef HAVE_IBV_VAR - mlx5dv_free_var(var); -#else - (void)var; - errno = ENOTSUP; -#endif -} - -__rte_cache_aligned -const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { - .version = MLX5_GLUE_VERSION, - .fork_init = mlx5_glue_fork_init, - .alloc_pd = mlx5_glue_alloc_pd, - .dealloc_pd = mlx5_glue_dealloc_pd, - .get_device_list = mlx5_glue_get_device_list, - .free_device_list = mlx5_glue_free_device_list, - .open_device = mlx5_glue_open_device, - .close_device = mlx5_glue_close_device, - .query_device = mlx5_glue_query_device, - .query_device_ex = mlx5_glue_query_device_ex, - .query_rt_values_ex = mlx5_glue_query_rt_values_ex, - .query_port = mlx5_glue_query_port, - .create_comp_channel = mlx5_glue_create_comp_channel, - .destroy_comp_channel = mlx5_glue_destroy_comp_channel, - .create_cq = mlx5_glue_create_cq, - .destroy_cq = mlx5_glue_destroy_cq, - .get_cq_event = mlx5_glue_get_cq_event, - .ack_cq_events = mlx5_glue_ack_cq_events, - .create_rwq_ind_table = mlx5_glue_create_rwq_ind_table, - .destroy_rwq_ind_table = mlx5_glue_destroy_rwq_ind_table, - .create_wq = mlx5_glue_create_wq, - .destroy_wq = mlx5_glue_destroy_wq, - .modify_wq = mlx5_glue_modify_wq, - .create_flow = mlx5_glue_create_flow, - .destroy_flow = mlx5_glue_destroy_flow, - .destroy_flow_action = mlx5_glue_destroy_flow_action, - .create_qp = mlx5_glue_create_qp, - .create_qp_ex = mlx5_glue_create_qp_ex, - .destroy_qp = mlx5_glue_destroy_qp, - .modify_qp = mlx5_glue_modify_qp, - .reg_mr = mlx5_glue_reg_mr, - .alloc_null_mr = mlx5_glue_alloc_null_mr, - .dereg_mr = mlx5_glue_dereg_mr, - .create_counter_set = mlx5_glue_create_counter_set, - .destroy_counter_set = mlx5_glue_destroy_counter_set, - .describe_counter_set = mlx5_glue_describe_counter_set, - .query_counter_set = mlx5_glue_query_counter_set, - .create_counters = mlx5_glue_create_counters, - .destroy_counters = mlx5_glue_destroy_counters, - .attach_counters = mlx5_glue_attach_counters, - .query_counters = mlx5_glue_query_counters, - .ack_async_event = mlx5_glue_ack_async_event, - .get_async_event = mlx5_glue_get_async_event, - .port_state_str = mlx5_glue_port_state_str, - .cq_ex_to_cq = mlx5_glue_cq_ex_to_cq, - .dr_create_flow_action_dest_flow_tbl = - mlx5_glue_dr_create_flow_action_dest_flow_tbl, - .dr_create_flow_action_dest_port = - mlx5_glue_dr_create_flow_action_dest_port, - .dr_create_flow_action_drop = - mlx5_glue_dr_create_flow_action_drop, - .dr_create_flow_action_push_vlan = - mlx5_glue_dr_create_flow_action_push_vlan, - .dr_create_flow_action_pop_vlan = - mlx5_glue_dr_create_flow_action_pop_vlan, - .dr_create_flow_tbl = mlx5_glue_dr_create_flow_tbl, - .dr_destroy_flow_tbl = mlx5_glue_dr_destroy_flow_tbl, - .dr_create_domain = mlx5_glue_dr_create_domain, - .dr_destroy_domain = mlx5_glue_dr_destroy_domain, - .dv_create_cq = mlx5_glue_dv_create_cq, - .dv_create_wq = mlx5_glue_dv_create_wq, - .dv_query_device = mlx5_glue_dv_query_device, - .dv_set_context_attr = mlx5_glue_dv_set_context_attr, - .dv_init_obj = mlx5_glue_dv_init_obj, - .dv_create_qp = mlx5_glue_dv_create_qp, - .dv_create_flow_matcher = mlx5_glue_dv_create_flow_matcher, - .dv_create_flow = mlx5_glue_dv_create_flow, - .dv_create_flow_action_counter = - mlx5_glue_dv_create_flow_action_counter, - .dv_create_flow_action_dest_ibv_qp = - mlx5_glue_dv_create_flow_action_dest_ibv_qp, - .dv_create_flow_action_dest_devx_tir = - mlx5_glue_dv_create_flow_action_dest_devx_tir, - .dv_create_flow_action_modify_header = - mlx5_glue_dv_create_flow_action_modify_header, - .dv_create_flow_action_packet_reformat = - mlx5_glue_dv_create_flow_action_packet_reformat, - .dv_create_flow_action_tag = mlx5_glue_dv_create_flow_action_tag, - .dv_create_flow_action_meter = mlx5_glue_dv_create_flow_action_meter, - .dv_modify_flow_action_meter = mlx5_glue_dv_modify_flow_action_meter, - .dv_destroy_flow = mlx5_glue_dv_destroy_flow, - .dv_destroy_flow_matcher = mlx5_glue_dv_destroy_flow_matcher, - .dv_open_device = mlx5_glue_dv_open_device, - .devx_obj_create = mlx5_glue_devx_obj_create, - .devx_obj_destroy = mlx5_glue_devx_obj_destroy, - .devx_obj_query = mlx5_glue_devx_obj_query, - .devx_obj_modify = mlx5_glue_devx_obj_modify, - .devx_general_cmd = mlx5_glue_devx_general_cmd, - .devx_create_cmd_comp = mlx5_glue_devx_create_cmd_comp, - .devx_destroy_cmd_comp = mlx5_glue_devx_destroy_cmd_comp, - .devx_obj_query_async = mlx5_glue_devx_obj_query_async, - .devx_get_async_cmd_comp = mlx5_glue_devx_get_async_cmd_comp, - .devx_umem_reg = mlx5_glue_devx_umem_reg, - .devx_umem_dereg = mlx5_glue_devx_umem_dereg, - .devx_qp_query = mlx5_glue_devx_qp_query, - .devx_port_query = mlx5_glue_devx_port_query, - .dr_dump_domain = mlx5_glue_dr_dump_domain, - .devx_query_eqn = mlx5_glue_devx_query_eqn, - .devx_create_event_channel = mlx5_glue_devx_create_event_channel, - .devx_destroy_event_channel = mlx5_glue_devx_destroy_event_channel, - .devx_subscribe_devx_event = mlx5_glue_devx_subscribe_devx_event, - .devx_subscribe_devx_event_fd = mlx5_glue_devx_subscribe_devx_event_fd, - .devx_get_event = mlx5_glue_devx_get_event, - .devx_alloc_uar = mlx5_glue_devx_alloc_uar, - .devx_free_uar = mlx5_glue_devx_free_uar, - .dv_alloc_var = mlx5_glue_dv_alloc_var, - .dv_free_var = mlx5_glue_dv_free_var, -}; diff --git a/drivers/common/mlx5/mlx5_glue.h b/drivers/common/mlx5/mlx5_glue.h deleted file mode 100644 index 81d6a22..0000000 --- a/drivers/common/mlx5/mlx5_glue.h +++ /dev/null @@ -1,309 +0,0 @@ -/* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2018 6WIND S.A. - * Copyright 2018 Mellanox Technologies, Ltd - */ - -#ifndef MLX5_GLUE_H_ -#define MLX5_GLUE_H_ - -#include -#include -/* Verbs headers do not support -pedantic. */ -#ifdef PEDANTIC -#pragma GCC diagnostic ignored "-Wpedantic" -#endif -#include -#include -#ifdef PEDANTIC -#pragma GCC diagnostic error "-Wpedantic" -#endif - -#include - -#include "mlx5_autoconf.h" - -#ifndef MLX5_GLUE_VERSION -#define MLX5_GLUE_VERSION "" -#endif - -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V42 -struct ibv_counter_set; -struct ibv_counter_set_data; -struct ibv_counter_set_description; -struct ibv_counter_set_init_attr; -struct ibv_query_counter_set_attr; -#endif - -#ifndef HAVE_IBV_DEVICE_COUNTERS_SET_V45 -struct ibv_counters; -struct ibv_counters_init_attr; -struct ibv_counter_attach_attr; -#endif - -#ifndef HAVE_IBV_DEVICE_TUNNEL_SUPPORT -struct mlx5dv_qp_init_attr; -#endif - -#ifndef HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT -struct mlx5dv_wq_init_attr; -#endif - -#ifndef HAVE_IBV_FLOW_DV_SUPPORT -struct mlx5dv_flow_matcher; -struct mlx5dv_flow_matcher_attr; -struct mlx5dv_flow_action_attr; -struct mlx5dv_flow_match_parameters; -struct mlx5dv_dr_flow_meter_attr; -struct ibv_flow_action; -enum mlx5dv_flow_action_packet_reformat_type { packet_reformat_type = 0, }; -enum mlx5dv_flow_table_type { flow_table_type = 0, }; -#endif - -#ifndef HAVE_IBV_FLOW_DEVX_COUNTERS -#define MLX5DV_FLOW_ACTION_COUNTERS_DEVX 0 -#endif - -#ifndef HAVE_IBV_DEVX_OBJ -struct mlx5dv_devx_obj; -struct mlx5dv_devx_umem { uint32_t umem_id; }; -struct mlx5dv_devx_uar { void *reg_addr; void *base_addr; uint32_t page_id; }; -#endif - -#ifndef HAVE_IBV_DEVX_ASYNC -struct mlx5dv_devx_cmd_comp; -struct mlx5dv_devx_async_cmd_hdr; -#endif - -#ifndef HAVE_MLX5DV_DR -enum mlx5dv_dr_domain_type { unused, }; -struct mlx5dv_dr_domain; -#endif - -#ifndef HAVE_MLX5DV_DR_DEVX_PORT -struct mlx5dv_devx_port; -#endif - -#ifndef HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER -struct mlx5dv_dr_flow_meter_attr; -#endif - -#ifndef HAVE_IBV_DEVX_EVENT -struct mlx5dv_devx_event_channel { int fd; }; -struct mlx5dv_devx_async_event_hdr; -#define MLX5DV_DEVX_CREATE_EVENT_CHANNEL_FLAGS_OMIT_EV_DATA 1 -#endif - -#ifndef HAVE_IBV_VAR -struct mlx5dv_var { uint32_t page_id; uint32_t length; off_t mmap_off; - uint64_t comp_mask; }; -#endif - -#ifndef HAVE_IBV_RELAXED_ORDERING -#define IBV_ACCESS_RELAXED_ORDERING 0 -#endif - -/* LIB_GLUE_VERSION must be updated every time this structure is modified. */ -struct mlx5_glue { - const char *version; - int (*fork_init)(void); - struct ibv_pd *(*alloc_pd)(struct ibv_context *context); - int (*dealloc_pd)(struct ibv_pd *pd); - struct ibv_device **(*get_device_list)(int *num_devices); - void (*free_device_list)(struct ibv_device **list); - struct ibv_context *(*open_device)(struct ibv_device *device); - int (*close_device)(struct ibv_context *context); - int (*query_device)(struct ibv_context *context, - struct ibv_device_attr *device_attr); - int (*query_device_ex)(struct ibv_context *context, - const struct ibv_query_device_ex_input *input, - struct ibv_device_attr_ex *attr); - int (*query_rt_values_ex)(struct ibv_context *context, - struct ibv_values_ex *values); - int (*query_port)(struct ibv_context *context, uint8_t port_num, - struct ibv_port_attr *port_attr); - struct ibv_comp_channel *(*create_comp_channel) - (struct ibv_context *context); - int (*destroy_comp_channel)(struct ibv_comp_channel *channel); - struct ibv_cq *(*create_cq)(struct ibv_context *context, int cqe, - void *cq_context, - struct ibv_comp_channel *channel, - int comp_vector); - int (*destroy_cq)(struct ibv_cq *cq); - int (*get_cq_event)(struct ibv_comp_channel *channel, - struct ibv_cq **cq, void **cq_context); - void (*ack_cq_events)(struct ibv_cq *cq, unsigned int nevents); - struct ibv_rwq_ind_table *(*create_rwq_ind_table) - (struct ibv_context *context, - struct ibv_rwq_ind_table_init_attr *init_attr); - int (*destroy_rwq_ind_table)(struct ibv_rwq_ind_table *rwq_ind_table); - struct ibv_wq *(*create_wq)(struct ibv_context *context, - struct ibv_wq_init_attr *wq_init_attr); - int (*destroy_wq)(struct ibv_wq *wq); - int (*modify_wq)(struct ibv_wq *wq, struct ibv_wq_attr *wq_attr); - struct ibv_flow *(*create_flow)(struct ibv_qp *qp, - struct ibv_flow_attr *flow); - int (*destroy_flow)(struct ibv_flow *flow_id); - int (*destroy_flow_action)(void *action); - struct ibv_qp *(*create_qp)(struct ibv_pd *pd, - struct ibv_qp_init_attr *qp_init_attr); - struct ibv_qp *(*create_qp_ex) - (struct ibv_context *context, - struct ibv_qp_init_attr_ex *qp_init_attr_ex); - int (*destroy_qp)(struct ibv_qp *qp); - int (*modify_qp)(struct ibv_qp *qp, struct ibv_qp_attr *attr, - int attr_mask); - struct ibv_mr *(*reg_mr)(struct ibv_pd *pd, void *addr, - size_t length, int access); - struct ibv_mr *(*alloc_null_mr)(struct ibv_pd *pd); - int (*dereg_mr)(struct ibv_mr *mr); - struct ibv_counter_set *(*create_counter_set) - (struct ibv_context *context, - struct ibv_counter_set_init_attr *init_attr); - int (*destroy_counter_set)(struct ibv_counter_set *cs); - int (*describe_counter_set) - (struct ibv_context *context, - uint16_t counter_set_id, - struct ibv_counter_set_description *cs_desc); - int (*query_counter_set)(struct ibv_query_counter_set_attr *query_attr, - struct ibv_counter_set_data *cs_data); - struct ibv_counters *(*create_counters) - (struct ibv_context *context, - struct ibv_counters_init_attr *init_attr); - int (*destroy_counters)(struct ibv_counters *counters); - int (*attach_counters)(struct ibv_counters *counters, - struct ibv_counter_attach_attr *attr, - struct ibv_flow *flow); - int (*query_counters)(struct ibv_counters *counters, - uint64_t *counters_value, - uint32_t ncounters, - uint32_t flags); - void (*ack_async_event)(struct ibv_async_event *event); - int (*get_async_event)(struct ibv_context *context, - struct ibv_async_event *event); - const char *(*port_state_str)(enum ibv_port_state port_state); - struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq); - void *(*dr_create_flow_action_dest_flow_tbl)(void *tbl); - void *(*dr_create_flow_action_dest_port)(void *domain, - uint32_t port); - void *(*dr_create_flow_action_drop)(); - void *(*dr_create_flow_action_push_vlan) - (struct mlx5dv_dr_domain *domain, - rte_be32_t vlan_tag); - void *(*dr_create_flow_action_pop_vlan)(); - void *(*dr_create_flow_tbl)(void *domain, uint32_t level); - int (*dr_destroy_flow_tbl)(void *tbl); - void *(*dr_create_domain)(struct ibv_context *ctx, - enum mlx5dv_dr_domain_type domain); - int (*dr_destroy_domain)(void *domain); - struct ibv_cq_ex *(*dv_create_cq) - (struct ibv_context *context, - struct ibv_cq_init_attr_ex *cq_attr, - struct mlx5dv_cq_init_attr *mlx5_cq_attr); - struct ibv_wq *(*dv_create_wq) - (struct ibv_context *context, - struct ibv_wq_init_attr *wq_attr, - struct mlx5dv_wq_init_attr *mlx5_wq_attr); - int (*dv_query_device)(struct ibv_context *ctx_in, - struct mlx5dv_context *attrs_out); - int (*dv_set_context_attr)(struct ibv_context *ibv_ctx, - enum mlx5dv_set_ctx_attr_type type, - void *attr); - int (*dv_init_obj)(struct mlx5dv_obj *obj, uint64_t obj_type); - struct ibv_qp *(*dv_create_qp) - (struct ibv_context *context, - struct ibv_qp_init_attr_ex *qp_init_attr_ex, - struct mlx5dv_qp_init_attr *dv_qp_init_attr); - void *(*dv_create_flow_matcher) - (struct ibv_context *context, - struct mlx5dv_flow_matcher_attr *matcher_attr, - void *tbl); - void *(*dv_create_flow)(void *matcher, void *match_value, - size_t num_actions, void *actions[]); - void *(*dv_create_flow_action_counter)(void *obj, uint32_t offset); - void *(*dv_create_flow_action_dest_ibv_qp)(void *qp); - void *(*dv_create_flow_action_dest_devx_tir)(void *tir); - void *(*dv_create_flow_action_modify_header) - (struct ibv_context *ctx, enum mlx5dv_flow_table_type ft_type, - void *domain, uint64_t flags, size_t actions_sz, - uint64_t actions[]); - void *(*dv_create_flow_action_packet_reformat) - (struct ibv_context *ctx, - enum mlx5dv_flow_action_packet_reformat_type reformat_type, - enum mlx5dv_flow_table_type ft_type, - struct mlx5dv_dr_domain *domain, - uint32_t flags, size_t data_sz, void *data); - void *(*dv_create_flow_action_tag)(uint32_t tag); - void *(*dv_create_flow_action_meter) - (struct mlx5dv_dr_flow_meter_attr *attr); - int (*dv_modify_flow_action_meter)(void *action, - struct mlx5dv_dr_flow_meter_attr *attr, uint64_t modify_bits); - int (*dv_destroy_flow)(void *flow); - int (*dv_destroy_flow_matcher)(void *matcher); - struct ibv_context *(*dv_open_device)(struct ibv_device *device); - struct mlx5dv_var *(*dv_alloc_var)(struct ibv_context *context, - uint32_t flags); - void (*dv_free_var)(struct mlx5dv_var *var); - struct mlx5dv_devx_uar *(*devx_alloc_uar)(struct ibv_context *context, - uint32_t flags); - void (*devx_free_uar)(struct mlx5dv_devx_uar *devx_uar); - struct mlx5dv_devx_obj *(*devx_obj_create) - (struct ibv_context *ctx, - const void *in, size_t inlen, - void *out, size_t outlen); - int (*devx_obj_destroy)(struct mlx5dv_devx_obj *obj); - int (*devx_obj_query)(struct mlx5dv_devx_obj *obj, - const void *in, size_t inlen, - void *out, size_t outlen); - int (*devx_obj_modify)(struct mlx5dv_devx_obj *obj, - const void *in, size_t inlen, - void *out, size_t outlen); - int (*devx_general_cmd)(struct ibv_context *context, - const void *in, size_t inlen, - void *out, size_t outlen); - struct mlx5dv_devx_cmd_comp *(*devx_create_cmd_comp) - (struct ibv_context *context); - void (*devx_destroy_cmd_comp)(struct mlx5dv_devx_cmd_comp *cmd_comp); - int (*devx_obj_query_async)(struct mlx5dv_devx_obj *obj, - const void *in, size_t inlen, - size_t outlen, uint64_t wr_id, - struct mlx5dv_devx_cmd_comp *cmd_comp); - int (*devx_get_async_cmd_comp)(struct mlx5dv_devx_cmd_comp *cmd_comp, - struct mlx5dv_devx_async_cmd_hdr *resp, - size_t cmd_resp_len); - struct mlx5dv_devx_umem *(*devx_umem_reg)(struct ibv_context *context, - void *addr, size_t size, - uint32_t access); - int (*devx_umem_dereg)(struct mlx5dv_devx_umem *dv_devx_umem); - int (*devx_qp_query)(struct ibv_qp *qp, - const void *in, size_t inlen, - void *out, size_t outlen); - int (*devx_port_query)(struct ibv_context *ctx, - uint32_t port_num, - struct mlx5dv_devx_port *mlx5_devx_port); - int (*dr_dump_domain)(FILE *file, void *domain); - int (*devx_query_eqn)(struct ibv_context *context, uint32_t cpus, - uint32_t *eqn); - struct mlx5dv_devx_event_channel *(*devx_create_event_channel) - (struct ibv_context *context, int flags); - void (*devx_destroy_event_channel) - (struct mlx5dv_devx_event_channel *event_channel); - int (*devx_subscribe_devx_event) - (struct mlx5dv_devx_event_channel *event_channel, - struct mlx5dv_devx_obj *obj, - uint16_t events_sz, - uint16_t events_num[], - uint64_t cookie); - int (*devx_subscribe_devx_event_fd) - (struct mlx5dv_devx_event_channel *event_channel, - int fd, - struct mlx5dv_devx_obj *obj, - uint16_t event_num); - ssize_t (*devx_get_event) - (struct mlx5dv_devx_event_channel *event_channel, - struct mlx5dv_devx_async_event_hdr *event_data, - size_t event_resp_len); -}; - -extern const struct mlx5_glue *mlx5_glue; - -#endif /* MLX5_GLUE_H_ */ diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index 2577ee5..c160e6b 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -38,6 +38,7 @@ CFLAGS += -O3 CFLAGS += -std=c11 -Wall -Wextra CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux CFLAGS += -I$(RTE_SDK)/drivers/net/mlx5 CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 CFLAGS += -D_BSD_SOURCE diff --git a/drivers/vdpa/mlx5/Makefile b/drivers/vdpa/mlx5/Makefile index ef34c0b..91c89d6 100644 --- a/drivers/vdpa/mlx5/Makefile +++ b/drivers/vdpa/mlx5/Makefile @@ -20,6 +20,7 @@ CFLAGS += -O3 CFLAGS += -std=c11 -Wall -Wextra CFLAGS += -g CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5 +CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux CFLAGS += -I$(RTE_SDK)/drivers/net/mlx5_vdpa CFLAGS += -I$(RTE_SDK)/lib/librte_sched CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5 -- 2.8.4