From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A9FAEA034C; Fri, 25 Feb 2022 00:25:32 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C62BA41156; Fri, 25 Feb 2022 00:25:28 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2073.outbound.protection.outlook.com [40.107.92.73]) by mails.dpdk.org (Postfix) with ESMTP id 8A6CE4114D for ; Fri, 25 Feb 2022 00:25:26 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OICNAdmAojGI1omLaCt6CDO3TAilyQL7qVTTGLJE1bG+K1fO9A5UrWN94MbC8eLj8ZC6jRVZzNWyLDvynqAZWryA6dXRVYxQ6RVnICoiqLSVTTNZY3AoKucdCTEMmStTIh+9eumeTrkM1Tz4/E3OM1+w8/QR64tUqTUSg0kQf6eIP3lfJBQVpT9Ih6GV1T0yQw4rpThVRuQh3JXe2yNI/+wIK5cMyQCoAYuqdpbwrxP6oFeOXzcN577qEmr+j7WBmNaQOt+1NqKcSDoFJ9u9qOBd0RwCXpNBVoyXEtZmxcg4kVvaPn7UJQWjUNzt0MImKoAl3XKRF1IvSS7qTDTVmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+1e1z+OTXqc9CG2yg/EQg+96B9tJxOE/Xahr0R1Ij0E=; b=V5r/pk91D/XBFz/OoKHNxuqtxtyliKINlMqybI9/WEBzGjWHnX2sfkyvC893KIJu+fpGYPbmDMZz/C3njAMtGZ3KHOZS54nHQovEss+RAWIGStZB4NbsMWxBdf4+31Tn+ybyDC6d3pSqChv+xjemjDlT0eHjf1sdg0xdtPpIHhiErVrb0bR4P/252ZWq5QS7QJVMteYoGHiW8Yb/WQcq2dvcoQihnUmbyk8Y1sji0Z9cJQqMrbjcpyeNWO91/Jkjxqhqo4yTq6eBj/wEjoIoTLQDdQWA3jPsF5iR9b+9MQV3FjWQGgriPd5ksfTKrXfArMPvU70hy1HL2cqWXMaJUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+1e1z+OTXqc9CG2yg/EQg+96B9tJxOE/Xahr0R1Ij0E=; b=pqML/5wkSEO2oaBkXhaFHCjlY+C4374gSQlqKKc7HoqZR0mGXoQci956G5pUuu4IH12xpxtAPLmzs/TmbYxOfdDgpGBoe5y9c0yZQRk7YHjYfA8NudtIjUHLjtQGsroOtafL3IbHRNvZnX91s6dwf5Zp56YhzEx9B49MuemklUV7tpq9yv7fdtkZJA4e8seCHGKg0VXg0B97M12mGqjIrbTmu1sUg+ROrAI1owYOtYn22ue7TUx7EFI+JlXu/1EHShea5x8k+qEUR7NBeZRGRnubaZq3K4wQekkvh9dyc2F4tr0l7rCH1yEp1GvJ6EkYG+uvANF2Ujl5i/1Rc66i+Q== Received: from MWHPR15CA0025.namprd15.prod.outlook.com (2603:10b6:300:ad::11) by MWHPR12MB1680.namprd12.prod.outlook.com (2603:10b6:301:10::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.24; Thu, 24 Feb 2022 23:25:22 +0000 Received: from CO1NAM11FT026.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ad:cafe::ca) by MWHPR15CA0025.outlook.office365.com (2603:10b6:300:ad::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.17 via Frontend Transport; Thu, 24 Feb 2022 23:25:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT026.mail.protection.outlook.com (10.13.175.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Thu, 24 Feb 2022 23:25:21 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Thu, 24 Feb 2022 23:25:21 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Thu, 24 Feb 2022 15:25:20 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9 via Frontend Transport; Thu, 24 Feb 2022 15:25:19 -0800 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Subject: [PATCH v3 2/6] common/mlx5: glue device and PD importation Date: Fri, 25 Feb 2022 01:25:07 +0200 Message-ID: <20220224232511.3238707-3-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220224232511.3238707-1-michaelba@nvidia.com> References: <20220223184835.3061161-1-michaelba@nvidia.com> <20220224232511.3238707-1-michaelba@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 69d57bff-5cf4-40ad-08f7-08d9f7eceda2 X-MS-TrafficTypeDiagnostic: MWHPR12MB1680:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QvqqVH8jKHMo+GfCivZSIQcAB/9FZZEYx1H6GsQQubuhiMWhyvKdeLCHSSccc2+wDBHP/u2FYKpG31uwov6N7tw+HwDmCL0XpKDydpNcoLoeJZbIoZu2TZ0tY/6eqOyxau8J2aVGXCwneETiFMZID4mSza1Tx9u9gYczKd+Jjcnw9Dhqd1iase/H9IT6DiPSgrfO6kNwAi/XuV0Yasg/AggXWfNn0YvQaVvfuqTmjD62N4QSsGZ9g6AVLqE93OKX7o+JcnOulsApH4QjOYgAbx/GVy2j2cd5zjLQUfC5HQg43PLcm6dsInzHMxqZJ1fD0TUEIjagjd+UVxmefUqkrjnx3/ItTWoE8IjkpC7fNKoFBUn3xuZikUWYsb4c7lwt4d09T2Ld6rIGKsWlm2KcLflTvgnPmbvKeEH7Ef8MHMUhd63/BRyPy35qYNMTBgdISDPsHREW95Q77xnGskPazg6L6nAUE/yjJKv9WPBjiCORYMYheEMgg/46kqbRHSog7ZuClLj+2MRQOZ+MIkkfnabzPfjLwJ8yrOj/GPs3ysb0A+BcpEHkg1JBeN3LZizz5VAFT6zvh4XgYU/sTtAzHWs/lIV1I2EYFg2w1e3Hk0WbNQRF01FyBYHip8DUBHNODQQNaWN40uHgZAg8mq63BlOMvjlVTn/fx/syu8rnyE5+F8CvirwKu0m1IdP43c60TXAmDdSIktKOLv1o/1CLDQ== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(6916009)(336012)(55016003)(36860700001)(54906003)(40460700003)(36756003)(2906002)(6666004)(7696005)(83380400001)(47076005)(5660300002)(8936002)(426003)(8676002)(2616005)(70586007)(4326008)(107886003)(82310400004)(81166007)(1076003)(316002)(26005)(186003)(86362001)(356005)(6286002)(508600001)(70206006)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Feb 2022 23:25:21.8141 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69d57bff-5cf4-40ad-08f7-08d9f7eceda2 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT026.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1680 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for rdma-core API to import device. The API gets ibv_context file descriptor and returns an ibv_context pointer that is associated with the given file descriptor. Add also support for rdma-core API to import PD. The API gets ibv_context and PD handle and returns a protection domain (PD) that is associated with the given handle in the given context. Signed-off-by: Michael Baum Acked-by: Matan Azrad --- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/linux/mlx5_glue.c | 41 +++++++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_glue.h | 4 +++ 3 files changed, 47 insertions(+) diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build index 4c7b53b9bd..ed48245c67 100644 --- a/drivers/common/mlx5/linux/meson.build +++ b/drivers/common/mlx5/linux/meson.build @@ -202,6 +202,8 @@ has_sym_args = [ 'mlx5dv_dr_domain_allow_duplicate_rules' ], [ 'HAVE_MLX5_IBV_REG_MR_IOVA', 'infiniband/verbs.h', 'ibv_reg_mr_iova' ], + [ 'HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR', 'infiniband/verbs.h', + 'ibv_import_device' ], ] config = configuration_data() foreach arg:has_sym_args diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c index bc6622053f..450dd6a06a 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.c +++ b/drivers/common/mlx5/linux/mlx5_glue.c @@ -34,6 +34,32 @@ mlx5_glue_dealloc_pd(struct ibv_pd *pd) return ibv_dealloc_pd(pd); } +static struct ibv_pd * +mlx5_glue_import_pd(struct ibv_context *context, uint32_t pd_handle) +{ +#ifdef HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR + return ibv_import_pd(context, pd_handle); +#else + (void)context; + (void)pd_handle; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_unimport_pd(struct ibv_pd *pd) +{ +#ifdef HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR + ibv_unimport_pd(pd); + return 0; +#else + (void)pd; + errno = ENOTSUP; + return -errno; +#endif +} + static struct ibv_device ** mlx5_glue_get_device_list(int *num_devices) { @@ -52,6 +78,18 @@ mlx5_glue_open_device(struct ibv_device *device) return ibv_open_device(device); } +static struct ibv_context * +mlx5_glue_import_device(int cmd_fd) +{ +#ifdef HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR + return ibv_import_device(cmd_fd); +#else + (void)cmd_fd; + errno = ENOTSUP; + return NULL; +#endif +} + static int mlx5_glue_close_device(struct ibv_context *context) { @@ -1402,9 +1440,12 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { .fork_init = mlx5_glue_fork_init, .alloc_pd = mlx5_glue_alloc_pd, .dealloc_pd = mlx5_glue_dealloc_pd, + .import_pd = mlx5_glue_import_pd, + .unimport_pd = mlx5_glue_unimport_pd, .get_device_list = mlx5_glue_get_device_list, .free_device_list = mlx5_glue_free_device_list, .open_device = mlx5_glue_open_device, + .import_device = mlx5_glue_import_device, .close_device = mlx5_glue_close_device, .query_device = mlx5_glue_query_device, .query_device_ex = mlx5_glue_query_device_ex, diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h index 4e6d31f263..c4903a6dce 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.h +++ b/drivers/common/mlx5/linux/mlx5_glue.h @@ -151,9 +151,13 @@ struct mlx5_glue { int (*fork_init)(void); struct ibv_pd *(*alloc_pd)(struct ibv_context *context); int (*dealloc_pd)(struct ibv_pd *pd); + struct ibv_pd *(*import_pd)(struct ibv_context *context, + uint32_t pd_handle); + int (*unimport_pd)(struct ibv_pd *pd); struct ibv_device **(*get_device_list)(int *num_devices); void (*free_device_list)(struct ibv_device **list); struct ibv_context *(*open_device)(struct ibv_device *device); + struct ibv_context *(*import_device)(int cmd_fd); int (*close_device)(struct ibv_context *context); int (*query_device)(struct ibv_context *context, struct ibv_device_attr *device_attr); -- 2.25.1