From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 510F3A034C; Tue, 22 Feb 2022 22:04:36 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B79AA41147; Tue, 22 Feb 2022 22:04:32 +0100 (CET) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2072.outbound.protection.outlook.com [40.107.223.72]) by mails.dpdk.org (Postfix) with ESMTP id 58D4941142 for ; Tue, 22 Feb 2022 22:04:31 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EsbCxvfXklfCgcqKQ9ylHcO6HWsAqSlj7Z22vkj9CXBDJv16akYGvbBf0cHGBuCPUUi5vntfr/bcQ0lgcnZ2JGq4pf8d+IqlM5GOaAMcPIdfj8/P0hx+OKaXWmaQssKWlNNjCHpNHNFcIZyfSzH6JAlXKqleuxT5UmxAy9X1yxsvQgEFVcuI3N8tD2KZG65UNqHYIS9tr2Cz92QnX6XvWWTbmA/wlGQh58WscCV2N/FFnYhtrLjvdcoGYIZQYxDGlSikvE7jrhkmX1mmydxaPJFLMvp5HCvfhSqmvnuw6lhOrcrxp6cy2gP258vkt3Ig9NBObnQ6kPZqRz2UB1e4IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PrShDu9D673FvzdOk5pWzOBHE9oOYLp8YYkFtHYNKVg=; b=g1kAk8kA4Jmas6dTL2VfCcFWKe2psj76HhaKo2bJZI6CV1mkIxytbBKPqsxT5s9Uid4JDbvGTNtO0Lv15BdZ9+qS0Ehbk2lX/r6gYdfXjbv+V7brR1NftHvN1N90B/5VoOSgkVB73gVxGqo5q0YYG3tT9KnDNcARwITv4ITQvABxcZsgMb196CkG4l/vQPfTaa5donAnGxj4dsDGqcdKJUgTMlGcXww1cJT8raXxgx1IDACgQyDR37ADkb0I5Jv5r/Qpvv9DsBE2qXNDQkYWgHU2ZnOw9Phg7mixMmb1cpUa0WcAzZa+Xe+VmoJnYkZ20UQrH7gGaagDt0jKWYDOQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PrShDu9D673FvzdOk5pWzOBHE9oOYLp8YYkFtHYNKVg=; b=LnFAISKXoQjaCs8t8GnQ+Ydf44ZPSDmQUVjtd+xiy1Fb4qmGAJ8/CcMjfrZYPTGUDS2XBphud1auED6060dLjSwzE7KNCLezwL1MiG8N34Zn6lL2E+wL7ZCqDgXBw75NBHiLksM7pKfL1wr8p0aDVI0z5zZ6Niq5sWKK1VjraPIMAOu8r/ffZxzU/nPuZqaa6ggGDltJXfBzWxh0QZqFkey7/iyNV2VnRq8fEIQGBkwo9KlXxKqXzuvNlxzEbKB6m79SaH5frsTYH6fLTr79VJyr4CjRWhxAnqmUwlqLw6eaP+h08ePE7ZI8yz18PGlrz+YCBgwu3MhoH7oUXr2Riw== Received: from MWHPR14CA0040.namprd14.prod.outlook.com (2603:10b6:300:12b::26) by DM4PR12MB5915.namprd12.prod.outlook.com (2603:10b6:8:68::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.17; Tue, 22 Feb 2022 21:04:29 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:300:12b:cafe::69) by MWHPR14CA0040.outlook.office365.com (2603:10b6:300:12b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5017.21 via Frontend Transport; Tue, 22 Feb 2022 21:04:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.5017.22 via Frontend Transport; Tue, 22 Feb 2022 21:04:27 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 22 Feb 2022 21:04:26 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Tue, 22 Feb 2022 13:04:25 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9 via Frontend Transport; Tue, 22 Feb 2022 13:04:24 -0800 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Subject: [PATCH 1/6] common/mlx5: glue device and PD importation Date: Tue, 22 Feb 2022 23:04:11 +0200 Message-ID: <20220222210416.2669519-2-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220222210416.2669519-1-michaelba@nvidia.com> References: <20220222210416.2669519-1-michaelba@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9427a0b9-8bea-421a-dd88-08d9f646e957 X-MS-TrafficTypeDiagnostic: DM4PR12MB5915:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /rYWaTxtVS4AVbgIB81zOdbS++nfJqchRJJv/eTwPMcg8RQdWehY6I+irjsoEDWdcq5ScZsM5Xv37zCcHbm6xbDBAKEJ6mFZtAXXCJ8dNzqmx8NEDwUtT6UgX2T0ccIxOPozmJvBfgC8TJCRCoEGyYUMfZbXQjyB37tWtApAeadb2mXqNjYijw/jbhxn4qUdZ/dXC9/BESrk+rBRAzEqxAAGnfY+A2nOzFg43LN3iv0UjeHDYN/mZM0zPswJJjrT/+IfFqIyzYiP8qemyFAE/1CHB7PH5QBh3ChjwM056MiSpljrrKPr8/u611GEqturVBpFRLK4CJ97RQrHNUFEGQGHqUOy9CLIXL7Y9FEX9zzpIB2G7OxwBB5tWKSWbLJfAK7mSaDVieiFPCO+kbBP3mBF20/wOgljC/4hVgFp/+lbfPue833yqc4j5UOmaPAq0YGmEp1UIqt7KZ+pDVXp8Pj4RKg9oNgHUdSo3wW1umXifI/T2zJUpJxeoiH/bF9nJUfJ0CjplE/CsrJ1WG6to/7eVpGtSXhNTvF0OFs/1TTPoW1z+8iSokLBsvz+qsrnD0oAqKGBO6p2NCHHrCJ9adEndmyHFpaFQVAlgdtKtdiNRVpoaiHkKvvu/9ZRVGQmGU5mDrSBM/PlAD4+EtCmcZ2VmIEtnk3/IE4mgFNJbEUEbv2NwjE8YMwOVasd9WP/yNMX9I/+2lt7jsrpKawiRA== X-Forefront-Antispam-Report: CIP:12.22.5.235; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(336012)(36860700001)(316002)(82310400004)(70206006)(4326008)(8676002)(70586007)(426003)(508600001)(55016003)(2616005)(5660300002)(7696005)(186003)(26005)(1076003)(6286002)(83380400001)(81166007)(86362001)(6666004)(107886003)(8936002)(36756003)(6916009)(40460700003)(356005)(2906002)(47076005)(54906003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2022 21:04:27.1535 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9427a0b9-8bea-421a-dd88-08d9f646e957 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.235]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5915 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for rdma-core API to import device. The API gets ibv_context file descriptor and returns an ibv_context pointer that is associated with the given file descriptor. Add also support for rdma-core API to import PD. The API gets ibv_context and PD handle and returns a protection domain (PD) that is associated with the given handle in the given context. Signed-off-by: Michael Baum --- drivers/common/mlx5/linux/meson.build | 2 ++ drivers/common/mlx5/linux/mlx5_glue.c | 41 +++++++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_glue.h | 4 +++ 3 files changed, 47 insertions(+) diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build index 4c7b53b9bd..ed48245c67 100644 --- a/drivers/common/mlx5/linux/meson.build +++ b/drivers/common/mlx5/linux/meson.build @@ -202,6 +202,8 @@ has_sym_args = [ 'mlx5dv_dr_domain_allow_duplicate_rules' ], [ 'HAVE_MLX5_IBV_REG_MR_IOVA', 'infiniband/verbs.h', 'ibv_reg_mr_iova' ], + [ 'HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR', 'infiniband/verbs.h', + 'ibv_import_device' ], ] config = configuration_data() foreach arg:has_sym_args diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c index bc6622053f..450dd6a06a 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.c +++ b/drivers/common/mlx5/linux/mlx5_glue.c @@ -34,6 +34,32 @@ mlx5_glue_dealloc_pd(struct ibv_pd *pd) return ibv_dealloc_pd(pd); } +static struct ibv_pd * +mlx5_glue_import_pd(struct ibv_context *context, uint32_t pd_handle) +{ +#ifdef HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR + return ibv_import_pd(context, pd_handle); +#else + (void)context; + (void)pd_handle; + errno = ENOTSUP; + return NULL; +#endif +} + +static int +mlx5_glue_unimport_pd(struct ibv_pd *pd) +{ +#ifdef HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR + ibv_unimport_pd(pd); + return 0; +#else + (void)pd; + errno = ENOTSUP; + return -errno; +#endif +} + static struct ibv_device ** mlx5_glue_get_device_list(int *num_devices) { @@ -52,6 +78,18 @@ mlx5_glue_open_device(struct ibv_device *device) return ibv_open_device(device); } +static struct ibv_context * +mlx5_glue_import_device(int cmd_fd) +{ +#ifdef HAVE_MLX5_IBV_IMPORT_CTX_PD_AND_MR + return ibv_import_device(cmd_fd); +#else + (void)cmd_fd; + errno = ENOTSUP; + return NULL; +#endif +} + static int mlx5_glue_close_device(struct ibv_context *context) { @@ -1402,9 +1440,12 @@ const struct mlx5_glue *mlx5_glue = &(const struct mlx5_glue) { .fork_init = mlx5_glue_fork_init, .alloc_pd = mlx5_glue_alloc_pd, .dealloc_pd = mlx5_glue_dealloc_pd, + .import_pd = mlx5_glue_import_pd, + .unimport_pd = mlx5_glue_unimport_pd, .get_device_list = mlx5_glue_get_device_list, .free_device_list = mlx5_glue_free_device_list, .open_device = mlx5_glue_open_device, + .import_device = mlx5_glue_import_device, .close_device = mlx5_glue_close_device, .query_device = mlx5_glue_query_device, .query_device_ex = mlx5_glue_query_device_ex, diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h index 4e6d31f263..c4903a6dce 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.h +++ b/drivers/common/mlx5/linux/mlx5_glue.h @@ -151,9 +151,13 @@ struct mlx5_glue { int (*fork_init)(void); struct ibv_pd *(*alloc_pd)(struct ibv_context *context); int (*dealloc_pd)(struct ibv_pd *pd); + struct ibv_pd *(*import_pd)(struct ibv_context *context, + uint32_t pd_handle); + int (*unimport_pd)(struct ibv_pd *pd); struct ibv_device **(*get_device_list)(int *num_devices); void (*free_device_list)(struct ibv_device **list); struct ibv_context *(*open_device)(struct ibv_device *device); + struct ibv_context *(*import_device)(int cmd_fd); int (*close_device)(struct ibv_context *context); int (*query_device)(struct ibv_context *context, struct ibv_device_attr *device_attr); -- 2.25.1