From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17528A034D; Tue, 11 Jan 2022 10:29:47 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 903D74121E; Tue, 11 Jan 2022 10:29:46 +0100 (CET) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2047.outbound.protection.outlook.com [40.107.243.47]) by mails.dpdk.org (Postfix) with ESMTP id A8149411AE for ; Tue, 11 Jan 2022 10:29:44 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wj8eGQnywObFpal3Imh/JUhMIzI7RXTuVtGOUKRK9IxBueuWXwg/setMgVPSTBptyTLxrwUEufTsfz1R4OgXB6e4W5fJ5XPn8OVRs/YsNzpSfsFA5kpcYbDJ44bY8NoHq854McE/j88AcoihJ3PLDuy6x4kpdFvfuVEL9BEFYv8iRBzPAmOAg1IYjJBPfD85uVJTSrh8HVkkc23Yaf+e4KOCkeDa2BX/lABbKACtNfnH/EJKgwduonfewYsQlrFTVsVCRzULTcXARntY5iuRkaNNGUw7tjlCp+ZGWaqgndW9osDA2uu71WJBDSxNBjLifPFV1asm4mrvrQ2fnp2SnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gYF/rCQEO+GThJxDZZmv4nM5KjeboTPip09VEz9JeuA=; b=DccRU50uuis4wR1F0ZAezNvGF5SynaG1kUGgxUcyVozpB9cEiFc+AFauPuDEhgSvaWsSb73TYTCBECtmDaksgIVt4kmkN/6MXo5vikrNPE46FawngW8gfUsY7IKV6kd+MIJFlEO9X8EzGOdkU39N/rS5oYfCD1a4RKKOYL2TR8Kdkpi1UTY6+N8lTBfGFZOeAFVL89xK6MFD5nwdkJbQ2oDpeUu1OQd0QUCZXsOdM7tFYb+VsQGdJ5mHEDQ1h4EMt242onkO8yg5wzNAQ4yDghoeoqnYLdGnqEgsxPM7I6t7SIqEFoYeuG8R9RG4e13FFpzafMHmYNQoDRcTvEtOZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gYF/rCQEO+GThJxDZZmv4nM5KjeboTPip09VEz9JeuA=; b=bO61fbX7ju80zU5u/U8tKq/hrDbkTKo88ZCfItT7dmWxLHXu8ERGn+wjmC3gnEPrLQ+LK376UtxahDSquqKjsJBPxGLZDJ3rcSGwLKZfU9DO3V/JNVJoSGhrn7r5CTlokpsNqGj8WItyAoarHxJN3FZ4CdDpYEuQRxu/0EdxN6EulHvnZ5QcnX4jDv8hSq8xtj4rjG0tapOpU7sphk2S9389ypcYjhnDzR8U28fu/OYyAVWSxWQngpJSDU1DeIHIF3X2YEmh77rf4WxqMvHtHEnvE4x8CGxt9aN6nLn4cXm0/wx578gTWRNTmACi8sdlcgLfhYO4gjoJk5su72r1dQ== Received: from BN9PR03CA0771.namprd03.prod.outlook.com (2603:10b6:408:13a::26) by CY4PR12MB1848.namprd12.prod.outlook.com (2603:10b6:903:11d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4867.11; Tue, 11 Jan 2022 09:29:42 +0000 Received: from BN8NAM11FT003.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13a:cafe::70) by BN9PR03CA0771.outlook.office365.com (2603:10b6:408:13a::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4867.9 via Frontend Transport; Tue, 11 Jan 2022 09:29:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by BN8NAM11FT003.mail.protection.outlook.com (10.13.177.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4867.7 via Frontend Transport; Tue, 11 Jan 2022 09:29:42 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 11 Jan 2022 09:29:41 +0000 Received: from nvidia.com (10.127.8.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.986.9; Tue, 11 Jan 2022 01:29:40 -0800 From: To: CC: Elena Agostini Subject: [PATCH v1 0/1] gpu/cuda: expose GPU memory with GDRCopy Date: Tue, 11 Jan 2022 17:39:28 +0000 Message-ID: <20220111173929.28746-1-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.127.8.10] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d09b6853-b7b0-41fc-b734-08d9d4e4e5e4 X-MS-TrafficTypeDiagnostic: CY4PR12MB1848:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tjDm9GFxX1SenRh2TEZlMz4Eh/IPTd/KTEOlmQqKZkKS/VXc3VAY/pFuZQyFAZpFvzJb9oQ4fEqNLawizPFN+1OKBiBBUsZC2N6P1CJSVFiJF2hOOFun93aqEAYW7cSVlub04IXpcGOHDWQ51/jlqmTXUiX011H5MnOC4ulJhGketxVhDPBtE2MBHnqyxwQkbzuYcsJH3kx62wL2I9ucJKYN0tcXTK16BExr9E8xR1V7YcaHpM988G5gdI20uhD36cdbJA+Jut0hVXgmHOoSVmRDoNa0PB8igt0xs/U03DXYbP2Srx5oDtCht3Jy82i7mKjZTFdl9pmJFZlmv9dimPH3ehA/K15mrIHu1wAKh9JQGM0ti24mHvOuTnATDPyBkngFULs+SO/2H4lkXBgw/GOSyybNsakY9z43z2870dfoinxBsNU5I7pnV4+Tvc38jvSDKdyfolii9rKGh7SBskE+1St4Otl7odFwOOCc1/E15KJW0GSpv5U+bdfyEI1uHkgiCINzGVCJ2lSdybhqO85bR1hwvrE4OuPIEnKHS8qnt+QNN3/ecusshA0FlgH0W6EJsoqnrqO7AESaE6wPOjyLNldEiywm/0e/YKOTBqha6bvKxhxcEGR+e5l2XgTHTpZQdCDmgfd/DdwHcmEfg7UV98FoUnKd9xEV8uLZnrTwqbZEQxqyuYVlnacFkz5bGlnqDgddxxJeg1xd6fqfPvO2aU44uO0b6g5duFtXnirEmDzZfs94mhFYSEOfvHPhpJVobyNV8kKTXPp3HFGoceD1ivWOcsCwrjlZ3aj2atuA1WyP88pcx1bOa55YdvJfGukP/oeAhWS/ubfOpVioZP7fD3/+O4IF0TGvJW4XnKzSAiREFcD7ipYaErMbBG33 X-Forefront-Antispam-Report: CIP:12.22.5.236; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(40470700002)(6666004)(316002)(83380400001)(1076003)(356005)(81166007)(2876002)(966005)(70206006)(2906002)(70586007)(5660300002)(186003)(8936002)(6916009)(40460700001)(7696005)(86362001)(4326008)(47076005)(55016003)(8676002)(107886003)(6286002)(426003)(36860700001)(36756003)(26005)(16526019)(336012)(82310400004)(508600001)(2616005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2022 09:29:42.2364 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d09b6853-b7b0-41fc-b734-08d9d4e4e5e4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.236]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT003.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1848 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Elena Agostini GPU CUDA implementation of the new gpudev functions to expose GPU memory to the CPU. Today GDRCopy library is required to pin and DMA map the GPU memory through the BAR1 of the GPU and expose it to the CPU. Goal here is to hide technical details GDRCopy library and expose the functionality through the generic gpudev layer. GDRCopy can be found here: https://github.com/NVIDIA/gdrcopy To build GPU CUDA driver with GDRCopy, you need to build DPDK indicating the gdrapi.h header file with -Dc_args="-I/path/to/gdrapi/". To execute you need to indicate the path to libgdrapi.so library with the environment variable GDRCOPY_PATH_L=/path/to/gdrcopy/lib/ If GDRCopy is not built with GPU CUDA driver, the GPU expose functionality will not be supported by the driver. This is an indipendent feature. All the other GPU CUDA driver capabilities are not affected if GDRCopy is not built. Signed-off-by: Elena Agostini --- Dependency on https://patches.dpdk.org/project/dpdk/patch/20220108000457.31104-1-eagostini@nvidia.com/ Elena Agostini (1): gpu/cuda: expose GPU memory with GDRCopy drivers/gpu/cuda/cuda.c | 101 +++++++++++++++++++++++++ drivers/gpu/cuda/gdrcopy.c | 139 +++++++++++++++++++++++++++++++++++ drivers/gpu/cuda/gdrcopy.h | 29 ++++++++ drivers/gpu/cuda/meson.build | 6 +- 4 files changed, 274 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/cuda/gdrcopy.c create mode 100644 drivers/gpu/cuda/gdrcopy.h -- 2.17.1