From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 589E9A0C43; Fri, 8 Oct 2021 19:44:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 41E1D41125; Fri, 8 Oct 2021 19:43:47 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2048.outbound.protection.outlook.com [40.107.92.48]) by mails.dpdk.org (Postfix) with ESMTP id 9DB42410FE for ; Fri, 8 Oct 2021 19:43:44 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Oi1JGdOQOSUCSxLNezDvGy4WDTxFTIDD+o51rIvauoVflxdpDkFPzVCNf+8gfaZwpVmSlGlmLjdJGOx8n2D+fh2I5NskjRZsn6TWHtFi5E7zH5e/fGvV3Xr3scHSlfJifBVPkeC1VK2o3UYTSgcOQaDcxSm5xooEpQB88LUVeXzosASWaHPRLizdZRnvaC12D0i7Og+VDo85sOgiGc+IZ0vDmNV0M5ndXB2KtF/w/MphxddjUFDrKGhglBrI4FzVWl41u/bNRDTKI1DORgokpFA1aNdWtLlW3GjRAAtH0XX2ihuICHw7GU9iYmDcN5cIk15/+fNKGT8yscEb5uMcIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RcJ8lKWpo6+elUvJVL104uFWcIiXQcFjM5BhbdJDUPo=; b=lxb1Q3l1T+tXTeMxC07OzLAq+RjBiAAYK18m17fLCXzlRu2TK8ZdXydGlalKYhnqATfiyRVxwOUnKAJKIM7p8FgrJZDSeIZaaPv3IZX+wsGd0l8rZOtE8QeCIilYNwctDG9UclBWy8JKWaSYc8n82uS44LtmEnAIXm9SoBk9JHctWd0+MoMuz5nCF6u3MebqXTeRA1IOL0o5ctHzXg/emzUJx5PAp+8cD7c76vFBBqSP2QRcOFHjYXwTsVe1QYkX9Hw9NUL5Gxd0gTrmP9WuiQXK60O2LOa1lyuKmHhCKz9X+SQGCr6+1q9dQ4kVEm1s+Df5MAprX9yoK/5a3cQYkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RcJ8lKWpo6+elUvJVL104uFWcIiXQcFjM5BhbdJDUPo=; b=shBhDy7OvpNwMF87ELMWZaB3ZjLz+IyLMAOP7u4jKSsYX0bhcmgmbJDBuwj+rUxVamfpzgcisMKy0P53WFrwlBkHRh0YMcvSyCIOgVLzUQ57e76zdYBwFB/rHik0Bk9Btu3JZY57SNxxFosMqM/+jTUxhgZ6qM2+Lp3RbSOnpIbzRp1/mUXfrUIllJqelrbHcO7lwQ2n2ljfk7NZ/GQsxkEtaLGIoGovt42fJgT7lTyXpnf+xtQJH/D1WuMUKGYlo599i06OEOaDaPzzcgrOyktZM59ZaLNCi6k7cm8L47o9QYODbpKw3HXLTi5UN2zLBCyBJXf1I/AfM/IQW480eg== Received: from MW4PR03CA0142.namprd03.prod.outlook.com (2603:10b6:303:8c::27) by DM5PR12MB2583.namprd12.prod.outlook.com (2603:10b6:4:b3::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.22; Fri, 8 Oct 2021 17:43:43 +0000 Received: from CO1NAM11FT047.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8c:cafe::97) by MW4PR03CA0142.outlook.office365.com (2603:10b6:303:8c::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 17:43:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT047.mail.protection.outlook.com (10.13.174.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 17:43:42 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 17:43:38 +0000 From: To: CC: Elena Agostini Date: Sat, 9 Oct 2021 01:53:46 +0000 Message-ID: <20211009015349.9694-7-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211009015349.9694-1-eagostini@nvidia.com> References: <20210602203531.2288645-1-thomas@monjalon.net> <20211009015349.9694-1-eagostini@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3a6d4725-0da9-4a19-4570-08d98a832b69 X-MS-TrafficTypeDiagnostic: DM5PR12MB2583: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Weooi5ZcV3D7M2l6eZTc85WmR2hGMBKMjDL/hmefKeORF3W/IRmvlBiUeFUTrDSXg/c1VZd0yKCPHmCgKjgA7wBFJ9QV6stxqsIg9SSIB920d9RnEppB0XbGL6qouFuj11jwPP0ELRDEscFzeVrEf5BGpVY3n36H/E1DPWGAHYGvBYoN+qOa3QXMScTs8OjiSncFruRqddzw1lr7IJAPCYRZlwcFfGs6SsyW9oNDi3cawgkXg41k3JFs2XGSbk+MjOzDES8nE7fIXXGMAlJ/IZQj0oWuOnLaU5SqQUyanZ3KgYeLXUfWCmlQBkKnTmgOoztmB5Waeyr4VOM+kgHWzv7tHSaK/LByHGYoqiiCzp3+DM/Fyf/7HDw2HS2w6OPH35Ka5SKMiSoZLC7TJGKV6RtdPMN3HZwXejPVvv2M4pA4nP9SIhShOBKErvvVgcY4lmqexdeCaESLSF6RyutHXF6e5qyV2UspUl+KJZbGJhNQIyexVcSpfm9ktDrp6vtjpPM0/7LVKclgeNDqa9St5WZYfbTcY76JXYzzHWq+4REDt5zqk6VPzhCLQlvSHHto4eVp5tWfZxCo/akL4Y8/wBkg0L8bBFsDZdNEKvdoJaBzE/DFgd959cKCHf37Bm3u0B9xLkEvr/nv9waecvycl3bQ3jWYG1fm/PYbu//hLacL+APDolfzlq4wHBw/m9nzX9aFhZhbJ+8mcPAkJzgLbg== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(6916009)(55016002)(6666004)(508600001)(2616005)(5660300002)(2876002)(82310400003)(6286002)(316002)(426003)(8676002)(83380400001)(2906002)(336012)(16526019)(86362001)(36860700001)(1076003)(70206006)(26005)(47076005)(36756003)(186003)(107886003)(7696005)(7636003)(8936002)(70586007)(356005)(4326008); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 17:43:42.2314 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3a6d4725-0da9-4a19-4570-08d98a832b69 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT047.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2583 Subject: [dpdk-dev] [PATCH v3 6/9] gpudev: add memory barrier X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Elena Agostini Add a function for the application to ensure the coherency of the writes executed by another device into the GPU memory. Signed-off-by: Elena Agostini --- doc/guides/prog_guide/gpudev.rst | 8 ++++++++ lib/gpudev/gpudev.c | 19 +++++++++++++++++++ lib/gpudev/gpudev_driver.h | 3 +++ lib/gpudev/rte_gpudev.h | 18 ++++++++++++++++++ lib/gpudev/version.map | 1 + 5 files changed, 49 insertions(+) diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index 9aca69038c..eb5f0af817 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -65,3 +65,11 @@ gpudev can register a CPU memory area to make it visible from a GPU device. Later, it's also possible to unregister that memory with gpudev. CPU memory registered outside of the gpudev library (e.g. with GPU specific library) cannot be unregistered by the gpudev library. + +Memory Barrier +~~~~~~~~~~~~~~ + +Some GPU drivers may need, under certain conditions, +to enforce the coherency of external devices writes (e.g. NIC receiving packets) +into the GPU memory. +gpudev abstracts and exposes this capability. diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index 1d8318f769..cefefd737a 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -624,3 +624,22 @@ rte_gpu_free(int16_t dev_id, void *ptr) } return GPU_DRV_RET(dev->ops.mem_free(dev, ptr)); } + +int +rte_gpu_mbw(int16_t dev_id) +{ + struct rte_gpu *dev; + + dev = gpu_get_by_id(dev_id); + if (dev == NULL) { + GPU_LOG(ERR, "memory barrier for invalid device ID %d", dev_id); + rte_errno = ENODEV; + return -rte_errno; + } + + if (dev->ops.mbw == NULL) { + rte_errno = ENOTSUP; + return -rte_errno; + } + return GPU_DRV_RET(dev->ops.mbw(dev)); +} diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h index 11015944a6..ab24de9e28 100644 --- a/lib/gpudev/gpudev_driver.h +++ b/lib/gpudev/gpudev_driver.h @@ -31,6 +31,7 @@ typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, void **ptr); typedef int (rte_gpu_free_t)(struct rte_gpu *dev, void *ptr); typedef int (rte_gpu_mem_register_t)(struct rte_gpu *dev, size_t size, void *ptr); typedef int (rte_gpu_mem_unregister_t)(struct rte_gpu *dev, void *ptr); +typedef int (rte_gpu_mbw_t)(struct rte_gpu *dev); struct rte_gpu_ops { /* Get device info. If NULL, info is just copied. */ @@ -45,6 +46,8 @@ struct rte_gpu_ops { rte_gpu_free_t *mem_free; /* Unregister CPU memory in device. */ rte_gpu_mem_unregister_t *mem_unregister; + /* Enforce GPU memory write barrier. */ + rte_gpu_mbw_t *mbw; }; struct rte_gpu_mpshared { diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index 3c276581c0..e790b3e2b7 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -387,6 +387,24 @@ int rte_gpu_register(int16_t dev_id, size_t size, void * ptr); __rte_experimental int rte_gpu_unregister(int16_t dev_id, void *ptr); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Enforce a GPU memory write barrier. + * + * @param dev_id + * Reference device ID. + * + * @return + * 0 on success, -rte_errno otherwise: + * - ENODEV if invalid dev_id + * - ENOTSUP if operation not supported by the driver + * - EPERM if driver error + */ +__rte_experimental +int rte_gpu_mbw(int16_t dev_id); + #ifdef __cplusplus } #endif diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index d4a65ebd52..d72d470d8e 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -13,6 +13,7 @@ EXPERIMENTAL { rte_gpu_init; rte_gpu_is_valid; rte_gpu_malloc; + rte_gpu_mbw; rte_gpu_register; rte_gpu_unregister; }; -- 2.17.1