From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C6687A0350; Mon, 21 Feb 2022 17:07:21 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5A5A04068C; Mon, 21 Feb 2022 17:07:21 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) by mails.dpdk.org (Postfix) with ESMTP id 3A1354013F for ; Mon, 21 Feb 2022 17:07:20 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WXEHhaqBMJU7o+JZhbmTuqAAVDeac1zfEWwojdMkrRteS1iDPeu4jFT+/GhhUXvRgo2Nw+Q7hyDgsk3zPl7p8jC6Tanv+mbBc+TEONv5xaUTjr2BuIpOyaorLgioku4U34HwDuTynkNiYSQzbhm/msGIKVatVZCYLV+v6TiB0e5SiUDBP/2ziqYcDWY34pfYTcA7B27X2nKytlKstiXDuIWY3a8n/93S8QpZeCe3bparDgQk6/pTU2+bIl/tsY9RXy+CUaYQUkKP820kNWZyKvbRk4nYr/4eGOuT1MZPO96+ZZ6p6ToiEBNaW8Tiv9814IVlyRRKn4HXjcfClIH44g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x9sRt+rsERsD9svEotulWFNC2TrLShmd6qO9Uv88dOM=; b=Ay8+AW1+9G/AUlOxo09wYwo9Fm6SfTRWY+XyBt8AKoCTi1I8wOh+hGXW1UVRrMjIdx5s5Ouw6bLdZpLaPAoQe3FSYFOrSp6gFlH1grpxXcAKzNEu8tmpuu6qNORSTWpONoT1NPO0JAEJ1rX0zctARJ6ahOoKbi5FQbKlLijoADy5/eDniGIk4YEva0D7bUt0NOfy4VV7pj02A4VX+wgKT4SgUEvbjmHoFfzwEk1AOiN6fsQ4Il6QOwYR7NaEooxQG5MJNOkbLjKqpwmdasS+0GUGhSSGZrsz8fiP5OdjdEYzr1T1kdjb2SOmQr9OrEM8rhLl7FW1l4wh4056zJ2aUw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x9sRt+rsERsD9svEotulWFNC2TrLShmd6qO9Uv88dOM=; b=kY6kfQHakuDEOOm6ehEdmmWx2/t0kVXblzTTR8BpZHFR75dI8GidyYZPrOnysJ+y8xOsn64LkMV4KVBzItr5WKALIdQO/HhXsp2ihVLc+WdQhMZsGePQb6XVOn19RZ6jlbrsIOfWHJcqK7M9V4Gf9xxlffHVTiIKYOXVmfBpP1/Vw9wHtUo6F1Cs2unjzavG7s+azH7E+h1Vqb5GHU5M8wLmZX7sdC8ukDEy1oWQOBjMxTV1NS7xmbYYbO9Srnyhjvzj5OjiXGcX9HYpJUkYTTK+tLaFGv5rOROdgJR++AoIfMoxLUUkit/sI+Iz8TxSo08dz/URMrwF+cNDgxou3g== Received: from MW4PR04CA0070.namprd04.prod.outlook.com (2603:10b6:303:6b::15) by BYAPR12MB3015.namprd12.prod.outlook.com (2603:10b6:a03:df::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.15; Mon, 21 Feb 2022 16:07:17 +0000 Received: from CO1NAM11FT045.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6b:cafe::4f) by MW4PR04CA0070.outlook.office365.com (2603:10b6:303:6b::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.14 via Frontend Transport; Mon, 21 Feb 2022 16:07:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by CO1NAM11FT045.mail.protection.outlook.com (10.13.175.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4995.15 via Frontend Transport; Mon, 21 Feb 2022 16:07:17 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Mon, 21 Feb 2022 16:07:16 +0000 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Mon, 21 Feb 2022 08:07:14 -0800 From: To: CC: Elena Agostini Subject: [PATCH v1] gpudev: use CPU map functionality in comm list Date: Tue, 22 Feb 2022 00:17:03 +0000 Message-ID: <20220222001703.20604-1-eagostini@nvidia.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3d9a5441-2882-41ad-5cb2-08d9f5543b84 X-MS-TrafficTypeDiagnostic: BYAPR12MB3015:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rqKtFKnT5FyLzT81ypq0hXUIquGb9FZbwRMjqIdTKCRLRm4J77n4JIIbbEbUWOIOkfYKqLhtxDbCap0xCQAJbpcKCi71mYOHCwN65wUHZU7DG36IJst5KiRO2/jfGOtW1PyQzvRz1azaKbJjnLxkxC9RfKD2v38eZo1SWv4WzB+hl32H0Oq9QJ70NrIACjiLGwuCsQIvXUajX/9nfFp2mxiWUxARRv5Hj5l2lD0scqCHcttme9WzBlVPKY1sn6aen2tskcseDwRB4NNyCCXa2FxyN4bEYBT6YRIwFbalxjlgkMsea5WUEdPt1mzoKgXJkh8Q8FEYb1kfrNazLUagdy9w4W2VFLoxsnHcW24kXCcU3k8AQcmCQikSyKfZ2oX+WljLoLisL4AzPWhoCVvehL5PP6oba9D3bv16MglyRN6S0aebwtn1uaWpsm873yEiVwOKrmZp2mxh8TwBtcGWgfaeY6cSHeRULSMhCo8+s/etWRJCGXVoHFHQ/k5Bf8Uf+W8P231VaXATqJg7OiTLoft9Ek3NqfPFvRK27oEid7N9wEXM4fMhZWkngrFrvfgl557wpuI+W/vJRNtOz6S/SVdAobj8nnNiGTayz7DNbauHOrSm7WZYyUlDCCxjXiRg52ncE9M9SmQkA0hhCqiDgLaR8JjN1NdoEks4ew9rlnQH0NNnudCKWt0/IiZmkYB1V2Mat1w/0tTtPWP48h0aQg== X-Forefront-Antispam-Report: CIP:12.22.5.234; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:mail.nvidia.com; PTR:InfoNoRecords; CAT:NONE; SFS:(13230001)(4636009)(40470700004)(36840700001)(46966006)(6916009)(1076003)(6666004)(70586007)(5660300002)(4326008)(316002)(107886003)(6286002)(16526019)(70206006)(7696005)(2616005)(508600001)(26005)(83380400001)(186003)(336012)(426003)(36756003)(47076005)(8676002)(8936002)(356005)(55016003)(36860700001)(40460700003)(2906002)(2876002)(81166007)(82310400004)(86362001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2022 16:07:17.2817 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3d9a5441-2882-41ad-5cb2-08d9f5543b84 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[12.22.5.234]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT045.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3015 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Elena Agostini rte_gpu_mem_cpu_map() exposes a GPU memory area to the CPU. In gpudev communication list this is useful to store the status flag. A communication list status flag allocated on GPU memory and mapped for CPU visibility can be updated by CPU and polled by a GPU workload. The polling operation is more frequent than the CPU update operation. Having the status flag in GPU memory reduces the GPU workload polling latency. If CPU mapping feature is not enabled, status flag resides in CPU memory registered so it's visible from the GPU. To facilitate the interaction with the status flag, this patch provides also the set/get functions for it. Signed-off-by: Elena Agostini --- app/test-gpudev/main.c | 8 ++- doc/guides/prog_guide/gpudev.rst | 2 +- lib/gpudev/gpudev.c | 116 +++++++++++++++++++++++++++++-- lib/gpudev/rte_gpudev.h | 49 ++++++++++++- lib/gpudev/version.map | 2 + 5 files changed, 166 insertions(+), 11 deletions(-) diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c index d4b8b8971d..f065e6cd81 100644 --- a/app/test-gpudev/main.c +++ b/app/test-gpudev/main.c @@ -324,7 +324,13 @@ simulate_gpu_task(struct rte_gpu_comm_list *comm_list_item, int num_pkts) * consume(comm_list_item->pkt_list[idx].addr); */ } - comm_list_item->status = RTE_GPU_COMM_LIST_DONE; + /* + * A real GPU workload function can't directly call rte_gpu_comm_set_status + * because it's a CPU-only function. + * A real GPU workload should implement the content + * of rte_gpu_comm_set_status() in GPU specific code. + */ + rte_gpu_comm_set_status(comm_list_item, RTE_GPU_COMM_LIST_DONE); return 0; } diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/gpudev.rst index 6223207a33..0ccf734dc0 100644 --- a/doc/guides/prog_guide/gpudev.rst +++ b/doc/guides/prog_guide/gpudev.rst @@ -216,7 +216,7 @@ about how to use functions in this library in case of a CUDA application. /* GPU kernel keeps checking this flag to know if it has to quit or wait for more packets. */ while (*quit_flag_ptr == 0) { - if (comm_list[comm_list_index]->status != RTE_GPU_COMM_LIST_READY) + if (comm_list[comm_list_index]->status_d[0] != RTE_GPU_COMM_LIST_READY) continue; if (threadIdx.x < comm_list[comm_list_index]->num_pkts) diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c index ce92d63257..3f6ea27417 100644 --- a/lib/gpudev/gpudev.c +++ b/lib/gpudev/gpudev.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "rte_gpudev.h" #include "gpudev_driver.h" @@ -847,6 +848,46 @@ rte_gpu_comm_create_list(uint16_t dev_id, return NULL; } + /* + * Use GPU memory CPU map feature if enabled in the driver + * to allocate the status flags of the list. + * Allocating this flag in GPU memory will reduce + * the latency when GPU workload is polling this flag. + */ + comm_list[0].status_d = rte_gpu_mem_alloc(dev_id, + sizeof(enum rte_gpu_comm_list_status) * num_comm_items, + rte_mem_page_size()); + if (ret < 0) { + rte_errno = ENOMEM; + return NULL; + } + + comm_list[0].status_h = rte_gpu_mem_cpu_map(dev_id, + sizeof(enum rte_gpu_comm_list_status) * num_comm_items, + comm_list[0].status_d); + if (comm_list[0].status_h == NULL) { + /* + * If CPU mapping is not supported by driver + * use regular CPU registered memory. + */ + comm_list[0].status_h = rte_zmalloc(NULL, + sizeof(enum rte_gpu_comm_list_status) * num_comm_items, 0); + if (comm_list[0].status_h == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + ret = rte_gpu_mem_register(dev_id, + sizeof(enum rte_gpu_comm_list_status) * num_comm_items, + comm_list[0].status_h); + if (ret < 0) { + rte_errno = ENOMEM; + return NULL; + } + + comm_list[0].status_d = comm_list[0].status_h; + } + for (idx_l = 0; idx_l < num_comm_items; idx_l++) { comm_list[idx_l].pkt_list = rte_zmalloc(NULL, sizeof(struct rte_gpu_comm_pkt) * RTE_GPU_COMM_LIST_PKTS_MAX, 0); @@ -862,8 +903,7 @@ rte_gpu_comm_create_list(uint16_t dev_id, rte_errno = ENOMEM; return NULL; } - - RTE_GPU_VOLATILE(comm_list[idx_l].status) = RTE_GPU_COMM_LIST_FREE; + comm_list[idx_l].num_pkts = 0; comm_list[idx_l].dev_id = dev_id; @@ -873,6 +913,17 @@ rte_gpu_comm_create_list(uint16_t dev_id, rte_errno = ENOMEM; return NULL; } + + if (idx_l > 0) { + comm_list[idx_l].status_h = &(comm_list[0].status_h[idx_l]); + comm_list[idx_l].status_d = &(comm_list[0].status_d[idx_l]); + + ret = rte_gpu_comm_set_status(&comm_list[idx_l], RTE_GPU_COMM_LIST_FREE); + if (ret < 0) { + rte_errno = ENOMEM; + return NULL; + } + } } return comm_list; @@ -910,6 +961,14 @@ rte_gpu_comm_destroy_list(struct rte_gpu_comm_list *comm_list, return -1; } + ret = rte_gpu_mem_cpu_unmap(dev_id, comm_list[0].status_d); + if(ret == 0) { + rte_gpu_mem_free(dev_id, comm_list[0].status_d); + } else { + rte_gpu_mem_unregister(dev_id, comm_list[0].status_h); + rte_free(comm_list[0].status_h); + } + rte_free(comm_list); return 0; @@ -920,6 +979,7 @@ rte_gpu_comm_populate_list_pkts(struct rte_gpu_comm_list *comm_list_item, struct rte_mbuf **mbufs, uint32_t num_mbufs) { uint32_t idx; + int ret; if (comm_list_item == NULL || comm_list_item->pkt_list == NULL || mbufs == NULL || num_mbufs > RTE_GPU_COMM_LIST_PKTS_MAX) { @@ -943,7 +1003,39 @@ rte_gpu_comm_populate_list_pkts(struct rte_gpu_comm_list *comm_list_item, RTE_GPU_VOLATILE(comm_list_item->num_pkts) = num_mbufs; rte_gpu_wmb(comm_list_item->dev_id); - RTE_GPU_VOLATILE(comm_list_item->status) = RTE_GPU_COMM_LIST_READY; + ret = rte_gpu_comm_set_status(comm_list_item, RTE_GPU_COMM_LIST_READY); + if (ret < 0) { + rte_errno = EINVAL; + return -rte_errno; + } + + return 0; +} + +int +rte_gpu_comm_set_status(struct rte_gpu_comm_list *comm_list_item, + enum rte_gpu_comm_list_status status) +{ + if (comm_list_item == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + + RTE_GPU_VOLATILE(comm_list_item->status_h[0]) = status; + + return 0; +} + +int +rte_gpu_comm_get_status(struct rte_gpu_comm_list *comm_list_item, + enum rte_gpu_comm_list_status *status) +{ + if (comm_list_item == NULL || status == NULL) { + rte_errno = EINVAL; + return -rte_errno; + } + + *status = RTE_GPU_VOLATILE(comm_list_item->status_h[0]); return 0; } @@ -952,14 +1044,21 @@ int rte_gpu_comm_cleanup_list(struct rte_gpu_comm_list *comm_list_item) { uint32_t idx = 0; + enum rte_gpu_comm_list_status status; + int ret; if (comm_list_item == NULL) { rte_errno = EINVAL; return -rte_errno; } - if (RTE_GPU_VOLATILE(comm_list_item->status) == - RTE_GPU_COMM_LIST_READY) { + ret = rte_gpu_comm_get_status(comm_list_item, &status); + if (ret < 0) { + rte_errno = EINVAL; + return -rte_errno; + } + + if (status == RTE_GPU_COMM_LIST_READY) { GPU_LOG(ERR, "packet list is still in progress"); rte_errno = EINVAL; return -rte_errno; @@ -974,9 +1073,14 @@ rte_gpu_comm_cleanup_list(struct rte_gpu_comm_list *comm_list_item) comm_list_item->mbufs[idx] = NULL; } - RTE_GPU_VOLATILE(comm_list_item->status) = RTE_GPU_COMM_LIST_FREE; + ret = rte_gpu_comm_set_status(comm_list_item, RTE_GPU_COMM_LIST_FREE); + if (ret < 0) { + rte_errno = EINVAL; + return -rte_errno; + } RTE_GPU_VOLATILE(comm_list_item->num_pkts) = 0; rte_mb(); return 0; } + diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h index 9802bff2a5..25ec55b4e4 100644 --- a/lib/gpudev/rte_gpudev.h +++ b/lib/gpudev/rte_gpudev.h @@ -124,8 +124,10 @@ struct rte_gpu_comm_list { struct rte_gpu_comm_pkt *pkt_list; /** Number of packets in the list. */ uint32_t num_pkts; - /** Status of the list. */ - enum rte_gpu_comm_list_status status; + /** Status of the list. CPU pointer. */ + enum rte_gpu_comm_list_status *status_h; + /** Status of the list. GPU pointer. */ + enum rte_gpu_comm_list_status *status_d; }; /** @@ -489,7 +491,7 @@ void *rte_gpu_mem_cpu_map(int16_t dev_id, size_t size, void *ptr); * @param dev_id * Reference device ID. * @param ptr - * Pointer to the GPU memory area to be unmapped. + * Pointer to the memory area to be unmapped. * NULL is a no-op accepted value. * * @return @@ -679,6 +681,46 @@ __rte_experimental int rte_gpu_comm_populate_list_pkts(struct rte_gpu_comm_list *comm_list_item, struct rte_mbuf **mbufs, uint32_t num_mbufs); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Set status flag value of a communication list item. + * + * @param comm_list_item + * Communication list item to query. + * @param status + * Status value to set. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + */ +__rte_experimental +int rte_gpu_comm_set_status(struct rte_gpu_comm_list *comm_list_item, + enum rte_gpu_comm_list_status status); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice. + * + * Get status flag value of a communication list item. + * + * @param comm_list_item + * Communication list item to query. + * Input parameter. + * @param status + * Communication list item status flag value. + * Output parameter. + * + * @return + * 0 on success, -rte_errno otherwise: + * - EINVAL if invalid input params + */ +__rte_experimental +int rte_gpu_comm_get_status(struct rte_gpu_comm_list *comm_list_item, + enum rte_gpu_comm_list_status *status); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice. @@ -701,3 +743,4 @@ int rte_gpu_comm_cleanup_list(struct rte_gpu_comm_list *comm_list_item); #endif #endif /* RTE_GPUDEV_H */ + diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map index 5bc5d154cd..b23e3fd6eb 100644 --- a/lib/gpudev/version.map +++ b/lib/gpudev/version.map @@ -12,8 +12,10 @@ EXPERIMENTAL { rte_gpu_comm_destroy_flag; rte_gpu_comm_destroy_list; rte_gpu_comm_get_flag_value; + rte_gpu_comm_get_status; rte_gpu_comm_populate_list_pkts; rte_gpu_comm_set_flag; + rte_gpu_comm_set_status; rte_gpu_count_avail; rte_gpu_find_next; rte_gpu_info_get; -- 2.17.1