From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E24BA0C4E; Mon, 8 Nov 2021 16:16:34 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5BEF641122; Mon, 8 Nov 2021 16:16:34 +0100 (CET) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2058.outbound.protection.outlook.com [40.107.237.58]) by mails.dpdk.org (Postfix) with ESMTP id 52F7540E28 for ; Mon, 8 Nov 2021 16:16:33 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NxztsLW88iReURJ1v+giUQIzcJg/1x61jrAmPXmr/2B3oeM/5WZWXoHJsq5DBT/PVQ0mJ9nZFy0kC3VO0a/4u1nFcCliPR1AgLEhzfoe0qJ40EbKz9bHNKKOVly0QQI8DLLsJcCbB//QXf6hausnfMEEcnOc5CAecaFoAX+apvBErxLL3Bjx6jZ8RMhunSbLQjsguGWewRLN8Zhvrz2JJcfyyp3lvJKRRvTf87NE0GpxCUUcvaDEbYHWHf4ekbn5sCdV73qszmtuHiwTmiyloSwGIcxj5BNmKBiwXBKkox3OFzJ3BUwchVC54Kh3WiCy1rIf8dDPnK69s96NlZSqvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cLoymPyKXi3rfIJHLQV1a8111FBvFsjenxTlQIFIYhU=; b=bwMMmsXDoOvDeQ6ZKMuAcFo6eLzypF69vopz5Wzfxy9eKz02qzKluEkWkOIbItmvlAqXs2sVOE5vXZO9Js9dJOIaM9gXfGFHF058dK+IWypLSTXfv8guetL5nPFFwkk7fe+AvHotHgEycV7IrYZzX4GfD99cl5OMMh4wHL0rM4BNXs84RncXYp+9GGwoIHP+gOm8NVWJs5HtFiMvTZKWrCtNEFqpKABl7SVaIxJlfXvKP0QPQ7pJFrLKwCInMs3GYKLMkDeinO1rpn/4PFgGzn0IVSCgFe/ptaZbhHMuO8SJwVJmqBI8ZuOGFNdNZqk5R6dEfxczihT6Uf4+cFXrXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cLoymPyKXi3rfIJHLQV1a8111FBvFsjenxTlQIFIYhU=; b=MggTARPjwi7yoqIqcYPcRnevfnJbW4TflYzEf0ObKKW1JSD4RQ4hR5r6tYzGq3crCrIejDntnvkOZCDj/mOyNsp1AnwNWNT29tKihxo5ol3lVy77Ter7HBpY+sVgurMKRApGwf5YtF9rKaLqiVxylkNJzSa/dT+rEaXZu6JMJH6QHnA6JvE/Ly0KKeusrVU3l2ebDkgMC5FVrsUIz/49eiAv5VSaY/0r+reuUEJb8qojvKfaHoBP9s4NUx7Wbf8e8fg0nGnV4LBCOFHpuYbUIDVU/IZ3MB1LK4RCg/Fj+QIWQ4vzYIVvx5DpXTWfr4BeiCjeVwgp/oc+G9db3PoGsw== Received: from DM6PR12MB4107.namprd12.prod.outlook.com (2603:10b6:5:218::7) by DM6PR12MB2731.namprd12.prod.outlook.com (2603:10b6:5:45::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.13; Mon, 8 Nov 2021 15:16:30 +0000 Received: from DM6PR12MB4107.namprd12.prod.outlook.com ([fe80::98ea:e961:8212:62f4]) by DM6PR12MB4107.namprd12.prod.outlook.com ([fe80::98ea:e961:8212:62f4%6]) with mapi id 15.20.4669.016; Mon, 8 Nov 2021 15:16:30 +0000 From: Elena Agostini To: =?iso-8859-1?Q?Mattias_R=F6nnblom?= , "dev@dpdk.org" CC: NBU-Contact-Thomas Monjalon Thread-Topic: [dpdk-dev] [PATCH v3 5/9] gpudev: add memory API Thread-Index: AQHXvGwMNWxX3539ZEGCf8+Ww3ELDavqgDmAgA9tv10= Date: Mon, 8 Nov 2021 15:16:30 +0000 Message-ID: References: <20210602203531.2288645-1-thomas@monjalon.net> <20211009015349.9694-1-eagostini@nvidia.com> <20211009015349.9694-6-eagostini@nvidia.com> <41c0dedd-c572-3b37-5fcb-67c90a3048cb@lysator.liu.se> In-Reply-To: <41c0dedd-c572-3b37-5fcb-67c90a3048cb@lysator.liu.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: lysator.liu.se; dkim=none (message not signed) header.d=none;lysator.liu.se; dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2ceba312-e60f-4368-15ef-08d9a2cabdfc x-ms-traffictypediagnostic: DM6PR12MB2731: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4jrXJCqQExziHA40aNnKv14ru9OdChQwQgGTQ7vHKjx0k2awaz6rKVivenk1g8IIFXS3T5lIjUcPrRDW5e2mMiWa1oGO1LrpjjLfQ8RfgsqiuZS7oi15aPTYnlshvHTggGt37YOsdZ/dp0VXUsFHN+dM1UYRZFmt+snm53nHArYuFmsHctKwmFuNLWoccugFPILfY0C9xGVOtumuSuqjX09HVhej8OJlGHB4i20j7JqqYwDROtg1AW9QuxMI+hKexiiVX4lIKcNpfbRrgFs1OkdPg2IHKTcsetmFTyPMQd9blsSLvIp4dMxo22s2d1bnzFgkSLbf2necqGC/yguO8qE0zQS6OaBIHRPb4yT9RRIiB5KYCBGVw8/olH9N9Avf88Bgjd0rQSm4N7eOcpkunGxxsDdo6byJ2GiK0OdoiJZkxnRh5enI2bfO0fBU2fGnR825r+0ca/yRKyRqYgG5waEElaB2tK5K5hGtSHo5MR6QZjXjSqd9zIGD5y+rxaseZBZ0r/WprUnGg/9b6Z+ILzSW8b+E3/KBWiMxB9QMCE75FjuJLkLG9NqZ/3w05RYXaWByJe6x/UYl6Lke82g81miNhb25tBoh1W//zUOb9NEh8ghzb40EmWm4BymN5XrDJ/5hegvqhoQ5VuSV96rVEqrvzWi05sVVr0RBi4ADRkDvVHABjA1BQl4wjW/2DUOh1PWLfItbhFjkTtTFx21a9A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR12MB4107.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(8936002)(2906002)(110136005)(86362001)(71200400001)(53546011)(66574015)(33656002)(9686003)(316002)(296002)(52536014)(186003)(38070700005)(26005)(122000001)(4326008)(30864003)(7696005)(5660300002)(55016002)(83380400001)(66476007)(66556008)(66446008)(6506007)(64756008)(91956017)(76116006)(8676002)(508600001)(66946007)(38100700002)(579004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?qFPn+tknMXjL7MebukeuhSosgFD8J8IClV4ieRGz2Idbj2chTSUJyPgzB7?= =?iso-8859-1?Q?Np8+mlgHqF1xsbTvjNe6s3EHAeAxi7OPY6nvwaqeIvRn8mYVEgOw1q0MCV?= =?iso-8859-1?Q?nUzvkNlKR7/MLD2lHJ+0PIltyldkbl8lXuPkGq++udz05yfU7YwtiRd4/8?= =?iso-8859-1?Q?aKNCeRRtcsy4QanJ3t0QL6XRPngI51R0El2UO/3wdlwa48CHZGDC8OBZ5s?= =?iso-8859-1?Q?aXbJQGbI2Fn5oKVtavVWBSeJJk4tdgwE7pBnxAyeNrQ3dFUejPpwfT8UjO?= =?iso-8859-1?Q?9BxtSYTOk43wGOtn5lv5dxy4D3XnxQgZkGFKfS7K8MD8c8rliRT9BY9Tce?= =?iso-8859-1?Q?y/2WG8vW4Z7GtjDEvnJ0A3933Yc1RrY9cMAUG9Sqqzv8tglz36tNYUgcrG?= =?iso-8859-1?Q?9XBjkZDdwnqqUaFFhavCRJvrC9/cDhdNqCdq/qS3/B/9dS4V5ejwQqMZ9b?= =?iso-8859-1?Q?e6JDmRdFLxubyW+hQOlQFTv5T6DWi8+4qX7B4XQcga+Ce5/5zo9SMQXNlg?= =?iso-8859-1?Q?KqDU76znnjgZeQWH+GjwxuWrqVMkSsRcMIxr3TozPmTaBGRD5tuswG7sOZ?= =?iso-8859-1?Q?SJS0rlyeb98WaeQpeLkVr4cYvvSbSShXDn8MRvd7XgBtPPf/KVGItUJkhh?= =?iso-8859-1?Q?8O6HjPnJkKOinY+T0qZnFPDmWdZoOqfPKpJQPfm+FaowuRBDcRJzxN8l4w?= =?iso-8859-1?Q?LQs5jp8BbdEhCUSYNriC7DknFWzdByJt9wMjOKdYqaUbOeTxJnjSZZ9iHg?= =?iso-8859-1?Q?gYyD2v1MsTBawlkcom6w2sj7pNrE7XVkCBq/UHaPJ6pIz1QSYi+ABZmSFe?= =?iso-8859-1?Q?Gkn/mstcpiSPpqzlMNqI8k9682Y062pVbiM7S/OerckSedFn/GqX3hz23b?= =?iso-8859-1?Q?O1DeAxgfxhZ6zCzCUh4j/X4umTwpHCcNQWApC0QhlyRfOUWXGSj+QVF80q?= =?iso-8859-1?Q?Zm2cYP2CoQ4ZZltqaoL3TqW2XrfY6+87xmhyxJbghfxlZUh6SbwSWO+u3w?= =?iso-8859-1?Q?E8fRtdKxOflbRPXiUph5u5Jh0XZyr7kYHQWyIimdGGnrdCNkZQO+2PJmEU?= =?iso-8859-1?Q?DPnNyJPzbt3p4f9eaJThecco+sBskwEfBDcSqr/ECRjcrQDol+Gkb+r5az?= =?iso-8859-1?Q?L9KiNBERm2JfS2kjaRB9Lvcngy9SHudAbhMJZtizXIfkCJBRfLmhKB/8o5?= =?iso-8859-1?Q?iAriHJVVqa/2qiPYwofdF0taFxraC3J9Wsizw+eplyim7PVfikd+Yzm68L?= =?iso-8859-1?Q?4Z9ZuB/4AmTx5zWFEW49S7Po58fu4zGPxQUde5yZhoIsfqXhXAJ75xAwWR?= =?iso-8859-1?Q?wr7qFxZFVnL9OEmWXxL60fiGJgtX6zvjrPzOpH2dv6GCSnlPKsDtzFaiFl?= =?iso-8859-1?Q?u61j5jkXGmysTBcbhfIrrNkn03vIO/eXLMj7IBkOja82EqvE/8Wa897QkJ?= =?iso-8859-1?Q?UHliyvdRAASEETJuE6KGcRbQENxt/2m/E8eABQJjoFVX9y8pE5YShwStRw?= =?iso-8859-1?Q?RxOi2sTIkme7Urh/JOIpzqsac0+hePwgLpMTS1qC8klM7KfWggfFYG2p0B?= =?iso-8859-1?Q?veKgEaxC0N/c3VBSjjTFIIIG2ZhcpitB5ap6rwlTIm092tcK5zzVO9evkJ?= =?iso-8859-1?Q?1MJKXpODFoHvuzqd2Bao6bsnnskwWyiXZJaPZrJ0pKMsyA4v84Gl5oLv78?= =?iso-8859-1?Q?vv0C1jM2dt6Bo5fGTcNQYc/Iic//uQlEUyQqvIrw?= MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4107.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2ceba312-e60f-4368-15ef-08d9a2cabdfc X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Nov 2021 15:16:30.2959 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: AGnMLLZBKNwBIUSV5vxzwyM75tIaL7cqEYWEPF2pKy9m4YfZ6wqOZOqFEFgO+i1AIICeuvVYbnwLTJtG+QXfog== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2731 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-dev] [PATCH v3 5/9] gpudev: add memory API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > From: Mattias R=F6nnblom > Date: Friday, 29 October 2021 at 21:38 > To: Elena Agostini , dev@dpdk.org > Cc: NBU-Contact-Thomas Monjalon > Subject: Re: [dpdk-dev] [PATCH v3 5/9] gpudev: add memory API> > > On 2021-10-09 03:53, eagostini@nvidia.com wrote: > > From: Elena Agostini > > > > In heterogeneous computing system, processing is not only in the CPU. > > Some tasks can be delegated to devices working in parallel. > > Such workload distribution can be achieved by sharing some memory. > > > > As a first step, the features are focused on memory management. > > A function allows to allocate memory inside the device, > > or in the main (CPU) memory while making it visible for the device. > > This memory may be used to save packets or for synchronization data. > > > > The next step should focus on GPU processing task control. > > > > Signed-off-by: Elena Agostini > > Signed-off-by: Thomas Monjalon > > --- > > app/test-gpudev/main.c | 118 ++++++++++++++++++++++++= + > > doc/guides/gpus/features/default.ini | 3 + > > doc/guides/prog_guide/gpudev.rst | 19 ++++ > > doc/guides/rel_notes/release_21_11.rst | 1 + > > lib/gpudev/gpudev.c | 101 +++++++++++++++++++++ > > lib/gpudev/gpudev_driver.h | 12 +++ > > lib/gpudev/rte_gpudev.h | 95 ++++++++++++++++++++ > > lib/gpudev/version.map | 4 + > > 8 files changed, 353 insertions(+) > > > > diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c > > index 6a73a54e84..98c02a3ee0 100644 > > --- a/app/test-gpudev/main.c > > +++ b/app/test-gpudev/main.c > > @@ -62,6 +62,110 @@ args_parse(int argc, char **argv) > > } > > } > > > > +static int > > +alloc_gpu_memory(uint16_t gpu_id) > > +{ > > + void * ptr_1 =3D NULL;> > Delete space between '*' and 'p'.> Thanks Mattias, I addressed all of your comments and I re-run checkpatch sc= ript In all the gpudev related files. > > + void * ptr_2 =3D NULL; > > + size_t buf_bytes =3D 1024; > > + int ret =3D 0;> > This initialization is redundant.> > > + > > + printf("\n=3D=3D=3D=3D=3D=3D=3D> TEST: Allocate GPU memory\n"); > > + > > + /* Alloc memory on GPU 0 */ > > + ptr_1 =3D rte_gpu_malloc(gpu_id, buf_bytes); > > + if(ptr_1 =3D=3D NULL) > > + {> > Misplaced braces.> > "if (" rather than "if(".> > > + fprintf(stderr, "rte_gpu_malloc GPU memory returned error= \n"); > > + return -1; > > + } > > + printf("GPU memory allocated at 0x%p %zdB\n", ptr_1, buf_bytes); > > + > > + ptr_2 =3D rte_gpu_malloc(gpu_id, buf_bytes); > > + if(ptr_2 =3D=3D NULL) > > + {> > Again, and throughout this file.> > > + fprintf(stderr, "rte_gpu_malloc GPU memory returned error= \n"); > > + return -1; > > + } > > + printf("GPU memory allocated at 0x%p %zdB\n", ptr_2, buf_bytes); > > + > > + ret =3D rte_gpu_free(gpu_id, (uint8_t*)(ptr_1)+0x700); > > + if(ret < 0) > > + { > > + printf("GPU memory 0x%p + 0x700 NOT freed because of memo= ry address not recognized by driver\n", ptr_1); > > + } > > + else > > + { > > + fprintf(stderr, "rte_gpu_free erroneusly freed GPU memory= 0x%p + 0x700\n", ptr_1); > > + return -1; > > + } > > + > > + ret =3D rte_gpu_free(gpu_id, ptr_2); > > + if(ret < 0) > > + { > > + fprintf(stderr, "rte_gpu_free returned error %d\n", ret); > > + return -1; > > + } > > + printf("GPU memory 0x%p freed\n", ptr_2); > > + > > + ret =3D rte_gpu_free(gpu_id, ptr_1); > > + if(ret < 0) > > + { > > + fprintf(stderr, "rte_gpu_free returned error %d\n", ret); > > + return -1; > > + } > > + printf("GPU memory 0x%p freed\n", ptr_1); > > + > > + return 0; > > +} > > + > > +static int > > +register_cpu_memory(uint16_t gpu_id) > > +{ > > + void * ptr =3D NULL; > > + size_t buf_bytes =3D 1024; > > + int ret =3D 0; > > + > > + printf("\n=3D=3D=3D=3D=3D=3D=3D> TEST: Register CPU memory\n"); > > + > > + /* Alloc memory on CPU visible from GPU 0 */ > > + ptr =3D rte_zmalloc(NULL, buf_bytes, 0); > > + if (ptr =3D=3D NULL) { > > + fprintf(stderr, "Failed to allocate CPU memory.\n"); > > + return -1; > > + } > > + > > + ret =3D rte_gpu_register(gpu_id, buf_bytes, ptr); > > + if(ret < 0) > > + { > > + fprintf(stderr, "rte_gpu_register CPU memory returned err= or %d\n", ret); > > + return -1; > > + } > > + printf("CPU memory registered at 0x%p %zdB\n", ptr, buf_bytes); > > + > > + ret =3D rte_gpu_unregister(gpu_id, (uint8_t*)(ptr)+0x700); > > + if(ret < 0) > > + { > > + printf("CPU memory 0x%p + 0x700 NOT unregistered because = of memory address not recognized by driver\n", ptr); > > + } > > + else > > + { > > + fprintf(stderr, "rte_gpu_free erroneusly freed GPU memory= 0x%p + 0x700\n", ptr); > > + return -1; > > + } > > + printf("CPU memory 0x%p unregistered\n", ptr); > > + > > + ret =3D rte_gpu_unregister(gpu_id, ptr); > > + if(ret < 0) > > + { > > + fprintf(stderr, "rte_gpu_unregister returned error %d\n",= ret); > > + return -1; > > + } > > + printf("CPU memory 0x%p unregistered\n", ptr); > > + > > + return 0; > > +} > > + > > int > > main(int argc, char **argv) > > { > > @@ -99,6 +203,20 @@ main(int argc, char **argv) > > } > > printf("\n\n"); > > > > + if(nb_gpus =3D=3D 0 > + { > > + fprintf(stderr, "Need at least one GPU on the system to r= un the example\n"); > > + return EXIT_FAILURE; > > + } > > + > > + gpu_id =3D 0; > > + > > + /** > > + * Memory tests > > + */ > > + alloc_gpu_memory(gpu_id); > > + register_cpu_memory(gpu_id); > > + > > /* clean up the EAL */ > > rte_eal_cleanup(); > > printf("Bye...\n"); > > diff --git a/doc/guides/gpus/features/default.ini b/doc/guides/gpus/fea= tures/default.ini > > index ec7a545eb7..87e9966424 100644 > > --- a/doc/guides/gpus/features/default.ini > > +++ b/doc/guides/gpus/features/default.ini > > @@ -8,3 +8,6 @@ > > ; > > [Features] > > Get device info =3D > > +Share CPU memory with device =3D > > +Allocate device memory =3D > > +Free memory =3D > > diff --git a/doc/guides/prog_guide/gpudev.rst b/doc/guides/prog_guide/g= pudev.rst > > index 7694639489..9aca69038c 100644 > > --- a/doc/guides/prog_guide/gpudev.rst > > +++ b/doc/guides/prog_guide/gpudev.rst > > @@ -30,6 +30,8 @@ Features > > This library provides a number of features: > > > > - Interoperability with device-specific library through generic handl= ers. > > +- Allocate and free memory on the device. > > +- Register CPU memory to make it visible from the device. > > > > > > API Overview > > @@ -46,3 +48,20 @@ that will be registered internally by the driver as = an additional device (child) > > connected to a physical device (parent). > > Each device (parent or child) is represented through a ID > > required to indicate which device a given operation should be execute= d on. > > + > > +Memory Allocation > > +~~~~~~~~~~~~~~~~~ > > + > > +gpudev can allocate on an input given GPU device a memory area > > +returning the pointer to that memory. > > +Later, it's also possible to free that memory with gpudev. > > +GPU memory allocated outside of the gpudev library > > +(e.g. with GPU-specific library) cannot be freed by the gpudev library= . > > + > > +Memory Registration > > +~~~~~~~~~~~~~~~~~~~ > > + > > +gpudev can register a CPU memory area to make it visible from a GPU de= vice. > > +Later, it's also possible to unregister that memory with gpudev. > > +CPU memory registered outside of the gpudev library > > +(e.g. with GPU specific library) cannot be unregistered by the gpudev = library. > > diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_no= tes/release_21_11.rst > > index 4986a35b50..c4ac5e3053 100644 > > --- a/doc/guides/rel_notes/release_21_11.rst > > +++ b/doc/guides/rel_notes/release_21_11.rst > > @@ -65,6 +65,7 @@ New Features > > * **Introduced GPU device class with first features:** > > > > * Device information > > + * Memory management > > > > * **Added new RSS offload types for IPv4/L4 checksum in RSS flow.** > > > > diff --git a/lib/gpudev/gpudev.c b/lib/gpudev/gpudev.c > > index f0690cf730..1d8318f769 100644 > > --- a/lib/gpudev/gpudev.c > > +++ b/lib/gpudev/gpudev.c > > @@ -6,6 +6,7 @@ > > #include > > #include > > #include > > +#include > > #include > > #include > > > > @@ -523,3 +524,103 @@ rte_gpu_info_get(int16_t dev_id, struct rte_gpu_i= nfo *info) > > } > > return GPU_DRV_RET(dev->ops.dev_info_get(dev, info)); > > } > > + > > +void * > > +rte_gpu_malloc(int16_t dev_id, size_t size) > > +{ > > + struct rte_gpu *dev; > > + void *ptr; > > + int ret; > > + > > + dev =3D gpu_get_by_id(dev_id); > > + if (dev =3D=3D NULL) { > > + GPU_LOG(ERR, "alloc mem for invalid device ID %d", dev_id= ); > > + rte_errno =3D ENODEV; > > + return NULL; > > + } > > + > > + if (dev->ops.mem_alloc =3D=3D NULL) { > > + GPU_LOG(ERR, "mem allocation not supported"); > > + rte_errno =3D ENOTSUP; > > + return NULL; > > + } > > + > > + if (size =3D=3D 0) /* dry-run */ > > + return NULL; > > + > > + ret =3D dev->ops.mem_alloc(dev, size, &ptr); > > + > > + switch (ret) { > > + case 0: > > + return ptr; > > + case -ENOMEM: > > + case -E2BIG: > > + rte_errno =3D -ret; > > + return NULL; > > + default: > > + rte_errno =3D -EPERM; > > + return NULL; > > + } > > +} > > + > > +int > > +rte_gpu_register(int16_t dev_id, size_t size, void * ptr) > > +{ > > + struct rte_gpu *dev; > > + > > + dev =3D gpu_get_by_id(dev_id); > > + if (dev =3D=3D NULL) { > > + GPU_LOG(ERR, "alloc mem for invalid device ID %d", dev_id= ); > > + rte_errno =3D ENODEV; > > + return -rte_errno; > > + } > > + > > + if (dev->ops.mem_register =3D=3D NULL) { > > + GPU_LOG(ERR, "mem registration not supported"); > > + rte_errno =3D ENOTSUP; > > + return -rte_errno; > > + } > > + > > + if (size =3D=3D 0 || ptr =3D=3D NULL) /* dry-run */ > > + return -EINVAL; > > + > > + return GPU_DRV_RET(dev->ops.mem_register(dev, size, ptr)); > > +} > > + > > +int > > +rte_gpu_unregister(int16_t dev_id, void * ptr) > > +{ > > + struct rte_gpu *dev; > > + > > + dev =3D gpu_get_by_id(dev_id); > > + if (dev =3D=3D NULL) { > > + GPU_LOG(ERR, "unregister mem for invalid device ID %d", d= ev_id); > > + rte_errno =3D ENODEV; > > + return -rte_errno; > > + } > > + > > + if (dev->ops.mem_unregister =3D=3D NULL) { > > + rte_errno =3D ENOTSUP; > > + return -rte_errno; > > + } > > + return GPU_DRV_RET(dev->ops.mem_unregister(dev, ptr)); > > +} > > + > > +int > > +rte_gpu_free(int16_t dev_id, void *ptr) > > +{ > > + struct rte_gpu *dev; > > + > > + dev =3D gpu_get_by_id(dev_id); > > + if (dev =3D=3D NULL) { > > + GPU_LOG(ERR, "free mem for invalid device ID %d", dev_id)= ; > > + rte_errno =3D ENODEV; > > + return -rte_errno; > > + } > > + > > + if (dev->ops.mem_free =3D=3D NULL) { > > + rte_errno =3D ENOTSUP; > > + return -rte_errno; > > + } > > + return GPU_DRV_RET(dev->ops.mem_free(dev, ptr)); > > +} > > diff --git a/lib/gpudev/gpudev_driver.h b/lib/gpudev/gpudev_driver.h > > index 9459c7e30f..11015944a6 100644 > > --- a/lib/gpudev/gpudev_driver.h > > +++ b/lib/gpudev/gpudev_driver.h > > @@ -27,12 +27,24 @@ enum rte_gpu_state { > > struct rte_gpu; > > typedef int (rte_gpu_close_t)(struct rte_gpu *dev); > > typedef int (rte_gpu_info_get_t)(struct rte_gpu *dev, struct rte_gpu_= info *info); > > +typedef int (rte_gpu_mem_alloc_t)(struct rte_gpu *dev, size_t size, vo= id **ptr); > > +typedef int (rte_gpu_free_t)(struct rte_gpu *dev, void *ptr); > > +typedef int (rte_gpu_mem_register_t)(struct rte_gpu *dev, size_t size,= void *ptr); > > +typedef int (rte_gpu_mem_unregister_t)(struct rte_gpu *dev, void *ptr)= ; > > > > struct rte_gpu_ops { > > /* Get device info. If NULL, info is just copied. */ > > rte_gpu_info_get_t *dev_info_get; > > /* Close device or child context. */ > > rte_gpu_close_t *dev_close; > > + /* Allocate memory in device. */ > > + rte_gpu_mem_alloc_t *mem_alloc; > > + /* Register CPU memory in device. */ > > + rte_gpu_mem_register_t *mem_register; > > + /* Free memory allocated or registered in device. */ > > + rte_gpu_free_t *mem_free; > > + /* Unregister CPU memory in device. */ > > + rte_gpu_mem_unregister_t *mem_unregister; > > }; > > > > struct rte_gpu_mpshared { > > diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h > > index df75dbdbab..3c276581c0 100644 > > --- a/lib/gpudev/rte_gpudev.h > > +++ b/lib/gpudev/rte_gpudev.h > > @@ -9,6 +9,7 @@ > > #include > > #include > > > > +#include > > #include > > > > /** > > @@ -292,6 +293,100 @@ int rte_gpu_callback_unregister(int16_t dev_id, e= num rte_gpu_event event, > > __rte_experimental > > int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Allocate a chunk of memory usable by the device. > > + * > > + * @param dev_id > > + * Device ID requiring allocated memory. > > + * @param size > > + * Number of bytes to allocate. > > + * Requesting 0 will do nothing. > > + * > > + * @return > > + * A pointer to the allocated memory, otherwise NULL and rte_errno i= s set: > > + * - ENODEV if invalid dev_id > > + * - EINVAL if reserved flags > > + * - ENOTSUP if operation not supported by the driver > > + * - E2BIG if size is higher than limit > > + * - ENOMEM if out of space > > + * - EPERM if driver error > > + */ > > +__rte_experimental > > +void *rte_gpu_malloc(int16_t dev_id, size_t size) > > +__rte_alloc_size(2); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Deallocate a chunk of memory allocated with rte_gpu_malloc(). > > + * > > + * @param dev_id > > + * Reference device ID. > > + * @param ptr > > + * Pointer to the memory area to be deallocated. > > + * NULL is a no-op accepted value. > > + * > > + * @return > > + * 0 on success, -rte_errno otherwise:> > I don't think you are supposed to set rte_errno if it's not needed, > which is not the case here (since you return the error code).> > > + * - ENODEV if invalid dev_id > > + * - ENOTSUP if operation not supported by the driver > > + * - EPERM if driver error > > + */ > > +__rte_experimental > > +int rte_gpu_free(int16_t dev_id, void *ptr); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Register a chunk of memory on the CPU usable by the device. > > + * > > + * @param dev_id > > + * Device ID requiring allocated memory. > > + * @param size > > + * Number of bytes to allocate. > > + * Requesting 0 will do nothing. > > + * @param ptr > > + * Pointer to the memory area to be registered. > > + * NULL is a no-op accepted value. > > + > > + * @return > > + * A pointer to the allocated memory, otherwise NULL and rte_errno i= s set: > > + * - ENODEV if invalid dev_id > > + * - EINVAL if reserved flags > > + * - ENOTSUP if operation not supported by the driver > > + * - E2BIG if size is higher than limit > > + * - ENOMEM if out of space > > + * - EPERM if driver error > > + */ > > +__rte_experimental > > +int rte_gpu_register(int16_t dev_id, size_t size, void * ptr); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice. > > + * > > + * Deregister a chunk of memory previusly registered with rte_gpu_mem_= register() > > + * > > + * @param dev_id > > + * Reference device ID. > > + * @param ptr > > + * Pointer to the memory area to be unregistered. > > + * NULL is a no-op accepted value. > > + * > > + * @return > > + * 0 on success, -rte_errno otherwise: > > + * - ENODEV if invalid dev_id > > + * - ENOTSUP if operation not supported by the driver > > + * - EPERM if driver error > > + */ > > +__rte_experimental > > +int rte_gpu_unregister(int16_t dev_id, void *ptr); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/gpudev/version.map b/lib/gpudev/version.map > > index 58dc632393..d4a65ebd52 100644 > > --- a/lib/gpudev/version.map > > +++ b/lib/gpudev/version.map > > @@ -8,9 +8,13 @@ EXPERIMENTAL { > > rte_gpu_close; > > rte_gpu_count_avail; > > rte_gpu_find_next; > > + rte_gpu_free; > > rte_gpu_info_get; > > rte_gpu_init; > > rte_gpu_is_valid; > > + rte_gpu_malloc; > > + rte_gpu_register; > > + rte_gpu_unregister; > > }; > > > > INTERNAL { > >