From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 85D07A0A0F; Fri, 4 Jun 2021 13:07:42 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EF6444068F; Fri, 4 Jun 2021 13:07:41 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 55DAE40147 for ; Fri, 4 Jun 2021 13:07:37 +0200 (CEST) IronPort-SDR: 1sXHVkW+zHHTaHKO0zy/Aj5gPPyoMKqN74TDav7j/CzS41VsCt//L1fg4lv+50ybKgCpBBcku6 Kfd9+U3d4Zug== X-IronPort-AV: E=McAfee;i="6200,9189,10004"; a="265434733" X-IronPort-AV: E=Sophos;i="5.83,248,1616482800"; d="scan'208";a="265434733" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jun 2021 04:07:30 -0700 IronPort-SDR: U+q//54rZD1YCJDEPEhBFb+O6TsHJUg5v4KLpMNlqyR+4/R0HSgiWMsQm2QANjvvmEAMEEtnUs 3+AWeZF2lXDg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,248,1616482800"; d="scan'208";a="439168190" Received: from orsmsx606.amr.corp.intel.com ([10.22.229.19]) by orsmga007.jf.intel.com with ESMTP; 04 Jun 2021 04:07:30 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Fri, 4 Jun 2021 04:07:30 -0700 Received: from orsmsx604.amr.corp.intel.com (10.22.229.17) by ORSMSX607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Fri, 4 Jun 2021 04:07:29 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4 via Frontend Transport; Fri, 4 Jun 2021 04:07:29 -0700 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.45) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.4; Fri, 4 Jun 2021 04:07:29 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MZw+wzcfDNkD6zIpAzw/QZJRX+/n4zZhAf4M6FKc1MIUG4ZoBRAYJL2BlEeD1awphcs/e2pDtt8q0AvZmNBZQ6LT0nNbJab7TofiQwAl/fxiaOOjhb9dX+gYWO25/9zCc1MhbY7xa3U6k5obzkxICxiPyujNeSj0ae+jqMNLmNeW8kgXX7XMeBvhc/237js26NXOvHEI303RfQxNPKdl8GQ7SFhY+giCQATElPL08xGd4wMUow9iiUTUzVpOW4s8eFPUIxnWhgNA69Ls6ba7nWvgeSNPC3+j8a7gcxt4C2xv0ykabe+LvjcIFgS/1lKBPdV1dzpAl94VqQlPK51Vqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nu/9aN9lu2gXhLIgL3riV+XtWOpEGrPJADcpalnlYJI=; b=NOgvAOpmxU/nr7cyPzj/fhPfX+QyOWmSdzttu8Ob0wns5/xibFy5M97cCqulIIKpoDqLw/98tqv9qj91oHhZ2Wf8Nby0BlPmDHgFJyimK91PHIPpVVmTaPKIAuLULNEqgjV9AcLTrL988ZrxnU7nEhTsOtG9k4K3q8J9zIDJl92K7LuLFtA8+tqKbQfGT5CS+7S0LlNaNsyQ6UlnTvecXZ1U1j6KCQxXzaqhniXyFUfW0o0VAal4L+MA8nG7ZO/x3P9sfdFAB8IYpXGMmysFDvWRi1g7bbHHzZ8SbSA4mMyLXFE+tTg12Fv5/cgLEK1t5w+HjOjBwLqRHLpU4rkcUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nu/9aN9lu2gXhLIgL3riV+XtWOpEGrPJADcpalnlYJI=; b=siHDQ75ty8XPTjsMcShrzyWoVOSn7rVXa7VKvqRNY7UYEq8AesMX2HTCydbaP4jtLTjt3kFti0oPh0QqxRkTo1WRrQsQxWHDbOAhMnRCZ6s1WMIj7Z6up92T56gVWBAjmx/GCkCk81hN4JR0RgL3yl7wWuQvhQwrFBhKppB9UmI= Received: from BN8PR11MB3795.namprd11.prod.outlook.com (2603:10b6:408:82::31) by BN6PR1101MB2260.namprd11.prod.outlook.com (2603:10b6:405:53::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4173.20; Fri, 4 Jun 2021 11:07:28 +0000 Received: from BN8PR11MB3795.namprd11.prod.outlook.com ([fe80::8807:6661:e425:102]) by BN8PR11MB3795.namprd11.prod.outlook.com ([fe80::8807:6661:e425:102%7]) with mapi id 15.20.4195.025; Fri, 4 Jun 2021 11:07:28 +0000 From: "Wang, Haiyue" To: Thomas Monjalon , "dev@dpdk.org" CC: Elena Agostini Thread-Topic: [dpdk-dev] [PATCH] gpudev: introduce memory API Thread-Index: AQHXV+8BBxLOVwnssEGBVBOBXmJb5asDsYIA Date: Fri, 4 Jun 2021 11:07:28 +0000 Message-ID: References: <20210602203531.2288645-1-thomas@monjalon.net> In-Reply-To: <20210602203531.2288645-1-thomas@monjalon.net> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.0.76 authentication-results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.142.30] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bbe49160-ba43-4df3-1e15-08d92748f104 x-ms-traffictypediagnostic: BN6PR1101MB2260: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vAn13Df5t94S0NSZPtoqzbBuWz5mrgmsJhmVVxnlsM28WGW13GMjMbDCB6v/r5qVa0Umv55kOcyDSnQy1kpvFweMWtnutA+vhATgh6IIM7SWRjuWR0fYh9D3XiQd39+Uym4z7DfrueUzq3ahqC0Ib4rvu/dZ4XlnyIUxpdNf41YoeeZ3fIyps6YimWEHpIpxSIvderDNe3kofsmy+HXEirK5CSM7jDbufzlMjwMu9QBRPuvsHjxpYUdrWo+iVFnkGtP3Z8/OCulGL1a9BENMO5Ic3BOENHIAmqAcHHimL2ejCeimKZ49WHNzrdgXwmHR3XFAdM5QrZ6P8iL58t1nQpKDtxu4VRlnRXhyBucJosXfDz9IObqu4go8Efpm0swGev8N1V11tLO6XMMPYbwUXkp1eq+OmuAWsovJ37ImrBwWp018NIJeUy2/bKXfCKTIJ6t+9six1O1NyXM+ppb3GY8DUQ2i7sBsFpG2mLRB9xXgIr9uoXizfZn2zEbLRnId6pYkl//37FjWQtYZ1SWh1qrC1m9ZZjlP6ysaLAZA4ieOz0NWNnhRNONk3HwPr1172j3lWVuylQ7PzD6k/yZ7tZxxGtjoB1DTXgXPug7lLmw= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN8PR11MB3795.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(346002)(396003)(366004)(136003)(376002)(39860400002)(33656002)(122000001)(71200400001)(38100700002)(110136005)(55016002)(66946007)(186003)(66476007)(478600001)(8936002)(8676002)(66446008)(86362001)(64756008)(83380400001)(5660300002)(7696005)(4326008)(53546011)(52536014)(26005)(6506007)(76116006)(2906002)(66556008)(316002)(9686003); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?yRmhLpBTkoHV9vvBh0wmriK/hzooLfBRg7wLcXJfiPmVoFDdUzYlrBbFO1Ar?= =?us-ascii?Q?A968+HdYgxoKYfaygIF336HB5BxVM86RzIRXcCfXz0AYcTMQXHPEsbBS2nHQ?= =?us-ascii?Q?kcJTvK6rtDu3vrGvJWPpxpBbr8jRcF6lU/JoyNeQhE5OmCNGcU+Vkc0gGeUs?= =?us-ascii?Q?mKdrPG6LnEJllBzRUq8C/SpmZFvWPZ+PW51fmY0WyF7a34eujvwTNgBSzazS?= =?us-ascii?Q?EVEbMH+eT8AirQ/czgYoN50xYESWqKxejXRVxFRNM/YhZ5N2b4tG2JddWIcA?= =?us-ascii?Q?hdmTwNZOIUEpImHnAuypGAE8fWUpoKnihe6lXUMgorAceyQjV/mJDmPTReuq?= =?us-ascii?Q?3QWG23fczXSCwNMv5m+8G9yVhcbG2SEN+TDjBeshZZrOK2AhZWoDykGb46AI?= =?us-ascii?Q?c8D8LphD4Kwz0yXzJJGWP5pVjl4i0A/EJa9qA+hehzsBmdE32VV+UMnNM8Bc?= =?us-ascii?Q?Jt7LwB3FxshaNKNiEv0KBnOrgxDBk7BkCnwuqeEqIX7UB0ckWVylfrEOIdTG?= =?us-ascii?Q?cdxYY7KSFz1sIIZNSYT6Jjxf09XL5zBivfWDrpEPRn8a1qjf/MI6PMDIL+XL?= =?us-ascii?Q?Ym3OYebI+FQvWHTi0REhxKsuFfSmO/BTqpu9IkoqWl2bOZUAUmRo8QcCznHE?= =?us-ascii?Q?snJsnDFD8cuo1sMZ92ztkE6gHA7wbztPasua74gx1usiMQlIsGdimMtEo4E/?= =?us-ascii?Q?swXwjghLUSR4kraVnaLTdhiEtJxrltiXEPIyiYYCVxl5TYsOhP8r587dc3zy?= =?us-ascii?Q?erq9ToseHtYWl3+CUgbHA4Us5522m0PcWm4+bB+B/NsDh1pfAM4dTSMSm5Xt?= =?us-ascii?Q?pnsNw8slGXp4JGo0DUM7Z6Zzwbjs+RgZpOJrkwXujjqG8e3ut3cvfbMMMbBc?= =?us-ascii?Q?pYFbsFDmDPiGmxq8+EnQUFC3rGzd81ET9olDo0Avcetl+mvmT1dy46mnNw48?= =?us-ascii?Q?ck3IKe3oqsOMfPzWvooafMVK8XkjdNarI4KpLVRlE6lzjF9H+UCbv0dOAKUR?= =?us-ascii?Q?x6xAp14EkLGwGrI43O5DvgiHZWaMjDeXyEaMTCdlLn+fkQ95el7e5gMVxUSl?= =?us-ascii?Q?PrWRes0YoiTjrbLP4DabqzMOi8fKN+IZtrXq3woNrNBX5UxFkCy1N/FVci23?= =?us-ascii?Q?5PDQFsSEYS6OijA6EZ6HoLYoHNloKAcmtvJEhses0lRl7t8YMScvS/ag5m0y?= =?us-ascii?Q?5rIIoae2nj/F863LySoZK2asRQ+Lp43dUkUf3Gc/aUm7ECNDQK3gkPpG+rE8?= =?us-ascii?Q?6LcX6vATkDoT4CavkNTsJXsdK7gez3Ku75KpYVEI+ZSUaX5v2bMOiVtsPH0K?= =?us-ascii?Q?/wTIVleaOp3QpY+SSBgvvABo?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN8PR11MB3795.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bbe49160-ba43-4df3-1e15-08d92748f104 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jun 2021 11:07:28.2714 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BkqmlKHkCHOXcsc4VSCGaUCsqaq2dFULIFdKRkO7VrYPdm5exhcip3Yj+d134kLyVJzi2tvKGJs2fY9l1E5aCA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2260 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] gpudev: introduce memory API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: dev On Behalf Of Thomas Monjalon > Sent: Thursday, June 3, 2021 04:36 > To: dev@dpdk.org > Cc: Elena Agostini > Subject: [dpdk-dev] [PATCH] gpudev: introduce memory API >=20 > From: Elena Agostini >=20 > The new library gpudev is for dealing with GPU from a DPDK application > in a vendor-agnostic way. >=20 > As a first step, the features are focused on memory management. > A function allows to allocate memory inside the GPU, > while another one allows to use main (CPU) memory from the GPU. >=20 > The infrastructure is prepared to welcome drivers in drivers/gpu/ > as the upcoming NVIDIA one, implementing the gpudev API. > Other additions planned for next revisions: > - C implementation file > - guide documentation > - unit tests > - integration in testpmd to enable Rx/Tx to/from GPU memory. >=20 > The next step should focus on GPU processing task control. >=20 > Signed-off-by: Elena Agostini > Signed-off-by: Thomas Monjalon > --- > .gitignore | 1 + > MAINTAINERS | 6 + > doc/api/doxy-api-index.md | 1 + > doc/api/doxy-api.conf.in | 1 + > doc/guides/conf.py | 8 ++ > doc/guides/gpus/features/default.ini | 13 ++ > doc/guides/gpus/index.rst | 11 ++ > doc/guides/gpus/overview.rst | 7 + > doc/guides/index.rst | 1 + > doc/guides/prog_guide/gpu.rst | 5 + > doc/guides/prog_guide/index.rst | 1 + > drivers/gpu/meson.build | 4 + > drivers/meson.build | 1 + > lib/gpudev/gpu_driver.h | 44 +++++++ > lib/gpudev/meson.build | 9 ++ > lib/gpudev/rte_gpudev.h | 183 +++++++++++++++++++++++++++ > lib/gpudev/version.map | 11 ++ > lib/meson.build | 1 + > 18 files changed, 308 insertions(+) > create mode 100644 doc/guides/gpus/features/default.ini > create mode 100644 doc/guides/gpus/index.rst > create mode 100644 doc/guides/gpus/overview.rst > create mode 100644 doc/guides/prog_guide/gpu.rst > create mode 100644 drivers/gpu/meson.build > create mode 100644 lib/gpudev/gpu_driver.h > create mode 100644 lib/gpudev/meson.build > create mode 100644 lib/gpudev/rte_gpudev.h > create mode 100644 lib/gpudev/version.map >=20 > +#include > + > +#include > + > +#include "rte_gpudev.h" > + > +struct rte_gpu_dev; > + > +typedef int (*gpu_malloc_t)(struct rte_gpu_dev *dev, size_t size, void *= *ptr); > +typedef int (*gpu_free_t)(struct rte_gpu_dev *dev, void *ptr); > + > +struct rte_gpu_dev { > + /* Backing device. */ > + struct rte_device *device; > + /* GPU info structure. */ > + struct rte_gpu_info info; > + /* Counter of processes using the device. */ > + uint16_t process_cnt; > + /* If device is currently used or not. */ > + enum rte_gpu_state state; > + /* FUNCTION: Allocate memory on the GPU. */ > + gpu_malloc_t gpu_malloc; > + /* FUNCTION: Allocate memory on the CPU visible from the GPU. */ > + gpu_malloc_t gpu_malloc_visible; > + /* FUNCTION: Free allocated memory on the GPU. */ > + gpu_free_t gpu_free; I'm wondering that we can define the malloc type as: typedef int (*gpu_malloc_t)(struct rte_gpu_dev *dev, size_t size, void **pt= r, unsigned int flags) #define RTE_GPU_MALLOC_F_CPU_VISIBLE 0x01u --> gpu_malloc_visible Then only one malloc function member is needed, paired with 'gpu_free'. > + /* Device interrupt handle. */ > + struct rte_intr_handle *intr_handle; > + /* Driver-specific private data. */ > + void *dev_private; > +} __rte_cache_aligned; > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Allocate a chunk of memory on the GPU. > + * > + * @param gpu_id > + * GPU ID to allocate memory. > + * @param size > + * Number of bytes to allocate. > + * @param ptr > + * Pointer to store the address of the allocated memory. > + * > + * @return > + * 0 on success, -1 otherwise. > + */ > +__rte_experimental > +int rte_gpu_malloc(uint16_t gpu_id, size_t size, void **ptr); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice. > + * > + * Allocate a chunk of memory on the CPU that is visible from the GPU. > + * > + * @param gpu_id > + * Reference GPU ID. > + * @param size > + * Number of bytes to allocate. > + * @param ptr > + * Pointer to store the address of the allocated memory. > + * > + * @return > + * 0 on success, -1 otherwise. > + */ > +__rte_experimental > +int rte_gpu_malloc_visible(uint16_t gpu_id, size_t size, void **ptr); Then 'rte_gpu_malloc_visible' is no needed, and the new call is: rte_gpu_malloc(uint16_t gpu_id, size_t size, void **ptr, RTE_GPU_MALLOC_F_C= PU_VISIBLE). Also, we can define more flags for feature extension. ;-) > + > +#ifdef __cplusplus > +} > -- > 2.31.1