From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 2F5D9A0A0F;
	Sat,  5 Jun 2021 13:09:43 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id A34F240147;
	Sat,  5 Jun 2021 13:09:42 +0200 (CEST)
Received: from mga17.intel.com (mga17.intel.com [192.55.52.151])
 by mails.dpdk.org (Postfix) with ESMTP id 9B30B4003E
 for <dev@dpdk.org>; Sat,  5 Jun 2021 13:09:40 +0200 (CEST)
IronPort-SDR: goIKHHO76LKrxCpYXMRppAHbOKM4TyDq6DWFTTr8Fu6skrs9cAqUCCG0Y6AVMK45hEJ/wFrhnd
 4ZiqNfE4hLjg==
X-IronPort-AV: E=McAfee;i="6200,9189,10005"; a="184797410"
X-IronPort-AV: E=Sophos;i="5.83,250,1616482800"; d="scan'208";a="184797410"
Received: from fmsmga003.fm.intel.com ([10.253.24.29])
 by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 05 Jun 2021 04:09:36 -0700
IronPort-SDR: kAU4fL6um5DYm9IEtJ5zYSA0yUY0tsyfekt42xgIxIXkrc4QB2m30n810XUiOnxkIpeG5esa7f
 yD7o4m4mA6eA==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.83,250,1616482800"; d="scan'208";a="475729691"
Received: from orsmsx606.amr.corp.intel.com ([10.22.229.19])
 by FMSMGA003.fm.intel.com with ESMTP; 05 Jun 2021 04:09:36 -0700
Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by
 ORSMSX606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.1.2242.4; Sat, 5 Jun 2021 04:09:35 -0700
Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by
 orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4
 via Frontend Transport; Sat, 5 Jun 2021 04:09:35 -0700
Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.169)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server
 (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 15.1.2242.4; Sat, 5 Jun 2021 04:09:35 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=AhkcKSu+FL+LYCV276gMInsfAzqeED9gBtazYQC0HoL0RbbsXecF1tgYD7/PlmgEsJ5nBGTMHRpVP+HIfT9CV7mjpWEZRgj56MeBimZqD0nXcAD64as4dGGIVmK/VcAK4eGcfT6tTqq+toou0Z7GrTBzAsI6D8YutLmGvvieX1qW6GATQpniv3xPCXzNOyII+rnMR6YL5IYBR9Nv1dWa8iHkUfOx/QY7hi+zOUn0hvwWimlM5th6XeWAhRi2tYgWbh5l7sF0ZgLGxuQJYGhisN+DXT3oH+GwdFkrLa0U6vCFHrdsFFTzht2Y3rGDWbgl0NX/8UbhcD+3sZBBW9ZqnQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bpqogM3Aks73rdEuGHF/4Cp0yl/F3WCmXQ6vaOCgYAg=;
 b=BnyEWR26lNy0gRBE7Zf4Efax04/sEJPredM9j0L53GULXfErbIb2mDO0f/a62oyprIXvDSrh7d2uZ/SxROhUrvUDPFYSh6ecYPt8MtgmKShnk1a/WMRnuJzU0tYKVEzrd+Q7d7EIjg2Iuy3UP8LOIU2fHUscKObug87ScIU68Kns0jh23hv0caqRwM5+wC/ya70UGtGkSknL9EN4xGH5mpm0M+n+ZDn0V9yaNBdsYmBDQNHEDqluhNEFxtbJNnsngEirQWweg3VqRiHJIXtaRaS5ULtwABalXFouDiuxzCWIuiK40JgH6EYwHhQvA4MxZJzW/Y/e3QkNq0hfvFro6Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; 
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=bpqogM3Aks73rdEuGHF/4Cp0yl/F3WCmXQ6vaOCgYAg=;
 b=n4gDxoVlEDmTeOTsrEPHV3l4ExSc7SsD9BDFvW9DdNR0/0hhCVSGEzgpeoCHo8W3ai2q8gkIUJKLZTxfcO6SzhzQAmwpN9FG0hpZncDzek+7Bu0Rc4PGgGsHSqFETVqsnlueAIZBVQHwW+wLJ6Nk9sajRSYv55irQt+mbgd71Bk=
Received: from BN8PR11MB3795.namprd11.prod.outlook.com (2603:10b6:408:82::31)
 by BN7PR11MB2740.namprd11.prod.outlook.com (2603:10b6:406:b7::15)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.24; Sat, 5 Jun
 2021 11:09:24 +0000
Received: from BN8PR11MB3795.namprd11.prod.outlook.com
 ([fe80::8807:6661:e425:102]) by BN8PR11MB3795.namprd11.prod.outlook.com
 ([fe80::8807:6661:e425:102%7]) with mapi id 15.20.4195.025; Sat, 5 Jun 2021
 11:09:24 +0000
From: "Wang, Haiyue" <haiyue.wang@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>
CC: "dev@dpdk.org" <dev@dpdk.org>, Elena Agostini <eagostini@nvidia.com>,
 "andrew.rybchenko@oktetlabs.ru" <andrew.rybchenko@oktetlabs.ru>, "Yigit,
 Ferruh" <ferruh.yigit@intel.com>, "jerinj@marvell.com" <jerinj@marvell.com>
Thread-Topic: [dpdk-dev] [PATCH] gpudev: introduce memory API
Thread-Index: AQHXV+8BBxLOVwnssEGBVBOBXmJb5asDsYIAgAAdKQCAAAozkIAADNIAgABA81CAAOgAgIAANFIA
Date: Sat, 5 Jun 2021 11:09:24 +0000
Message-ID: <BN8PR11MB379529D01D3D4622B91F2536F73A9@BN8PR11MB3795.namprd11.prod.outlook.com>
References: <20210602203531.2288645-1-thomas@monjalon.net>
 <1648109.dXcHqNedQ3@thomas>
 <BN8PR11MB379589288419BD793A0EB923F73B9@BN8PR11MB3795.namprd11.prod.outlook.com>
 <3569019.DFCUuWcX1B@thomas>
In-Reply-To: <3569019.DFCUuWcX1B@thomas>
Accept-Language: zh-CN, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.6.0.76
authentication-results: monjalon.net; dkim=none (message not signed)
 header.d=none;monjalon.net; dmarc=none action=none header.from=intel.com;
x-originating-ip: [192.198.143.9]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: aa23fc41-0c78-43e0-5de8-08d9281260cb
x-ms-traffictypediagnostic: BN7PR11MB2740:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <BN7PR11MB27403C3E3A3004DE3DCF1715F73A9@BN7PR11MB2740.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: Aff3ZO2Js8+MwAl8npCYJi47R4nggT8+yIUuThm9otyiRzvIfhBNCE4mu/AObw+bMfbcMVXMNdbC+vfnXf4bEIeIfbkoUTRyMtCJGNTvanLoHHxB8ZpOGWxCbIgGwuz2QkoOQ0ekc1lo4589hNIIJh58AlWT1x2P4O5yw+w2B1rFLATy/IsHXdEB2QrRvu4MdE5xI3pAVxIdn7Mv2ZHyux9kk9sIwaaECse3IsX5LPzqoJUs6THeeycNrOpJnvXvORAwkbO3B8evh8JJRC48mkcGWCMshR0kieDpmE2Y2OHkE+JW9UfV1Mk7drO5WYteg0QTdxzBOUYhdeR8M5Ch37BRTzrqxN6xiz4aIH9ECx2hZkgBYhZ+Ac8EpeLIXGdpynlEg9HwXZLV219upBteb/Pp1MUcoS7XVUNe7R3YQTI1EYuIukn3wnxv2amx3kcHzd2HM9gStseBrLmK33vsQoX9bhZCapxGX8jWQK58uaVEAdhtm1Pjcx4dWZg20M1Hj8V08zHeFTPUVn/JFa3xKE/ZBq0POn6KLs6UhQEnv0wMyK0eRskFu30ncmq6fV1FekNl15zDZul+flxboCk3nUm9DePsWQGe6cFLNH2/Y0A=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN8PR11MB3795.namprd11.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(39860400002)(376002)(366004)(346002)(396003)(136003)(52536014)(5660300002)(316002)(66476007)(66556008)(33656002)(86362001)(66946007)(76116006)(54906003)(71200400001)(2906002)(6916009)(8676002)(66446008)(9686003)(64756008)(8936002)(55016002)(83380400001)(53546011)(6506007)(7696005)(478600001)(26005)(122000001)(4326008)(186003)(38100700002);
 DIR:OUT; SFP:1102; 
x-ms-exchange-antispam-messagedata: =?us-ascii?Q?7Kv0b+8Ea7rsv/IN3eXVRlBStInd9wCYsNjaFSQPPe1dmgSGreWvDMnnxb8H?=
 =?us-ascii?Q?QOihCZYn6psYtkmVqsKE7QwiHfbwdNivrb9b1n8FVcNKv0e1KYlJp1Nt6WSz?=
 =?us-ascii?Q?0m8gxN6+8sVB/mUZ/9hq+0oar3UQBmP8Vw6BQSZ72xhuKC4kEJstFsWpNMrC?=
 =?us-ascii?Q?z8cKp6t7xFnbNozhnFU64CuLdjpxXImLbNpkMMSbRGV0npPGn2nnwH1rA4bv?=
 =?us-ascii?Q?zyYy0Eylb1qDJDoStgiTlcWpe19TifX3cjXwLweURcqyvSHlBqNhbpOMa4xE?=
 =?us-ascii?Q?IOCAakZMYsD9wh7Ww3fuSc/kDwxyBjjgislui0PKVaKGisBG/l+XVBqsJBb4?=
 =?us-ascii?Q?NZVR7XKu+yJHUH5D1+yMEBek6UO+lUGnFcDd+iwA5pg1JlCBYmqHGfgI+BRl?=
 =?us-ascii?Q?xRJu4A8RMnSaprIE0cstfC4tzRYCh/MQA/nXwRRDK1Y+SPUfjihYGRxmFadz?=
 =?us-ascii?Q?wytGKaOqOYR2ovSdhT41MTDhApLibRlewoaxn7cLeuASNdgK+1H7sEKrJieM?=
 =?us-ascii?Q?qLytkgsedE59R5wWWHLh6y5IFj9mmF15nc88rni504Qp4woScDO02krlsGRG?=
 =?us-ascii?Q?9o8CjF1Ltuek9V4kKjWylTwsgXAVfVkP8xS9fzfWP+IiNo6ExkGHd/1LzbZG?=
 =?us-ascii?Q?kZzifhDkF2EzfMrHlIQXr6ut4HhrYFWUJMMk/dp5sKgtD0BrTdj6t6bVcJjg?=
 =?us-ascii?Q?zyrSN1TZekgnXomhH5UJnSEzhePD6OZYxc0tbaRON9KyHFJkpko7gtk+bA93?=
 =?us-ascii?Q?hn/MKXpkNjTlGB/SMLAqzqAYMPOHTwlKOlU+BolZgV09bkua5n62xxOCfGfO?=
 =?us-ascii?Q?tP7pfOxH67zGaTRmxuCvbC1HhwTeeFgPAY55Pd1YmUSQBXtqcqAR1zdbfJp3?=
 =?us-ascii?Q?P92z/UIg3QQKwmPjqHNCbdQfv/Vxp2ey/023aJBFxg94SZVxg9w57OdQfnN5?=
 =?us-ascii?Q?LjoEaudYkwtqrdYta+BoRzI5legZUK+dW5X9egglfkr1pvrMwnIkYHJLDQOK?=
 =?us-ascii?Q?VgGdQlNNTHmOpespGwpBF70AI13tK0BcnU3X3h1HVElJ2yNbD7FmTZ+F1SNT?=
 =?us-ascii?Q?wxQ3KRsqkbhGi1RK0MkRZRtribHWIdhPYqLqUoVQkCGJGrzUIhTkgbIYjzDD?=
 =?us-ascii?Q?+GpMzZkD/3gwUaQWlCnlguvM1Ln7W+AYVWfzkJtmK5CQyB5qhLzZ1+DzG7vJ?=
 =?us-ascii?Q?oe0rFqNvhqNl3GfVZrirHMLFbM4tOrhZm9ErrFhG6VI+t2I1wQWXJPVRVmP1?=
 =?us-ascii?Q?dDKt8j6qx/tLljvUgKJtwMdX7YXk0cLuhgeiRDBUzn/WJ6izs8XOgTY/JxVM?=
 =?us-ascii?Q?hTgcmgk0VGxCfjso21MeduZ1?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN8PR11MB3795.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: aa23fc41-0c78-43e0-5de8-08d9281260cb
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jun 2021 11:09:24.7639 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vz+A1a6Em8tdgvzdv2lNY9525AIVMUeiNznZlNqmD2Tw4VhoVQPYvrhfxH62nnPuLeHG+L87EeiN3MRuFdksuA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR11MB2740
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [PATCH] gpudev: introduce memory API
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Saturday, June 5, 2021 15:49
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org; Elena Agostini <eagostini@nvidia.com>; andrew.rybchenko=
@oktetlabs.ru; Yigit, Ferruh
> <ferruh.yigit@intel.com>; jerinj@marvell.com
> Subject: Re: [dpdk-dev] [PATCH] gpudev: introduce memory API
>=20
> 04/06/2021 20:04, Wang, Haiyue:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 04/06/2021 15:25, Wang, Haiyue:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > Another question is about the function rte_gpu_free().
> > > > > How do we recognize that a memory chunk is from the CPU and GPU v=
isible,
> > > > > or just from GPU?
> > > > >
> > > >
> > > > I didn't find the rte_gpu_free_visible definition, and the rte_gpu_=
free's
> > > > comment just says: deallocate a chunk of memory allocated with rte_=
gpu_malloc*
> > > >
> > > > Looks like the rte_gpu_free can handle this case ?
> > >
> > > This is the proposal, yes.
> > >
> > > > And from the definition "rte_gpu_free(uint16_t gpu_id, void *ptr)",=
 the
> > > > free needs to check whether this memory belong to the GPU or not, s=
o it
> > > > also can recognize the memory type, I think.
> > >
> > > Yes that's the idea behind having a single free function.
> > > We could have some metadata in front of the memory chunk.
> > > My question is to confirm whether it is a good design or not,
> > > and whether it should be driver specific or have a common struct in t=
he lib.
> > >
> > > Opinions?
> > >
> >
> > Make the GPU memory to be registered into the common lib API with the m=
etadata
> > like address, size etc, and also some GPU specific callbacks like to ha=
ndle how
> > to make GPU memory visible to CPU ?
> >
> > And the memory register can be like the exist external memory function:
> >
> > int
> > rte_extmem_register(void *va_addr, size_t len, rte_iova_t iova_addrs[],
> > 		unsigned int n_pages, size_t page_sz)
>=20
> How do you specify the device ID

I mean that take the current external memory register as an example, it is =
not
a real proto-type.

The GPU memory management library can provide the this kind of API for GPU =
driver
to register its memory at probe time or start time ? =20

> I may have missed something.
>=20



>=20