From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140070.outbound.protection.outlook.com [40.107.14.70]) by dpdk.org (Postfix) with ESMTP id F33B11B43C for ; Sun, 17 Feb 2019 07:23:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p3NNBj4bhzeeNkhuIy5UgVBFT/Q9frKcbwI34iwBSkk=; b=mpuWvpWcDvbSEO7V7Fjlmg65ZJat6rKG4Twm2Lt1bDcoERlO5aegMf9DIMNznVySSDjRp7ifmJ/ZiqOVNsNsvev18B/gUP/F3IVWxVzgCm12ulwmbx76eHYvnoj9ZPmzMCeJ9mnsqMzP11k5YVBbcUlsN284AtApechB92ctDd8= Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com (52.133.22.13) by AM6PR0502MB3783.eurprd05.prod.outlook.com (52.133.17.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1622.19; Sun, 17 Feb 2019 06:23:56 +0000 Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::3c1a:f2ea:1e07:a3f9]) by AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::3c1a:f2ea:1e07:a3f9%7]) with mapi id 15.20.1622.018; Sun, 17 Feb 2019 06:23:56 +0000 From: Shahaf Shuler To: =?iso-8859-1?Q?Ga=EBtan_Rivet?= CC: "anatoly.burakov@intel.com" , Yongseok Koh , Thomas Monjalon , "ferruh.yigit@intel.com" , "nhorman@tuxdriver.com" , "dev@dpdk.org" Thread-Topic: [PATCH 3/6] bus: introduce DMA memory mapping for external memory Thread-Index: AQHUw42zZufhWV3+PkCK3R9dCCgPAKXd2cBwgAF6ioCABDXGIA== Date: Sun, 17 Feb 2019 06:23:56 +0000 Message-ID: References: <323319abdbdc238c3586dafe9ad49dab554d6e64.1550048188.git.shahafs@mellanox.com> <20190213111702.wj3cqg6skttqduc4@bidouze.vm.6wind.com> <20190214140053.relxge6hcjcuylpy@bidouze.vm.6wind.com> In-Reply-To: <20190214140053.relxge6hcjcuylpy@bidouze.vm.6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3c8a6bdb-338c-40f8-7adf-08d694a07edf x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600110)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM6PR0502MB3783; x-ms-traffictypediagnostic: AM6PR0502MB3783: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-exchange-diagnostics: =?iso-8859-1?Q?1; AM6PR0502MB3783; 23:CyioQ0IEzjYishDgGhuIyPw2A+Finx6WX4vLT?= =?iso-8859-1?Q?NxaLo+qUiqyT15/c8JPihxEfMCm7Uw6EvWCfJr03FUWWzW5IHSQCreuC4M?= =?iso-8859-1?Q?d54CKNqjQBZG9XD+BNBGAvtm9FAuHB0VOunOhQeVR01dj0m7KDKtkpn+PJ?= =?iso-8859-1?Q?gtJDOFlgf/ZrxqOLTyWUxnwWODWUIWIUhc+Cge1bW/+aq/8a8gEQa7VZOM?= =?iso-8859-1?Q?9UtjW1Fqny98+tDGA3o0ynKqipO9vD33ZrzJNePlUYzFI6CtnUeCM1BUrg?= =?iso-8859-1?Q?wExW7yVjC6/SIotjrO3Xh15SAt7Bmo13/OPGrl5Iu/2jE0pT1nU6VOv3pt?= =?iso-8859-1?Q?asDAh1w/M5a6gKuCUXbyn4sjNaD/72z3+8ULEa0JwJ37M9KtYkHVoFnkc9?= =?iso-8859-1?Q?OFmw3eXTDvqpmJJVFYQHfP3PtRPUBttffvD3I9YplYYOvNhDim5KXePUqq?= =?iso-8859-1?Q?AdLZkClZRCtbCU92RorehilIuVF2vAyg1bM/e1RwFpjjqydmrped2P7Tqw?= =?iso-8859-1?Q?gHY4szWflBo2eUVMr69XXFDZZeEMjRrllCEte6kpJZpDoumacnY5KbtdyI?= =?iso-8859-1?Q?IvQyr0cPAv5d4WP2AT7V/3uLOnjg/jnY4n1X/ZPfw9i2ebwpzqujmHbex/?= =?iso-8859-1?Q?jGQ0W2hg0OmIWs9dICvRKnCDKa+mYtBs/ZOQAWCzF23MrOz+YN7pxzO5Kv?= =?iso-8859-1?Q?U7SahMbY0x3n2HD8LjZJTp4syXEINKPnILsQT7TrSfBCSHKhsE7HXS3oKZ?= =?iso-8859-1?Q?SXfukHF5nkBDpKIsgHnu99D8qA3LbzrfaPE9m6TYaxgSANgLG32qgQrsVs?= =?iso-8859-1?Q?IbK4NLhAM2BLN5ggmEYlaqOmPNcWLPZ6iGt8G5Tbme7WJxon34LMWRTjtM?= =?iso-8859-1?Q?cMq8cbVlk45prPfWC6EeebDZFgKpjJD5GzFWT1seqVenibRDLJF6FKXJJQ?= =?iso-8859-1?Q?Gm/U14eqmQbz1zm153eiay+nzmxoAp+c4W7Tx+OT88dwsdM+BKHGHE7Y6W?= =?iso-8859-1?Q?92ah+lYNlsIgzHuj4yZfDLfZ6lt0pp0uJqDZlpBhH6Ns7cOqkV+nvvYEaq?= =?iso-8859-1?Q?I1osBcOgJ4X2CQyvIIhCrBbHuStVOkxMalbWq0x30njX9KLSSBXjwiiV2i?= =?iso-8859-1?Q?BHC4gLxwTHAgJz/PcY6dWwLosfGNvxJuxFialec7NpfTafn4JW71pYsfMI?= =?iso-8859-1?Q?myPdIGKem8U5AK1YjFUYwpMlnbCgzmRECUGFI10LqO18/DeHMAYyZwICKB?= =?iso-8859-1?Q?Y3+OR1x+h+JlTKiml0J9+J4L5bzmkLvnuBfui5+Y0nAauekhk/6ClWg+hS?= =?iso-8859-1?Q?5wW1ZrABnYyMLomdrg8Tftf07?= x-microsoft-antispam-prvs: x-forefront-prvs: 0951AB0A30 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(136003)(346002)(376002)(366004)(396003)(189003)(199004)(33656002)(446003)(3846002)(486006)(11346002)(476003)(6116002)(6916009)(66574012)(105586002)(478600001)(106356001)(26005)(6506007)(6346003)(102836004)(186003)(93886005)(76176011)(14454004)(66066001)(316002)(54906003)(99286004)(2906002)(7696005)(68736007)(6246003)(71190400001)(71200400001)(229853002)(9686003)(53936002)(55016002)(8676002)(256004)(14444005)(8936002)(305945005)(81166006)(81156014)(97736004)(5660300002)(25786009)(74316002)(86362001)(7736002)(6436002)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR0502MB3783; H:AM6PR0502MB3797.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: BrIDe4iznKtG6qgLPRV3N9QCbkLrhA7XsWinMB9Mpynj+TVDnoujM85ANv7sz/c26tmppeaAn+3nivktUQrx7WwtWsN21DubrwysWxLtJas7xfCaRpbKvG6ukymwkjub9m+ZbO4UAchm7bXDOU6h+rprXB04Rryf5eJ4QW6izoIoqC6OKTfHWCS4xaJOUr4mwaWaFYPvvK0dCkV1HcXIlpWlVCFkfG1jcGtACXS2T7kdKocTWKN3mICX52UJrz1lCF64W6nJ6UetCts4YM1vpBvaSuGb2kZFuuCUaMFCixhsjcgrGaFuINUEzfHmbfjA6OGvFpzQjlaZAJ1LFD1ZrYWv3vgIBv5c7SrM0uPB8UiXIdcfOGNFPaZMP2GFsWr+sp97M6MIOHa86Qmh2aR+hEKQy9/3s+oxnGLlVLDYXUc= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3c8a6bdb-338c-40f8-7adf-08d694a07edf X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Feb 2019 06:23:56.2556 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB3783 Subject: Re: [dpdk-dev] [PATCH 3/6] bus: introduce DMA memory mapping for external memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Feb 2019 06:23:58 -0000 Thursday, February 14, 2019 4:01 PM, Ga=EBtan Rivet: > Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for external > memory >=20 > On Wed, Feb 13, 2019 at 07:07:11PM +0000, Shahaf Shuler wrote: > > Wednesday, February 13, 2019 1:17 PM, Ga=EBtan Rivet: > > > Subject: Re: [PATCH 3/6] bus: introduce DMA memory mapping for > > > external memory [...] > > > > > > How are those other vendors' devices mapped initially right now? Are > > > they using #2 scheme instead? Then the user will remap everything usi= ng > #3? > > > > It is not a re-map, it is a completely different mode for the memory > management. > > The first question to ask is "how the application wants to manage its > memory" ? > > If it is either #1 or #2 above, no problem to make the mapping internal= on > the "other vendor devices" as they can register to the memory event > callback which trigger every time new memory is added to the DPDK memory > management system. > > For #3 the memory does not exists in the DPDK memory management > system, and no memory events. Hence the application needs to explicitly c= all > the dma MAP. > > The change on this patch is just to make it more generic than calling o= nly > VFIO. > > >=20 > Right! I mostly used #1 ports and never really thought about other kind o= f > memory management or how they might follow a different logic. >=20 > Do you think this could be used with a lot of sequential mapping/unmappin= gs > happening? It much depends how efficient is the driver mapping and unmapping.=20 In most cases, mapping is heavy operation.=20 >=20 > I'm thinking for example about a crypto app feeding crypto buffers, being > able to directly map the result instead of copying it within buffers migh= t be > interesting. But then you'd have to unmap often. >=20 > - Is the unmap() simple from the app PoV? Yes, just call rte_bus_dma_unmap.=20 >=20 > - Must the mapping remain available for a long time? It must remain as long as you need the device to access the memory. On your= example, it should remain till the crypto dev finished writing the buffers= .=20 >=20 > - Does the app need to call tx_descriptor_status() a few times or > does dma_unmap() verify that the mapping is not in use before > unmapping? I think it is a matter of driver implementation.=20 In general, it is application responsibly to make sure the memory is no lon= ger needed before unmapping, just like I don't destroy today mempool being = used by some rxq. It can be done by any means the application has, not only= tx_descriptor_status. Driver can protect bad application and warn + fail the call on such case, h= owever it is not a must.=20 >=20 > -- > Ga=EBtan Rivet > 6WIND