From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00056.outbound.protection.outlook.com [40.107.0.56]) by dpdk.org (Postfix) with ESMTP id C526C1B4F7 for ; Sun, 2 Dec 2018 06:48:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tBgcv0g0SNKF0HVFAPx1fI3MOUYdONjgN/9Zt0Y7PpE=; b=NyqIfZRmX+8YoVhW4XgRW1ExsLA3FClDxmZLmQCyA/Tq9WLpYNCSpLVsxojXVptunif5jBcVt0HGZqduKRQK3ygkXuM8Mtpa9eIbSgbpABs/X5rHxt5L5dRFGX+5yy9DkHSp0I8u0u4mZfNjfHSDezH/MwkPjzfIyj+H7dWytmU= Received: from DB7PR05MB4426.eurprd05.prod.outlook.com (52.134.109.15) by DB7PR05MB4202.eurprd05.prod.outlook.com (52.134.107.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1382.18; Sun, 2 Dec 2018 05:48:37 +0000 Received: from DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::f59c:f6f0:665b:8b16]) by DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::f59c:f6f0:665b:8b16%4]) with mapi id 15.20.1361.019; Sun, 2 Dec 2018 05:48:37 +0000 From: Shahaf Shuler To: Anatoly Burakov , "dev@dpdk.org" CC: Yongseok Koh , Thomas Monjalon , "shreyansh.jain@nxp.com" Thread-Topic: [PATCH 0/4] Allow using external memory without malloc Thread-Index: AQHUh+o+XbGH/p29B0WfQneI9d+4qaVq9C5g Date: Sun, 2 Dec 2018 05:48:37 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR05MB4202; 6:9PmeV6OsURZq90fxJit7LYMBLDk7daI/xUrV0jMke4MMeh6NeC1SnDpzxSH/rFcVXIAyyhuV1fy+E78WbNKi+nPv6LcMMaSL5SqFW5rCEzKdKErHacdhD3qW4DwIXNIMY6UOWBVfXdJiIDKEn6VFGdPB+S/d6Ky2c3iFlyO1YO84yaEdWZqXOQzNOfZ2W3FL4S3IXFFlmVBAqMOQzcQventUzSS7cha8dbevudp1z7C0TQpCMIaMQUA3BWvrxc+/Jb5Ao0f8mDD8G+yE6pdTX5n9ifTz7/3FIKboTtfT3K1tC6HdVKIXjuWyQJOxwfET4R54MgD+BpTtHOSlBpq/9sHLlNJ7T6J0ZIrIYrSKa2IMGotS7eMMI7kWrrGdbesJuoBPjws4WEGHNBypKblIRtDhoX7Qr9IlViGlNbsS5xdCrNYSDb56aDATljJrTMtmiVuQjVPMe4LCodyUmJXSug==; 5:4E75+u9dyfKTtxbNWgeIkOhVbT5Fa1hp8q54/jZ8Vwq7r4/vPykCkNdZT9xSfGccpU48UQDjDSvwJPoSX4sUX0PnHZhbtctjjzoX9virOHTajPHuAvoqC/enVw+9XbRUhO36iwf/SMkfZX0PJQ0jYeYj5xciEH5gbA+BOdamfW4=; 7:c3hHXHLn6aix8zr7MbAQz6I7vjna0EWBfkTx8D8zeljUtenCLl2IAsCuf9WkEMnKjVwgnFyKGf67QS0tEF5w8ntvSyF75P0zVuC1xDJUnTvhHzN9mIFn1dDBa8FWwkeiM3Kye7JpuBssP+FnC1qsmA== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 00c7b4d0-ce3d-4aa3-03bf-08d65819ce27 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390098)(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:DB7PR05MB4202; x-ms-traffictypediagnostic: DB7PR05MB4202: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(3231455)(999002)(944501485)(52105112)(10201501046)(6055026)(148016)(149066)(150057)(6041310)(20161123560045)(20161123558120)(20161123564045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DB7PR05MB4202; BCL:0; PCL:0; RULEID:; SRVR:DB7PR05MB4202; x-forefront-prvs: 087474FBFA x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(346002)(376002)(136003)(396003)(366004)(189003)(199004)(5660300001)(74316002)(305945005)(7736002)(14454004)(4326008)(97736004)(8676002)(81166006)(81156014)(8936002)(9686003)(106356001)(55016002)(6246003)(6306002)(105586002)(53936002)(71190400001)(6436002)(6116002)(3846002)(33656002)(71200400001)(66066001)(966005)(86362001)(229853002)(2906002)(102836004)(486006)(6506007)(45080400002)(476003)(11346002)(446003)(478600001)(2501003)(54906003)(110136005)(316002)(186003)(256004)(26005)(99286004)(68736007)(25786009)(76176011)(7696005); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR05MB4202; H:DB7PR05MB4426.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: FkNfEAGZJmd/HexxNzNmaZkHgbEiZ9TNI/pwInttYxvEFc/FIrA+d9+/J3SFa88oaABFtoipc+RoLjbbVFXWIEcXwjdcR0wk++oKIpFT3/g9NG7c/zUOTsvS2tvFF+Pbn1ZphLsBdkd5uViCFge7fcsx7/U37ziJU2IE3qmleapROlV13bPjojLhQ4ohvlFDA0Zl+Cl4MgKYxDTwKeLOq3/iC7qSO6xVkIYEfL0bxhDX364+6nO3ySd+64b7wYILlbWOFv10woXOM0H0H6LaG7gtqLhX1MH/OTu85LtPoBDHw5l+8iXARgbd0BSazxIgoQJXPUJbCeY9CzmgeVNma29RbA4VTUUx510UTD4GPqw= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 00c7b4d0-ce3d-4aa3-03bf-08d65819ce27 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2018 05:48:37.4031 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR05MB4202 Subject: Re: [dpdk-dev] [PATCH 0/4] Allow using external memory without malloc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Dec 2018 05:48:39 -0000 Hi Anatoly,=20 Thursday, November 29, 2018 3:49 PM, Anatoly Burakov: > Subject: [PATCH 0/4] Allow using external memory without malloc >=20 > Currently, the only way to use externally allocated memory is through > rte_malloc API's. While this is fine for a lot of use cases, it may not b= e suitable > for certain other use cases like manual memory management, etc. >=20 > This patchset adds another API to register memory segments with DPDK (so > that API's like ``rte_mem_virt2memseg`` could be relied on by PMD's and > such), but not create a malloc heap out of them. >=20 > Aside from the obvious (not adding memory to a heap), the other major > difference between this API and the ``rte_malloc_heap_*`` external memory > functions is the fact that no DMA mapping is performed automatically. >=20 > This really draws a line in the sand, and there are now two ways of doing > things - do everything automatically (using the ``rte_malloc_heap_*`` API= 's), > or do everything manually (``rte_extmem_*`` and future DMA mapping API > [1] that would replace ``rte_vfio_dma_map``). This way, the consistency o= f > API is kept, and flexibility is also allowed. >=20 As you know I like the idea. One question though, do you see a use case for application to have external= ly allocated memory which needs to be registered to the DPDK subsystem howe= ver not being used for DMA? My only guess would be so helper libraries which requires the memory alloca= tion from user (however it doesn't seems like a good API).=20 If no use case, maybe it is better to merge between the two (rte_extmem_* a= nd rte_dma_map) to have a single call for app to register and DMA map the m= emory. The rte_mem_virt2memseg is not something application needs to unders= tand, it is used internally by PMDs or other libs.=20 > [1] > https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fma > ils.dpdk.org%2Farchives%2Fdev%2F2018- > November%2F118175.html&data=3D02%7C01%7Cshahafs%40mellanox.co > m%7C007a9234feaf42c82f6508d656015eb1%7Ca652971c7d2e4d9ba6a4d1492 > 56f461b%7C0%7C0%7C636790961244424277&sdata=3DYqwcPEEhJM3I7Toe > Ne%2BGcbeo%2FmPbYEnNFckoA7ES2Hg%3D&reserved=3D0 >=20 > Note: at the time of this writing, there's no release notes > template, so no release notes updates in this patchset. > They will be added in later revisions. >=20 > Anatoly Burakov (4): > malloc: separate creating memseg list and malloc heap > malloc: separate destroying memseg list and heap data > mem: allow registering external memory areas > mem: allow usage of non-heap external memory in multiprocess >=20 > .../prog_guide/env_abstraction_layer.rst | 63 +++++++-- > lib/librte_eal/common/eal_common_memory.c | 116 > +++++++++++++++++ > lib/librte_eal/common/include/rte_memory.h | 122 > ++++++++++++++++++ > lib/librte_eal/common/malloc_heap.c | 104 +++++++++++---- > lib/librte_eal/common/malloc_heap.h | 15 ++- > lib/librte_eal/common/rte_malloc.c | 115 +++++++---------- > lib/librte_eal/rte_eal_version.map | 4 + > 7 files changed, 434 insertions(+), 105 deletions(-) >=20 > -- > 2.17.1