From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-eopbgr30088.outbound.protection.outlook.com [40.107.3.88]) by dpdk.org (Postfix) with ESMTP id 998A85F32 for ; Mon, 19 Mar 2018 09:59:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=dR1Xkhh6IThT1bi4OY7gv916LtXAXsAkBrcrL27BN7c=; b=eqUcy6rGCBpPA7GgGrnLqybjKsjKd0Y1emR84SWaWP8NznXmUVHR+l3M6jT+vTXD5/dL+0nhsIJpQVN6d0kkYvInhMXFCtDZtiVWIXJCuWUB6l8LFIE+p86JVqqIjwcYY8MAtEmI4wmiN9P4ffPg78WCWmaarvzOkbBfAXklhoc= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=shreyansh.jain@nxp.com; Received: from mail-wr0-f172.google.com (209.85.128.172) by AM5PR0402MB2771.eurprd04.prod.outlook.com (2603:10a6:203:98::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.588.14; Mon, 19 Mar 2018 08:59:03 +0000 Received: by mail-wr0-f172.google.com with SMTP id d10so17734643wrf.3 for ; Mon, 19 Mar 2018 01:59:03 -0700 (PDT) X-Gm-Message-State: AElRT7GW2TNpfvs1ru28BDo7RdMd1Tz49tEWvEi/mVinClsDMWDegpw0 cTLTH07UDglgti6Ure34dyUwFi14l/lq6fn2AGY= X-Google-Smtp-Source: AG47ELstrgroVXKluQBB5lFnwFf12Ok9TBf/A+yOvadMLvc+XC7jHWyUnTO0/skqruIv4OjSuZ2hW05FgsW1Zm9mPrc= X-Received: by 10.223.171.213 with SMTP id s79mr8030561wrc.52.1521449941153; Mon, 19 Mar 2018 01:59:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.30.65 with HTTP; Mon, 19 Mar 2018 01:58:30 -0700 (PDT) In-Reply-To: References: From: Shreyansh Jain Date: Mon, 19 Mar 2018 14:28:30 +0530 X-Gmail-Original-Message-ID: Message-ID: To: Anatoly Burakov Cc: dev@dpdk.org, Thomas Monjalon Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [209.85.128.172] X-ClientProxiedBy: AM5PR04CA0020.eurprd04.prod.outlook.com (2603:10a6:206:1::33) To AM5PR0402MB2771.eurprd04.prod.outlook.com (2603:10a6:203:98::13) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 04b0d76e-7e5a-4d10-c716-08d58d77a9dc X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:AM5PR0402MB2771; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0402MB2771; 3:FUfzv0HB9MTojAfDcKdX9lAyMEvB3Mjrlr/78WeNF5B+fLhRBVJDaPBnih2mQNEelMKnUgNdUyqDI2F8bUIpyxs+bGql2qEbULngR2/ohKaCiGsVYKUTyvqxALUJbs/Edd0ooB1LPvjo7QRgODPBvzYBsYiR00N2Jfr2r3CZFK3DsbjlTa79xcIL8RWcz87uaErict9X1rX6cb0w99IcjDWVYZxDenzu68rBDuFiI+i33reEP+ntT2W/bIHRtUDr; 25:hj1+h2IJsN7CeDAJCrzjJStHv+vsmA8NvnrcEX2MRTHBGxsxn6OPIBPMxc0Ne2AUcUYH9BYPsL/GXYVXPIFLPQ0s0yteqD+QFDLC4xzul/wgDeYhJNxD/yIGBh8lv5TrqnjaFxJ+/FBrtvNJL0JC19uSqS15B9qtPk4TpzYw78KCckdEzeupse39jjSsTWPBzN2UEtgNfNNErveh0cPGfp/LiaDtNqkOdO5hpr630fVYsHwP7sf23BA5aV94DI+LEiqISprIe2zk8DxhEX7zqFJxFyP+/ghUgxsUb9k+hqL+pIAF3rHpa9qpBj7CmnBIF2qCjIsQTdRlT8UI0gmL+w==; 31:NAqr0hxw/yLbsVhxSnWuMgbRaoDY2713wo4j4UaWehNuQNhDKOecbrSjXXI6ObvhXCIUUkHU1/pKuG9ke7tTUd0snmxbnTXa+jKnu7CLJ7XwsUGqDzet4jd7emzJRPhWueY/80IPAm27Ytmxbw0l5OfSrU/E/2ZyysDSInuF4CPprNEe2nZSqY+RVrv2dBce2sfWY/C9RBE8IaG+AzWoRLyldfpztEuyy+QVAoxiVCE= X-MS-TrafficTypeDiagnostic: AM5PR0402MB2771: X-Microsoft-Exchange-Diagnostics: 1; AM5PR0402MB2771; 20:wpV1JNmpNQkq81yJZaC6p88Gc7dBfLAjHDykaztyArj0co7xLAVQmYV6HMxxg8XHUiUvfCTei4iu0BZFCyJyMP8yA7wHyQ3U8YX43zGD/3ZIX90SsBTLhrbvSQwzC+jXt/T84j2sheeCDv4eImalIU+Fi4qGK+hnZ9xScxoIkFsBVnomrBtnE2disxm1+Tvp0SR+RAX/Q4gyqrsFhgm3yZnbUn/wMI8Il6V3JUxUkJm62Kr6YraxDxxNReNkTlLO8AjpMtE0sXS0yi5cdVW/hlOARavVC2dkVyhjZXF8E9CbZFPeKMKJOI/iydSLAjWkRo63zdceXVPb2ZoAV/bs9RH2znDV6WTpPreJgHJIugCmaW9PC6irD03iS9R+ftU0ziaAa6hcQrtSxiqEzDdY8ApU5ZJNokYhGZiCYdV0TMZHqfLEMn1H4St9lnyax/EkapXOxLY4JHLaFBCcUPU1J5qPTknXAzlUJv7u9f+p4kd6uqTyE0x3X/sqRTfr2wyK; 4:oFmIXUMrfUJS2pUu2+NhWO2/9RD3E3EWcRG6NrjFr9mYUyTRBaalJvAIB8fTThQ6TPyyp0TpY82z1IlL36kkHTO4Pmvy6cK54U5GjE6m0e4hZ6+G//F/F8Sv21wSnIzzkSj4AXl5k3Uj7qudK9ypqDGfwyHrywnl7JuwhFk9FJDVxMmc5UnYPnBv+lSvqJ4EeI4RsqHnkq071SmzJ0GKZTRHVCd7waXMYHRUhWWXR0J0raqjhTgNAnkdU2n9Un+FfKqFVTITBQkVWknr1uRVQolIIlt3QIRl85SIgF4uauUjLMPiI8wexNFme4XC/mgdmfF9NHnDGWwrz0vNJClu1CYgBm1wW4hkbDiQgAw8DNw= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(131327999870524)(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(3231221)(944501300)(52105095)(10201501046)(6055026)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123558120)(6072148)(201708071742011); SRVR:AM5PR0402MB2771; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0402MB2771; X-Forefront-PRVS: 06167FAD59 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(346002)(39380400002)(376002)(39860400002)(366004)(189003)(199004)(53376002)(229853002)(6306002)(305945005)(478600001)(6862004)(53936002)(4326008)(7736002)(95326003)(2950100002)(186003)(498394004)(23676004)(16586007)(9896002)(107886003)(66066001)(9686003)(86362001)(6246003)(106356001)(59536001)(26005)(68736007)(98316002)(52116002)(966005)(42186006)(33896004)(76176011)(316002)(386003)(59450400001)(61266001)(47776003)(53546011)(93516011)(5820100001)(97736004)(61726006)(69596002)(8936002)(122856001)(55446002)(81156014)(6116002)(3846002)(50466002)(5660300001)(8676002)(81166006)(105586002)(2906002)(55456009); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0402MB2771; H:mail-wr0-f172.google.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTTVQUjA0MDJNQjI3NzE7MjM6ZXJGS0QvSWFtcXBLbElFQnZpWCs2WGQw?= =?utf-8?B?UEFaYmNTUTdNZk5FbXJBYjVsNyswelFnaW9kL3dEL1BlTUhzbmc5M3FXMGpE?= =?utf-8?B?VVVzT21ZSW83WnN2OU9CZ0xldWp4RmVRakJDSFUrL3FpVy9IVU5CSnhOVytq?= =?utf-8?B?WEtraW9VZE9BZTVPMi91c0NRdzZFbHUyMVVkVnp5YkMyOFlhQ1g4aDYwQllE?= =?utf-8?B?WW55OGoxRVVhZGVQNnhETTRUU2p6NkZpclFmZUJFYWo1YkUxWGl1Z2FGTU94?= =?utf-8?B?SGNEZTViZDNJaVN0ckdlRW5LT2htNWNYUVFKUFQ2VlJTaFZqZW90TW95NDZR?= =?utf-8?B?L2s2enRTM1ZXaFZKK2FycWI3Y1poY09RQ3hlZTgvSXBOQThEZ1VGZEVjTFRp?= =?utf-8?B?dFhCeEI2eTBBNlhDMkJtT0hqaktTL0haK3RYZE9rWTV5Mjl2MXJ1Y2NMSHg0?= =?utf-8?B?RE5oSC9jeEVYL0x0KzV3cCtOd2ZHbUtKRTRRUTRCZ0Y4VGhEOFYrSlRBTmtj?= =?utf-8?B?MzROdWtaeG52OWVSNXNZR0dhblpHR3lDTHVwNXJUbG9qeGxoVDMwdUVwdERF?= =?utf-8?B?aUQzVWhUZVJmMjYzRStla3lFOE5LZURGQ3FTV3BEUWhCZ3B3WEJGUXNoTVlt?= =?utf-8?B?UEE2VkN2TngwOGJuVDd2OWk1Smo0NWROM3dIUE11QkZ0R3Yrd1ZLN1BkZ1RU?= =?utf-8?B?RXp4ekNMMjlzYkxIK2o5TFRabDhLVTBWdk1ZeDlPVUV3RE5YeFZETGg2OXhO?= =?utf-8?B?ZXVSTkx0THBHOEk1TVo1TUoyVSs1Q2x1cVRVUnZTTHdXY2VxZk5JaHFCNGIz?= =?utf-8?B?akdDVllCS3l6RnhjWjdqYWkzUFRTdEllWGRrdTJKbjJyeGd0cGRRSTlmZG5G?= =?utf-8?B?RG5EZmxQMDRsb1pOc1Y1bW5kTGhMbXZ2R2VaUUJtQkRLT3ZQbEZidy9VMHlN?= =?utf-8?B?Q3E1d1htWGR2UzRKMjFZQ3JENWx1YmlWQmpPWWdnMkxsb0Vqekt5amdQOFBJ?= =?utf-8?B?N3RBZmp3MnlaM0NUVVdHeWc1NzRRc3dXRldwWmtWMEJRczI5RHJzUktFZ0w1?= =?utf-8?B?c0h1KzNhbG5BMy9iRjVhZ2NSL2o2M3pBYzdBSndnVmlmRUVkdlFQeTdTamVy?= =?utf-8?B?ZmR3NTNlZSs4WlUxQnppaUh6RFZzZGs3alFkQlgxdTBSRnlIZUg0Rmx1UGwv?= =?utf-8?B?ejZhYU1GZjlTYm9yLyttbks5TEd2a0xyZjBnUWJlSmZkTWUrQUJqVXJQN0hG?= =?utf-8?B?OFdWZnVTQkIyc3F4emJoSVpqeXc3UGtlRkhDSnovY1RoOTRjVnVEMEo0cmdq?= =?utf-8?B?eGJEZjFPQmUxUEpEU3NRTVB1ZnVHSnowNXZ2WVAvQk9vbTUxdzlaTUx3ZC84?= =?utf-8?B?SEFwR0JZbnlYSzg5T3Jpa0NWYm9CbFpTTW1oQTRURmtuUmQvelFzbENKcVJy?= =?utf-8?B?WE9zTEFrNG9uWFFrWU0wdEVTVTg2Y2hLRTNyK29JMDRFamNud0ZFd1FFTVFr?= =?utf-8?B?STV1bWFTeWxkYy8yZzhHWEJjVDdKSW80N3pZa3Q4U0wzZFgyWi9QMFIxaEli?= =?utf-8?B?ZXhXTU1jVGduK0E2Y0RqYVdlR1BkdnVIRWtIcEgyWk42VVBRaVpWdHRCbWVh?= =?utf-8?B?UVkwRlRrQWhjbXVuSmJiMUJOTW8wa2hINDNvRnJVUmJpOHIzT0JhTENrOTM5?= =?utf-8?B?cG5CVkVNZHh1Q3o5amsxMVVrbFU0RzZycTlpa2d3Y3NkQ01mSithZFlBTmtS?= =?utf-8?B?Q0doZXI2ZFhZQndWRk1FV2E3MGY4LzhrbGd6aGJXMW43T3BPdGJLT2hSMWQ1?= =?utf-8?B?UXlkcEhDRFlCbU80NVg3bG9rUFFsRE1XelpMK1Q0aXZLOWdXbW5BVklQMTc1?= =?utf-8?B?YTFJbEord25ja2FRelV6ZFJsUzlGUDN5UVBxclIzNWd2bmU2eHB5OFRUd2RJ?= =?utf-8?Q?FO2TLXc2dre9rNAXP9gnbrftkq6fgkGg=3D?= X-Microsoft-Antispam-Message-Info: CB5h5AO2OBQ/nLtXh6+gUg4TY660L11wEFDBqNNZ3rinU1z64ircFhOUE7MxjArtHpzZey7BThwwGasVypjmQEfGSFKYromRttvcJjcWH3rb1S1IuIb70ntl04+12+jQfjeYtpzfM6j6WtQmQBRJwAA3nme0AX7ZtLUkdmtL3ZJTzFipTi21Jh/xa+hZZDlT X-Microsoft-Exchange-Diagnostics: 1; AM5PR0402MB2771; 6:Wte8UnFxXdg65SN08jIiuL09eer0wDcMD5A2XM91k7zRfks11QTq1Sv7WmQubBnaCVUrVQrAchU56hLp3gKAr1c/P8fKfDr/5W7QtFslV7DA2Nvah0GGWsJkXT+RO4rJloQ6R49ZtI9asvc3wi7CW8m0jtsBLn+S9OdjE2UBAiMAxtLskvduOfb/0J8us3aemY7nlYY9/RcIRmDpnTg/iRiJUU3G56oS1fgBn0tT/fMpC3DTUyyN4cnDUFpd6OTjvJtWWB34g8gj12e7/jmtqkiw8y6BHlue7eJR337pHCnEJrjmFm5Moxv1pmj263rpgqLiC5z76V8H5vd1U5CfEigtqU+gQn5rgpCEQ7lKS6g=; 5:ZhGDIkzBd5A2ZZvUjwefYES+5Gi8GOsZdv4GyL0eQri0N02XQOiNTJ9rG7BlcLyKnxhBZnW4sMiegUP4p4zgd/1RulTh51SnvZzDLKIL36qpZ7plewGtppz4c/hx7E4yv6CJnKgtN8jGJmoz5+iOaGqvtGF7DDau4dnx+Hw5aaE=; 24:OdBaG0dbTh/tZ4KB7w/5J3IbpS7J8TUBArUCiJuIS5Vg2HrruPP6MamEUblqz+3pSu35KKSazOmEnfKZsm5fntSuhRqazrVaz1VKAFhkDvw=; 7:FQVyMsKz7nDjZ8nYQZ69g0uHVxJG5M8NOtMhbsDSP43akfaTT5fcqCXav1cxYw3Kd+91IvleHbsaa+85zkCbQqcJ4LMamgiNcGnLi5HT1Vr1r+Qc+5Tfi7xivTWP5O6YRBXqREKywVlcnmTzSWz0mJ4w6ysZ0iPZ9+uxm4l397u3SGZ9gapfJpmoaCSrYRByYmBYnZoPmal7UhRUe6iwbp50+uGMbEyK4Z/OQvvRC2Qw/wNOJfaFCt7ybNd7iISs SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2018 08:59:03.3226 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 04b0d76e-7e5a-4d10-c716-08d58d77a9dc X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0402MB2771 Subject: Re: [dpdk-dev] [PATCH v2 00/41] Memory Hotplug for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2018 08:59:04 -0000 Hi Anatoly, On Wed, Mar 7, 2018 at 10:26 PM, Anatoly Burakov wrote: > This patchset introduces dynamic memory allocation for DPDK (aka memory > hotplug). Based upon RFC submitted in December [1]. > > Dependencies (to be applied in specified order): > - IPC bugfixes patchset [2] > - IPC improvements patchset [3] > - IPC asynchronous request API patch [4] > - Function to return number of sockets [5] > > Deprecation notices relevant to this patchset: > - General outline of memory hotplug changes [6] > - EAL NUMA node count changes [7] > > The vast majority of changes are in the EAL and malloc, the external API > disruption is minimal: a new set of API's are added for contiguous memory > allocation for rte_memzone, and a few API additions in rte_memory due to > switch to memseg_lists as opposed to memsegs. Every other API change is > internal to EAL, and all of the memory allocation/freeing is handled > through rte_malloc, with no externally visible API changes. > > Quick outline of all changes done as part of this patchset: > > * Malloc heap adjusted to handle holes in address space > * Single memseg list replaced by multiple memseg lists > * VA space for hugepages is preallocated in advance > * Added alloc/free for pages happening as needed on rte_malloc/rte_free > * Added contiguous memory allocation API's for rte_memzone > * Integrated Pawel Wodkowski's patch for registering/unregistering memory > with VFIO [8] > * Callbacks for registering memory allocations > * Multiprocess support done via DPDK IPC introduced in 18.02 > > The biggest difference is a "memseg" now represents a single page (as opposed to > being a big contiguous block of pages). As a consequence, both memzones and > malloc elements are no longer guaranteed to be physically contiguous, unless > the user asks for it at reserve time. To preserve whatever functionality that > was dependent on previous behavior, a legacy memory option is also provided, > however it is expected (or perhaps vainly hoped) to be temporary solution. > > Why multiple memseg lists instead of one? Since memseg is a single page now, > the list of memsegs will get quite big, and we need to locate pages somehow > when we allocate and free them. We could of course just walk the list and > allocate one contiguous chunk of VA space for memsegs, but this > implementation uses separate lists instead in order to speed up many > operations with memseg lists. > > For v1 and v2, the following limitations are present: > - FreeBSD does not even compile, let alone run > - No 32-bit support I just read on announce@dpdk.org [1] that an early merge to this series is expected. So, would this limitation be fixed before merge? Or, has it already been fixed in github repo? [1] http://dpdk.org/ml/archives/announce/2018-March/000182.html [...] - Shreyansh