From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EFC5745910; Thu, 5 Sep 2024 17:34:47 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D5DD042EB1; Thu, 5 Sep 2024 17:34:47 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2084.outbound.protection.outlook.com [40.107.236.84]) by mails.dpdk.org (Postfix) with ESMTP id 55A6342E5F for <dev@dpdk.org>; Thu, 5 Sep 2024 17:34:46 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eFbE1lueNdMS5pNb2LS2w6nhJ0afgLLcynkq6CS8LdSV/z2XhdTAt6YlyyxdplrkAuB82wdDBsUcJpDnInLPFzqUExmvNG91TQoaD9PMe/hQKA04NyBcTn0mhIpGLRf3cXm7t4jG3rocoUl2ZGmX7p4Ik8Vmyce15WcF7Np7XVrkF/9Xr3ioP/t0Mk8ll9zG2Qu1AcFwSji/tggAAUaxwfNxMfz32sFlHDlsbT1mxomXRfUIXycfil9fgeHIZQYEuM1vIoHTELIHZR3fNy4q5WdR4Din9wuxlUUej3Ptd2qZTmCIT6ejg3CjrSNGQSXAKoM6vPA+GVdP+mxKO73rXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b3SqY8ModNvP+dNGa1HHen6QdoDZWWDa0po9WDqF+ms=; b=X+xPIikkXNCe2a+xCnz8HVfxyPSlJykU2yk4DuFvvOn93YpWYKFZly23Bz3Xejbr/gWhVOWTCfLoUvfUBI/10UbJ4PdGj2FBSKkrSvioMx1WYSM0Nh8VDf477koatAdGmmJ/ih7ITy2pGVavG+w1EzB13trS7ba3H9aFQsb0qmL9EulnZ0yJP5ckD/JHhTmsEakc6JMxVT2OIGYxupuD4eiRVrsPa/Dsz5Mq8oY3AgQdk49STkaZr/DUmBZYubS0bM00ElQxfEDQ9WeE38H14KafrAKNayBC8x6HY3gx06aJiDhTeLkKoOzSlgZGNT0YimGsVscpymOoUhVBkpzUeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b3SqY8ModNvP+dNGa1HHen6QdoDZWWDa0po9WDqF+ms=; b=V1R1oHZCUStFV/NmMn+3v3w8CVw0J1iPMza0JxoTyGdS/tGxCjpLp2kj/bgOXw2BVaPzG2eOj1/ExBpx9pG/tTbd6tJZ0+89l1VZ1K0BlLDZKPf5vIOAl9UVnM9Uox4LQwMCwq+HjMvdjQxiaUYzKIjBtD15BxaLs96OwPpvrsU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by IA0PR12MB8227.namprd12.prod.outlook.com (2603:10b6:208:406::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.25; Thu, 5 Sep 2024 15:34:40 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::ebfb:2f9f:f9ca:82cd]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::ebfb:2f9f:f9ca:82cd%4]) with mapi id 15.20.7918.019; Thu, 5 Sep 2024 15:34:40 +0000 Message-ID: <462f4550-698e-4f49-9280-3a3708448337@amd.com> Date: Thu, 5 Sep 2024 16:34:33 +0100 User-Agent: Mozilla Thunderbird Subject: Re: [RFC 0/2] introduce LLC aware functions To: "Burakov, Anatoly" <anatoly.burakov@intel.com>, "Varghese, Vipin" <vipin.varghese@amd.com>, dev@dpdk.org Cc: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= <hofors@lysator.liu.se> References: <20240827151014.201-1-vipin.varghese@amd.com> <288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com> <65f3dc80-2d07-4b8b-9a5c-197eb2b21180@amd.com> <8addd7f6-fac8-45ec-a44f-f81eb008cc36@intel.com> <db3af9ba-6b7b-4c43-bce0-d85d222cfa99@amd.com> <3edc8a89-7d10-47f4-8f95-856c2a7fc7ba@intel.com> <3eae1577-f06f-48f2-863a-faf70b97bc72@amd.com> <a4fae3a0-bab8-4539-9a19-08f90ee4f542@intel.com> Content-Language: en-US From: Ferruh Yigit <ferruh.yigit@amd.com> Autocrypt: addr=ferruh.yigit@amd.com; keydata= xsFNBGJDD3EBEAC/M7Tk/DfQSmP1K96vyzdhfSBzlCaGtcxNXorq4fALruqVsD3oi0yfyEz9 4YN8x7py0o9EL8ZdpOX0skc0AMCDAaw033uWhCn0GLMeGRKUbfOAPvL6ecSDvGD7CJIO9j0J eZUvasBgPdM/435PEr9DmC6Ggzdzt8IuG4PoLi5jpFSfcqxZFCCxLUDEo/w0nuguk2FTuYJg B2zEZ4JTBZrw7hIHiFh8D8hr6YA6a5uTofq1tr+l048lbtdFUl8TR0aIExVzE4Z8qKZlcE+9 RQaewjK5Al1jLE4sHdmd3GN+IvgDF3D/fLsi25SKJDeGSdeHkOmaX0qGeM4WKIfU6iARRCiQ N3AmBIxZ/A7UXBKLaOyZ+/i3sE6Wb53nrO4i8+0K2Qwyh6LjTeiJAIjYKN43ppxz3DaI+QwQ vI+uyHr4Gg0Da9EPPz/YyKauSeOZCfCB5gIfICO0j6x0SCl8uQ2nLpjxcZkf0gjcwUzP3h+S 3x6NfDji9YEij0zczW/dcSpGgZ6vsFpPrtnP9ZXy6J53yp0kJtOJoOlkEFFdU2yCZnCDseum CoudmGLZVvS0/DzHDJejq+3kK3FDGktZBOxZIIpal+nFqS7lVgOZc4+huVv3jyhzoAUOEyXA XK5j6o7g8STUY+z33QNnHpdLvecMwuzmvqy0jR54yAbZ64mB9QARAQABzSNGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBhbWQuY29tPsLBlwQTAQgAQQIbAwULCQgHAgYVCgkICwIEFgID AQIeAQIXgAIZARYhBEm7aYjps5XGsPHCElRTPtCKKm/6BQJkdyEEBQkE3meNAAoJEFRTPtCK Km/6UdcP/0/kEp49aIUhkRnQfmKmNVpcBEs4NqceNCWTQlaXdEwL1lxf1L49dsF5Jz1yvWi3 tMtq0Mk1o68mQ7q8iZAzIeLxGQAlievMNE0BzLWPFmuX+ac98ITBqKdnUAn6ig5ezR+jxrAU 58utUszDl16eMabtCu76sINL5izB8zCWcDEUB4UqM8iBSQZ7/a7TSBVS0jVBldAORg1qfFIs cGMPQn/skhy3QqbK3u3Rhc44zRxvzrQJmhY6T1rpeniHSyGOeIYqjpbpnMU5n1VWzQ4NXvAD VDkZ4NDw6CpvF4S2h2Ds7w7GKvT6RRTddrl672IaLcaWRiqBNCPm+eKh4q5/XkOXTgUqYBVg Ors8uS9EbQC/SAcp9VHF9fB+3nadxZm4CLPe5ZDJnSmgu/ea7xjWQYR8ouo2THxqNZtkercc GOxGFxIaLcJIR/XChh9d0LKgc1FfVARTMW8UrPgINVEmVSFmAVSgVfsWIV+NSpG9/e90E4SV gMLPABn1YpJ8ca/IwqovctqDDXfxZOvCPOVWTzQe/ut767W+ctGR1kRkxWcz470SycOcY+PW VRPJd91Af0GdLFkwzZgNzkd6Gyc9XXcv4lwwqBLhWrBhqPYB0aZXIG1E/cVTiRp4dWpFHAFD DcuLldjIw93lCDsIeEDM9rBizGVMWEoeFmqSe7pzGTPXzsFNBGJDD3EBEAC8fBFQHej8qgIG CBzoIEd1cZgPIARlIhRudODXoNDbwA+zJMKtOVwol3Hh1qJ2/yZP11nZsqrP4fyUvMxrwhDe WBWFVDbWHLnqXMnKuUU1vQMujbzgq/4Rb9wSMW5vBL6YxhZng+h71JgS/9nVtzyaTtsOTrJi 6nzFSDx6Wbza2jYvL9rlK0yxJcMEiKwZQ/if4KcOesD0rtxomU/iSEv6DATcJbGXP6T93nPl 90XksijRKAmOwvdu3A8IIlxiSSVRP0lxiHOeR35y6PjHY2usfEDZZOVOfDfhlCVAIBZUZALv VmFOVSTYXeKgYa6Ooaf72+cHM3SgJIbYnevJfFv8YQW0MEAJ/IXE7B1Lk+pHNxwU3VBCrKnA fd/PTvviesuYRkrRD6qqZnINeu3b2DouVGGt2fVcGA38BujCd3p8i7azoGc7A6cgF7z9ETnr ANrbg1/dJyDmkDxOxVrVquTBbxJbDy2HaIe9wyJTEK2Sznpy62DaHVY+gfDQzexBXM10geHC IIUhEnOUYVaq65X3ZDjyAQnNDBQ4uMqSHZk8DpJ22X+T+IMzWzWl+VyU4UZXjkLKPvlqPjJk 1RbKScek5L2GhxHQbPaD76Hx4Jiel0vm2G+4wei8Ay1+0YRFkhySxogU/uQVXHTv63KzQMak oIfnN/V2R0ucarsvMBW+gwARAQABwsF8BBgBCAAmAhsMFiEESbtpiOmzlcaw8cISVFM+0Ioq b/oFAmR3IPsFCQTeZ44ACgkQVFM+0Ioqb/qINhAAtcor9bevHy22HvJvXX17IOpPSklZJAeQ Az43ZEo5kRlJ8mElc2g3RzYCvL/V3fSiIATxIsLq/MDtYhO8AAvklxND/u2zeBd7BkRZTZZX W1V1cM3oTvfx3LOhDu4f2ExQzCGdkzbXTRswSJIe1W0qwsDp+YPekbrsKp1maZArGeu+6FuW honeosIrWS98QJmscEhP8ooyJkLDCCOgEk+mJ/JBjzcJGuYn6+Iy/ApMw/vqiLGL1UWekcTA g18mREHqIR+A3ZvypIufSFB52oIs1zD/uh/MgmL62bY/Cw6M2SxiVxLRsav9TNkF6ZaNQCgn GqifliCEMvEuLZRBOZSYH2A/PfwjYW0Ss0Gyfywmb2IA990gcQsXxuCLG7pAbWaeYazoYYEQ NYmWatZNMAs68ERI2zvrVxdJ/fBWAllIEd0uQ4P05GtAHPdTIDQYp545+TPV7oyF0LfXcsQs SFVZE6igdvkjfYmh+QOrHGZvpWXLTmffVf/AQ81wspzbfxJ7sYM4P8Mg5kKOsaoUdyA/2qVe cMh1CLUHXF1GlofpGbe1lj4KUJVse5g3qwV7i9VrseA8c4VIZewdIjkzAhmmbxl+8rM/LKBH dZUMTzME5PFCXJIZ83qkZQ795MTe2YScp9dIV7fsS5tpDwIs7BZNVM1l3NAdK+DLHqNxKuyO 8Zk= In-Reply-To: <a4fae3a0-bab8-4539-9a19-08f90ee4f542@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: DUZP191CA0051.EURP191.PROD.OUTLOOK.COM (2603:10a6:10:4fa::28) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|IA0PR12MB8227:EE_ X-MS-Office365-Filtering-Correlation-Id: 0deffe2a-03cc-426c-4573-08dccdc04070 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?MDVBanphT3VWSnBWSEhiM1JPNXlZZ2pQTXk4NFR2N1pIOXNDRitKRldiQ0to?= =?utf-8?B?Nkl0ZTJ3bGY3OXZTeXhLbi9DMnRPcVJnSTRyVllNTXJzNVNxTG9uRXRtSUl2?= =?utf-8?B?QVR4M1RLbzJXNVBSeWFZdnhQU29MUUE1RGJ2b0tCbTkyMnc1ejBXVGhoL3RU?= =?utf-8?B?czViY2pMZGpOMkVMUXUxU3FMWjVNdUZTQ3dpZjlIZnRLWHRYam5GSTJlNFho?= =?utf-8?B?MEhDOXVKbk9mN0V5ZEw4UWk1WlF3bGJ1Z1lYTG9NMUJoSEpHdzE0eXRLWDZN?= =?utf-8?B?dklhZTBEYkNJa3FSMFlyYk5RU01GM2s2YnRSNCtUU2RiTWRmWFo5UGJaMXNm?= =?utf-8?B?cXVyNFYvRnBRZjFDWk9RY3ZrTSs3Qm8rRnYwWHBoVWx4ME1GaUtWTFl1aENk?= =?utf-8?B?MGtIRTlKc3RzTmtIdENYL3JsYmtQR3g4YmxlYUxYM09Uc1hjeHUyMkxMRi9P?= =?utf-8?B?TmNybUVDMjU4VU8vV2hYOStJSEo4djZOQXV5Wmt1L0ZLcFVaeW4zTEI2WFdN?= =?utf-8?B?UGpYYTduZzZHUHE4UStRcjFOd0VjSjhtdXBNZUJzS2dHdzQvb3ViREFqSHBo?= =?utf-8?B?N1hxYWRTcHBmOHhnanQzWU93S0d2M1FZTDBXUnIzSW9EOXdSWURUbWxFTmMy?= =?utf-8?B?NVRkeUdtbEFNdU5nQ3RSMDNGV0dCKzZrV1N1K1ZtRjVsMUNmT3o3SEV5RTAx?= =?utf-8?B?SUQzNGpWY0laYzhPWjBrMFV4UXMxZm9qeE9kaTNGbDdyQmFXMGtjWUlTQzMy?= =?utf-8?B?dFh1Rk5GNVhuNGxwQ3Rhd3BpaE1DU1dnVG5yR0h4VThiR25mL3Nvc2NnU1lm?= =?utf-8?B?bnYveUdXUVIwNDNUaGtCeG9iT3RlS2xHQnlrbmtjU1ptSmFSa09yaHpJME5x?= =?utf-8?B?Z1RqMUpRQnlSMngySEZOcTdra0c5QnBzempoclN1MlhpYzdzWDdKaThIamEv?= =?utf-8?B?ejNlYm4wZGNyZGZPYXlxUFRoYUtXOGNtRVRyZ1hKblBMWVBqRUU2citwVVdr?= =?utf-8?B?SXpDMHVGYzhBL1pIOVY1aEhqUndUeVltZW9jTU1KeEFxWXliQ25TMjlIVmpB?= =?utf-8?B?OE9kYTJ6Nm8rSGoxZmx3SndtbWRXaEE3eXlhY0h6UXJZNktYUXlveXN0VUxN?= =?utf-8?B?allYVkRqTWdqNE9aZUlPT1JuMmdob0JweG9FY2t2ZEZTOGE4NWpNdTdVQ1FQ?= =?utf-8?B?QXdEbTlsOVQ3TlRZb2hSN2gzNFpoZlBMdmM2TWN2Si9GYkRrME9MQmRrTVM2?= =?utf-8?B?WXd1dnI5VTdSMlA4NUtwTHN6NXJDUWtuTTRLRjA4MHRHMkFsczBHa0Y3bk5C?= =?utf-8?B?SWRCZ0ovNDE4cEQrTDBXM0VNek1WMDR6a28waEZuMmxnc0xqZDMyeURMU0VB?= =?utf-8?B?bHhleG5zb0dlNUwwMnhlRGRMY0JtUjUrZWxML2pqOVM2QU95d2hTNnA5dWlK?= =?utf-8?B?dkVGR2FVVkIwdnpzY295KzMwYy81TzZtRVlaak04YzZ2VzJqelJzVzVjOTBC?= =?utf-8?B?ZFhNL1ZIdzd5UWYydDhwbWhXcWNPVEthSFZlWDBqZDNCUkppUXcrK0FVbHNt?= =?utf-8?B?dnpDbUNTaTNUM0ZWc1BtWi9jeG5uWWRiRmxQd1VkQjA1YU4vTVdFTzVsY3lr?= =?utf-8?B?NHYrVkc4eHRHdmJ4N2tVTHZSVTlGaEJ4aUxXTURuMlZBY1UxdEExV3Mrc3hp?= =?utf-8?B?MVliMjFSZVBkMjAxelBmK2haKzdpQS8zbWowTVFxQWdGYnNwVjNIcDd4aTYy?= =?utf-8?B?VzBDQkI3LzhtVXIyTUNoYVU3M2MvemlFK3ppL3dkYmJ4SkZOVi81NUtod1Q2?= =?utf-8?B?enVOeS9ldml4SlFDNFV2dz09?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZnhOa3ZKbDhtb05wR2N2N0NRMGEyQ09wYkNBSU9GejBoY3J5Yk1UcjgrWWdF?= =?utf-8?B?TGJZTVZtaGNqanZLZGJ6dFlGRUM1ZXFLVkdyQVg0bGxzamhKL3BpbFIrdVp6?= =?utf-8?B?VlRoTVd3V3J0ZlRBZSt1UnN0M2tzNkpUNlFQY1huSlBhcDNiRGthZ3YxNUt1?= =?utf-8?B?TXluaG1aNTFQY1JxK1lXV1JWOVhqSUZrSVA4d2V5NDRPYkJFNjRFM2x6YnpT?= =?utf-8?B?aTlSQzl3QWp2alZUaVBXSUhRaFE2a2ZISSs0SlJVengyNmR4NmRsaFhHdTV4?= =?utf-8?B?NlkvNVQrSmFUYVB6a0t5STlNNGcwY25NSFd0V3JveEcyZGVTMk4wUWcwL3I0?= =?utf-8?B?a2tEd01yVkJlSUJTZldEWlBoTjZDN1MzY0JlVE5tOFZqUlBiY2NuNW4vMGNx?= =?utf-8?B?a0IrYVlTejczZEZoT1pRNWhLNTQ1S1l1VHQxb2ZXKzRIQUZRQlUvTk5FeUFh?= =?utf-8?B?aVZVcTNwVG8rUkM1YkZ3c1pCMjhqd3ZXUHNyZ0thYnBpQU1UYTVyNzh1MTE3?= =?utf-8?B?RnVjQ2VKZ3poN3ZmMTBPL2FZMThpd1l5c1JSWStBRnpuSkVuQW13VSs1UXd3?= =?utf-8?B?SHdGVjhPOHVvN2E1TEtoK2o2dkg3cC8xUk5KQ1VkK08wOEMyalhpREgyR3hB?= =?utf-8?B?ZjBnaFRGMWl4SjhFcU5VS2oxaU1JV3lmbVZjekhtb2ZoWEZjZG9PbFJDbElY?= =?utf-8?B?dUkzS0pRWmlyQjhFZklWU0M1NlpIUGlOM2RSQ20rdmxxYUtMOEZHMml0OTVt?= =?utf-8?B?Z0VnK1VlU0pqTVpZdFFqTURGSWtBQVhHNXk3M2QyZHprNHRNM04zYm1qN0xT?= =?utf-8?B?eGxOZDNlbEhiN0NxaUkvWVUzN1lMNWxhdHJ2VmYyYytVdjZiclVvem9HdnlM?= =?utf-8?B?VE9ROTVqZWJLUWJPdzhjeXVTMUFuUkxoc2hIVU5GZGRCd0Q3bWd5Ui8yeE1v?= =?utf-8?B?bFJQTnNhL05qSU92enI0SzB5Z2RCTjFmajRFZTBMdTJ1MjYrODEvUURSS2Q5?= =?utf-8?B?N2lKWUIrbUF2YnFXZXM1NnI1d1EyMjNiWVdRLzVKMmU4cUpZY3pBZzMremxP?= =?utf-8?B?R3p6cDBWRXMzZm1aSExTQWdtR25LMXI4SDRGTHJDZE5nZ2psaUFOMG5Cb2t6?= =?utf-8?B?cmg1cUU3c2dNaEZGc0ZKa1hKeHZxbExBdUsyemcrMEJBK05zbGg1MmpqMmNk?= =?utf-8?B?Y1loQitNOGdLQjlsbElCRlBPMGVMbDYvT3ViNFlCQVRsQlNFTTNjRi9Vb1ND?= =?utf-8?B?T1lGQVF6Y0JkWWcwbWpiWjV2Zm4rLy8rRFFTMVk5MGFXSjZ3bExLOHRDek0y?= =?utf-8?B?NTU0REd1YmN3NVFIVmJoRFRnNWZvb1VwZDkzekdvQzI3K00vRytacnYvUGJq?= =?utf-8?B?bWlsbEVzUGN3TFFBbVEvdEtnZWVrSDZkcG4yT3JETnZBcS8rdEdlOEl4QjdB?= =?utf-8?B?RHRQcmkzRDRzZE9VMGJmVld6K21EZ1ZsVGZYMXo5RWVHMlllTFYyQ0t4WG9P?= =?utf-8?B?Zkx5dytmcVhyWWpRdWduS0VsUisrUi9lMTBOQWxTc0h5Zml2K0JyQ0x4UVp0?= =?utf-8?B?Z205QTZWektPYUd0ME9Kak4xQTUwQVRkWGNSeTlzRW9hK2oweG92K3FTU2Fm?= =?utf-8?B?M1craUZHUVVqVDBCT0oydU1XNGFIVnJQN3ZmQVhuSTh5MFc3eWZ6UWEwcG5T?= =?utf-8?B?b2JTUStmQjFFNjZheHVqQ1dGTHk3aWJHVzlWdXpzc1R5Vm4wSFFiTCtNbU12?= =?utf-8?B?cTdtYU9OYjBBWDRZdU9YRzhXVW9VQ3BTNkRPYnFpejNFdmtnaHQ5c2FURTJk?= =?utf-8?B?SFY0MnFUdHFQL1BtK0lrWjcxSXlscWR1bXkxRFVBSlBkWWpUbkhzejlOUmlw?= =?utf-8?B?Kytic0t5aUVOMnBVN3ptNUxDSEY5SmlHWjZZMldRNmFENVdtOWpXbjduZFNo?= =?utf-8?B?WWhLTERMOTdZTkx5UVdHL2NxU3Yzb0NHVUNZdDJBY1JBWUxGTFFCM3p2Mnll?= =?utf-8?B?Q25jVVcxUHdENERXNmpDaml5bGNVd3hjMm5RSXVGMjFKVXF1WjNNa21qVndt?= =?utf-8?B?V0hJUlRWVGRCcGRwalNTUFA2NE1SM0VTeGE2bkdVWmJrdHlPQ1JnUzQxMEFS?= =?utf-8?Q?MS2/4Luu2NEJ00ByRwALTCgKR?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0deffe2a-03cc-426c-4573-08dccdc04070 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Sep 2024 15:34:39.9653 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 558Fn+3a+thZa9Vra3qurhi9ysvmW2vvQKlqWARl8933eoKtAFFV32q/bxH6IpIr X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8227 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On 9/5/2024 3:45 PM, Burakov, Anatoly wrote: > On 9/5/2024 3:05 PM, Ferruh Yigit wrote: >> On 9/3/2024 9:50 AM, Burakov, Anatoly wrote: >>> On 9/2/2024 5:33 PM, Varghese, Vipin wrote: >>>> <snipped> >>>>>> > > Hi Ferruh, > >>> >>> I feel like there's a disconnect between my understanding of the problem >>> space, and yours, so I'm going to ask a very basic question: >>> >>> Assuming the user has configured their AMD system correctly (i.e. >>> enabled L3 as NUMA), are there any problem to be solved by adding a new >>> API? Does the system not report each L3 as a separate NUMA node? >>> >> >> Hi Anatoly, >> >> Let me try to answer. >> >> To start with, Intel "Sub-NUMA Clustering" and AMD NUMA is different, as >> far as I understand SNC is more similar to more classic physical socket >> based NUMA. >> >> Following is the AMD CPU: >> ┌─────┐┌─────┐┌──────────┐┌─────┐┌─────┐ >> │ ││ ││ ││ ││ │ >> │ ││ ││ ││ ││ │ >> │TILE1││TILE2││ ││TILE5││TILE6│ >> │ ││ ││ ││ ││ │ >> │ ││ ││ ││ ││ │ >> │ ││ ││ ││ ││ │ >> └─────┘└─────┘│ IO │└─────┘└─────┘ >> ┌─────┐┌─────┐│ TILE │┌─────┐┌─────┐ >> │ ││ ││ ││ ││ │ >> │ ││ ││ ││ ││ │ >> │TILE3││TILE4││ ││TILE7││TILE8│ >> │ ││ ││ ││ ││ │ >> │ ││ ││ ││ ││ │ >> │ ││ ││ ││ ││ │ >> └─────┘└─────┘└──────────┘└─────┘└─────┘ >> >> Each 'Tile' has multiple cores, and 'IO Tile' has memory controller, bus >> controllers etc.. >> >> When NPS=x configured in bios, IO tile resources are split and each seen >> as a NUMA node. >> >> Following is NPS=4 >> ┌─────┐┌─────┐┌──────────┐┌─────┐┌─────┐ >> │ ││ ││ . ││ ││ │ >> │ ││ ││ . ││ ││ │ >> │TILE1││TILE2││ . ││TILE5││TILE6│ >> │ ││ ││NUMA .NUMA││ ││ │ >> │ ││ ││ 0 . 1 ││ ││ │ >> │ ││ ││ . ││ ││ │ >> └─────┘└─────┘│ . │└─────┘└─────┘ >> ┌─────┐┌─────┐│..........│┌─────┐┌─────┐ >> │ ││ ││ . ││ ││ │ >> │ ││ ││NUMA .NUMA││ ││ │ >> │TILE3││TILE4││ 2 . 3 ││TILE7││TILE8│ >> │ ││ ││ . ││ ││ │ >> │ ││ ││ . ││ ││ │ >> │ ││ ││ . ││ ││ │ >> └─────┘└─────┘└─────.────┘└─────┘└─────┘ >> >> Benefit of this is approach is now all cores has to access all NUMA >> without any penalty. Like a DPDK application can use cores from 'TILE1', >> 'TILE4' & 'TILE7' to access to NUMA0 (or any NUMA) resources in high >> performance. >> This is different than SNC where cores access to cross NUMA resources >> hit by performance penalty. >> >> Now, although which tile cores come from doesn't matter from NUMA >> perspective, it may matter (based on workload) to have them under same >> LLC. >> >> One way to make sure all cores are under same LLC, is to enable "L3 as >> NUMA" BIOS option, which will make each TILE shown as a different NUMA, >> and user select cores from one NUMA. >> This is sufficient up to some point, but not enough when application >> needs number of cores that uses multiple tiles. >> >> Assume each tile has 8 cores, and application needs 24 cores, when user >> provide all cores from TILE1, TILE2 & TILE3, in DPDK right now there is >> now way for application to figure out how to group/select these cores to >> use cores efficiently. >> >> Indeed this is what Vipin is enabling, from a core, he is finding list >> of cores that will work efficiently with this core. In this perspective >> this is nothing really related to NUMA configuration, and nothing really >> specific to AMD, as defined Linux sysfs interface is used for this. >> >> There are other architectures around that has similar NUMA configuration >> and they can also use same logic, at worst we can introduce an >> architecture specific code that all architectures can have a way to find >> other cores that works more efficient with given core. This is a useful >> feature for DPDK. >> >> Lets looks into another example, application uses 24 cores in an graph >> library like usage, that we want to group each three cores to process a >> graph node. Application needs to a way to select which three cores works >> most efficient with eachother, that is what this patch enables. In this >> case enabling "L3 as NUMA" does not help at all. With this patch both >> bios config works, but of course user should select cores to provide >> application based on configuration. >> >> >> And we can even improve this effective core selection, like as Mattias >> suggested we can select cores that share L2 caches, with expansion of >> this patch. This is unrelated to NUMA, and again it is not introducing >> architecture details to DPDK as this implementation already relies on >> Linux sysfs interface. >> >> I hope it clarifies a little more. >> >> >> Thanks, >> ferruh >> > > Yes, this does help clarify things a lot as to why current NUMA support > would be insufficient to express what you are describing. > > However, in that case I would echo sentiment others have expressed > already as this kind of deep sysfs parsing doesn't seem like it would be > in scope for EAL, it sounds more like something a sysadmin/orchestration > (or the application itself) would do. > > I mean, in principle I'm not opposed to having such an API, it just > seems like the abstraction would perhaps need to be a bit more robust > than directly referencing cache structure? Maybe something that > degenerates into NUMA nodes would be better, so that applications > wouldn't have to *specifically* worry about cache locality but instead > have a more generic API they can use to group cores together? > Unfortunately can't cover all usecases by sysadmin/orchestration (as graph usecase one above), and definitely too much HW detail for the application, that is why we required some programmatic way (APIs) for applications. And we are on the same page that, the more we can get away from architecture details in the abstraction (APIs) better it is, overall intention is to provide ways to application to find lcores works efficiently with each other. For this what do you think about slightly different API *, like: ``` rte_get_next_lcore_ex(uint i, u32 flag) ``` Based on the flag, we can grab the next eligible lcore, for this patch the flag can be `RTE_LCORE_LLC`, but options are wide and different architectures can have different grouping to benefit most from HW in a vendor agnostic way. I like the idea, what do you think about this abstraction? * Kudos to Vipin 😉