From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A4D074591C; Fri, 6 Sep 2024 10:44:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 72DF342E12; Fri, 6 Sep 2024 10:44:31 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) by mails.dpdk.org (Postfix) with ESMTP id EA2534029E for ; Fri, 6 Sep 2024 10:44:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725612269; x=1757148269; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=CXj7XXCe4ZtRdNID3/p26hTLSR/8yq+0Fiqktm6Fyc8=; b=IXKR7v5pT37zBxF1tAv3UuMxEsbhFp4nPeVHpCSTQYJyRH3Uww8QCEuW B38odetSpESmT94HFbI9ObPUxYfW6KYC9RMvoIUjc8hauhMCIcEgjPs7N yx7jrOlTNXgy5TracV1vjpJ127bnets79EA3EIrf+P3PtKEFj24bJHjBd YZnGrNAiG58FhWBV5m3ERsfSjDdJ/jLovPQY01WLLbc5xLAIkG7HgZK+L GhochvfXtl2ZS9zv7tZzfcfDpnGfY4TxG3GO+HCnoddo0svy3juxYTgu8 wB0e10YuLKm+vI6/BOkyx5X87H7Evl7yFEjVMtqCFKBiLKYNk0m36iUAt w==; X-CSE-ConnectionGUID: YC3O60SpTLW+Rrnh6r5zMQ== X-CSE-MsgGUID: cKv97P/YTXOw81+1AwUyyw== X-IronPort-AV: E=McAfee;i="6700,10204,11186"; a="34960973" X-IronPort-AV: E=Sophos;i="6.10,207,1719903600"; d="scan'208";a="34960973" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Sep 2024 01:44:28 -0700 X-CSE-ConnectionGUID: S0RAhWfCTY6pIatWYQByMQ== X-CSE-MsgGUID: isflt5DmTCW4LAi+hcYSMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,207,1719903600"; d="scan'208";a="65541827" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmviesa006.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 06 Sep 2024 01:44:27 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 6 Sep 2024 01:44:27 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Fri, 6 Sep 2024 01:44:27 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.169) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 6 Sep 2024 01:44:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f2j9oFW+MbYFfa7nZNxxvFgDrpzx0SflAJuMG72MePEma0VjIiFX66Sjholn0jNdhbF2DmnU3gOR+k3EpOfSP+DTWNxccSkZXzmZgICAC3y1i/PSEElKH4HF/Ymd466cDRZ1VGZ28yn1QJ5pWQ2k/oxddKjO/+exvAXQmAdkthBJ6DVOv0B27/5RErJbJ0mqKB0VodQx/U9f6P5hNz2U376/4UaSBvPYnL/x3KAtj2RHfd1kNPSBydyPR1yiVeQIy5oy1BIODAC9AOWr1jNNsBrpIhbEgshS7DA4aydcmzPLfjyOJL3VI0SO7Fh7SkDBKmndgNHJnBbEM0ujgsG9Qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x6a0KWgbCI5tnxiXXzIeZocpG1RICITTNiz0lNsPg8E=; b=FDOrGNuyJ7neTVTq8+jTw9OOrRfTtwKZpfyw69B5tdsyqpEE2JRw7HpjioYWRxECGIqzSo0s+8EJrNybboRL8SB61hqCL+Ne+Q9Ktxh5F6viPGQMDhV90F9m33K3X3V40Ieti+vo5+49L8yr6m6AsFmfEZ/hY2TNjHKWPGl8+EgrezuLqqnhlHnkaJuqoy/FzaqZmZktRmq1QOyXZ/VMTb1+iqP/ojVl9Y4ApjElloRbML0Ay7T5u0IMJZRsh1P1bHLa5FqJB3+8OrOErIjl7SJYdjfh9Uv80+TJtKj+JK46rU5TkvLn1WwCitJAtfuK5hbWDkA3TP+0IoLsWGUG/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by CY8PR11MB7900.namprd11.prod.outlook.com (2603:10b6:930:7a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Fri, 6 Sep 2024 08:44:25 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::21e4:2d98:c498:2d7a]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::21e4:2d98:c498:2d7a%4]) with mapi id 15.20.7918.024; Fri, 6 Sep 2024 08:44:25 +0000 Message-ID: <16b78eca-1861-45ec-8d7b-af99b57af824@intel.com> Date: Fri, 6 Sep 2024 10:44:20 +0200 User-Agent: Mozilla Thunderbird Subject: Re: [RFC 0/2] introduce LLC aware functions To: Ferruh Yigit , "Varghese, Vipin" , CC: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= References: <20240827151014.201-1-vipin.varghese@amd.com> <288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com> <65f3dc80-2d07-4b8b-9a5c-197eb2b21180@amd.com> <8addd7f6-fac8-45ec-a44f-f81eb008cc36@intel.com> <3edc8a89-7d10-47f4-8f95-856c2a7fc7ba@intel.com> <3eae1577-f06f-48f2-863a-faf70b97bc72@amd.com> <462f4550-698e-4f49-9280-3a3708448337@amd.com> Content-Language: en-US From: "Burakov, Anatoly" In-Reply-To: <462f4550-698e-4f49-9280-3a3708448337@amd.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: DB8PR06CA0066.eurprd06.prod.outlook.com (2603:10a6:10:120::40) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|CY8PR11MB7900:EE_ X-MS-Office365-Filtering-Correlation-Id: 8971285f-89fc-4da2-b6f9-08dcce501cd1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?Nk1odWFxZ1JmOW5rajd0QUpjVkljdVFPN0cxek14T3R4Y3E3TjhyK2dGWDRC?= =?utf-8?B?MnB2RkxQbzdmemp0VzlBYmpxZFowY1BVYWNtbkZpUHhBcmFGVHlqbytNRHlJ?= =?utf-8?B?SHdTZ3J5WFFiV2RvWlJZOXROSDlHTzZ2ZlZxbi94ZEtteEY4Sm1rbW1VZU0r?= =?utf-8?B?NVRkMU91RjJuRFN1VnNRU3RWeXJmNVZ1NlpLOEVMcEFES0xmc0NuWFZiQ2tK?= =?utf-8?B?NHVGZXVFNGtqTmo4ZUZKbGRLcFJzYkV3dGFRSWdmbTA2TzlTU3dWbEtrdFkr?= =?utf-8?B?QkRzUUlqNXRwaUplNDQ4OXNoVkJmMHdub0xrdm9hY2pOSThoL2Q5MEswdHIw?= =?utf-8?B?MS9oc25lVEczNmdtcGFucG02NzZDTlBpSXRNbmlzWU82TGxLUnRoZE9rdStM?= =?utf-8?B?Qmt3a3hBdU1hZm9ySTR1RVdrV1pFeTRsRUdGaUlSY01hVlljOTZMZFluR0g1?= =?utf-8?B?djI2Yisrc3dqeERiMlloZ2I1cXdvbWtxS0V1UWtyUVNZU3p4R0FRUVg2UFZX?= =?utf-8?B?TmlPMVZCV3RiSUlxWElrU3dwS2NTWmdDUWc0VFBkeG5PbG1lYXF6VzFnOENS?= =?utf-8?B?UndCY0ZVMVcwS0tDaWo4MFZnWXcxUmRCTU1zTjcxcVdtU2dmSzY1RWdlRUxr?= =?utf-8?B?bHhudGVKVHFpOFlRcU1QcmJWTDFFbnMxWitWN1locWNvNXMzZmRkNFgyejFL?= =?utf-8?B?a251WkpLOVJVMHlnQmh0WFpTRGUwSElua1paRnMwalRELzdnTVJVN1RDeGdj?= =?utf-8?B?WVFFd2pVcTA4RUNUQmc1QnFFRlNaYjZqakVPWlZCdFhXekNrTmdtenNwSTNr?= =?utf-8?B?MUdiT1lBZDNhbmFSMHN2TW5oZGRyaDJwMU9RNGFoUUtZcVZIMFM1anVhMGJC?= =?utf-8?B?L0ZPQ21rUTVkMHJOZ0xWa2ltWE1QNWc0Wnp1a0VFOElIbVhBQlhPSDhqbHVa?= =?utf-8?B?bW5jOStoK2twZUdTNzUrd205elh0cWRLcVJzU0N6L1ozTHpSd3pKYVNSRW5Y?= =?utf-8?B?Rlk5dllwNnczZDUzV1FrRkFBT2Q5VFhWcGEwZDlHSFFIZzB2eVhzTGk5ZUd2?= =?utf-8?B?OTQ0M2N2ZjJVM2c1N0R4RWN6RlFrRDdSNTNPVXZlclNLZnpLM0lwM09mU1lK?= =?utf-8?B?eHY5T1BZNGllM21xSW8yNXJ6SUFINlNPUUV3QmoxQWEyajBHazZ2ZUk2Smtu?= =?utf-8?B?Tmw2TXdDdE5ZdDJJbmJQL01MbGsxM0d6dllrSnJlNjlscWwyeDk5ZXhFRlpl?= =?utf-8?B?dTNJMnBVQk45UnNRZGI3TG5HTzNRNU0vRUQzNjZHV1BTeUltSytSRVJ2WDNE?= =?utf-8?B?dFRqVlZrYmYreEdrcVlQelBaQmdHNmsyUXFOeEJTRUtkT3VNTmZDRmw3N3Jo?= =?utf-8?B?TjYwUkRTUFcxMkZ6WGxmZDJBS2IrRWhEbFBDc0dlUUJ3UldZTk42aDAzdHVs?= =?utf-8?B?LzhEai9yN0NGU3hMRUNtMmxaeVBOV3F1TlVwSmMrUnh2Vmh0UEM0c2ltdVI5?= =?utf-8?B?Ymh0UDZNaldsZTVFWHNDVDBSTGZOblBiWmYrRHY4Sjl2TUhGKzdNWFd4VFNE?= =?utf-8?B?OEx4UHBURkx1Tkc2aU1hZkREZHVackZhYk9jS0U5SjRTMFdrSG1IRXVHSGtx?= =?utf-8?B?bjV3ckJpWUkrTVZZR0E3NVhvQ084cSt0VWYzdzY5aGRlZ0ZtMjJRb0hncWVQ?= =?utf-8?B?cCtlRmd6SGIyUjlqVDQ0TTNVa3puOS9JekpCWUZxbmdaNE15MGNFQ05mR04v?= =?utf-8?B?TVF6blg0L3JiR0tOOG9pUEJ5ZnRxZVBuMERUamROTzR1N3NzY1BodjVJeWhI?= =?utf-8?B?cjdQUEtDb0Nzd1FhTTcwUT09?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(366016); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bDBaWEZXWldOaVRLK2s4M3BNUXZtUUttdUI0TTIzQVV6OHBQTEJwdnRjVUU5?= =?utf-8?B?d2k0WkoxUmhCbEI1aFEwZ3lRdFNsS05MdVdOZW1odFFVZzRibGNlUlBLVFUr?= =?utf-8?B?blNQZGR6M2JoUEFJVHRYSFB4eHoya0lLMnpLbS8xajlRMS8rakt1Sm5paktU?= =?utf-8?B?dTh6aEpOaW1wWDJtMEhpT1hQM1d1R1hRSk5naElNdUU1anZrN3EwZE1Fekk4?= =?utf-8?B?dUN0WkhsdVZLcnJwNnpFRjVvTjRsdFZzUlFZSmtaUDVYdTZOelA1VDVnVm5H?= =?utf-8?B?ZGVGMDlldFdxSDdtSTZ2RDkzS2VNSjFSWUVCdnRqSkpBbDQzZUZ6d2I2eXYy?= =?utf-8?B?eXNmellEc1pBWTVjek1VMis3elI0SlJRTHVjcFZWNlQwTEVodENpVGNYdmtM?= =?utf-8?B?THNqRFg3ejZlVVVTU1l6N2RLLzhvRmdZZFlxWDczZTVNSGxmQjAzT0I1WFJu?= =?utf-8?B?dm1XTEJnKzVHWXAwVkhqeDRUbldSb1ZLWTA4VjBXREdTZzJSa1R0Q2tkcnhq?= =?utf-8?B?dzZaSjl1R3djSVJVckRRWGlKbnU3V1E4SVJrVDZNcDhpSlpLVGo3aUVXeS9h?= =?utf-8?B?K202SXJ5YWNxWElHcjdhZ1lBVHlHK2VhOU5yTlhJYlpVSFpwOU5tRUd4SXNP?= =?utf-8?B?STZMUFhEU2NnanVULzdpRUU4MlNteHYxTENsZEtmM2lPdGpxd2tXNy9lZGJQ?= =?utf-8?B?MkRJL3N0TmxjeGNBTHM1NkR4cFl2dTh4NGIxcEpOOStUa2JUVnZvN1hSRE84?= =?utf-8?B?QndTcTdydkhOS0sycTdMeEpXMm5mN1F0RmdXcU9SVHgzRytmdDRSWWxnNE1u?= =?utf-8?B?a1VPRGN1RlhabWYzbE1aTkIyeGt5c0ZHWElNdGg3eUljZjNtbmQ5SWR0cXlF?= =?utf-8?B?TE96bGlTVEhDS3VVVEhRUVVXQ3JabENCVWJrcENlT3dHR3B4bHVCSTdJbFE3?= =?utf-8?B?VUxQUmVyeEg3NEJLQVNFWHB1QWVMWG5SQkdBVXZGc2hzZGE2YitoS1YxcTFu?= =?utf-8?B?SjVMY1hjZGxnYzZjNXhwc0dGMlpZcEt4SGRMU2dkUmNNL21DaXU0dGxQcktt?= =?utf-8?B?cmFVY3RMWUhKMGN5V2paNjhYVXNJUUcwUUIzOWdldjVocEdhMG8vZXE1OVZI?= =?utf-8?B?alFrRk8wZThid1VEREpucWpUald1M1NCdUozSnN2ZjFZMERrL2RGbFllRWpO?= =?utf-8?B?aVlBbHhzbUc2YzJrOUdkaW5LbjdnNGR4T0t2bDV1SjMyK1JmemhYcUhPZ3Ri?= =?utf-8?B?WGxXTDdPZUliMkh6T2dUL3FMMXJ0TVUvNXNpNFFIR2hTUGpSTHJEejJEcnp1?= =?utf-8?B?aUNZajdPZ2lSWVRqazN1TGRoMzFuNS9YSHFFZGlXV2V2d1lCeFdPemRHV2Js?= =?utf-8?B?OWhHam9jdlhZWGxiSGR2U080Ukt0L2graVlTQkpBM2Y2MEVpQXF6OXpZMGZq?= =?utf-8?B?MTcxZ2xRaStaRUF0ajlBZUh4NlM2ZXdQR1RBMVZSd3I3UGE5STBQVDRCZHo3?= =?utf-8?B?bDVMMDRTOFlpZy91QlZ6Zi9rR3BITnZWdmZkYW96dnI3VUN3cDBTZE40cmtE?= =?utf-8?B?QWRsVkJEZEdXRy9vTVgzUmNPQjNWelRkYjdlYmR4ejdmcy9jMkZrQ09hZ1o2?= =?utf-8?B?U0RybUVXT1lsQ2o5UDJUeUN2WUUzUitkVDMxakg4ZDJtQitSY2x5SDV1SXhO?= =?utf-8?B?dS9xMmx2bnZaWmo0c09lMkFTdXY4TUY1RzBtelVhdS9UZTl3Zk85UzlBSzRO?= =?utf-8?B?N3dFb3FmSUNZcFgwRE4yZmhRR3ZDQVp5eDEramMxcGcxcHFNd3pOR04ydXQy?= =?utf-8?B?emhjbURSQUlCWnpMY05ZM3VxSGxCM3k1SFJOM3AyTGt5NkpnaEMzd25tR0M0?= =?utf-8?B?V1RtTi9iWjkvcE02SjdJRmhpS1hnWS81c2xCUmtrTTQ1Zi9FNHdRbVE5aFJP?= =?utf-8?B?em1SY0s0VVNzL1JQZ0UyMUdqNC9NOFV4bWpySGFJUHl0UHQ4bmxQaEZzS1dU?= =?utf-8?B?cG02elp0WkI4VE16VGpab216RHJwK3ZrR3h0RFl4QW1MaElTRVJNak9xM0J5?= =?utf-8?B?L0ZRTlU3YXAxNXRNVkxYbFNjT0lZcDdVT3FqNEg5Slc4NDBRMXhCUWp3dWl4?= =?utf-8?B?WTU4azEvd3dUcUxNcEJHL3FnbG5vS1BiOVlzSCt6QU5VUVA5cXpFS1V5RDZp?= =?utf-8?B?NkE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 8971285f-89fc-4da2-b6f9-08dcce501cd1 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2024 08:44:25.7137 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: bn1i5MUAH5kcHTjkNFdlAUukdD5qzDdW2rdQnht80gU2PnfamIJtYMHSf9EWZeKgMJCIoQxpmyQxmZAYzgAuvjVBeAEvFcXdHCAPPCzhgJU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB7900 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >> Yes, this does help clarify things a lot as to why current NUMA support >> would be insufficient to express what you are describing. >> >> However, in that case I would echo sentiment others have expressed >> already as this kind of deep sysfs parsing doesn't seem like it would be >> in scope for EAL, it sounds more like something a sysadmin/orchestration >> (or the application itself) would do. >> >> I mean, in principle I'm not opposed to having such an API, it just >> seems like the abstraction would perhaps need to be a bit more robust >> than directly referencing cache structure? Maybe something that >> degenerates into NUMA nodes would be better, so that applications >> wouldn't have to *specifically* worry about cache locality but instead >> have a more generic API they can use to group cores together? >> > > Unfortunately can't cover all usecases by sysadmin/orchestration (as > graph usecase one above), and definitely too much HW detail for the > application, that is why we required some programmatic way (APIs) for > applications. > > And we are on the same page that, the more we can get away from > architecture details in the abstraction (APIs) better it is, overall > intention is to provide ways to application to find lcores works > efficiently with each other. > > For this what do you think about slightly different API *, like: > ``` > rte_get_next_lcore_ex(uint i, u32 flag) > ``` > > Based on the flag, we can grab the next eligible lcore, for this patch > the flag can be `RTE_LCORE_LLC`, but options are wide and different > architectures can have different grouping to benefit most from HW in a > vendor agnostic way. > I like the idea, what do you think about this abstraction? > > * Kudos to Vipin 😉 > Hi Ferruh, In principle, having flags for this sort of thing sounds like a better way to go. I do like this idea as well! It of course remains to be seen how it can work in practice but to me it certainly looks like a path worth exploring. -- Thanks, Anatoly