From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9BAA5458E9; Mon, 2 Sep 2024 16:17:46 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3C7B040651; Mon, 2 Sep 2024 16:17:46 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.21]) by mails.dpdk.org (Postfix) with ESMTP id E7C984042C for ; Mon, 2 Sep 2024 16:17:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725286665; x=1756822665; h=message-id:date:subject:to:references:from:in-reply-to: content-transfer-encoding:mime-version; bh=bwbGY5ivwa2f8ymG5ZOA9Hth1+LeEIjMHCkBfTVWUpY=; b=Yv8iTV1FTDScQ8Iitjfc9z7878V0qU+1KwvP7Pw+s/vU5WDo7nS9aiaN Zg3Z27RxX4Zcm53QLDgo4y7m5EyYmYf8t3b0ZuvLvHuHphZDkgFulKtop 3bRaio4dpL5XxLV7wLT/n8A2zRWf0VMd3cwTUENShQM2ZE5q9IFtWRkXL ZcEL7elRlbBXbaRySOH4/AlckK+yBjSAt/xV8AzVSRMOpXuXHyffFEZ92 dr3f/1j0XZr0R4y7XR8sJkuBCC3QQKb+B8wbf9PigGiVkI9/TxM3rQvZJ x9obhsxxkjaz2S+iKleBM06jRiq1ohzcuWmCHf6NQ4jtj5BVRAdl/3Ttc g==; X-CSE-ConnectionGUID: SnMDOqZYSvmLNEBnLRsEBg== X-CSE-MsgGUID: i4sX88tGQA60wInUcS0jWg== X-IronPort-AV: E=McAfee;i="6700,10204,11183"; a="23826107" X-IronPort-AV: E=Sophos;i="6.10,195,1719903600"; d="scan'208";a="23826107" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by orvoesa113.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Sep 2024 07:17:43 -0700 X-CSE-ConnectionGUID: wsS1SkO9SJCBONIy4hdWxg== X-CSE-MsgGUID: T1jJcr5dS0S9LUPA7Vc84w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,195,1719903600"; d="scan'208";a="65351443" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orviesa008.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 02 Sep 2024 07:17:42 -0700 Received: from orsmsx611.amr.corp.intel.com (10.22.229.24) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 2 Sep 2024 07:17:42 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx611.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 2 Sep 2024 07:17:42 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.44) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 2 Sep 2024 07:17:42 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jVkrim2KZVeru3bUY4Ld1+ETdhD9gp8NbQBJB5G4TnpgTgQZ/0xkyWKbN+ZQlEO28vAaRDLiz8TPXtYMrumCtvYZnbEjYI9tJaZtOrxpEL5Xw3b4r2jqeCpCSkiUX9fKZv3SAH0bBBoYqPX6WNcWHi/GqPj0uBMqeTObHXKR9tHZD01vxfV/X1ogaDnjmXpkysI8pRu9BpTAKumUB3jnmktGRT5O8kKfryZ4xpLHDvKutQzSHtHRf0TR8JsQtEpIWM/nDjvQlNk2HnvReasIFT+J8uWT1FKwgCJvHu6t7Dh+p1ubYPKbQe5x3TIQnr3ONUG2/AP3bkNcUYOqG1A4Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7yW+wdo9b+oU7CxEMwawnV3q8UhS4vhNhB/3Olyws9U=; b=DNmHEXWUOb0XzimrDokTA8RT7SIVwRqy0YIq1ufMdwVTX3xn03adPWeoB72gVHZPr61cZW+8pk7cvkkTRXeHlLJp1S+qC1seDMkve619fguA+kHGlxFxFfAA3NxeIfRqGq6+QBiBso9c9ofmIR+YDYPge1g8v/M3ALKXFMx/hTaUHZW5/M6i9TQ1JOMVH1w6t6dYOgnDWQcQkvTpwS59FM8LPHrJ1/KyQiYDIDTOBPIBZHvijtsrtjXVgNAwD2gBUzWsLb++IoUqw3JKUp6ECaa7YR0AxOYRQqp18wNZrh5+3t8Y66L72ZO51f4MyVR+wa/XW8Ow0VYMu8D0ebXJCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) by IA0PR11MB7403.namprd11.prod.outlook.com (2603:10b6:208:431::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.25; Mon, 2 Sep 2024 14:17:39 +0000 Received: from DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::21e4:2d98:c498:2d7a]) by DM4PR11MB6502.namprd11.prod.outlook.com ([fe80::21e4:2d98:c498:2d7a%4]) with mapi id 15.20.7918.024; Mon, 2 Sep 2024 14:17:39 +0000 Message-ID: <8addd7f6-fac8-45ec-a44f-f81eb008cc36@intel.com> Date: Mon, 2 Sep 2024 16:17:13 +0200 User-Agent: Mozilla Thunderbird Subject: Re: [RFC 0/2] introduce LLC aware functions To: "Varghese, Vipin" , , References: <20240827151014.201-1-vipin.varghese@amd.com> <288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com> <65f3dc80-2d07-4b8b-9a5c-197eb2b21180@amd.com> Content-Language: en-US From: "Burakov, Anatoly" In-Reply-To: <65f3dc80-2d07-4b8b-9a5c-197eb2b21180@amd.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: DB8PR09CA0010.eurprd09.prod.outlook.com (2603:10a6:10:a0::23) To DM4PR11MB6502.namprd11.prod.outlook.com (2603:10b6:8:89::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6502:EE_|IA0PR11MB7403:EE_ X-MS-Office365-Filtering-Correlation-Id: 14969790-72d8-4dec-1f92-08dccb59f4cf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?B?Y0VuREFHM243UjBxNU5tcVBva0RaWEppbWh3YWRIWk5MNHMwSU5WQ3hSNFpx?= =?utf-8?B?dFpmMVN5OERSWW9naUUwQ3FCRStuaUNQb1NtdFZmaHNZd2JEWWJ1c3ZoQzFN?= =?utf-8?B?Q3k2VW5HT1pkcXI2b0RJeno3bEdsTjl2eWhEQ1Ewcmg0NEhzL1drL0t4clBO?= =?utf-8?B?cWFYNGFMcmcxLzdjY1RUVVRpeXhmTmIvY1I0WDhVenFWOHhTV0RvUi9lZDF3?= =?utf-8?B?N1Q2djB6OCtobVBiZWNaL0xtVkxIVkFXZ1RmRDdrUnlmbzdSckhKaFppU3FR?= =?utf-8?B?T05UcDJvcWZ3eGx0Sy9WbGZIdW9URlNYc2NSelBUMHJFZGRBT1EwYlNqY1VK?= =?utf-8?B?MnlLM3VpcUs0WlFHUkx3OHhlS1Y1R3g1aGpDcFlWVmZDTXdHKzBvcm16bHZG?= =?utf-8?B?L3oyRXFGaE5WeFlYU3BadEVmQzVoelkwcHZ2NEpmMHoyZ2dDWXBRa3IwaDZP?= =?utf-8?B?cGJXWkJKd1dsS2NVRUVGbndnbngrY2gzY0FtOVg3UnQ0WTNBTnFyRjFrdlNk?= =?utf-8?B?ZGwrTTV5TjZQaDlBQjdSbnhhekZXNldhTUZPSjRNRGNOK0U0d0h4Z2QxMFNj?= =?utf-8?B?SXlEeDR5K1VIaDF6NmVPZTIwRnBsM1dhYW9oU24wUmlZWnhjTWc0SEhjcktT?= =?utf-8?B?aUh4Q2dtMlg0bW10NVRVajByOFQzQWM1c0lzY2dhRUFNZXcyNVg0aUxLRVFv?= =?utf-8?B?TVNQWDdHc3hBZ3I2VXJNT0xOT1UrT2VSektPM0xNN1l3Z0RybWdUbldFOUFy?= =?utf-8?B?QUhrdmpEajN5bGEveEZJcElGdEYwRFBtS285aGZkaEx0dFBQVWl2M3dKUFkv?= =?utf-8?B?Nm5xUFRtQytES0N6WWpEcGxya3h4MlNtbVBwRjlFV1I1cFpmNTVRRVJGZ2ZD?= =?utf-8?B?TElNVGxPU2ZlVnlCOUhHcHY1dGtwOHJqYnI5UUlTRGxscGFuRlRib05nMkF0?= =?utf-8?B?cDNOV1RZQzZrTnp5Q1ZObWdHdS9xKy9DNlNlWWF3Y3hHUEFqbUVtUWxETndT?= =?utf-8?B?aTVvQit1WktZMW9WUkFpQllSQ0pFNFNKK2VNdENtY0w3a1JUZWdvU3BiblA2?= =?utf-8?B?UTdXRFFVOERRb0xLdjByQVdJaGpBQlIyMnhpVTRNampuejl5VG5iemprbFE0?= =?utf-8?B?ajNOQ0luKzRub3dCaXhpQjdhNS9oalVWZnQrSnJ6cmQzZEd2WjVYak9waW5x?= =?utf-8?B?anJQWGRUTlVJU29jUmZZOWNvd05qTFUwNG9DenhGc0tFaEY2QWd3R1dldnBE?= =?utf-8?B?ajh4QW5jNmkvdGtjaEFNejJLMVlZM0E5bGV1Y1FuN2VsejhzTkxZYTQxYTVO?= =?utf-8?B?dUUxUWFuZmV3SVk0RDFNUy9EVHljSEEzWmU2WExzeTBQSTJ4S2l0QjJYZ3JL?= =?utf-8?B?N1ZWMU1FWkMxUHJNYldaMEtqakVLS3c3alFuOU1jOU4rejZHenhKVFlaeWIx?= =?utf-8?B?b3c4T0prSVJsV2ZYVW8zVEM4MTdLMWo4ejFYUlVHS2p6Tzc0NlNyYjJlZzhQ?= =?utf-8?B?MG1CdzdQQWd5enh6bmZjcEJ2aFpRZnFaUDBDVUJmRlUvbnZhN2R1a0JWMXJI?= =?utf-8?B?eVZsd2NtU0dmYzNRcEN2T2Z2bVV6UnA2TU40OWZKUkdkNjBpRXhxb3Zyb2JW?= =?utf-8?B?WXN2TDREbmNqQUxBaVVyRDJiMEtHL0RnSGlLQ1BYWjJjemxpek56V3pvNWFo?= =?utf-8?B?TllPQ3lTNmhLNUdWM3E3TWlGcGwyUlRlaWpxYlppcU5UemVvZEpFN0Fkdmpw?= =?utf-8?Q?6ubZ3lLdHcwUcRn7RV7mqzVFFYA6v2zbHWBVJOV?= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB6502.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(366016)(376014); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?em8rVnV0NUFKL2xkTzBuUDV6NUFIOXBQT0llbEVRdVE3UndzeGx2THprM25p?= =?utf-8?B?a2ZRcVR0eGZtQnZZVk5mbjl1cVp5ZjFFTlZ2b3dKM1NiaGNrOE0rVkFVZ3VM?= =?utf-8?B?ZkdyNXJwTzBsK3c3MURYUCs5b1Z6QjczMjdMUktWNDF1SjlwYTVFbm5pR0lh?= =?utf-8?B?REhiUGE0VnlXMVZsQ3FWTWdXS2ZpOFVNanc1VFBNaGtUSXdSenR1alhTbGNp?= =?utf-8?B?U0VsQVh6Mk1aVFdVQ0FGTFUrc3BYelRFUVRNR2xmd0VKMCs1V29neXlmSGF5?= =?utf-8?B?QU4rQUY1a0dJT000SGlaVXA1d3UrTkQ0R3J1Qncvcjh0VFgvK1pwdVBhZnVn?= =?utf-8?B?UkVKL2tzRm5wbE9Bc1ZjNWdsc2p1dlFXQjF5SGhkblh1QzhrRWFDVFlKNjlG?= =?utf-8?B?VkZoWnpBYUkyZHJ5SFZPcVZaenU3NldTL2tnem5QVkpiUTBGLzBDazhMVkZP?= =?utf-8?B?UGFGQjFoTG1zUTJreXhKOVMvTVpxaHh4cjdmQThqdWxyRG1aMXV1dHhqVkg4?= =?utf-8?B?Mk5nT05vNDYyTWtnQURWdk9VVlNzVUUwTjdmUEp3Z2hPeXQ3ZzdrY0xGc2lv?= =?utf-8?B?eW1WTVNpV0plSmszLy9icXg5cng2UXdZTXdUd29DVGIycmM1MmJXTVJEVERX?= =?utf-8?B?cFpsbzIxMVkwQXd2ZVd4VTV4dWY3NEIyanplVE4rUFpRblc2MFlsYjBmbjNi?= =?utf-8?B?ZnZTMkRCRlV5YktBRTZCdVd0bXl1VWJXWVZBcUYzeXlpVjduaExxcFJGVmhC?= =?utf-8?B?OWtjbmZHUjV4MlNRNUxhbmUwTXUrRDh3RE95SytNTFQrdU8zQVBoVFZpL2tk?= =?utf-8?B?ak5aRXhCS0FGTGZhYUlmdzdNckZnVkRIZmZXZlFYc211akZOSjNpbkRIRDI4?= =?utf-8?B?UzNxYWtsejJ0WnhPTUZQVzNRb21oaE9paTlNM1NCKzdBdEdqcnBBa2V3R01h?= =?utf-8?B?bDczL3NuS0NFZmF4Z1ZnQmlsbVZ4OCtIa0c1MmpubUl3MzJxeFJ0U2lxZTJn?= =?utf-8?B?THJWZm1UVVMxQkhta2lKaFYrQjdRQTI5ejVCb1p4dkh3WjJ2ejQ3NWpYa29s?= =?utf-8?B?ZFlQYSt2c0dJTElwZXY3bFlzYTdOZXJNMkZpN1FoZVR6Tm1xTXkyWEh3Rk81?= =?utf-8?B?eTlTVWIxMXRaSno5RGtuVEpjWFNnWnpiMGswRFpvZk1VMXJLYjZIb1ZPVi9n?= =?utf-8?B?QnA1TjZUeUdDb0J2K0FWS1UxVUtvN3hxSFJnM3h0NThINkxFR2tBNFVKRGo1?= =?utf-8?B?aXVHSG4xd0xxTmphemJqOThQWFpIS0Y4bTE5SHpQUUNRdk44blRoZHZ3MzYx?= =?utf-8?B?RmJNSzEwclZrSjdTYWdCTGdRNUN1ZU83WnlrYldsTkRpNng3L2pzUHdNQ3o5?= =?utf-8?B?QUZWU0NNSkdhQlJuRDhDcjIyOEcxMUdORXdPRE5lSVJBdU5YbTFKeFRZU1Bi?= =?utf-8?B?eEYwbkE2eFpEbGMwSXR2eU9hRFJTdFVZRXoyU2o2VklxcDMxN2ZocTNNSVBy?= =?utf-8?B?VmlzMVNJc3cxd3NUdjBuRzdsZ0QzYTZLOTV6ZVZyRW94RlRGUTdSMHYwYU83?= =?utf-8?B?T0VaZk1hNUNwRUhMczJJemlLRXVCaDk2ZUlXTkQ0OXFFcHd4STFkczIzMVh6?= =?utf-8?B?MnU1T3JVTnlYVlhDRW5LQXNJS1JUKzh6bFdUMHFNZzVOTE1xZ1BDK2xhMGYx?= =?utf-8?B?RVhSUHdHUDFHSEk2V2gxSFgvZjhHTmZoMnNFUnhZYUtZZjJqY3ZBQ1RvOFJL?= =?utf-8?B?SEVZUUZDSTEyYkJWN2F4VHpFRTNCcElmcW92ZFJzTlc4bU4wc3pteDdKQlo3?= =?utf-8?B?ajExWW1RWnpSWFVQQUF5QWdrUlpiRE5aS0dqbi9KNjdEN0FHblgwdFFKa2Rq?= =?utf-8?B?amFSR25FSDlFcjhQT2tkS2tubDVzVHA0SUkxYUFtQzNyREpkb1ZXTStyeDlU?= =?utf-8?B?QVRjNjU5M2R6czI0OW01RXI0R3Q5ZHlnL2I1d1JLN1FjTC82VFYwOUNRQ3Ni?= =?utf-8?B?U1RzZnd2MGpRdGRvbmhRUnhpdW0xdWc2MjRTY1ZQczZncTdCREZZQ3NYYXFZ?= =?utf-8?B?Z1VlZGI0L3ZxcGYydHJqSndoVmxwSjdncXI0SXJDUng5c0tON2dVejQvR1o3?= =?utf-8?B?c3V4a05Hd3hRWEhDRHUydjZCU1hvQ3YzT01VN2FPeXltdGlOWHlxREpNdVFG?= =?utf-8?B?NFE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 14969790-72d8-4dec-1f92-08dccb59f4cf X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6502.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2024 14:17:20.1871 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: h/OMfXm9GswVzZLj9hMcIvIAl/+70B1SAXALk/xb2Zo9GRG95oDRubUskubD8YY0HBLLszdOls9gBabJYEjoYU9oDzGx+3WeTyb4JM2RSDk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7403 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/2/2024 3:08 AM, Varghese, Vipin wrote: > > > Thank you Antaloy for the response. Let me try to share my understanding. > >> I recently looked into how Intel's Sub-NUMA Clustering would work within >> DPDK, and found that I actually didn't have to do anything, because the >> SNC "clusters" present themselves as NUMA nodes, which DPDK already >> supports natively. > > yes, this is correct. In Intel Xeon Platinum BIOS one can enable > `Cluster per NUMA` as `1,2 or4`. > > This divides the tiles into Sub-Numa parition, each having separate > lcores,memory controllers, PCIe > > and accelerator. > >> >> Does AMD's implementation of chiplets not report themselves as separate >> NUMA nodes? > > In AMD EPYC Soc, this is different. There are 2 BIOS settings, namely > > 1. NPS: `Numa Per Socket` which allows the IO tile (memory, PCIe and > Accelerator) to be partitioned as Numa 0, 1, 2 or 4. > > 2. L3 as NUMA: `L3 cache of CPU tiles as individual NUMA`. This allows > all CPU tiles to be independent NUMA cores. > > > The above settings are possible because CPU is independent from IO tile. > Thus allowing 4 combinations be available for use. Sure, but presumably if the user wants to distinguish this, they have to configure their system appropriately. If user wants to take advantage of L3 as NUMA (which is what your patch proposes), then they can enable the BIOS knob and get that functionality for free. DPDK already supports this. > > These are covered in the tuning gudie for the SoC in 12. How to get best > performance on AMD platform — Data Plane Development Kit 24.07.0 > documentation (dpdk.org) > . > > >> Because if it does, I don't really think any changes are >> required because NUMA nodes would give you the same thing, would it not? > > I have a different opinion to this outlook. An end user can > > 1. Identify the lcores and it's NUMA user `usertools/cpu-layout.py` I recently submitted an enhacement for CPU layout script to print out NUMA separately from physical socket [1]. [1] https://patches.dpdk.org/project/dpdk/patch/40cf4ee32f15952457ac5526cfce64728bd13d32.1724323106.git.anatoly.burakov@intel.com/ I believe when "L3 as NUMA" is enabled in BIOS, the script will display both physical package ID as well as NUMA nodes reported by the system, which will be different from physical package ID, and which will display information you were looking for. > > 2. But it is core mask in eal arguments which makes the threads > available to be used in a process. See above: if the OS already reports NUMA information, this is not a problem to be solved, CPU layout script can give this information to the user. > > 3. there are no API which distinguish L3 numa domain. Function > `rte_socket_id > ` for CPU tiles like AMD SoC will return physical socket. Sure, but I would think the answer to that would be to introduce an API to distinguish between NUMA (socket ID in DPDK parlance) and package (physical socket ID in the "traditional NUMA" sense). Once we can distinguish between those, DPDK can just rely on NUMA information provided by the OS, while still being capable of identifying physical sockets if the user so desires. I am actually going to introduce API to get *physical socket* (as opposed to NUMA node) in the next few days. > > > Example: In AMD EPYC Genoa, there are total of 13 tiles. 12 CPU tiles > and 1 IO tile. Setting > > 1. NPS to 4 will divide the memory, PCIe and accelerator into 4 domain. > While the all CPU will appear as single NUMA but each 12 tile having > independent L3 caches. > > 2. Setting `L3 as NUMA` allows each tile to appear as separate L3 clusters. > > > Hence, adding an API which allows to select available lcores based on > Split L3 is essential irrespective of the BIOS setting. > I think the crucial issue here is the "irrespective of BIOS setting" bit. If EAL is getting into the game of figuring out exact intricacies of physical layout of the system, then there's a lot more work to be done as there are lots of different topologies, as other people have already commented, and such an API needs *a lot* of thought put into it. If, on the other hand, we leave this issue to the kernel, and only gather NUMA information provided by the kernel, then nothing has to be done - DPDK already supports all of this natively, provided the user has configured the system correctly. Moreover, arguably DPDK already works that way: technically you can get physical socket information even absent of NUMA support in BIOS, but DPDK does not do that. Instead, if OS reports NUMA node as 0, that's what we're going with (even if we could detect multiple sockets from sysfs), and IMO it should stay that way unless there is a strong argument otherwise. We force the user to configure their system correctly as it is, and I see no reason to second-guess user's BIOS configuration otherwise. -- Thanks, Anatoly