From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 1E286458C8;
Mon, 2 Sep 2024 03:08:40 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id A5A7840269;
Mon, 2 Sep 2024 03:08:39 +0200 (CEST)
Received: from NAM10-MW2-obe.outbound.protection.outlook.com
(mail-mw2nam10on2086.outbound.protection.outlook.com [40.107.94.86])
by mails.dpdk.org (Postfix) with ESMTP id 384D6400D7
for <dev@dpdk.org>; Mon, 2 Sep 2024 03:08:33 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;
b=b1UxlkvdkzKF7aim2vXJdAaR7tKM2FwgKhFOK4wrfWH+Aekka+s9wI2AZgBZJF/UVKFoz57zWpWBKEowzNJ8FnUMaBo9dmIy8ONsmPOZIVUJjwaoJGZ7Q5m8ytlcg70sSM0cEv0kkYQxmXHhf4Qrn4BvuNwDYLOCcqTOWx3hi8KNnaJqjXVeNVKlAg7f4DESyffLcau8N7M03od9UW4ViDhm4sVCgZENamcZgKZjicsOpr5ogvwyn6usNEyLdok5SqP0bp1bzMYr35Fm6mPvUSurpPfZoxsIEeMkQK2V2nwBBmIATQw66S1+HFNAxoRX77TQ/I9P+YID02buV9gVew==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector10001;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=x2R/aDukjFRHH2WaNEEKSo8/H+J1KMWOrpVYlNkwiF8=;
b=vHTD3IQO3+rkeLxdgti7MUsy1SnQ5vFrvDyHfPz/1nxaqM3i/VGLtL2m1JTKikLXwHKqI7hYOUqhzO4n5xcKi0oZZ5o3A5m0LIcZKSF4KzZ2vkknpbbX7xEnZOMgp8YEIGZPLyucQM1dc3xfoNl4/3vWAyzZVM1DU3D+mLVaEWLwv/kBfi4iGaB9fqVtV64OTyEs7Aq2MkCIar6GWGzQdx31npr4KPu1zf7kGajXEzVnjXu/xgwngpfIz1U1bXd40ttJcuCP8aka0Pwud3nVc6KED/iedmpa4E9KFPcdVlwz/map6+3JX9aVRPdfsNXO2+YurjgWV3lah0QEjggptg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass
header.d=amd.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=x2R/aDukjFRHH2WaNEEKSo8/H+J1KMWOrpVYlNkwiF8=;
b=IHToJEa6javqedYVfBMTBLwWM8aAmaDhsE60oyaIsUkywOvQd2Io7vkl54imLacuu3E9PJmur2J4L4u1oUuiAIdbqIffCXOr6kqOfLJ53C5bzb9IS/iULIq9fMXPEyQ0R2u3z/jadkELSEAhoHOu4sIuJDoR8y0qR/IWmNLzlF0=
Authentication-Results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=amd.com;
Received: from PH7PR12MB8596.namprd12.prod.outlook.com (2603:10b6:510:1b7::6)
by IA1PR12MB6556.namprd12.prod.outlook.com (2603:10b6:208:3a0::14)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.23; Mon, 2 Sep
2024 01:08:29 +0000
Received: from PH7PR12MB8596.namprd12.prod.outlook.com
([fe80::a011:943d:7291:8069]) by PH7PR12MB8596.namprd12.prod.outlook.com
([fe80::a011:943d:7291:8069%5]) with mapi id 15.20.7918.024; Mon, 2 Sep 2024
01:08:28 +0000
Content-Type: multipart/alternative;
boundary="------------VITCIWUFQtMLPc17thORNsaM"
Message-ID: <65f3dc80-2d07-4b8b-9a5c-197eb2b21180@amd.com>
Date: Mon, 2 Sep 2024 06:38:20 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC 0/2] introduce LLC aware functions
Content-Language: en-US
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>, ferruh.yigit@amd.com,
dev@dpdk.org
References: <20240827151014.201-1-vipin.varghese@amd.com>
<288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com>
From: "Varghese, Vipin" <vipin.varghese@amd.com>
In-Reply-To: <288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com>
X-ClientProxiedBy: BM1PR01CA0161.INDPRD01.PROD.OUTLOOK.COM
(2603:1096:b00:68::31) To PH7PR12MB8596.namprd12.prod.outlook.com
(2603:10b6:510:1b7::6)
MIME-Version: 1.0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: PH7PR12MB8596:EE_|IA1PR12MB6556:EE_
X-MS-Office365-Filtering-Correlation-Id: 87028420-c76f-44de-0719-08dccaebc12a
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016;
X-Microsoft-Antispam-Message-Info: =?utf-8?B?dkR5WVg3Ri85MXRhSmFoRFFtd0NtelVHWGhXZmZrTCtla00yYzNxY1FPaWFq?=
=?utf-8?B?c2QyTXhqaGU2VDA2bHNjc000UW1oZElza1gyQ0puZDRpaGZaMTlNL21OZ2VL?=
=?utf-8?B?aVQraUNidUNxNDJPRlV2VjRRRU5kRjlKeVM4K25tdG9MK2VDRHVpL0M1MTVi?=
=?utf-8?B?K1RySWgyUkRHYml6K1JPL3luc2dvTFgwdnY3c3ZPMTZWc05VcjQza3NCYlM1?=
=?utf-8?B?ZFJZdmxlQVlsNmgrYXhHM2pBWEMrd2xBMVFHZ3Y0UzVvRHl1VHlueXdsaUg1?=
=?utf-8?B?QlJWWWhobjl6Vlp0TWh6bXJzcHVPbVRla2tjUlFGRmRyUmpuUkdQazRMODlw?=
=?utf-8?B?MFdHdGNNdXFnbHFOK1hjL0o1SXVhS3hqRUJ6MXJCSC9wWWxabW1UdCtyYTQy?=
=?utf-8?B?ak5WWkE5RVdpMFE5T3dWcXpvWHNRdU1QYzJEUTk3N2wzZzcwRWNMRk1HQXEr?=
=?utf-8?B?RWdsQlVVN1hmZFZZWTUzdEorczl2bHF0WmFKengzbEhoZUdLWnk2YlV3dzhO?=
=?utf-8?B?WGtpcnVHMWVxNVAvVHBrelA0a1RzdGtmR3ErVDFJWlk3d1dkdkFoNUZzenBG?=
=?utf-8?B?UVYzN3dHZVptN3JWL0tLbzJYTHVOYzRZeHRISENrOHVkeEcwcFlOLzZXNDRj?=
=?utf-8?B?aG1kRzA5Rk1xSVVqVjM1aEFYc2ZGaHhYZEozYktBYml5RE9veGU5NTVTU25k?=
=?utf-8?B?NW5uNy9mN05JR0NJamI1VklTd0w0SkxsenA3M1ZkdnRpc3BJczNhTHBZMVVZ?=
=?utf-8?B?emdoVEo5d280V1gxMGQ5SVJSYmFnTXV1YzVFUFRWUXVFVjBZSEltYVVLdCti?=
=?utf-8?B?aC9VUFAwdy9nQlFVWGQ3eDl1L0paOHEzQ0piWjVQVmp3T3YwU0w2dFRnS01C?=
=?utf-8?B?OFhkK004TXZreENMMEttVnVVZDFuYUVFM29PSzR4MjJObkMxQStUaGZ2VFYv?=
=?utf-8?B?NmxPYUtYSDhjQkZBbTdIU1pZZWNtcUVYelN5Z3FFY2g2Qm11b1BKN2hrUlZm?=
=?utf-8?B?MVR3MzRFQ1dsdGJqMy9kNGVIK2QwdmcxdzBLY3EwMTBwSmdCeTYrZGtkdFEw?=
=?utf-8?B?elNVWnh6MUxFZWNHMW8wMjVydlVGVTlDdEZqK2xTejU0YXBYNFh6NkxBajBH?=
=?utf-8?B?b0NuZFdaUWNwRlZtQjNPaWFJWEV5eTFnSlNSR1dWNW1DbVFIVlhDMnA4VTkr?=
=?utf-8?B?Y3BDMHBxK0FjVjJVeFljcHZsL1BVTENXOTcxSXNiYk1wdktQM2hPNFF0WWs1?=
=?utf-8?B?cS9TRTEzT2FSUE5Rdk5aejFMMXBWN01IYXRjN3p6YVIrcy9URnd4Y1Ficklx?=
=?utf-8?B?Z2FYUUdNNk9DSEJheFRSeGtRK1ZJbDlvdVpoT3B6akRYbnZQTkZIbXpaZEQv?=
=?utf-8?B?YVlsS2ovc1hrSnI0YVZVdjBRZ0djWGpRaE1TOVB6ZElhdkEvNHdCV2lXZS9W?=
=?utf-8?B?VEpyaktuckp0dURiU0wzZloySzZ6Q2pwVitNNFBzZ0hQU1BkUnBLVFF3bC9k?=
=?utf-8?B?N3pxSGFwaGNkQTlPRDhGTHdxRitob0xiK3BJaENiWTdGOTFKczM5NVc5dFl0?=
=?utf-8?B?KzFIelZSRVR4d1ZhM2VzUEpLN0pqTnZralVKeUxhdWxUYzNkcE51Z1dla0dw?=
=?utf-8?B?bVZCZC9iNSs4Z2VuWGJya3Q0R1ZUL291clZtQ2N4RkdpOWNBR2w1eU5WZEN2?=
=?utf-8?B?OStjZjgzOG5DNlViTWx4djJocHExNW9HTmhMM1pLcGJMa0ZqMmlNMjkxd3ls?=
=?utf-8?Q?ypxha9qx86XQhN1WTA=3D?=
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:PH7PR12MB8596.namprd12.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230040)(376014)(1800799024)(366016); DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V2NpSm5DYVR3aTdKOWtrRGNDdDhGcVl6Y1huS2tiaEhwTk9CSlJDSHdIeDlm?=
=?utf-8?B?em4vMjZ6TmVNNmV4YWhDWUdtb2Y0MG1UMU04aVFqK1pSbmp4amwwOTA4NGgw?=
=?utf-8?B?YjlrWktuUVBKZXBKeEJnMkxMV3lQOFNNeW53ZDF3Mis5SFJiWXI3WHZRM0gz?=
=?utf-8?B?TnVocGw5cnZFZEZXTTNaNXlVbGQzcFBEZnNSenhyTW1jM1FYNTdOYjZZMUUy?=
=?utf-8?B?aXFtajZHcFltYXRibER3UEY2MXd5TXZBMTVhdGY1Mk9maU1xK2xPODlkbmln?=
=?utf-8?B?eDZSUmxOa1pFVnlKc1BMa25PVDlLTGFSZzdTSE12NkFKV0doWmRON3kzcTh3?=
=?utf-8?B?cGlaMzJvaVpRZTFPaUVCcG1vcVA3SkNMYTVxWDExR0JLMVNDS3hESlc2Nk5h?=
=?utf-8?B?OEpGYnFXTWgxbll5WWVVS3RyU1ZDT1EzODVyNk9sZVcvWFZuQ29KWGhoRThu?=
=?utf-8?B?QVdlcHhjbkcxY3dBOWtENDhmRjBkTkYxNXAyYTdPTlRiRkhFUlRETnc0QU5s?=
=?utf-8?B?dnZSR0w3U2s5QU1CSmdXS3A4S3dWcUNaaUswWkNPdDJJRTJ1TWk0NFdhRDcx?=
=?utf-8?B?cVdXaExpaGFNREZoOG9JSC80enhWVkE1WC9EK1BqV2lRdXZnSUlLS1pNaE1w?=
=?utf-8?B?WmJGL2NMN282SWozT3hvRjlrenJoV29UM1B1UlBaMDV2azhiS0hlbmZsbmtQ?=
=?utf-8?B?RlBlN1R4c3NnV2cyK3Rpa3luVzFlWUdxN2R3REZQTGdtb0lBVmxsODF0NVlU?=
=?utf-8?B?ZDh1MUFjRnpKLzFGQU4wYVo2bERvakxkL3Q2Mmd6dVN0amtJNUdIa2gxQ095?=
=?utf-8?B?N29CazVzRnc4UnYxY1FMSkxJTk1Lcm55ZnE3Zy9sODVBa1MzbXZNT2orM1lZ?=
=?utf-8?B?emFLNFlRVWZ5ZkhFVmwyMHNOdm5yOVV3ZE4yN2s4RXgySjJIbXJ0a1V1eE9C?=
=?utf-8?B?SVVHc2JIYk92YXd6MzhnSDRJcERJMEJLeHNzMTRIWkg1RFFsbSttOUpZMFEy?=
=?utf-8?B?NTQ0bFVHSTJrWEhOY2VyVlRsakc3UVFZWnBCUGRpd25wTHBYeWF4NmlraG1C?=
=?utf-8?B?Q0huZWgxUUN2NGpNQUMzaG83UXp4NnBKSmtadjBwNjFWTElJY1NvUVdtNklV?=
=?utf-8?B?RUNwZGFsTEIxbVpCd1dvZG0vM3lHclpOTjliY2xBZ05jd3lGRFd3NFAySTF5?=
=?utf-8?B?VjFXdER5WnZvT0g5OXByYjNQWTJ0aWtqMFFlSnpnU0ZnVjlPSzZCOEpoZCtQ?=
=?utf-8?B?RkFjS0FKUXRJUVpqclEvRmozV1JwMjQrWHMrSFVYQWczeXp5cGtLVk9mZVpq?=
=?utf-8?B?Y3ZPRWUweFVVU2J4T2MzbGJNYmtHRkdEMnVOck4wRXpJSFpzQm1hRnR5SEQ5?=
=?utf-8?B?N0tRQTRrRU9HL0FSSEhNL25oQnRjZEVEQi9tdU9UNS9QQ2dUQTRRdFpyU1Nh?=
=?utf-8?B?QThVTmJWc1JHRHpZU2k2SHZWLytIUXB6eFBjbS9SQUgwamhPMDBER1RZUWIx?=
=?utf-8?B?UUZlUDRhZWVwMnZvbURPSlY5bmJ0M0JNM2xhM3Z3WGNxN294Nmw2U01DVUNq?=
=?utf-8?B?bnZWK2VvZC9LdnF5cWN6QzVVZ2xyMWR0VXBaRHc0UktsVEpzZ2FJeithVmZ5?=
=?utf-8?B?eDFocURXUkRmMWhlNDl2R0cyL3NjSlV3TTdhUTE0dHpubkk0aG1pVXgvZ29T?=
=?utf-8?B?bGhOVVRjWnJtQmVueWtwUXN1anhnVTVodkt6bVN3Z1NPQ2xYUkUvQXBsdDFF?=
=?utf-8?B?Y3hRZi9tMUV3dGRaR1dBc2pYYitTNjRqYmIyOExmc21DTmVkVHNkV0Z1Qm5l?=
=?utf-8?B?SThERmNpb3JJR2p0ZEQ3RzZpRzRKd2loSmZ6ZmVCQ1hGZ3JkRWNXQnNFdU9D?=
=?utf-8?B?RVg2WDBPS2pjOUI2Q2FnUnBOYWszNVcxR1p2amRCZjk0ZW5Pa2xIT0pJNENH?=
=?utf-8?B?bzFNa2pDWVROTVJMM2JwUjRuN0Y4bXBDMnJPWjhiM3A3Q21DdVNzMFhTdUZ1?=
=?utf-8?B?ZUFxQWd1NHo0djRTM2t2bFdLeHNtNS9nelJnT1JHbDRUSEVnSEsxUG54elNx?=
=?utf-8?B?UlVuQW9RcWxKR3NSOVQrSm5oZGFOY0lSV2VzVWxZcE9Nd01kODVJS2RYcFNU?=
=?utf-8?Q?gbEn6kL/jD/RYMxu7xcwcng60?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 87028420-c76f-44de-0719-08dccaebc12a
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB8596.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2024 01:08:28.7593 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WPmZ1CiDJnagQJmtC6zWKW9BPc/5DwnJJ6+CwHTkkJteuu1rWj+Cz9ZCG/o/3ctkfLztFe59f3yX5BLiBX9uvw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6556
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
--------------VITCIWUFQtMLPc17thORNsaM
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
<Snipped>
Thank you Antaloy for the response. Let me try to share my understanding.
> I recently looked into how Intel's Sub-NUMA Clustering would work within
> DPDK, and found that I actually didn't have to do anything, because the
> SNC "clusters" present themselves as NUMA nodes, which DPDK already
> supports natively.
yes, this is correct. In Intel Xeon Platinum BIOS one can enable
`Cluster per NUMA` as `1,2 or4`.
This divides the tiles into Sub-Numa parition, each having separate
lcores,memory controllers, PCIe
and accelerator.
>
> Does AMD's implementation of chiplets not report themselves as separate
> NUMA nodes?
In AMD EPYC Soc, this is different. There are 2 BIOS settings, namely
1. NPS: `Numa Per Socket` which allows the IO tile (memory, PCIe and
Accelerator) to be partitioned as Numa 0, 1, 2 or 4.
2. L3 as NUMA: `L3 cache of CPU tiles as individual NUMA`. This allows
all CPU tiles to be independent NUMA cores.
The above settings are possible because CPU is independent from IO tile.
Thus allowing 4 combinations be available for use.
These are covered in the tuning gudie for the SoC in 12. How to get best
performance on AMD platform — Data Plane Development Kit 24.07.0
documentation (dpdk.org)
<https://doc.dpdk.org/guides/linux_gsg/amd_platform.html>.
> Because if it does, I don't really think any changes are
> required because NUMA nodes would give you the same thing, would it not?
I have a different opinion to this outlook. An end user can
1. Identify the lcores and it's NUMA user `usertools/cpu-layout.py`
2. But it is core mask in eal arguments which makes the threads
available to be used in a process.
3. there are no API which distinguish L3 numa domain. Function
`rte_socket_id
<https://doc.dpdk.org/api/rte__lcore_8h.html#a7c8da4664df26a64cf05dc508a4f26df>`
for CPU tiles like AMD SoC will return physical socket.
Example: In AMD EPYC Genoa, there are total of 13 tiles. 12 CPU tiles
and 1 IO tile. Setting
1. NPS to 4 will divide the memory, PCIe and accelerator into 4 domain.
While the all CPU will appear as single NUMA but each 12 tile having
independent L3 caches.
2. Setting `L3 as NUMA` allows each tile to appear as separate L3 clusters.
Hence, adding an API which allows to select available lcores based on
Split L3 is essential irrespective of the BIOS setting.
>
> --
> Thanks,
> Anatoly
>
--------------VITCIWUFQtMLPc17thORNsaM
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p><Snipped></p>
<p>Thank you Antaloy for the response. Let me try to share my
understanding.<br>
</p>
<blockquote type="cite" cite="mid:288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com">I
recently looked into how Intel's Sub-NUMA Clustering would work
within
<br>
DPDK, and found that I actually didn't have to do anything,
because the
<br>
SNC "clusters" present themselves as NUMA nodes, which DPDK
already
<br>
supports natively.
<br>
</blockquote>
<p>yes, this is correct. In Intel Xeon Platinum BIOS one can enable
`Cluster per NUMA` as `1,2 or4`.</p>
<p>This divides the tiles into Sub-Numa parition, each having
separate lcores,memory controllers, PCIe</p>
<p>and accelerator. <br>
</p>
<blockquote type="cite" cite="mid:288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com">
<br>
Does AMD's implementation of chiplets not report themselves as
separate
<br>
NUMA nodes? </blockquote>
<p>In AMD EPYC Soc, this is different. There are 2 BIOS settings,
namely</p>
<p>1. NPS: `Numa Per Socket` which allows the IO tile (memory, PCIe
and Accelerator) to be partitioned as Numa 0, 1, 2 or 4.</p>
<p>2. L3 as NUMA: `L3 cache of CPU tiles as individual NUMA`. This
allows all CPU tiles to be independent NUMA cores.</p>
<p><br>
</p>
<p>The above settings are possible because CPU is independent from
IO tile. Thus allowing 4 combinations be available for use.</p>
<p>These are covered in the tuning gudie for the SoC in <a href="https://doc.dpdk.org/guides/linux_gsg/amd_platform.html">12.
How to get best performance on AMD platform — Data Plane
Development Kit 24.07.0 documentation (dpdk.org)</a>.</p>
<p><br>
</p>
<blockquote type="cite" cite="mid:288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com">Because
if it does, I don't really think any changes are
<br>
required because NUMA nodes would give you the same thing, would
it not?
<br>
</blockquote>
<p>I have a different opinion to this outlook. An end user can</p>
<p>1. Identify the lcores and it's NUMA user
`usertools/cpu-layout.py`</p>
<p>2. But it is core mask in eal arguments which makes the threads
available to be used in a process.</p>
<p>3. there are no API which distinguish L3 numa domain. Function `<a class="el" href="https://doc.dpdk.org/api/rte__lcore_8h.html#a7c8da4664df26a64cf05dc508a4f26df" style="color: rgb(61, 87, 140); font-weight: bold; text-decoration: none; font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal;">rte_socket_id</a><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initi
al; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span>
` for CPU tiles like AMD SoC will return physical socket.</span></span></p>
<p><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span><br>
</span></span></p>
<p><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span>Example:
In AMD EPYC Genoa, there are total of 13 tiles. 12 CPU tiles
and 1 IO tile. Setting </span></span></p>
<p><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span>1.
NPS to 4 will divide the memory, PCIe and accelerator into 4
domain. While the all CPU will appear as single NUMA but each
12 tile having independent L3 caches. <br>
</span></span></p>
<p><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span>2.
Setting `L3 as NUMA` allows each tile to appear as separate L3
clusters.</span></span></p>
<p><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span><br>
</span></span></p>
<p><span style="color: rgb(0, 0, 0); font-family: Roboto, sans-serif; font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(249, 250, 252); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;"><span>Hence,
adding an API which allows to select available lcores based on
Split L3 is essential irrespective of the BIOS setting.<br>
</span></span></p>
<p><br>
</p>
<blockquote type="cite" cite="mid:288d9e9e-aaec-4dac-b969-54e01956ef4e@intel.com">
<br>
--
<br>
Thanks,
<br>
Anatoly
<br>
<br>
</blockquote>
</body>
</html>
--------------VITCIWUFQtMLPc17thORNsaM--