From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8F7DB43DCA; Mon, 8 Apr 2024 10:11:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F2ACD40289; Mon, 8 Apr 2024 10:11:01 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) by mails.dpdk.org (Postfix) with ESMTP id D3D0040041 for ; Mon, 8 Apr 2024 10:10:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1712563860; x=1744099860; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=473J22UILsBP3kMbbjvx2K1mUjAbXP4otn2M/Pc8e1M=; b=aSbhBWx1Rnvi+LDwiapCl6ujuWh2I5ppxVSWQjXBN+A1Te+liwf/MXoT xe2cRh0KBq/0EnJ4D8yxDBaeV2fUpp+/qihBdouWBTjkik3EAYLA77YJD hVWaZHRwjPyY3MuS70v4bMf1ILLW/NxFJIZkveIpOU69KLMf6FWQ4Ctwu 2MSf/7Znb2FUNqZXNNPPZJlqCslCgNV8iUbHlg9e9ftu3ByNP1iFqnnyY dAZNPEuNz3exI21InIAmu4zevwiDiLaYP+9nys6ftv73H+KfLnoz3ir2E FHIGHFBDQQjIyOgH6Wva6f2tIPcdYonacBdeYNftFx0b6FIp26D73XeJr g==; X-CSE-ConnectionGUID: c2Al3aTTR1ieabgHJwhZ7Q== X-CSE-MsgGUID: WpdCQPcRSWeXfmLKvgeNNA== X-IronPort-AV: E=McAfee;i="6600,9927,11037"; a="8406275" X-IronPort-AV: E=Sophos;i="6.07,186,1708416000"; d="scan'208";a="8406275" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Apr 2024 01:10:57 -0700 X-CSE-ConnectionGUID: FOp2xJAGQwC7X72h0hl9QQ== X-CSE-MsgGUID: /HTKZDdwQASDRCm6S0DXMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,186,1708416000"; d="scan'208";a="19933147" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmviesa008.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 08 Apr 2024 01:10:57 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 8 Apr 2024 01:10:56 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Mon, 8 Apr 2024 01:10:56 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 8 Apr 2024 01:10:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BPVB9YEm4t5gdmrj+h+BMaZTaiTQT4ghHro/FBIqgZjRSlD1hqrKSkadhxplc/3EpuWJeyCqJXEcsxhuvsrPRl/onk1BkyoyNW8XhhmkVNnnf07bEbNfL72BpzewVs7QLpskzPeciyquTXyN99y8qzannrCJq18S+oeFDrwmD8Fs9XJRh2GeZVezEap1afiAEpd4M0M9LPD6MrrB+chDXCtVncOhlcMowqpuegaIXJF6UOYOBnSN3yvJLt/jA1KSZ+0/xge9jUa1sAzG8aBTf2JOQf5LM0+4jsl6OlJkQ4XJpctrSqeweeufqqKr57yVL5RbnqOnwgRHs159YFjwSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WYHgNRZRJhDClLDI5s+qy9tASri7WmdbnL0i0hf1Njo=; b=a37KUfHP0fMdZXTGw19Jftim+fbarQckxlVWnBY2FF1TMzzToicjYZB3vu5tc0gn8sORRAr0shn7uWpctSYiesJJd74yapidz6+SqreUhLV/4vT4Ir3Qv/q4cR3aaLPcZVCLwLaTD8KhCDfxc7yECqAZAPMy+rIllKgsvwsUY4d+WMD3lUTqrqWw4g3MdO4FseOffNC6gT9x7+6IMKMDUU5xDNa7XSuUQxuiD/0uLlmMCWgbG4EaAGjdnyCFckchwgwQvFYvCeNOPF5xEyxFFzZIGYJB690MUptZr55teiIcIxYY5hNEzKhs9owwXk6n8XPxaIsQly11k2WHS7mFNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA0PR11MB4637.namprd11.prod.outlook.com (2603:10b6:806:97::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7452.26; Mon, 8 Apr 2024 08:10:49 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::487e:e20c:ad88:9c0f]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::487e:e20c:ad88:9c0f%7]) with mapi id 15.20.7452.019; Mon, 8 Apr 2024 08:10:43 +0000 Date: Mon, 8 Apr 2024 09:10:38 +0100 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= CC: Patrick Robb , Nicholas Pratte , , , , , , , , Jeremy Spewock Subject: Re: [PATCH] dts: Change hugepage runtime config to 2MB Exclusively Message-ID: References: <20240404153106.19047-1-npratte@iol.unh.edu> <98CBD80474FA8B44BF855DF32C47DC35E9F36D@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35E9F376@smartserver.smartshare.dk> Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F376@smartserver.smartshare.dk> X-ClientProxiedBy: DU2PR04CA0265.eurprd04.prod.outlook.com (2603:10a6:10:28e::30) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA0PR11MB4637:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nSuaDuaa5NmP53krOzwHXmUaJE1E0l8GL/uwIg1d1s6aO9FsTipVBAe/6MXtY1AUuZJncLF8jf5qv0XzWtfJpvdeTD5JZP8koHImwM/25/bS6KTmmEUAZorjnsR4yf608DhxXFcBy2Nzj7ZL82+CMW8h19mOcuIutiT41VasI65M5+wUEpv6ujWAJULaJb882sU3ZY4kqAmlGLkfGA3SUfSFzDOowlBwsMbL9btSlU24HXn6EjTs3azK1Tozspe02mjYdMgZ8FjtIC/OY8fgR4ITbBoQaoVpnw/D/L7POK+/vVchk0+0lQO0UD1gXiFdXy/BZsZZdXzf6QuF9zmGIG2Px+Pb+nbaB+nGgwqNS1HLzYWma+V7z65IDY3q0eGFJMUX/P50XtSI7fhJ+m8dBTzxz39YWBQr8IXzTrsHIcsCr+a8EM9PeZIWqec/kod8mdu2dHYco8LE22PcQtLGsh6ngAhcJQRGQQ4c7hzShbAo4BmRzho1so3Kpx/zVLewZYa1/BDN8eWZZF8tPxHZaWT1jtG47M8V5B98sPC1GqC/6sZlW8+0dtSSDvLZO9eqRw3CQ/fwgnSQhMDeBByInwj8GQNRKDM68GPuQBmAA8Fay8pbC3cwp/UffAJA3FQJPbjS7gUw7CXbhv4Si7ugHUPx+xgvTQrMJDJitKjaTxo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(7416005)(366007)(1800799015)(376005); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZlFKZlJIVlNScThLUUs4OTY5ZlJTdm44Wmxqc1REZHo2ODZ3Q29jTGtMM3I4?= =?utf-8?B?ZVI1RnVxTXdmc2V0QXg4VkwrRFNmenBNcW8ra3Mxc3p6a3ZiQkZHUHRoUG8z?= =?utf-8?B?OGpHdS9nNmNFMmhTZVk0anBFaFhQcHhod3JHU2hTd2haWnZUYWJUNzJCb2R1?= =?utf-8?B?aW1EOEVQUnlmUExSUkpCcTNqMCtNdjhlbkMvNllxTFhYRC9DcHdHeTg1c1pT?= =?utf-8?B?ODZYOGNLWkYyK2FPRjBqY1l5WjZJK2d0bUlUNjZBSDlxYmZEZVU2cHhHQmpM?= =?utf-8?B?S1hpdGhBdWRLanFQRk1ZQkN4M2xZdlJxdmx5YlpMOURVaFRlOWlwdFVCdUFD?= =?utf-8?B?OHZNaUxiZUQ1c1hzV0FqdndvZG1HVHdRR29pbWFpUWhHRDA2aEozaHpOSmZL?= =?utf-8?B?YjZIUGtKWDdjM3k2eDVuWjdEcThmNEJDcjV5ODJoRUFsWGNrdHBqdnBqSFln?= =?utf-8?B?SGNTeElnWE5zNFFyelF2L2QvbUlyMzJzdU1CY29GdnZVMU9iL21NZ01zVitV?= =?utf-8?B?SmtDOHYwdW1ydWZNNWxUdUZ1eXZhVThHOUhMWW1EaTI4aWJCMklHWEo4T0Vq?= =?utf-8?B?MlY4RSswWGQ1TUlUTkRHcy95UUJJM2JoZmtuYkdNcFI2alhSbHBicUtQcHNZ?= =?utf-8?B?ZlBiMERoS3Q5TC9IaXR5aXozaDYzdzB0Z1BKWHlwNDVJb2t2QXFlN1dLMzRt?= =?utf-8?B?VVNPWWU0TzV6MWVpZHF0TkZsTGR5Qk5tUTRXR1NuQnU1SENhV0dQdm5XdjRj?= =?utf-8?B?Z0Juc2t4MlNNYmRyR2RkOUJJMUZlYVpZOUxUZWg2Nm5XMWhJNVZTMXBwa1ll?= =?utf-8?B?UDRyenNjaXdEVWFqUUdkUVdGSUptR0V6SmJOOWI5STZsSTZ6cVMzT2tEemdj?= =?utf-8?B?NWxRYkdCbHg1YXB3RDlFMndaSnZTZUo3QTc5OFhoaTdxSmZYUUhsNE9LVlNz?= =?utf-8?B?WjROWEJ3NmQ3SGxpZnhUVHhsVGx5U1hWcmhwOFRIZGhKamRKRWE3M2tML2F1?= =?utf-8?B?UTlLbXEycWd3RExMWklZWmpxYTl3UkdyK2d5U3RCcTk5TnFYZ2Y2WGNlNzd2?= =?utf-8?B?d28rZGl0V2JOV0ZJRnQ2ZmpLellvTmw2L1BLUVoxSmVVOUdIVnkzdFkveWJh?= =?utf-8?B?Q0hhTkp3RnZwcVNpT0F1eGhoZHE1bW1lVjNlUnpodDhMcjAwQmFlMlFyVnFq?= =?utf-8?B?ZS94UGR1YjZjR3ppYTBlNXFiMVJFdjdLRmVPMWdWS2RXOUlzb2hyTzhoNit1?= =?utf-8?B?L3BzaUZHa2JVSjkwQnlWYkgxdG1xYWlEWjhDbFV6UmdHSFBhSUJJTVgxWWZV?= =?utf-8?B?dVVNT3JNOHJHa1lWcm10UGpFbldRSUFhNjZyNGtlS3JnMEJVcWVxd01MYUU5?= =?utf-8?B?d3BXU1o4WlpSWm15L3RjUVlkQW9JNUFIME9aaFVmR0M1MFVqTnk3R3MvNTVx?= =?utf-8?B?RDdhSXhWN0NvSUZBeHVTRVZ5VUZjU3RXWXdWRG05ZllBR1dkc1ByeHJKTDVY?= =?utf-8?B?SjlPVmNnUWRHcFJVV3B4ZmhaaVBHL29XdmJRV0kxa0Iybml2b3JpVTZSRFFk?= =?utf-8?B?eTVRMklYeUExYkZDRVp4QUpjaWtUYy8xNytxRlk5aGdVRGJudGRuelh6bHlT?= =?utf-8?B?QmZKeUcyQkFQeW00dDdDWDAvRXkzcXlVUVBXVlRoeFVxRVVKYlY4WFJUUDln?= =?utf-8?B?TUQ2aWZiNVExdU1BNTlUcDBJa2xsa2hlUElTVHVmditxVmk0VkpuMXNSWEhQ?= =?utf-8?B?azl3MDZ5T2RtYkhjaFNOSU1sczJmTWM4d0dJNmZnSlA0Q0graUtPeXhiOVht?= =?utf-8?B?dHZyVkppektWK1ZKaVVqeGNGZzFrK2p2Ym9EcHUxQmhVTWdVWC9OVWV0TnVL?= =?utf-8?B?N0M1cytNaVpVTCswL2cvVERsUU5haVlvZjVvaVJVOHhKK2ZSbmxzTmxFaXdC?= =?utf-8?B?emlWcFBiZXZDSG9WbEc0b0w0ck9ZN0FFRGRuVU5oTmN4K3M4MFVTd3RlblJx?= =?utf-8?B?VzBodFhTYWtLR2tTV2wxL0x2dEVUbGlOWENhdGtTcjJKcERoT1lqMUJVZXBa?= =?utf-8?B?Z25kTTNYakRxV2hqOHF3c3ozT05pZ2hIbzdvL2dKRnJ5UklvVFZJMlYxd05w?= =?utf-8?B?WGE4TzNhUEZiSHM0d2lIUmMrRXo5WStxb0FWRCtxellGczRpSTEwSE1PSVo2?= =?utf-8?B?ekE9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 55a25c58-56c3-472e-70fe-08dc57a362b4 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2024 08:10:42.9003 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FFojAM5C97C6LSThuHNAWzVr1TZOALRzj7Peu8u4cmM65gFZKrPwKOTzPRd6g36g3SaCaSrWFFyzJZFba4ctYIeMgUx/m+jjZgMRnpByh4U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR11MB4637 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, Apr 07, 2024 at 12:04:24AM +0200, Morten Brørup wrote: > > From: Patrick Robb [mailto:probb@iol.unh.edu] > > Sent: Saturday, 6 April 2024 21.21 > > > > On Sat, Apr 6, 2024 at 4:47 AM Morten Brørup > > wrote: > > > > > > > > > This change seems very CPU specific. > > > > > > E.g. in x86 32-bit mode, the hugepage size is 4 MB, not 2 MB. > > > > > > I don't know the recommended hugepage size on other architectures. > > > > > > > Thanks Morten, that's an important insight which we weren't aware of > > when we initially discussed this ticket. > > > > We read on some dpdk docs that 1gb hugepages should be set at boot (I > > think the reason is because that's when you can guarantee there is gbs > > of contiguous available memory), and that after boot, only 2gb > > hugepages should be set. I assume that even for other arches which > > don't support the 2mb pages specifically, we still want to allocate > > hugepages using the smallest page size possible per arch (for the same > > reason). > > Correct; for very large hugepages, they need to be set at boot. > I don't remember why, but that's the way the Linux kernel works. > 2 MB is way below that threshold, and 1 GB is above. > > You can also not set nr_overcommit_hugepages for those very large hugepages, only nr_hugepages. > > > > > So I think we can add some dict which stores the smallest valid > > hugepage size per arch. Then during config init, use the config's arch > > value to determine that size, and set the total hugepages allocation > > based on that size and the hugepages count set in the conf.yaml. Or > > maybe we can store the list of all valid hugepgage sizes per arch > > (which are also valid to be set post boot), allow for a new > > hugepage_size value on the conf.yaml, validate the input at config > > init, and then set according to those values. I prefer the former > > option though as I don't think the added flexibility offered by the > > latter seems important. > > I agree; the former option suffices. > > A tiny detail... > ARM supports multiple (4, I think) different hugepage sizes, where the smallest size is 64 KB. > So you might want to choose another hugepage size than the smallest; but I still agree with your proposed concept of using one specific hugepage size per arch. > > Like x86_64, ARM also supports 2 MB and 1 GB hugepage sizes, and 2 MB hugepages is also the typical default on Linux. > > I don't know which hugepage sizes are supported by other architectures. > It might only be 32-bit x86 that needs a different hugepage size than 2 MB. > Even for 32-bit x86, I think most distros now use PAE mode to allow physical addresses >4GB, so even then 32-bit hugepages are 2MB rather than 4MB. /Bruce