From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1409943E2C; Tue, 9 Apr 2024 18:57:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9E60B402C7; Tue, 9 Apr 2024 18:57:37 +0200 (CEST) Received: from mail-ot1-f47.google.com (mail-ot1-f47.google.com [209.85.210.47]) by mails.dpdk.org (Postfix) with ESMTP id 2F23B4025D for ; Tue, 9 Apr 2024 18:57:36 +0200 (CEST) Received: by mail-ot1-f47.google.com with SMTP id 46e09a7af769-6ea0b7a563aso562667a34.1 for ; Tue, 09 Apr 2024 09:57:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1712681855; x=1713286655; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=N/iT0DNsk1dy44E2sQ/f+su3aXML4lP2P3rZV2i6654=; b=C0G8YORCJk58d6/wqRCgR5U8PjqULQv4UgZ8RR2/cpfkt5sBq7Jxqw06+1Tyx/fGA8 cYGB0dTMYZp9TriftWSi4/8x6AKWqR/GK1wjtAnQ2xd4h+bu9Eb8WIzxQn1yY49TjNhI Dkqnczw9QxAGWsh8Nz2cvYJ5aTmgntJjpDq5Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712681855; x=1713286655; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N/iT0DNsk1dy44E2sQ/f+su3aXML4lP2P3rZV2i6654=; b=B9akYHB917bB/7gP5LSWY/lvjg+nRSDM/863vUVdiD7sZVruA7WlFCDQSMBeaHsz6H VPiiRn4agn28vikmEUZRrOBLEykMxbNYd954Wdc5IeE2id9XVzQM9aBolzvYcTt7Ghr2 iYAnku5UHc6cjtAzKXBUi8/gY26o5BOI+/cowB7NTG7Y8BXyitcja876OR8eRIWUV2E4 V4DUbemicUsEf64Doy2gryjIi+AWSzoQoRFeEX+q0omNMp6VijVOuvTE3ioDKIqRwjgA 7u+Jnn6o1FtPcjfWQ4KJEyLgfuK6TnQdSNh6AQ18h4N0d/NduL5Rp1ratUdydPhWkNA9 C9gQ== X-Forwarded-Encrypted: i=1; AJvYcCWudA2OmeQxBQdxMeTw7bax6thzRbpfesAFnySy6ISHP18cxNRE+8VIealHNxQ4xeKRkHTEW1QsBiSUNB0= X-Gm-Message-State: AOJu0YxB6AUJRWBzitTNcT7AzABAdvlvDKZs7BThWKRruydU8hZrCgCP 7OrwoERj27PK48Yhu57W8L+9p1w+LLXyZRM2ktgTf7fOs4odhsMkfX8/qYVqR8lvVBdLPdYS8JM SCEW/65Tu/YU1eXwBVW7D20UPxothwkOfBc5nPQ== X-Google-Smtp-Source: AGHT+IG7tPUXmHZc8U77ZpHAQDX/uKJiWHIS+qQQeZzBQS47SjjYghn7Z2Hi1ZQqEH3q7jFUkrkdV/s88dxr8zdJZmo= X-Received: by 2002:a05:6870:b41d:b0:22e:77b6:4f9d with SMTP id x29-20020a056870b41d00b0022e77b64f9dmr118188oap.3.1712681855287; Tue, 09 Apr 2024 09:57:35 -0700 (PDT) MIME-Version: 1.0 References: <20240404153106.19047-1-npratte@iol.unh.edu> <98CBD80474FA8B44BF855DF32C47DC35E9F36D@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35E9F376@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35E9F37D@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F37D@smartserver.smartshare.dk> From: Nicholas Pratte Date: Tue, 9 Apr 2024 12:57:23 -0400 Message-ID: Subject: Re: [PATCH] dts: Change hugepage runtime config to 2MB Exclusively To: =?UTF-8?Q?Morten_Br=C3=B8rup?= Cc: Bruce Richardson , Patrick Robb , paul.szczepanek@arm.com, juraj.linkes@pantheon.tech, yoan.picchi@foss.arm.com, thomas@monjalon.net, wathsala.vithanage@arm.com, Honnappa.Nagarahalli@arm.com, dev@dpdk.org, Jeremy Spewock Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org I will change the default hugepage total back to 256. Much of the justification for changing the default amount was based on guidelines in the DPDK documentation. In the system requirements section, an example is provided demonstrating how to allocate 2GB of 2MB hugepages. We used this reasoning when setting the default amount as 2560 creates 5GB, or 2GB for each NUMA node, and some additional wiggle room. Much of this was also based on the Juraj's initial suggestion of 256 hugepages largely being arbitrary, but he had also referenced the aforementioned DPDK documentation during discussion of how many hugepages should be configured by default if nothing is set. Regardless, our suggestion of changing the default hugepage size was more of a light suggestion than anything else, so I will just remove this. I will also make the changes you suggested for linux_session.py. The plan I have is to add the parameter back in and hardcode the 2048 hugepage size to continue our 2MB logic, with the understanding that this can be changed in the future. On Mon, Apr 8, 2024 at 5:35=E2=80=AFAM Morten Br=C3=B8rup wrote: > > > From: Bruce Richardson [mailto:bruce.richardson@intel.com] > > Sent: Monday, 8 April 2024 10.11 > > > > On Sun, Apr 07, 2024 at 12:04:24AM +0200, Morten Br=C3=B8rup wrote: > > > > From: Patrick Robb [mailto:probb@iol.unh.edu] > > > > Sent: Saturday, 6 April 2024 21.21 > > > > > > > > On Sat, Apr 6, 2024 at 4:47=E2=80=AFAM Morten Br=C3=B8rup > > > > wrote: > > > > > > > > > > > > > > > This change seems very CPU specific. > > > > > > > > > > E.g. in x86 32-bit mode, the hugepage size is 4 MB, not 2 MB. > > > > > > > > > > I don't know the recommended hugepage size on other architectures= . > > > > > > > > > > > > > Thanks Morten, that's an important insight which we weren't aware o= f > > > > when we initially discussed this ticket. > > > > > > > > We read on some dpdk docs that 1gb hugepages should be set at boot = (I > > > > think the reason is because that's when you can guarantee there is = gbs > > > > of contiguous available memory), and that after boot, only 2gb > > > > hugepages should be set. I assume that even for other arches which > > > > don't support the 2mb pages specifically, we still want to allocate > > > > hugepages using the smallest page size possible per arch (for the s= ame > > > > reason). > > > > > > Correct; for very large hugepages, they need to be set at boot. > > > I don't remember why, but that's the way the Linux kernel works. > > > 2 MB is way below that threshold, and 1 GB is above. > > > > > > You can also not set nr_overcommit_hugepages for those very large hug= epages, > > only nr_hugepages. > > > > > > > > > > > So I think we can add some dict which stores the smallest valid > > > > hugepage size per arch. Then during config init, use the config's a= rch > > > > value to determine that size, and set the total hugepages allocatio= n > > > > based on that size and the hugepages count set in the conf.yaml. Or > > > > maybe we can store the list of all valid hugepgage sizes per arch > > > > (which are also valid to be set post boot), allow for a new > > > > hugepage_size value on the conf.yaml, validate the input at config > > > > init, and then set according to those values. I prefer the former > > > > option though as I don't think the added flexibility offered by the > > > > latter seems important. > > > > > > I agree; the former option suffices. > > > > > > A tiny detail... > > > ARM supports multiple (4, I think) different hugepage sizes, where th= e > > smallest size is 64 KB. > > > So you might want to choose another hugepage size than the smallest; = but I > > still agree with your proposed concept of using one specific hugepage s= ize per > > arch. > > > > > > Like x86_64, ARM also supports 2 MB and 1 GB hugepage sizes, and 2 MB > > hugepages is also the typical default on Linux. > > > > > > I don't know which hugepage sizes are supported by other architecture= s. > > > It might only be 32-bit x86 that needs a different hugepage size than= 2 MB. > > > > > > > Even for 32-bit x86, I think most distros now use PAE mode to allow phy= sical > > addresses >4GB, so even then 32-bit hugepages are 2MB rather than 4MB. > > In order to save you from more work on this patch, we could assume/hope t= hat all architectures support 2 MB hugepages, and drop the discussed per-ar= chitecture size dict. > At least for until the assumption doesn't hold true anymore. > > Then I only have two further comments to the patch: > > 1. Don't increase from 256 to 2560 hugepages (in dts/conf.yaml), unless t= here is a reason for it. If there is, please mention the increase and reaso= n in the patch description. > 2. Don't remove the size parameter from _configure_huge_pages() (in /dts/= framework/testbed_model/linux_session.py); it might be useful later. >