From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09C5E463BF for ; Tue, 11 Mar 2025 14:00:19 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B28804027F; Tue, 11 Mar 2025 14:00:18 +0100 (CET) Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by mails.dpdk.org (Postfix) with ESMTP id AE0E140263 for ; Tue, 11 Mar 2025 14:00:17 +0100 (CET) Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2ff784dc055so7174841a91.1 for ; Tue, 11 Mar 2025 06:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741698017; x=1742302817; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=6t52YDdmW9ztsB0pAUDOEI2xSDCr5AXW8sBKgUIrKM0=; b=MBBN9t9pBhI7BLMGw07PhhCeXO17bwMSzg6QDwkWF33XyvRlvYiTlCh5SILfKhXF7S v9X/+p8GjaMS+wQJnHpmemokAZOtsbl73N6U5JCFQ6sSgEUgOrQNIBB/1s9OUH676ctR NdizTPXOsGDTWmF+Q8jMyciMb7SRnhaUFrCkGDy7uHrIMaClg5/Fd52GWLqQqvpifrDL 3yDuEG8ABjR+t1L+P87kjnYmL8V2m/EB6WNVytCy5YdOezQT+5qup18B+tGHMGWaNL7M iKnVlqHcCEi0MEgtSWe43NrM/MP7azEJs+tKWdPl/yKBVxBKpO9q7fKoz9/6FC1e4E+U 5eRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741698017; x=1742302817; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6t52YDdmW9ztsB0pAUDOEI2xSDCr5AXW8sBKgUIrKM0=; b=lKPSnLWE6T5+YEiJU/X362rMcu37CamcxDdygKKvwkSEjg6wJ7rrsaE2mWXClv/SQP x8HPV54t791tg4EmsCrcSlvGWbk5AtTGd4KmEHzVCfxZHi21gG/wuWSORjGAGACNLKij RwZudhFu9B4cINqW3TcQEZVngvyubhRqfByATai+fGYadfrJnO3mrL4rkpu7zezLuUJD aUKiXWbep2iYMaql9NkW2iueNJ/us5cUX2GjIOvUvmDKvtyqbhxSRryQ6Y3/nDUZc6Pp njp+HGUE7fCIoPWlm6x8AikpAQN49pdAZx8JMNSGHfojl74xJXBReIJKvsO8Hx8L1WU8 ibjg== X-Gm-Message-State: AOJu0YzrTniVmf9MzDLaNYoZUZ3437CvLspGhaCE2cM3cpN1l/Z0Qbkx n78rEK+R3wKZHG52IhHnROKYjQOI76ni+KNj21yHlhjWt9QL9drmKSKUkb6PP4jSV2p8ibKz4yx 6ZtS6q2JjssSjyURBLz64V+XdsxLy2Q29 X-Gm-Gg: ASbGncs98kmmOylu4zTFB5Igda+8DBqNcrPRNxJndCbUguhmnWiF0mKYbwsqEAxAH9Y o1VIKOjYZnFs36+OjWJn32LEgbvMa7XE1nJj4EzEZqkA7RBT2dTDVHT1MirR1+m0Tsld1teSbG/ z/pAhZN6AwVFgPYH6Nh1+im0281qo= X-Google-Smtp-Source: AGHT+IERIfNaULfN+WmO+ziqw8WH7kjedvugbBIsXNLaGz5WB7YH9jrr3x94Tet0hRCg0/y7DVN0tUSXgfwCtPZSg+U= X-Received: by 2002:a17:90b:1647:b0:2ff:58e1:2bc9 with SMTP id 98e67ed59e1d1-300ff351f72mr4689950a91.25.1741698016503; Tue, 11 Mar 2025 06:00:16 -0700 (PDT) MIME-Version: 1.0 References: <20250310142140.4841e75b@hermes.local> <60c59649-a393-4d45-81e2-faad68b8a506@mahan.org> In-Reply-To: <60c59649-a393-4d45-81e2-faad68b8a506@mahan.org> From: Igor Gutorov Date: Tue, 11 Mar 2025 15:59:40 +0300 X-Gm-Features: AQ5f1JrQL6EUm_KD65IjmYdVr0e2jxkmc_OkADR_mG2zX61cHwDHp_g1R_v5-BU Message-ID: Subject: Re: Calculating number of HugeTLBs required To: Patrick Mahan Cc: users@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Tue, Mar 11, 2025 at 9:01=E2=80=AFAM Patrick Mahan wro= te: > > On 3/10/25 2:21 PM, Stephen Hemminger wrote: > > On Tue, 4 Mar 2025 11:22:50 -0800 > > Patrick Mahan wrote: > > > >> Morning, > >> > >> This might be simple question, but it has my curiosity itching. > >> > >> I did a quick scan through the documentation, but I did not see any go= od > >> guidelines for determining the number of HugeTLB based on the number o= f PMDs and > >> number of RX/TX queues. > >> > >> I'm am looking at three different platforms, one has 2 ports (ixgbe), = one has 3 > >> ports (1 e1000 and 2 i40es) and a third has 2 ports (1 e1000 and 1 Cav= ium liquidIO). > >> > >> I'm trying to come up with some means of defining the HugeTLB requirem= ents other > >> than trial and error. > >> > >> Thanks, > >> > >> Patrick > > > > There is on exact way to estimate this. But for most applications the l= argest memory footprint > > is the mbuf pool. For sizing the mbuf pool you need to account for all = the NIC's, queues, and descriptor arrays > > as well as any internal staging buffers. > > That's what I thought, but I was hoping for some group wisdom here. I am= trying > to construct the various startup (two are systemd and one is still sysvin= it) to > try and calculate this based on, as you pointed out, the # of NICS, the #= of > queues, etc. The code was already using HugeTLBs for other stuff (RiB/Fi= B, > database) that I had written code to calculate that information based on = the # of > entries in the database, the maximum # of routes, etc at boot time to res= erve. > The move to DPDK adds more complexity to this as we are looking at levera= ging > more CPU cores, which may mean more queues, which means more packets, etc= . > > Right now I've done some, back of the envelope, calculations, but that is= not a > way to dynamically approach this. > > Anyways, thanks for responding... > > Patick Hello Patrick, One very crude way I use is to allocate "more than enough" pages. Say, 32 1G pages. Then, check `/sys/devices/system/node/node0/hugepages/hugepages-1048576kB/free_hugepage= s`, it should say 32 (note `node0` in the path, adjust accordingly). Start your application and check `free_hugepages` again. The difference is how many hugepages your application uses. I then adjust the total number of allocated pages to remove unused ones. As you've said, this is somewhat a trial and error approach, but practically requires only 1 application startup/shutdown, and could probably be automated. -- Regards, Igor