From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id A65F14626D
for <public@inbox.dpdk.org>; Wed, 19 Feb 2025 23:28:34 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 633914014F;
Wed, 19 Feb 2025 23:28:33 +0100 (CET)
Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com
[209.85.167.45]) by mails.dpdk.org (Postfix) with ESMTP id 9973740041
for <users@dpdk.org>; Wed, 19 Feb 2025 23:28:31 +0100 (CET)
Received: by mail-lf1-f45.google.com with SMTP id
2adb3069b0e04-5439a6179a7so374058e87.1
for <users@dpdk.org>; Wed, 19 Feb 2025 14:28:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1740004111; x=1740608911; darn=dpdk.org;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:subject:cc:to:from:date:from:to:cc:subject:date
:message-id:reply-to;
bh=tKqZf7e8OgjpL6BF2gg0z75WpCX0NTaFV/u1oUYKIA0=;
b=a21GY8bqFOINm0H1UhmzC4VO3HIL+Q4eHqMJNDa/WChvr9EyQJOhwdtsay9RgZ4j1R
SaZnKsqvCoquGQjheUMmfGZHaqxzXcMx+qTwPbiAuGmoxg4J276SPQEgRE4RMcM7RRKv
IfdEDVU3pOsf98r3hq2DR/QjDgcSQrunpVB0SMeoOUmqpCcfumzPz/w4Jq51n0EGd0WW
l2yARP08y5wU34qhNs2JVDTrGyHX5Yjq2WzWsIgkc7WDxdWBBmgDlgMi48GQ12XH99tY
kF4ZbAo4+Gtaglxz1q4vAQzGreSeJrV/kfOHhHJGYG6UaT7Ogl+8oseYZmIdpARPb1tx
MukA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1740004111; x=1740608911;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=tKqZf7e8OgjpL6BF2gg0z75WpCX0NTaFV/u1oUYKIA0=;
b=uM6qIegeB62m4XCQIjrpKBvtr+SQxtLK5Z0sRRR9TZjMIA2kqi+J4Ydtg/SSumyUxJ
ibQI/iirHXHib9Id7SQzNQOci6MekVRL243NZBEMr18n7ybjcJpHjSvG0IIZENW9mcpI
5fJ4bjnAVjar6hGUvcIWc2j32ZZVNpUfszus1Xw/atkhrb/7dXc7WoZhWyE7y5Pw9pPl
4LFCls348dVrnOQNHVkQHWRVUfp10s0KIT9gl51dvRX+UPR4FfKGFn20N+8NOAYMTVMf
rldMdRGrYjSnzPfbJKoiLPVpFwrjoo77ri9EnahzNCJ4pbTJYDE62NcIsyJsyzbr2HO9
/aCA==
X-Gm-Message-State: AOJu0YxgjvaB2j+OSBEh4S2MH6vEOg+/WvfIoyE8eBj+3OajHBftwKiE
UB0Lq3Tb6y+fENDraD8Mt0JKg4sw2xJ4LPdNdwn8vxP/1i2T4NYp
X-Gm-Gg: ASbGncsoDVzHx01sljMqpSrtdGtcdpsiUD7gS2WFIaKsgSBtQgjb5Zh6WICc6wO6vAP
JaXv77Akq5EF8KjofXpM+X9wXqKE5B8GaqtkCL0oX/SBiwG/sih/r/dQ8ueqUMuGWelIsNChZyA
wOeVmxzTXhkOWLRbYeVdZjumak9HIS2TtU/4rJMsilu96W61zQA12+KZHCnbaST4IJwZla+8pwe
5vtcvPvxMpCriCpXqzD8l1OuQOxgJWfmkJ35jfZXHCzcdikj8+mkg/W/vDv9WPo2A8CgoSJcXO2
FxNmT2C6eh2y51BRZQQdz3dof0ZjF8giPWvn24ZsAEUCZUFb+0bSQMqDxf+55yE=
X-Google-Smtp-Source: AGHT+IEy5rGc3n33dp8aUBo99h+L0VI2P23s9vmtdZ9Ukcq6zncU7KL7KLucO45V2hTgfeY0fd9fbA==
X-Received: by 2002:a05:6512:2399:b0:545:25cc:23cd with SMTP id
2adb3069b0e04-546e4f0aa88mr289461e87.16.1740004110382;
Wed, 19 Feb 2025 14:28:30 -0800 (PST)
Received: from sovereign (broadband-109-173-43-194.ip.moscow.rt.ru.
[109.173.43.194]) by smtp.gmail.com with ESMTPSA id
2adb3069b0e04-545254f7072sm2031834e87.127.2025.02.19.14.28.29
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Wed, 19 Feb 2025 14:28:29 -0800 (PST)
Date: Thu, 20 Feb 2025 01:28:28 +0300
From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
To: Lucas <lucascrijns@gmail.com>
Cc: users@dpdk.org
Subject: Re: Increase DPDK's virtual memory to more than 512 GiB
Message-ID: <20250220012828.21ae8b76@sovereign>
In-Reply-To: <CADTTPn8Cgu3ShDWWGdd9whQQAmiFH9x9MaweZ4DR3XHdcww6Eg@mail.gmail.com>
References: <CADTTPn8Cgu3ShDWWGdd9whQQAmiFH9x9MaweZ4DR3XHdcww6Eg@mail.gmail.com>
X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
<mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
<mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org
2025-02-19 15:22 (UTC+0100), Lucas:
> I am creating an application where I want to cache packets in a ring for a
> certain duration. To do so at high speed (100 Gb/s) for tens of seconds, I
> would like to create a mempool with a size bigger than 500 GiB.
> I know that in rte_config.h, we have this excerpt:
> ```
> #define RTE_MAX_MEMSEG_LISTS 128
> ...
> #define RTE_MAX_MEM_MB_PER_LIST 32768
> ```
> 4'194'304 MiB of addressable memory, which is more than enough. However,
> there is a virtual memory limit on DPDK processes on x86_64, which appears
> to be 512 GiB.
> at https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html it is
> listed that RTE_MAX_MEM_MB controls that.
> In the meson.build file in the config directory, we have:
> ```
> if dpdk_conf.get('RTE_ARCH_64')
> dpdk_conf.set('RTE_MAX_MEM_MB', 524288)
> else # for 32-bit we need smaller reserved memory areas
> dpdk_conf.set('RTE_MAX_MEM_MB', 2048)
> endif
> ```
> Changing this value does not seem to change the amount of virtual memory
> that DPDK allocates. It appears that no headers or C-files actually
> reference this value.
https://elixir.bootlin.com/dpdk/v24.11.1/source/lib/eal/common/eal_common_dynmem.c#L113
> What would I need to change to allow more virtual memory than 512 GiB and
> be able to allocate a mempool with a size bigger than that?
Increase RTE_MAX_MEM_MB_PER_LIST and RTE_MAX_MEM_MB_PER_TYPE to 512 GB.
See the big comment in the linked function code.
Note that RTE_MAX_MEM_MB must accommodate for all memory types,
e.g. for a 2-processor x86_64 system with 2M and 1G hugepages supported
it must be 2 NUMA nodes x 2 huggepage sizes x 512 GB = 2048 GB.