From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A65F14626D for ; Wed, 19 Feb 2025 23:28:34 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 633914014F; Wed, 19 Feb 2025 23:28:33 +0100 (CET) Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by mails.dpdk.org (Postfix) with ESMTP id 9973740041 for ; Wed, 19 Feb 2025 23:28:31 +0100 (CET) Received: by mail-lf1-f45.google.com with SMTP id 2adb3069b0e04-5439a6179a7so374058e87.1 for ; Wed, 19 Feb 2025 14:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740004111; x=1740608911; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=tKqZf7e8OgjpL6BF2gg0z75WpCX0NTaFV/u1oUYKIA0=; b=a21GY8bqFOINm0H1UhmzC4VO3HIL+Q4eHqMJNDa/WChvr9EyQJOhwdtsay9RgZ4j1R SaZnKsqvCoquGQjheUMmfGZHaqxzXcMx+qTwPbiAuGmoxg4J276SPQEgRE4RMcM7RRKv IfdEDVU3pOsf98r3hq2DR/QjDgcSQrunpVB0SMeoOUmqpCcfumzPz/w4Jq51n0EGd0WW l2yARP08y5wU34qhNs2JVDTrGyHX5Yjq2WzWsIgkc7WDxdWBBmgDlgMi48GQ12XH99tY kF4ZbAo4+Gtaglxz1q4vAQzGreSeJrV/kfOHhHJGYG6UaT7Ogl+8oseYZmIdpARPb1tx MukA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740004111; x=1740608911; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tKqZf7e8OgjpL6BF2gg0z75WpCX0NTaFV/u1oUYKIA0=; b=uM6qIegeB62m4XCQIjrpKBvtr+SQxtLK5Z0sRRR9TZjMIA2kqi+J4Ydtg/SSumyUxJ ibQI/iirHXHib9Id7SQzNQOci6MekVRL243NZBEMr18n7ybjcJpHjSvG0IIZENW9mcpI 5fJ4bjnAVjar6hGUvcIWc2j32ZZVNpUfszus1Xw/atkhrb/7dXc7WoZhWyE7y5Pw9pPl 4LFCls348dVrnOQNHVkQHWRVUfp10s0KIT9gl51dvRX+UPR4FfKGFn20N+8NOAYMTVMf rldMdRGrYjSnzPfbJKoiLPVpFwrjoo77ri9EnahzNCJ4pbTJYDE62NcIsyJsyzbr2HO9 /aCA== X-Gm-Message-State: AOJu0YxgjvaB2j+OSBEh4S2MH6vEOg+/WvfIoyE8eBj+3OajHBftwKiE UB0Lq3Tb6y+fENDraD8Mt0JKg4sw2xJ4LPdNdwn8vxP/1i2T4NYp X-Gm-Gg: ASbGncsoDVzHx01sljMqpSrtdGtcdpsiUD7gS2WFIaKsgSBtQgjb5Zh6WICc6wO6vAP JaXv77Akq5EF8KjofXpM+X9wXqKE5B8GaqtkCL0oX/SBiwG/sih/r/dQ8ueqUMuGWelIsNChZyA wOeVmxzTXhkOWLRbYeVdZjumak9HIS2TtU/4rJMsilu96W61zQA12+KZHCnbaST4IJwZla+8pwe 5vtcvPvxMpCriCpXqzD8l1OuQOxgJWfmkJ35jfZXHCzcdikj8+mkg/W/vDv9WPo2A8CgoSJcXO2 FxNmT2C6eh2y51BRZQQdz3dof0ZjF8giPWvn24ZsAEUCZUFb+0bSQMqDxf+55yE= X-Google-Smtp-Source: AGHT+IEy5rGc3n33dp8aUBo99h+L0VI2P23s9vmtdZ9Ukcq6zncU7KL7KLucO45V2hTgfeY0fd9fbA== X-Received: by 2002:a05:6512:2399:b0:545:25cc:23cd with SMTP id 2adb3069b0e04-546e4f0aa88mr289461e87.16.1740004110382; Wed, 19 Feb 2025 14:28:30 -0800 (PST) Received: from sovereign (broadband-109-173-43-194.ip.moscow.rt.ru. [109.173.43.194]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-545254f7072sm2031834e87.127.2025.02.19.14.28.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Feb 2025 14:28:29 -0800 (PST) Date: Thu, 20 Feb 2025 01:28:28 +0300 From: Dmitry Kozlyuk To: Lucas Cc: users@dpdk.org Subject: Re: Increase DPDK's virtual memory to more than 512 GiB Message-ID: <20250220012828.21ae8b76@sovereign> In-Reply-To: References: X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org 2025-02-19 15:22 (UTC+0100), Lucas: > I am creating an application where I want to cache packets in a ring for a > certain duration. To do so at high speed (100 Gb/s) for tens of seconds, I > would like to create a mempool with a size bigger than 500 GiB. > I know that in rte_config.h, we have this excerpt: > ``` > #define RTE_MAX_MEMSEG_LISTS 128 > ... > #define RTE_MAX_MEM_MB_PER_LIST 32768 > ``` > 4'194'304 MiB of addressable memory, which is more than enough. However, > there is a virtual memory limit on DPDK processes on x86_64, which appears > to be 512 GiB. > at https://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html it is > listed that RTE_MAX_MEM_MB controls that. > In the meson.build file in the config directory, we have: > ``` > if dpdk_conf.get('RTE_ARCH_64') > dpdk_conf.set('RTE_MAX_MEM_MB', 524288) > else # for 32-bit we need smaller reserved memory areas > dpdk_conf.set('RTE_MAX_MEM_MB', 2048) > endif > ``` > Changing this value does not seem to change the amount of virtual memory > that DPDK allocates. It appears that no headers or C-files actually > reference this value. https://elixir.bootlin.com/dpdk/v24.11.1/source/lib/eal/common/eal_common_dynmem.c#L113 > What would I need to change to allow more virtual memory than 512 GiB and > be able to allocate a mempool with a size bigger than that? Increase RTE_MAX_MEM_MB_PER_LIST and RTE_MAX_MEM_MB_PER_TYPE to 512 GB. See the big comment in the linked function code. Note that RTE_MAX_MEM_MB must accommodate for all memory types, e.g. for a 2-processor x86_64 system with 2M and 1G hugepages supported it must be 2 NUMA nodes x 2 huggepage sizes x 512 GB = 2048 GB.