From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF44543F80; Sat, 4 May 2024 08:51:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8884E402D5; Sat, 4 May 2024 08:51:25 +0200 (CEST) Received: from dkmailrelay1.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id CF23E4014F for ; Sat, 4 May 2024 08:51:23 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesys.local [192.168.4.10]) by dkmailrelay1.smartsharesystems.com (Postfix) with ESMTP id 9DA1E209B2; Sat, 4 May 2024 08:51:23 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: RE: Need help with reducing VIRT memory Date: Sat, 4 May 2024 08:51:21 +0200 X-MimeOLE: Produced By Microsoft Exchange V6.5 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35E9F41B@smartserver.smartshare.dk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Need help with reducing VIRT memory Thread-Index: Adqcu3BiXzYpqqoETgaEa9S0ksiqXwAEGosAAAF8anAAJajQIAAM690AAAAQZAAAAJ6IAAATRKZA References: <20240502230352.7bb1ef0d@sovereign> <20240503235442.70b4b056@sovereign> <20240503235632.27ea47f4@sovereign> From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: "Lombardo, Ed" , "Dmitry Kozlyuk" Cc: X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > From: Lombardo, Ed [mailto:Ed.Lombardo@netscout.com] > Sent: Friday, 3 May 2024 23.19 >=20 > Hi Dmitry, > Thank you, I will give this a try. Many arrays in DPDK are statically allocated at the build-time max array = size, so tweaking DPDK's build-time configuration parameters will reduce = the memory consumption. E.g. we tweak these (using DPDK 17.11): CONFIG_RTE_MAX_NUMA_NODES CONFIG_RTE_MAX_LCORE CONFIG_RTE_MAX_ETHPORTS CONFIG_RTE_MAX_QUEUES_PER_PORT >=20 > Another angle is how can I calculate or determine when DPDK is > initialized how much total memory DPDK will use. Then I can provide > this to our application core and then they can take the DPDK memory = max > requirement and subtract it from the total memory and this amount of > memory can be locked down by the app core. The application does not > want the locked down memory to be used by anything else. (just an = idea > I thought of). I would love having this ability too. However, it depends on too many = parameters, both build time and run-time, to be truly predictable. You = have to run your application's DPDK initialization and configuration = stages, and then measure it. That will help provide an estimate. You can check the memory consumption after calling rte_eal_init() and at = different stages of application initialization, like this: struct rte_malloc_socket_stats stats, stats_sum; unsigned int socket; memset(&stats_sum, 0, sizeof(stats_sum)); /* Iterate through all initialized heaps */ for (socket=3D0; socket< RTE_MAX_NUMA_NODES; socket++) { if (rte_malloc_get_socket_stats(socket, &stats) < 0) continue; stats_sum.heap_totalsz_bytes +=3D stats.heap_totalsz_bytes; stats_sum.heap_freesz_bytes +=3D stats.heap_freesz_bytes; stats_sum.heap_allocsz_bytes +=3D stats.heap_allocsz_bytes; } printf("DPDK hugemem used after rte_eal_init(): %.01f MB\n", (float)stats_sum.heap_allocsz_bytes / (float)(1024 * 1024)); >=20 > Thanks, > Ed >=20 > -----Original Message----- > From: Dmitry Kozlyuk > Sent: Friday, May 3, 2024 4:57 PM >=20 > 2024-05-03 23:54 (UTC+0300), Dmitry Kozlyuk: > > #define RTE_MAX_MEM_MB_PER_LIST 2024 // see item 4 below >=20 > Typo: 2048