[PATCH] config: limit lcore variable maximum size to 4k

[PATCH] config: limit lcore variable maximum size to 4k

[David Marchand]

OVS locks all pages to avoid page faults while processing packets.
1M for each lcore translates to allocating 128M with default build
options on x86.
This resulted in OOM while running unit tests in parallel.

At the moment, the more demanding DPDK user of lcore variable is
rte_service, with a 2112 bytes object.

Limit the lcore variable maximum size to 4k which looks more reasonable.

Fixes: 5bce9bed67ad ("eal: add static per-lcore memory allocation facility")

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
 config/rte_config.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config/rte_config.h b/config/rte_config.h
index 498d509244..5f0627679f 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -41,7 +41,7 @@
 /* EAL defines */
 #define RTE_CACHE_GUARD_LINES 1
 #define RTE_MAX_HEAPS 32
-#define RTE_MAX_LCORE_VAR 1048576
+#define RTE_MAX_LCORE_VAR 4096
 #define RTE_MAX_MEMSEG_LISTS 128
 #define RTE_MAX_MEMSEG_PER_LIST 8192
 #define RTE_MAX_MEM_MB_PER_LIST 32768
-- 
2.47.0

RE: [PATCH] config: limit lcore variable maximum size to 4k

[Morten Brørup]

> From: David Marchand [mailto:david.marchand@redhat.com]
> Sent: Friday, 8 November 2024 19.18
> 
> OVS locks all pages to avoid page faults while processing packets.
> 1M for each lcore translates to allocating 128M with default build
> options on x86.
> This resulted in OOM while running unit tests in parallel.
> 
> At the moment, the more demanding DPDK user of lcore variable is
> rte_service, with a 2112 bytes object.
> 
> Limit the lcore variable maximum size to 4k which looks more
> reasonable.

4 KB is not future proof.

Here's an example where 16 KB is cutting it close:
https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9F7D0@smartserver.smartshare.dk/

Depends on how we are going to use it. 4 KB suffices if we only want to use it for "small" structures.

Would 64 KB work as a compromise?

RE: [PATCH] config: limit lcore variable maximum size to 4k

[Morten Brørup]

> From: Morten Brørup [mailto:mb@smartsharesystems.com]
> Sent: Friday, 8 November 2024 19.35
> 
> > From: David Marchand [mailto:david.marchand@redhat.com]
> > Sent: Friday, 8 November 2024 19.18
> >
> > OVS locks all pages to avoid page faults while processing packets.

It sounds smart, so I just took a look at how it does this. I'm not sure, but it seems like it only locks pages that are actually mapped (current and future).

> > 1M for each lcore translates to allocating 128M with default build
> > options on x86.
> > This resulted in OOM while running unit tests in parallel.

Is the root cause the lcore variables library itself, or the unit test using a lot of memory for testing the lcore variables?
We don't want to fix the library if the problem is elsewhere.

> >
> > At the moment, the more demanding DPDK user of lcore variable is
> > rte_service, with a 2112 bytes object.
> >
> > Limit the lcore variable maximum size to 4k which looks more
> > reasonable.
> 
> 4 KB is not future proof.
> 
> Here's an example where 16 KB is cutting it close:
> https://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9F7D0@smart
> server.smartshare.dk/
> 
> Depends on how we are going to use it. 4 KB suffices if we only want to
> use it for "small" structures.
> 
> Would 64 KB work as a compromise?