From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 08FB2A0524; Thu, 4 Feb 2021 12:39:21 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E744A2407A2; Thu, 4 Feb 2021 12:39:20 +0100 (CET) Received: from mail-il1-f171.google.com (mail-il1-f171.google.com [209.85.166.171]) by mails.dpdk.org (Postfix) with ESMTP id 77E8524079D for ; Thu, 4 Feb 2021 12:39:19 +0100 (CET) Received: by mail-il1-f171.google.com with SMTP id d6so2256667ilo.6 for ; Thu, 04 Feb 2021 03:39:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=scfykKsbN7GZMpFDLbbd96G+YhHZVxiawX9/fgMi7us=; b=Izr9xFW+ES//ekJK+et+QgCKmXtDILzuP0woC4qUpSc5e5OkOpD0gwsdAWajxDq0dQ 7XG9LMSMr0d8ch3iwR8aRSyL40MNWLvOu2owHOoB90g10RXQUPzM9I0n6tnjAMtVQpdb HHUIi+eiJ72d1xQvh06qSZgb16ZLECAB980yKy0H1BFMeDKVvpX12Dcx97RAwYlfrkNa 5Zcz0m4yOeccIq8V2f9JCC5unlV8V8GDQiMSt9CXtcFJ2OAwofjhXztSyeeIqZsMHq0t JOyeXajdb1tFTctlUL6d8JMqdv/d7M8y3uBklFYBqusD5nsHrZ+3PqJqRpA6BIouXDuO IFpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=scfykKsbN7GZMpFDLbbd96G+YhHZVxiawX9/fgMi7us=; b=BBHqyXPNYh7bYTWp41DkAL0NwY7dixqWCTqOQkS23t+ULq9cZijt906T916KEvsXtu osjHDxm6ZqBjAUueY048HWrTgyjVmOJNh+MahUtNT9e39HCf6aIiE8DtzcdqDTYMedzG qsGxRDYg9NcmuSKwA5ty1w3OAIa1LuZCc6tTuG+x0aUmXIRKB6lFmd/NZwhCk1MRz0eB Y/OJ7HC2b205c9IB9BPh4Qi8/vXWtuOjx4T/VIfw6vWBtmRwBeQJF+8LdFpI+bs/34SO qROJcbdLi9+ObLA9pZ2Sagjxv0cv4dpCBmR1JQZVu1nGgqT0DBYzSC4yqsM6+eI4ks92 i7VA== X-Gm-Message-State: AOAM533C0mzHQyMLMqBqAcHjfpC2I/dz8T0v2CX/N0+GeLW1c50zMyza pCrIqAK3frHl5AuK/FfZYmul/6PFd6Y1oTX+808= X-Google-Smtp-Source: ABdhPJylJ1bofpBBRW3l947runQFPYazKgztJOm34hsVGOTWO46TX2kcnarZ9xSW5r2sS7iR4BWGQlGJLqvmeeiYL4s= X-Received: by 2002:a92:c7b0:: with SMTP id f16mr6598172ilk.162.1612438758904; Thu, 04 Feb 2021 03:39:18 -0800 (PST) MIME-Version: 1.0 References: <20210203211818.3047146-1-thomas@monjalon.net> <2455385.Mn0tqrlHz1@thomas> In-Reply-To: <2455385.Mn0tqrlHz1@thomas> From: Jerin Jacob Date: Thu, 4 Feb 2021 17:09:02 +0530 Message-ID: To: Thomas Monjalon Cc: dpdk-dev , Bruce Richardson Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH] config: increase default maximum number of NUMA nodes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Feb 4, 2021 at 3:58 PM Thomas Monjalon wrote: > > 04/02/2021 07:19, Jerin Jacob: > > On Thu, Feb 4, 2021 at 2:49 AM Thomas Monjalon wrote: > > > > > > AMD CPU can present a high number of NUMA nodes. > > > The default should be 32 for better compatibility. > > > > The typical configuration is 4 nodes[1] for AMD. Just wondering, Is it > > an exception case? if so, Do we need to consume more memory for normal > > cases? > > > > [1] > > https://developer.amd.com/wp-content/resources/56308-NUMA%20Topology%20for%20AMD%20EPYC%E2%84%A2%20Naples%20Family%20Processors.PDF > > As you can read in > https://www.dell.com/support/kbdoc/fr-fr/000137696/amd-rome-is-it-for-real-architecture-and-initial-hpc-performance > there is an option "CCX as NUMA Domain. > This option exposes each CCX as a NUMA node. > On a system with dual-socket CPUs with 16 CCXs per CPU, > this setting will expose 32 NUMA domains." > and > "Enabling this option is expected to help virtualized environments." I see. > > I would not say it is exceptional. > And in my understanding, the memory cost is not so high for DPDK. > Do you see some large arrays depending on RTE_MAX_NUMA_NODES? Not quite a lot. lib/librte_efd/rte_efd.c: struct efd_online_chunk *chunks[RTE_MAX_NUMA_NODES]; lib/librte_eal/linux/eal_memory.c: uint64_t memory[RTE_MAX_NUMA_NODES]; lib/librte_eal/linux/eal.c: char * arg[RTE_MAX_NUMA_NODES]; lib/librte_eal/common/eal_common_dynmem.c: uint64_t memory[RTE_MAX_NUMA_NODES]; lib/librte_eal/common/eal_common_dynmem.c: int cpu_per_socket[RTE_MAX_NUMA_NODES]; lib/librte_eal/common/eal_private.h: uint32_t numa_nodes[RTE_MAX_NUMA_NODES]; /**< List of detected NUMA nodes. */ lib/librte_eal/common/eal_internal_cfg.h: uint32_t num_pages[RTE_MAX_NUMA_NODES]; lib/librte_eal/common/eal_internal_cfg.h: volatile uint64_t socket_mem[RTE_MAX_NUMA_NODES]; /**< amount of memory per socket */ lib/librte_eal/common/eal_internal_cfg.h: volatile uint64_t socket_limit[RTE_MAX_NUMA_NODES]; /**< limit amount of memory per socket */ lib/librte_eal/windows/eal_lcore.c: struct socket_map sockets[RTE_MAX_NUMA_NODES]; lib/librte_node/ip4_lookup.c: struct rte_lpm *lpm_tbl[RTE_MAX_NUMA_NODES]; Acked-by: Jerin Jacob > >