Re: EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list

[Stephen Hemminger <stephen@networkplumber.org>]

On Fri, 8 Sep 2023 14:31:22 +0000 (UTC)
Don Trotter <djtrotter@yahoo.com> wrote:

>  On Thursday, September 7, 2023 at 09:26:34 PM CDT, jiangheng (G) <jiangheng14@huawei.com> wrote:
>  
>  
>  
> Hi
>  
> I've had the same error. It may be caused by this parameter:
>  
> https://github.com/DPDK/dpdk/blob/v22.11/config/rte_config.h#L33C9-L33C33
>  
> If you use 2MB hugepages, the max memory size is 2MB * 8192 = 16GB. This error will occur when you allocate more than 16GB of memory per numa.
>  
>   
>  
> But should not have this error if you use 512 MB of memory, can you show RTE_MAX_MEMSEG_PER_LIST in your dpdk?
>  
> Check whether the other sizes hugepage  is used, Theoretically, this error indicates that the rte_eal_init init fails and the program should exit.
>    My initial statement was that the log  was seen "from EAL during EAL init", but that is not right. The call to rte_eal_init() is successful. Later I allocate the memory, using rte_mempool_create("node_pool", (128 * 1024 * 1024 - 1), 256, 256, 0, NULL, NULL, NULL, NULL, rte_socket_id(), 0). That is when I see the message. And the call to allocate the memory is successful, and I am processing packets and allocating nodes just fine. Here is the information from rte_config.h.
> $ grep RTE_MAX_MEM  ./cn98xx-release-output/build/dpdk/config/rte_config.h#define RTE_MAX_MEMSEG_LISTS 128#define RTE_MAX_MEMSEG_PER_LIST 8192#define RTE_MAX_MEM_MB_PER_LIST 32768#define RTE_MAX_MEMSEG_PER_TYPE 32768#define RTE_MAX_MEM_MB_PER_TYPE 65536#define RTE_MAX_MEMZONE 2560
>  
>   
>  
> 发件人: Don Trotter <djtrotter@yahoo.com>
> 发送时间: 2023年9月8日 5:33
> 收件人: Stephen Hemminger <stephen@networkplumber.org>
> 抄送: users@dpdk.org
> 主题: Re: EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list
>  
>   
>  
> On Thursday, September 7, 2023 at 03:21:12 PM CDT, Stephen Hemminger <stephen@networkplumber.org> wrote:
>  
>   
>  
>   
>  
> On Thu, 7 Sep 2023 19:58:35 +0000 (UTC)
>  
> 
> Don Trotter <djtrotter@yahoo.com> wrote:
> 
> >  To clarify, the log message when my application called rte_mempool_create() to create the "node_pool", and the call succeeded.
> > Thanks,Don Trotter
> >    On Thursday, September 7, 2023 at 01:54:08 PM CDT, Don Trotter <djtrotter@yahoo.com> wrote: 
> > 
> >  Hi,
> > I have recently started using DPDK. I am working on a project on OcteonTX2 with DPDK 11.23.01. I am seeing this message from EAL during EAL init, but everything is working.
> >     EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list
> > The system has 96GB of memory.  These 2 pools get created and everything works fine.
> > mempool <mbuf_pool>@0x13fed3e00  flags=10  socket_id=0  pool=0x1140300000  iova=0x13fed3e00  nb_mem_chunks=1  size=65535  populated_size=65535  header_size=128  elt_size=10200  trailer_size=40  total_obj_size=10368  private_data_size=128  ops_index=0  ops_name: <cn9k_mempool_ops>  avg bytes/object=10368.558602
> > mempool <node_pool>@0x1575d8180  flags=10  socket_id=-1  pool=0x19ffffd00  iova=0x1575d8180  nb_mem_chunks=2  size=134217727  populated_size=134217727  header_size=128  elt_size=256  trailer_size=0  total_obj_size=384  private_data_size=0  ops_index=3  ops_name: <ring_mp_mc>  avg bytes/object=384.000094
> > You read that right. I’ve got 128*1024*1024 256 byte buffers created for data.
> > I also see there is still heap left, although pretty low after.
> > Heap id:0        Heap name:socket_0        Heap_size:55834574848,        Free_size:2403644544,        Alloc_size:53430930304,        Greatest_free_size:536870016,        Alloc_count:293,        Free_count:5,
> > Linux free stats.
> > tmp# free -h -w              total        used        free      shared     buffers       cache   availableMem:           95Gi        88Gi       6.5Gi       444Mi          0B       470Mi       599MiSwap:            0B          0B          0B
> > Is there anything wrong with that EAL log? Is there a lurking problem?
> > Thanks,Don Trotter  
>  
> 
> >
> >   
> 
> Did you setup hugepages?
> How many and what size?
> Is this a NUMA system?
> 
> Also 11.23.01 seems like a weird release number.
> The DPDK release numbering scheme is year followed by month. I.e. 22.11 was released in November of 2022
>  
>   
>  
>   
>  
> # cat /sys/kernel/mm/hugepages/hugepages-524288kB/nr_hugepages
>  
> 176
>  
>   
>  
> Yes it is a NUMA system.
>  
>   
>  
> You are correct sir. The SDK is SDK11.23.01 (2023-01) and DPDK is 22.11.
>  
>   
>  
> Thanks, Don Trotter
>  
>   
> 
>    

You may need to add hugepages on each NUMA node.