From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 89A8342549 for ; Fri, 8 Sep 2023 16:31:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 58916402B7; Fri, 8 Sep 2023 16:31:26 +0200 (CEST) Received: from sonic318-20.consmr.mail.gq1.yahoo.com (sonic318-20.consmr.mail.gq1.yahoo.com [98.137.70.146]) by mails.dpdk.org (Postfix) with ESMTP id 98EB840285 for ; Fri, 8 Sep 2023 16:31:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694183483; bh=uck2qvJHf+CGoRwwIdIi/xvXmBTJsEdk0HIaOBjMKyA=; h=Date:From:Reply-To:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=VDFsuz3ctzUGpIq0ntS9bmj5mTU+ADpiUqsihrOb5xGrjuppHjOpZQQovd7SX7s/BXNBq4/xCzB8NjpHtiE503weG69Lc+WaoP+S6SWjk6UbPJwg/vSa1u/lWj4dbr/kKO/dII1yqcDJenIlIJfmbD5pX3RnJ8S3is3l29P6jFwGfm5YwN+sx9G8ZdJb/F3wV1QUEGrSsj+AMDDn9X2UGmdcR4GGPpEfgZmRwMLolbeHMUlLtN3W40jRYIJ9BcgP3Fy++ELz2zRy97kIXQDQqGXqmBgyqYVgTLTHiNv9X+dtG6OSjijZofEoEx5/zb6vYuwbdtrr8NO3c7kwiziBBA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1694183483; bh=mzn4qlI43vh/0xUXOg0kBZEJQG0Db98Wv2TM453vtbK=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=URnk6cEBUKtKfCyxqDrGGWtHD02o8ogP9zYeATS6bBc9OWiCK+RPZG5C4nXs651c+A4kX30B1Q5xJ4iDD28+uHPA1A5/fo1G5BBgBYSwmLM2tUhXCOp5veATz7UaryaRBjytXguGQY4wChbZJI+dI4I8aOvbzserHrQ3OWRbr955pAtFHcSG3aqdlkFwZ5kSMp1F7e93/QTwSMyIncjsprdwAL7vep2aW9IcOSbL5H8AgH9s4tq7B0JM5mYtXy0yKN1NxU1B4TUGQzNqVQN7tep6Lg4S9NLyjbiwPlA7okbLDQ8CgkQEzbhltClUxSR5rjf2iTYVUB3OXeb8I3r21A== X-YMail-OSG: uRZKVekVM1kQQ1VO8XeVpxIaEb12LtrP.H87XYDOcBJv0.eCnmvn3cvF9lxS7Xk FJrQ.UkKW3ZdF2dJ.vhLjKMJRJxpBMU5.gkyZRp79O3hTyyNUfvqGdHwDEAKe45vrSNT946kMXxQ 08ZUFbJZkm_zN53NV99KMewqHGMJNUKCGyg_t_wzwUah2WHsMRMfIsTTYOYNBL_i6eqUshkUhedY Wx2TrtStgB_wQVjBQ9kv809nt.0aaZoKC.Exq_2ZbxaGRaX0rz_qYz_ZLIl3C6t8Y3ZaUNULSq1R GB2EyteHFIHFxeh8gZNgSIJ0sI2bH9pp.Wk2annz.U_T6KCoNCq6DqI325o1oPIRWV9To9D3HYgb KbStrthrMFWjKjPKLajMA4EwdWrZotUdPsFTZlJbnns2V_qBdJagdi.41wJ6UFy08Zpvwun4a4km 7CJHdKuF5b4C0WTVrzJL6YhJ9OvKTEs33jZJUbJOTEcB2AjzAU4En2dP2t2MOtV.1INVXfyiotrp TmZpqFtOf.EO3OYD2LxfRTYAgCWCQlC4o3HGvAzurQfWz9Au_KgL_rmar._Sa06yogDnQS6ViTiu 1tQuFFl0481OoktoRS4BsiIBd8eQP4DbghROq0RqUivBItZDAB1pAgbkJjYCBxTCqyUHtlRy07fs QTjYRcGjU.JaftYxnxcXJOgioueeIr7Iq8my4kwFQuHY5_CDDaxtSRn2wihawHmK3gPzVGsP.kCp nyMzIROyhI3JyxUE7oHmW5stQXu5rr0DcaqxQD3b9Na0Vq34.2RrBfG7_tcGrYCJUmSMcGgEF.qo vhhhxUmpSzLB5oAlFAs.6_ei_5kE5UiLPBxwKVvPiX2UCPfzWy._mcTI7dUFAjUi10zkmG9AjoMD 0a.ZZc_hlysUa.kovtUqyT8Hydlna4xkZMXMOh0jaltgnSOBv3zc3dVh.7CXysKq6rdkejX9qCfr evbkzc5.qvFGY8ET8lnvnMH08Ux4IQhtrLMBAnldaSRHdRs5AymfZk5DIxr9E3w5J6YBVlxiMSqh bs7h1r7fzRRscCz5R7r8xucJxxXn66H.o7YH5JsfliJ6x29mVci0lHUzvFR83si6PrzhUusXv8WZ EZfZym9mHZ5LZm1GkLLWvNR6Y4JtZtsjAmz99yrnrUWvia56AB7S8.5YfIS77A7izT7GXpJ5g0vJ mdHb_JgQu5C7LW3YdpuN.1_YUiA3zGRbxIV5Tk4OgRKxAA08EMfaDWmYfalwdgxQ24uxXt17.hMu UpRNq1YXYIzhSQD2reH8XxEjwg4TqjBE6iqDwrbX..5ZwrGQ.t6OrfXA6LhMHYdD6mpcz2vwri2y X9rMYFdAqtjmrNvy06XRWu8OnmUIiq9x1hzNzFQlCmibjC6zEulLBP2MRmgunsQL7asD2s4zotE4 yMQtSzGkAcX_3iJ9qRG4TKsAMDFzj5YR_e.L7QohkXRXYpgW_zzezcr.YpyWhmZ1..SLeUbWUFv7 FmeyybgnZTbjmfSpApRZdRI7DNthBrQAqkXvmhcGQKDz05LVIKku9ieFS70a7C2vnnB_PzKWD8Vk 2cv2elcAn4G1xS7x8eA5dvFebWDHUFpWb6NTPb95SI5BGQDHD4IH88NiztGvNCRsid0OX1BWTqxe Eg8yQFMHwQ7GX2SQ6zzEMnqapsLuBSCmsxVRaBsqa_mCGHWxn.Dn3.Wxo3Itq7zrlaAkb6lk.xVh tGDQ1UkrC0V1jvPneOqEJ60baEZqn.68uRwhUGSHClcNi8Rg89iXJDINZDEM.7anY5PLg897j92v LdGzXAWtIlzCfKqw2.Zlr1RoU8rlVgYlgdxHXZn_sDUzaY12UdRN26a3YGUZVC1EY8g9wyns_OhF 7A2yG9AvfcHbsDCxhag_PvJIPVgu2K0NQFgPlEz.Z__QOyF6lY1QydJCiADVBZgzk69UMgTF.TVy cTy8DcAhhAFTLGv9KGSLxxP1rmwC.6P0edWeW_CK6joHyEaus0HYBFBrjKntqPUevIcp60BO6aqE O0mKhjr9xSHiFV1efIMW5i8XzNuiheGNJMJK2ChhSvj29QXS3Oq7SqTMPF9MFLNB.Q8RPNfMyAK. J1G1L_55aWpZq112iYkTzN7ztBdoqsNfst_Gk8HaUVQfxmeGbNKqzW_Hb.yBT0r45QkD6ia.BMaL Pe8F0ZDoCmXeOx5dbMqp67CUrjJMgRIdorLVubl_GSd_IDUvRfMYVxHmiOy8ycA3Q71.xhoRyimz _pK5IKCpuBDexWg0X4zgO06FSSi18vap89LSz4FV9OgG1lg_jR9_Zv1GsGBJuJkn2 X-Sonic-MF: X-Sonic-ID: 87c91980-e87f-41c0-93c2-11fb6e63b1d4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic318.consmr.mail.gq1.yahoo.com with HTTP; Fri, 8 Sep 2023 14:31:23 +0000 Date: Fri, 8 Sep 2023 14:31:22 +0000 (UTC) From: Don Trotter To: Stephen Hemminger , "jiangheng (G)" Cc: "users@dpdk.org" Message-ID: <1466053359.5447492.1694183482188@mail.yahoo.com> In-Reply-To: <22952d8e22994649b2c8f1b22997e0ce@huawei.com> References: <22952d8e22994649b2c8f1b22997e0ce@huawei.com> Subject: Re: EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5447491_939065372.1694183482184" X-Mailer: WebService/1.1.21763 YMailNorrin X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Don Trotter Errors-To: users-bounces@dpdk.org ------=_Part_5447491_939065372.1694183482184 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thursday, September 7, 2023 at 09:26:34 PM CDT, jiangheng (G) wrote: =20 =20 =20 Hi =20 I've had the same error. It may be caused by this parameter: =20 https://github.com/DPDK/dpdk/blob/v22.11/config/rte_config.h#L33C9-L33C33 =20 If you use 2MB hugepages, the max memory size is 2MB * 8192 =3D 16GB. This = error will occur when you allocate more than 16GB of memory per numa. =20 =C2=A0 =20 But should not have this error if you use 512 MB of memory, can you show RT= E_MAX_MEMSEG_PER_LIST in your dpdk? =20 Check whether the other sizes hugepage =C2=A0is used, Theoretically, this e= rror indicates that the rte_eal_init init fails and the program should exit= . =C2=A0My initial statement was that the log=C2=A0 was seen "from EAL duri= ng EAL init", but that is not right. The call to rte_eal_init() is successf= ul. Later I allocate the memory, using=C2=A0rte_mempool_create("node_pool",= =C2=A0(128 * 1024 * 1024 - 1), 256, 256, 0, NULL, NULL, NULL, NULL, rte_soc= ket_id(), 0). That is when I see the message. And the call to allocate the = memory is successful, and I am processing packets and allocating nodes just= fine. Here is the information from rte_config.h. $ grep RTE_MAX_MEM=C2=A0 ./cn98xx-release-output/build/dpdk/config/rte_conf= ig.h#define RTE_MAX_MEMSEG_LISTS 128#define RTE_MAX_MEMSEG_PER_LIST 8192#de= fine RTE_MAX_MEM_MB_PER_LIST 32768#define RTE_MAX_MEMSEG_PER_TYPE 32768#def= ine RTE_MAX_MEM_MB_PER_TYPE 65536#define RTE_MAX_MEMZONE 2560 =20 =C2=A0 =20 =E5=8F=91=E4=BB=B6=E4=BA=BA: Don Trotter =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2023=E5=B9=B49=E6=9C=888=E6=97=A5 5:3= 3 =E6=94=B6=E4=BB=B6=E4=BA=BA: Stephen Hemminger =E6=8A=84=E9=80=81: users@dpdk.org =E4=B8=BB=E9=A2=98: Re: EAL: eal_memalloc_alloc_seg_bulk(): couldn't find s= uitable memseg_list =20 =C2=A0 =20 On Thursday, September 7, 2023 at 03:21:12 PM CDT, Stephen Hemminger wrote: =20 =C2=A0 =20 =C2=A0 =20 On Thu, 7 Sep 2023 19:58:35 +0000 (UTC) =20 Don Trotter wrote: >=C2=A0 To clarify, the log message when my application called=C2=A0rte_mem= pool_create() to create the "node_pool", and the call succeeded. > Thanks,Don Trotter >=C2=A0 =C2=A0 On Thursday, September 7, 2023 at 01:54:08 PM CDT, Don Trott= er wrote:=C2=A0 >=C2=A0 >=C2=A0 Hi, > I have recently started using DPDK. I am working on a project on OcteonTX= 2 with DPDK 11.23.01. I am seeing this message from EAL during EAL init, bu= t everything is working. > =C2=A0 =C2=A0=C2=A0EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suit= able memseg_list > The system has 96GB of memory.=C2=A0 These 2 pools get created and everyt= hing works fine. > mempool @0x13fed3e00=C2=A0 flags=3D10=C2=A0 socket_id=3D0=C2= =A0 pool=3D0x1140300000=C2=A0 iova=3D0x13fed3e00=C2=A0 nb_mem_chunks=3D1=C2= =A0 size=3D65535=C2=A0 populated_size=3D65535=C2=A0 header_size=3D128=C2=A0= elt_size=3D10200=C2=A0 trailer_size=3D40=C2=A0 total_obj_size=3D10368=C2= =A0 private_data_size=3D128=C2=A0 ops_index=3D0=C2=A0 ops_name: =C2=A0 avg bytes/object=3D10368.558602 > mempool @0x1575d8180=C2=A0 flags=3D10=C2=A0 socket_id=3D-1=C2= =A0 pool=3D0x19ffffd00=C2=A0 iova=3D0x1575d8180=C2=A0 nb_mem_chunks=3D2=C2= =A0 size=3D134217727=C2=A0 populated_size=3D134217727=C2=A0 header_size=3D1= 28=C2=A0 elt_size=3D256=C2=A0 trailer_size=3D0=C2=A0 total_obj_size=3D384= =C2=A0 private_data_size=3D0=C2=A0 ops_index=3D3=C2=A0 ops_name: =C2=A0 avg bytes/object=3D384.000094 > You read that right. I=E2=80=99ve got 128*1024*1024 256 byte buffers crea= ted for data. > I also see there is still heap left, although pretty low after. > Heap id:0=C2=A0 =C2=A0 =C2=A0 =C2=A0 Heap name:socket_0=C2=A0 =C2=A0 =C2= =A0 =C2=A0 Heap_size:55834574848,=C2=A0 =C2=A0 =C2=A0 =C2=A0 Free_size:2403= 644544,=C2=A0 =C2=A0 =C2=A0 =C2=A0 Alloc_size:53430930304,=C2=A0 =C2=A0 =C2= =A0 =C2=A0 Greatest_free_size:536870016,=C2=A0 =C2=A0 =C2=A0 =C2=A0 Alloc_c= ount:293,=C2=A0 =C2=A0 =C2=A0 =C2=A0 Free_count:5, > Linux free stats. > tmp# free -h -w=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 total=C2= =A0 =C2=A0 =C2=A0 =C2=A0 used=C2=A0 =C2=A0 =C2=A0 =C2=A0 free=C2=A0 =C2=A0 = =C2=A0 shared=C2=A0 =C2=A0 =C2=A0buffers=C2=A0 =C2=A0 =C2=A0 =C2=A0cache=C2= =A0 =C2=A0availableMem:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A095Gi=C2=A0 = =C2=A0 =C2=A0 =C2=A0 88Gi=C2=A0 =C2=A0 =C2=A0 =C2=A06.5Gi=C2=A0 =C2=A0 =C2= =A0 =C2=A0444Mi=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0B=C2=A0 =C2=A0 =C2=A0 = =C2=A0470Mi=C2=A0 =C2=A0 =C2=A0 =C2=A0599MiSwap:=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 0B=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0B=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 0B > Is there anything wrong with that EAL log? Is there a lurking problem? > Thanks,Don Trotter =20 > >=C2=A0 Did you setup hugepages? How many and what size? Is this a NUMA system? Also 11.23.01 seems like a weird release number. The DPDK release numbering scheme is year followed by month. I.e. 22.11 was= released in November of 2022 =20 =C2=A0 =20 =C2=A0 =20 # cat /sys/kernel/mm/hugepages/hugepages-524288kB/nr_hugepages =20 176 =20 =C2=A0 =20 Yes it is a NUMA system. =20 =C2=A0 =20 You are correct sir. The SDK is SDK11.23.01 (2023-01) and DPDK is 22.11. =20 =C2=A0 =20 Thanks, Don Trotter =20 =C2=A0 =20 ------=_Part_5447491_939065372.1694183482184 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Th= ursday, September 7, 2023 at 09:26:34 PM CDT, jiangheng (G) <jiangheng14= @huawei.com> wrote:


Hi

=20

I've had the same error. It may = be caused by this parameter:

=20

https://github.com/DPDK/dpdk/blob/v22.11/confi= g/rte_config.h#L33C9-L33C33

=20

If you use 2MB hugepages, the ma= x memory size is 2MB * 8192 =3D 16GB. This error will occur when you alloca= te more than 16GB of memory per numa.

=20

 

=20

But should not have this error i= f you use 512 MB of memory, can you show RTE_MAX_MEMSEG_PER_LIST in your dp= dk?

=20

Check whether the other sizes hu= gepage  is used, Theoretically, this error indicates that the rte_eal_= init init fails and the program should exit.

=20
 
My initial statement was that the log  = was seen "from EAL during EAL init", but that is not right. The call to rte_eal= _init() is successful. Later I allocate the memory, using rte_me= mpool_create("node_pool", (128 * 1024 * 1024 - 1), 256, 256, 0, = NULL, NULL, NULL, NULL, rte_socket_id(), 0). That is when I see the message= . And the call to allocate the memory is successful, and I am processing pa= ckets and allocating nodes just fine. Here is the information from rte_conf= ig.h.
<= br>
$ grep RTE_MAX_MEM  ./cn98xx-release-outpu= t/build/dpdk/config/rte_config.h
#define RTE_MAX_MEMSEG_LISTS 128=
#define RTE_MAX_MEMSEG_PER_LIST 8192
#define RTE_MAX_M= EM_MB_PER_LIST 32768
#define RTE_MAX_MEMSEG_PER_TYPE 32768
<= div>#define RTE_MAX_MEM_MB_PER_TYPE 65536
#define RTE_MAX_MEMZONE= 2560

=20

 

=20

=E5=8F=91=E4=BB=B6=E4=BA=BA:= Don Trotter <djtrotter= @yahoo.com>
=E5=8F=91=E9=80=81=E6=97=B6=E9= =97=B4: 2023=E5=B9=B4= 9=E6=9C=888=E6=97= =A5 5:33
=E6=94=B6=E4=BB=B6=E4=BA=BA: Stephen Hemminger <stephen@networkplumber.org>
=E6=8A=84=E9=80=81: users@dpdk.org
=E4=B8=BB=E9=A2=98: Re: EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable mems= eg_list

=20

 <= /span>

=20

On Thursday, September 7, 2023 at 0= 3:21:12 PM CDT, Stephen Hemminger <stephen@networkplumber= .org> wrote:

=20

 

=20

 

=20

On Thu, 7 Sep 2023 19:58:35 +0000 (= UTC)

=20


Don Trotter <djtrotter@yahoo.com<= span lang=3D"EN-US" style=3D"font-family:sans-serif;color:#26282A;">> wrote:

>  To clarify, the log message when my application called rte_= mempool_create() to create the "node_pool", and the call succeeded.
> Thanks,Don Trotter
>    On Thursday, September 7, 2023 at 01:54:08 PM CDT, Don Tr= otter <djtrotter@yahoo.com> wrote: 
> 
>  Hi,
> I have recently started using DPDK. I am working on a project on Octeo= nTX2 with DPDK 11.23.01. I am seeing this message from EAL during EAL init,= but everything is working.
>     EAL: eal_memalloc_alloc_seg_bulk(): couldn't find s= uitable memseg_list
> The system has 96GB of memory.  These 2 pools get created and eve= rything works fine.
> mempool <mbuf_pool>@0x13fed3e00  flags=3D10  socket_id= =3D0  pool=3D0x1140300000  iova=3D0x13fed3e00  nb_mem_chunks= =3D1  size=3D65535  populated_size=3D65535  header_size=3D12= 8  elt_size=3D10200  trailer_size=3D40  total_obj_size=3D103= 68  private_data_size=3D128  ops_index=3D0  ops_name: <cn9k_mempool_ops>  avg bytes/object=3D10368.558602
> mempool <node_pool>@0x1575d8180  flags=3D10  socket_id= =3D-1  pool=3D0x19ffffd00  iova=3D0x1575d8180  nb_mem_chunks= =3D2  size=3D134217727  populated_size=3D134217727  header_s= ize=3D128  elt_size=3D256  trailer_size=3D0  total_obj_size= =3D384  private_data_size=3D0  ops_index=3D3  ops_name: <ring_mp_mc>  avg bytes/object=3D384.000094
> You read that right. I=E2=80=99ve got 128*1024*1024 256 byte buffers c= reated for data.
> I also see there is still heap left, although pretty low after.
> Heap id:0        Heap name:socket_0    &= nbsp;   Heap_size:55834574848,        Free_size:24= 03644544,        Alloc_size:53430930304,    &= nbsp;   Greatest_free_size:536870016,        Alloc= _count:293,        Free_count:5,
> Linux free stats.
> tmp# free -h -w              total&= nbsp;       used        free   = ;   shared     buffers       cache&= nbsp;  availableMem:           95Gi = ;       88Gi       6.5Gi    &n= bsp;  444Mi          0B      &= nbsp;470Mi       599MiSwap:        =     0B          0B      &= nbsp;   0B
> Is there anything wrong with that EAL log? Is there a lurking problem?=
> Thanks,Don Trotter

=20


>
> 

Did you setup hugepages?
How many and what size?
Is this a NUMA system?

Also 11.23.01 seems like a weird release number.
The DPDK release numbering scheme is year followed by month. I.e. 22.11 was= released in November of 2022

=20

 

=20

 

=20

# cat /sys/kernel/mm/hugepages/hugepages-524288kB= /nr_hugepages

=20

176

=20

 

=20

Yes it is a NUMA system.

=20

 

=20

You are correct sir. The SDK is SDK11.23.01 (2023= -01) and DPDK is 22.11.

=20

 

= =20

Thanks, Don Trotte= r

=20

 
=

=20
------=_Part_5447491_939065372.1694183482184--