From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D96C34554D for ; Tue, 2 Jul 2024 17:08:43 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B1EE540A6F; Tue, 2 Jul 2024 17:08:43 +0200 (CEST) Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by mails.dpdk.org (Postfix) with ESMTP id B7F97402AD for ; Tue, 2 Jul 2024 17:08:41 +0200 (CEST) Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-7066a3229f4so2765351b3a.2 for ; Tue, 02 Jul 2024 08:08:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1719932921; x=1720537721; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=66Hp4xotBIYbNlyah0OQHNCVzdraY1cyTyCZkWRQ/dw=; b=YHYsWrq+rNrCcZrOIKQ4Iz0j1Stz9pvysC4Lg+Pmd/eJaWgJsky5fUolyLEKkAau2R 4gcNe/8+x77QgMBhyea0WZKKfhDUApsockasLtiKm+e+m5G+q/bHIl2q28+EUxMFmYdd KQbngdnEPtNEeoGugq1OISJB7vhaQ0JJ5sYm4r0adExC7hFMGzKvf0KtPrqWyauewrRd FwTpdFiaspEI5C9XHF1ypmy8xuwTriwbSXI3Cc0pDw3cuzPZKbIqOZLPMOyns/6HTvLA QnmwVECvma83pFtxgAWeMcCCGKccnpFsz5FQEPhOxREbOfnGZ62JpHWJvgSJgwXr7Wj0 MjEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719932921; x=1720537721; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=66Hp4xotBIYbNlyah0OQHNCVzdraY1cyTyCZkWRQ/dw=; b=JerHpCXKG1oleFaoUQE97uiF6f8ua0XjqyUHfeY/Yvq8QRgef+Mte9Y9MV2QraChLc lmYzIcly4x9A9vA9aGuSh6Kir2egXYIIYjOXRIpFfeIxSJgWL4v6l6rhhLuqT9bSKMNQ ucHdTqILqqAizbS1ZsIfycRDGzUNKjjqO5rp/NeZ4H9Cg9M3s8I8QhuDxRXY7x/IronK lNVw6sR8QWOc4eIuQxXnO9B1o9wkUBQ68CNsBDxnWaq96DIzNdaN5h0Whk5ZVA4SpkDn q1Ac02Jjk9/4R+0lLNgCxUNepffkxC+OmAzbEICLUxZhKQE23fleox1dslXLwH+LLJb7 TxAQ== X-Gm-Message-State: AOJu0Yx7++tK9fMlTmvovnfgB9BTuoV07LvAUftQtrjIvEpq+am7F9vC M9uwt5D/jpe39S7fx6b6SkbiiHpg3IlnfmqB2Cwv1AcrxwY+HKMwAd7pMtl4zuqft6kPLNsVaHQ vuwQ= X-Google-Smtp-Source: AGHT+IHrisYHlqMUgntq8V+ymtODyTZjMZ3r5xCT2BkmtAHynXL+ovitwPrwuf0HU+omgVb7hWPsUw== X-Received: by 2002:a05:6a00:10d5:b0:704:209a:c59e with SMTP id d2e1a72fcca58-70aaad391ffmr7333287b3a.9.1719932920556; Tue, 02 Jul 2024 08:08:40 -0700 (PDT) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-708044ad3besm8607548b3a.169.2024.07.02.08.08.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jul 2024 08:08:40 -0700 (PDT) Date: Tue, 2 Jul 2024 08:08:37 -0700 From: Stephen Hemminger To: =?UTF-8?B?5rSq5YWo?= Cc: users@dpdk.org Subject: Re: Issue with Cannot allocate memory when using 32-bit DPDK application Message-ID: <20240702080837.5e925cbb@hermes.local> In-Reply-To: <5073ea6b.82662.19072f2bb7d.Coremail.hongquan@iie.ac.cn> References: <5073ea6b.82662.19072f2bb7d.Coremail.hongquan@iie.ac.cn> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Tue, 2 Jul 2024 18:17:08 +0800 (GMT+08:00) =E6=B4=AA=E5=85=A8 wrote: > Dear DPDK community, >=20 >=20 > I am encountering an issue when attempting to run a 32-bit DPDK applicati= on on Linux. Specifically, I am facing a "Cannot allocate memory" error dur= ing initialization. While I can mitigate this issue by using the `--no-huge= ` option, it adversely affects the performance of my application. >=20 >=20 > Here is the error output I receive: >=20 >=20 > ``` > sudo ./app -l 0-1 --proc-type=3Dprimary --file-prefix=3Dpmd1 --vdev=3Dnet= _tap001,iface=3Dtap001 --no-pci > EAL: Detected CPU lcores: 2 > EAL: Detected NUMA nodes: 1 > EAL: Detected shared linkage of DPDK > EAL: Multi-process socket /var/run/dpdk/pmd1/mp_socket > EAL: Selected IOVA mode 'PA' > EAL: Cannot get a virtual area: Cannot allocate memory > EAL: Cannot allocate VA space for memseg list, retrying with different pa= ge size > EAL: Cannot allocate VA space on socket 0 > EAL: FATAL: Cannot init memory > EAL: Cannot init memory > app: main.c:284: main: Assertion `(ret =3D rte_eal_init(argc, (char **) a= rgv)) >=3D 0' failed. > Aborted > ``` >=20 >=20 > When debugging with `--log-level=3Deal,8`, the relevant portion of the ou= tput indicates attempts to allocate memory: >=20 >=20 > ``` > EAL: Attempting to preallocate 2048M on socket 0 > EAL: Ask a virtual area of 0xc000 bytes > EAL: Virtual area found at 0xeb077000 (size =3D 0xc000) > EAL: Memseg list allocated at socket 0, page size 0x800kB > EAL: Ask a virtual area of 0x80000000 bytes > EAL: Cannot mmap((nil), 0x80200000, 0x0, 0x22, -1, 0x0): Cannot allocate = memory > EAL: Cannot get a virtual area: Cannot allocate memory > EAL: Cannot allocate VA space for memseg list, retrying with different pa= ge size > EAL: Cannot allocate VA space on socket 0 > EAL: FATAL: Cannot init memory > EAL: Cannot init memory > app: main.c:284: main: Assertion `(ret =3D rte_eal_init(argc, (char **) a= rgv)) >=3D 0' failed. > Aborted > ``` >=20 >=20 > System information: > - Hugepages configured: `echo 1024 > /sys/kernel/mm/hugepages/hugepages-2= 048kB/nr_hugepages` > - Hugepages mounted: `mount -t hugetlbfs hugetlbfs /dev/hugepages` > - NUMA node information: `numactl --hardware` > - available: 1 nodes (0) > - node 0 cpus: 0 1 > - node 0 size: 7896 MB > - node 0 free: 3915 MB > - node distances: > - node 0: 10 >=20 >=20 > DPDK version: 22.03 > Distribution: Ubuntu 22.04 > Kernel information: Linux hq-virtual-machine 6.5.0-35-generic #35~22.04.1= -Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2 x86_64 x86_64 x86_64 G= NU/Linux >=20 >=20 > I believe the issue stems from the attempt to preallocate 2048M on socket= 0, but using `-m` or `--socket-mem` options did not resolve the problem. >=20 >=20 > Could you please provide guidance on how to properly configure DPDK to av= oid this memory allocation issue while maximizing performance? What CPU architecture? Traditionally on 32 bit x86 has 3GB for userspace an= d 1GB is the shadow kernel. Some other architecture may do the same thing. In userspace, you then have some memory for programs code, data and stack. = The DPDK EAL init then tries to map all of available huge pages (2G) and fails = to find enough contiguous virtual address space to fit. Try smaller amount of huge pages.