From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 2DDAA44191
for <public@inbox.dpdk.org>; Thu, 5 Sep 2024 00:23:12 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id A255F4025C;
Thu, 5 Sep 2024 00:23:11 +0200 (CEST)
Received: from mx0a-00196b01.pphosted.com (mx0b-00196b01.pphosted.com
[67.231.157.166])
by mails.dpdk.org (Postfix) with ESMTP id A2C1B40151
for <users@dpdk.org>; Thu, 5 Sep 2024 00:23:10 +0200 (CEST)
Received: from pps.filterd (m0096262.ppops.net [127.0.0.1])
by mx0b-00196b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 484LIk5T011638;
Wed, 4 Sep 2024 18:23:10 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h=
cc:content-type:date:from:message-id:mime-version:subject:to; s=
netscout.com.09.24.2020; bh=2P7SgI94O+wtH6wcpFb/4BWCYc1/0D2fB0kI
TUbI1MY=; b=vK/VVsMsCWzy7RDFGe2LQOR7jOL+syEaXUwfLt0NeSUgEJFLQY9Z
PDI9l/pNKiNbZoN7vZNDj9VxfNwpNxY/Vwr1plpxQmFn3byrQV9v/Z07VyiNv1Hs
8z4X4mYDcCgASU0980NLTSN9O5BhHsH57Cp0Ia1v3Q2FMufMNXlEhmqhZknqpGG9
Cg7Oiebe2W/kgytfQg3/+SQAsjeKpN0xZ09Ch+rZSfsDzb3P3kJEz9GOEVM4bkWI
RM9vJJofdNCRBv859g4rjdmWM4u8Z2g9NxzChMAaYyaeGLu17xC94xliDEUNpweG
WkMY4MtD5+G7IkYkloUxHRLTNjRzpcnL7w==
Received: from nam04-mw2-obe.outbound.protection.outlook.com
(mail-mw2nam04lp2177.outbound.protection.outlook.com [104.47.73.177])
by mx0b-00196b01.pphosted.com (PPS) with ESMTPS id 41eyb5r26q-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Wed, 04 Sep 2024 18:23:09 -0400 (EDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none;
b=BzeuiyWfgwLjRnlp+nkVBjoAFhnMT2iGNVVoatG+3BtNFlNU60fF5oqqmFhlUM2BPEG6VF0v2TSDROG9ysgRdUBUgBrCHSorioIScAeJHK8dLpG1etAJyhamRTgXmW2eYksVelfnW3Q8zPeqetQsILdtc4rlekuAQWK7NohKeWri/AYuGfRV1A0jWZx3bhg00FurZYjVGIctT/EYzUdyzia24QA0CAeljg9Y1l9zTAhb6ytpvSloQ7xzC53Ht3dKzGKRZSL8gvwknr3fEHnVmcwo9PtYiwIDQGNxFP3zSYFbrqAXzBbnZ8lLMAg8+pDEk7SstKpqfGzKR2cSGNHlbQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector10001;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=2P7SgI94O+wtH6wcpFb/4BWCYc1/0D2fB0kITUbI1MY=;
b=RsZXWD5AJUjQd8MxDbkYrQxYV8O2+gU9OqmEsXyHwtpKTzhrdExJizj7Z9EnNrmM2TlCDQW7o/6zXXqZfpA/VGjzahvV3jUsVSpcXNnjsr++LfvMX5kSs2Lrbw5yuTAJj48/QC//MfR/kUJddQSqy5G8vjG7aJJnK48D3qs2k8ihGwSD9R12mpP28guVP7oU8RoQfjFR8QVT8Jg3lDAg/CovjOJQRnOlPWK3z36k+33nVHrfV4p9HBw+ufBxKxrstARNdYq3xTSxf/EIcE12rNF7E8sM2rX23m/KlYGLnnOnPfaec58P+iXZzeS27bcc4I2d80kgnEwgn8MovXjD1A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com;
dkim=pass header.d=netscout.com; arc=none
Received: from CH3PR01MB8470.prod.exchangelabs.com (2603:10b6:610:1a4::21) by
CO1PR01MB8939.prod.exchangelabs.com (2603:10b6:303:275::15) with
Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.7918.27; Wed, 4 Sep 2024 22:23:07 +0000
Received: from CH3PR01MB8470.prod.exchangelabs.com
([fe80::80c4:7216:f070:e5fd]) by CH3PR01MB8470.prod.exchangelabs.com
([fe80::80c4:7216:f070:e5fd%4]) with mapi id 15.20.7918.024; Wed, 4 Sep 2024
22:23:06 +0000
From: "Lombardo, Ed" <Ed.Lombardo@netscout.com>
To: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
CC: "users@dpdk.org" <users@dpdk.org>
Subject: hugepage allocation mapping failure
Thread-Topic: hugepage allocation mapping failure
Thread-Index: Adr/FMQTwzM00r+tRkmT2Kzap3l71g==
Date: Wed, 4 Sep 2024 22:23:06 +0000
Message-ID: <CH3PR01MB847096E58EAF53571CE9115B8F9C2@CH3PR01MB8470.prod.exchangelabs.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: CH3PR01MB8470:EE_|CO1PR01MB8939:EE_
x-ms-office365-filtering-correlation-id: e84cd805-2bbd-4f2b-da11-08dccd30267b
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0; ARA:13230040|1800799024|376014|366016|38070700018;
x-microsoft-antispam-message-info: =?us-ascii?Q?03RXWJgSAOuQYoWiINwvR+rLX5R60m58AjnFdQV/of+YRkMB237GPc1HBM/I?=
=?us-ascii?Q?0wBdSp6Voi0KCnqrfkfgasyJk2QX8QR7x4L2bYO+I154jGkWR4pu1ywzTqa9?=
=?us-ascii?Q?yPjr4McXBo3/Gh2hdmap/ScXZDI5gs94IdnNo0njVoRyhrnJdSi/kkDYxYjf?=
=?us-ascii?Q?+oAD/0TG/F5cfTQcD/HSNEPoLLGLaR7atpBtt5UUwPTCv/djm+elg4s0OoTl?=
=?us-ascii?Q?fbvULaiMFDfhZRvmAfzvst7ew0K3X67i1y5MrVFxh4//G18QZTtuWS5ayJMX?=
=?us-ascii?Q?OUDJ5Zb89IhZfXXtyIMdRCV/MvnWiw0WX0JepINfA0Urr7rOPkI6xAwdoeqF?=
=?us-ascii?Q?iqBzGozuGYAP6uhUei2wzYmxQlTc7PbwZH2lEt+IUEGVk0B7WrvLl7QWzknA?=
=?us-ascii?Q?33UkG9EoOeEZuTcbn6YRKdXQAfLqO7l9d8jp8xshm3tNJz7mb0uxR4MA01np?=
=?us-ascii?Q?FyQwq29BLbg2oTiAGXf3F6DzFOXvacnHhO80zst4q5EGFFFFy/1aQvJnmCn0?=
=?us-ascii?Q?WfRZEpf+mqndzk0bdpslbR1KLmIT1STKmYbrpSZz+DC1iBhCONjcyXm0Bury?=
=?us-ascii?Q?XOMnIIgBxcgfk39ZwK0ktZDeOCDJnZUWN3njRbTmhlD15i7PPQEQeRhjVxmW?=
=?us-ascii?Q?a3qnSddwqNEbz4byW1n7dq3EwA/X5DWab9F+GIUqfvq0HOZ1mD7zRxAIvi0i?=
=?us-ascii?Q?vH1mXoL2udusrTNiZyF0aZEL7sS5Py1hVj68YlZDj11mmAyQ4OME7Zv+Gx62?=
=?us-ascii?Q?w6wqXnKJgeEcwOV58ZSk8+YBE80aEc1YXEILTuaczysZc5yk//QsPAPaBeDE?=
=?us-ascii?Q?YwAaf6guQu66C8QYTKzEWPg25GFVOP2XGY60wmr901sdx5VyWMH9WCijMabu?=
=?us-ascii?Q?u6uMEpS8hpUd/vNPc2XJxoYtwAk1Azf1Wg3NerYwi4XxnLoS7KPsXXo4NXY1?=
=?us-ascii?Q?VHoHBOCZOBKawVbcbDUuNZQeR+Z+lT/xxPLdnlBZGYqtChYhC+sjxeAfLy2E?=
=?us-ascii?Q?aLErFL4cHQ4cr5naFWaYsCAYx7Yxf+2DK+r0CrBnWgCZB7PEfj/DN9FQWVnt?=
=?us-ascii?Q?WeVHUCATg3sUJiAeByogfttAZ1DaBxjBY33+8FxQ3oI1Htu6P5cDxLg6rjgw?=
=?us-ascii?Q?5U3RofeY056VlcRdcLQ7Hsk92TbdMMwgc6G+Js1g06khGZOl5J6zj1C1pSWs?=
=?us-ascii?Q?YC+4bEwgZfFgVi76Y06ixsxL/xi4TTaYy3HsmAqJ++ecqkRgnoAox0qh3dlO?=
=?us-ascii?Q?/9KvmV3DdXye9PPRkh2QPILEc3/wL64RQqLRMxCbXEKWlnhLiWLg0D6FsGgS?=
=?us-ascii?Q?SElL+upBO4QJ3jzss+t9YUdWSThtCBK4rL33YXbfU47TxDgxrBhEKiYwHO3t?=
=?us-ascii?Q?Qsl3jsRRCND43bWj85xVwAjU31rPSyFcsOQ96yEjgewl5sPoqA=3D=3D?=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:CH3PR01MB8470.prod.exchangelabs.com; PTR:; CAT:NONE;
SFS:(13230040)(1800799024)(376014)(366016)(38070700018); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?t0oG1iqLwWwACusfPf2n8ZNzjSJNmohrrYVLoTuJlBtkumMJ3zRYoJIQyehM?=
=?us-ascii?Q?Pwb5SxErVstXNRTn7OHVf5nsWlI6n5FNu/HME3qYEm1Aue6O/ezSw3Ofg03P?=
=?us-ascii?Q?euJHoNAbMhjkg2QB/NO/tXjlUOkirr3tGbRcsLTrTbUWhMXmtmBy04skJy2Y?=
=?us-ascii?Q?+O04+Pl1IUtoij8t8+U5UvTt30Blvn5J/Ki80OC1c9Kvr2UtviWE8sDDj0FV?=
=?us-ascii?Q?slWfNsDgQU69SWjB4PZcUCBn2LOWvGm123fphIQKZiJBvmuS2ifK5QOFqRZI?=
=?us-ascii?Q?v7zjEwjS//XoYwtXirqmt32S5ZWpagKSrq+2w3kjFNQyDKsJQfXyivH/V71K?=
=?us-ascii?Q?Rnkcfi+lbbXFYS3wbQiPNXFHbZaTKtzH0QEZUoAeJjki8TZINPl3mJFYbq6g?=
=?us-ascii?Q?3u4aq3u1Xto2HUaAEmn/pa6VhIbNKiQgkMXdsjjMPrl9jcZOKXpkNv7QFA4r?=
=?us-ascii?Q?K1GR/a/MvfE2pMMhtH9L2iWFENpkZI1q6Wnfla1E8ihv9CatM/3epF3r4fn8?=
=?us-ascii?Q?Ok/vOQDsnWa5bL5HOEO+Z0dF2hRM74uor7FBl03wK69kNaWLktvcahH5IFr5?=
=?us-ascii?Q?Qih1711aEt9LXpVVx1O+Lpahj0Z8c093IKvJKKMBjAMxpgKFPzmM5ImHy5Cl?=
=?us-ascii?Q?EoemBfKV9QBmRyikYILCYwlPAg8Vf6oyRPGh2fxIRpbxhDEw8Zb53tBt/Bgk?=
=?us-ascii?Q?RzHJDHctdStB9X2XqsLzXyhsPS7hSrr4Eco4y37hwnge/yS7x2xokHgMkofa?=
=?us-ascii?Q?4tu+aFDsXOIKbJJae7Kp7B4ChFZP/pm4BXB+GTpRnd5a0vjitQKWKrqI+FUN?=
=?us-ascii?Q?Yro83/xgebO7ph3octvjqjngQ1e2fstX4p4IyNMBf8c60kfwF4ukypxJbomN?=
=?us-ascii?Q?0Cjagw/CY6ug4TTvl4SIYkqy4eUtsc+g/6G7cupI1FW4W544KW8khGiGr7sR?=
=?us-ascii?Q?IaNnJLuAOTAk57PRGvEIRFIDjwyteWkuoOR5h9YwBKZ62PQy68CgeynV2uJ0?=
=?us-ascii?Q?ENxdjzQoW1lU8NmHhuyt1NP96Q//y37g8g2KmRjM5C7YNaVLJ+6kWXiHW5yb?=
=?us-ascii?Q?UOM6kG2lHBT88NuLteZUqi38JHO7rsqlTDNjIg9WJtYbbb70Xfo/V1iTia+o?=
=?us-ascii?Q?oY5q7jzn9bQCRpPTGlbZy6sWlm7M4M1/bYaU7B/BlLS/38uo7elIlaj1WNny?=
=?us-ascii?Q?VH1FRwbNwbwjr5jCQKSCC4hoyr/y9JhVXXVSD350q05ryTbEVydu+0jkan7M?=
=?us-ascii?Q?8spndUDHfRs9rglMHAZX6HZo3iK5RGxUEMtBBUZwwkGMJjvOhGdu17vziBoM?=
=?us-ascii?Q?5bjgqarGYI3iXY8bLRbbMT4ityCOdogjMXGq6jNGyD75FI97dAEMjy/OhmhD?=
=?us-ascii?Q?mJFw03pjFqz+wM0EcjEP4vbl6MDcv9HUCUpmFhPeaz/tyfXOYklm4zF+M/c1?=
=?us-ascii?Q?FtC9MT8nkJ+8z+ZqsU5QsRO5opgpGxfvJ7qHIIg8t9RXmXYUHDWlXQArlKWY?=
=?us-ascii?Q?nBs9ejBz0/5/urbBsRuPqr2j5JrGtu1hTybP13+dFEjjkcEKiZcxg9su5s8+?=
=?us-ascii?Q?tMjeFzm4qxmI0JwUalInMB6CDgP/+OdJ8rKaXTR+?=
Content-Type: multipart/alternative;
boundary="_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_"
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: W50s6heP3VOUruHs/uxOxhFrtSQPrbZLpJkCKxMfkhnswoUV4w/EBbL/QxI0NEW5q51PZKRLssXjVaW3Ve6+X29/894tz/JoXAPVaoANHaVby0wio6tYSNstP5SVvKNXStBbGXPn+8DYnjJwcVOsaih0h7+KpHbKdo2WNyGVGLFFNDvV05tKpdKkRrBOCcI+IDm3yJ/o4JOiK43m8uzENpEiyxyOh5jNdwb+LnGx1vMsG1oEHzpobzqt5IHuU2q9AXpDFMOn4hNZA9xuNYOJHVeoymYAWtSr905fueWXbK/rou8tdSQ8vSFypNTFZaN7BMy7isJxjn2CyxbOMy2lBpebiZuq0Kt1D+0m6wF9wAfb05H1UkSCjcKTn4nyYt0LcHSetrBR1194a2+vpfpRcpsrBPQ6uBsqJPSro0cvw1P0oO0GVpGvXvjg3NAyCnsSogUD8VENujXOytUrt1qhXFsX3eraflZyhhmtmdX2EhF7SF2QYBqccYLENa6ye8933VqDh9gI9pIyEamTa92+kAeFnPaRuun88Ta64e54j2iZ8UL59lxAM+OIMNN0o6JX+ZX/F4BI9rjEEKYVuNwlnCPy+5J4syL/ZjhNmz8CCPSNXLXI1xcx+fMjEFwMuxrN
X-OriginatorOrg: netscout.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CH3PR01MB8470.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e84cd805-2bbd-4f2b-da11-08dccd30267b
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Sep 2024 22:23:06.7262 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: g87px1tm0TYyibqNOBHZQBZB8X5kzOa35vD9+420YK1WUsShp87xsJjGgLpmwyvjP4o4QHNyctgrgJva5tKang==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB8939
X-Proofpoint-ORIG-GUID: XI0fZ0SmSffBc2qnhpdbw0eWdaqcIfwa
X-Proofpoint-GUID: XI0fZ0SmSffBc2qnhpdbw0eWdaqcIfwa
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy
score=0 phishscore=0
suspectscore=0 clxscore=1011 mlxlogscore=999 lowpriorityscore=0
malwarescore=0 mlxscore=0 spamscore=0 impostorscore=0 adultscore=0
bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx
scancount=1 engine=8.21.0-2407110000 definitions=main-2409040170
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
<mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
<mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org
--_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hi Dmitry,
I hope you don't mind if I reach out to you for hugepage memory mapping to =
memseg list issue that intermittently occurs.
We are seeing on occasion the DPDK allocation of hugepages fail.
DPDK version 22.11.2
Oracle 91 OS with kernel 5.14.0-284
The VM is configured with 32GB memory and 8 vCPU cores.
Setup for 2 x 1GB =3D 2GB hugepage total
We dynamically allocate hugepages before our application starts, is not don=
e in grub but done in a bash script.
I turned on EAL debug in our application, which shows debug messages during=
EAL init.
Enable dpdk log EAL in nsprobe.
EAL: lib.eal log level changed from info to debug
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 0 on socket 0
EAL: Detected lcore 2 as core 0 on socket 0
EAL: Detected lcore 3 as core 0 on socket 0
EAL: Detected lcore 4 as core 0 on socket 0
EAL: Detected lcore 5 as core 0 on socket 0
EAL: Detected lcore 6 as core 0 on socket 0
EAL: Detected lcore 7 as core 0 on socket 0
EAL: Maximum logical cores by configuration: 128
EAL: Detected CPU lcores: 8
EAL: Detected NUMA nodes: 1
EAL: Checking presence of .so 'librte_eal.so.23.0'
EAL: Checking presence of .so 'librte_eal.so.23'
EAL: Checking presence of .so 'librte_eal.so'
EAL: Detected static linkage of DPDK
EAL: Ask a virtual area of 0x2000 bytes
EAL: Virtual area found at 0x100000000 (size =3D 0x2000)
[New Thread 0x7fed931ff640 (LWP 287600)]
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
[New Thread 0x7fed929fe640 (LWP 287601)]
EAL: PCI driver net_iavf for device 0000:00:05.0 wants IOVA as 'PA'
EAL: PCI driver net_ice_dcf for device 0000:00:05.0 wants IOVA as 'PA'
EAL: PCI driver net_iavf for device 0000:00:06.0 wants IOVA as 'PA'
EAL: PCI driver net_ice_dcf for device 0000:00:06.0 wants IOVA as 'PA'
EAL: Bus pci wants IOVA as 'PA'
EAL: Bus vdev wants IOVA as 'DC'
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: Module /sys/module/vfio not found! error 2 (No such file or directory)
EAL: VFIO modules not loaded, skipping VFIO support...
EAL: Ask a virtual area of 0x2e000 bytes
EAL: Virtual area found at 0x100002000 (size =3D 0x2e000)
EAL: Setting up physically contiguous memory...
EAL: Setting maximum number of open files to 1024
EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
EAL: Creating 1 segment lists: n_segs:2 socket_id:0 hugepage_sz:1073741824
EAL: Ask a virtual area of 0x1000 bytes
EAL: Virtual area found at 0x100030000 (size =3D 0x1000)
EAL: Memseg list allocated at socket 0, page size 0x100000kB
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x140000000 (size =3D 0x80000000)
EAL: VA reserved for memseg list at 0x140000000, size 80000000
EAL: Creating 1 segment lists: n_segs:1024 socket_id:0 hugepage_sz:2097152
EAL: Ask a virtual area of 0xd000 bytes
EAL: Virtual area found at 0x1c0000000 (size =3D 0xd000)
EAL: Memseg list allocated at socket 0, page size 0x800kB
EAL: Ask a virtual area of 0x80000000 bytes
EAL: Virtual area found at 0x1c0200000 (size =3D 0x80000000)
EAL: VA reserved for memseg list at 0x1c0200000, size 80000000
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0
EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0
EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0
EAL: Requesting 2 pages of size 1024MB from socket 0 <<<< Same on good a=
nd bad
EAL: Attempting to map 1024M on socket 0 <<<< here, on good VM it stat=
es Attempting to map 2048M on socket 0, we have one numa node or 1 socket.
EAL: Allocated 1024M on socket 0 <<<< here, it allo=
cated the 1024M on socket 0.
EAL: Attempting to map 1024M on socket 0 <<<< here, attempts to map la=
st 1G to socket 0.
EAL: Could not find space for memseg. Please increase 1024 and/or 2048 in c=
onfiguration. <<<
EAL: Couldn't remap hugepage files into memseg lists <<<<
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
//good
EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0
EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0
EAL: Requesting 2 pages of size 1024MB from socket 0
EAL: Attempting to map 2048M on socket 0
EAL: Allocated 2048M on socket 0
EAL: Added 2048M to heap on socket 0
Could it be that the hugpages are not contiguous and reboot clears this iss=
ue, not able to confirm.
I tried rebooting the VM 10 times and could not get it to fail.
Tried multiple VMs and sometimes fails.
Seen on VMWare VM and openStack VMs.
Few months back you helped me reduce the VIRT memory of our application.
I added the following before building the dpdk static libraries that are us=
ed in our application build.
#define DPDK_REDUCE_VIRT_8G // is used to select the reduced MSL, etc red=
uctions.
#if defined(DPDK_ORIGINAL) // original, VIRT: 36.6 GB
#define RTE_MAX_MEMSEG_LISTS 128
#define RTE_MAX_MEMSEG_PER_LIST 8192
#define RTE_MAX_MEM_MB_PER_LIST 32768
#define RTE_MAX_MEMSEG_PER_TYPE 32768
#define RTE_MAX_MEM_MB_PER_TYPE 65536
#endif
#if defined(DPDK_REDUCE_VIRT_8G) // VIRT: 5.9 GB
#define RTE_MAX_MEMSEG_LISTS 2
#define RTE_MAX_MEMSEG_PER_LIST 1024
#define RTE_MAX_MEM_MB_PER_LIST 2048
#define RTE_MAX_MEMSEG_PER_TYPE 1024
#defin
e RTE_MAX_MEM_MB_PER_TYPE 2048
#endif
We provide to rte_eal_init() the following arguments:
'app_name, -c0x2, -n4, --socket-mem=3D2048, --legacy-mem, --no-telemetry'
What do you suggest to eliminate this intermittent map to memseg list issue=
?
Thanks,
Ed
--_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#467886" vlink=3D"#96607D" style=3D"word-wrap:=
break-word">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi Dmitry,<o:p></o:p></p>
<p class=3D"MsoNormal">I hope you don’t mind if I reach out to you fo=
r hugepage memory mapping to memseg list issue that intermittently occurs.<=
o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">We are seeing on occasion the DPDK allocation of hug=
epages fail.<o:p></o:p></p>
<p class=3D"MsoNormal">DPDK version 22.11.2<o:p></o:p></p>
<p class=3D"MsoNormal">Oracle 91 OS with kernel 5.14.0-284<o:p></o:p></p>
<p class=3D"MsoNormal">The VM is configured with 32GB memory and 8 vCPU cor=
es.<o:p></o:p></p>
<p class=3D"MsoNormal">Setup for 2 x 1GB =3D 2GB hugepage total<o:p></o:p><=
/p>
<p class=3D"MsoNormal">We dynamically allocate hugepages before our applica=
tion starts, is not done in grub but done in a bash script.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">I turned on EAL debug in our application, which show=
s debug messages during EAL init.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><b>Enable dpdk log EAL in nsprobe.<o:p></o:p></b></p=
>
<p class=3D"MsoNormal">EAL: lib.eal log level changed from info to debug<o:=
p></o:p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 0 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 1 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 2 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 3 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 4 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 5 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 6 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Detected lcore 7 as core 0 on socket 0<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Maximum logical cores by configuration: 128<o:p=
></o:p></p>
<p class=3D"MsoNormal">EAL: Detected CPU lcores: 8<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Detected NUMA nodes: 1<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Checking presence of .so 'librte_eal.so.23.0'<o=
:p></o:p></p>
<p class=3D"MsoNormal">EAL: Checking presence of .so 'librte_eal.so.23'<o:p=
></o:p></p>
<p class=3D"MsoNormal">EAL: Checking presence of .so 'librte_eal.so'<o:p></=
o:p></p>
<p class=3D"MsoNormal">EAL: Detected static linkage of DPDK<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Ask a virtual area of 0x2000 bytes<o:p></o:p></=
p>
<p class=3D"MsoNormal">EAL: Virtual area found at 0x100000000 (size =3D 0x2=
000)<o:p></o:p></p>
<p class=3D"MsoNormal">[New Thread 0x7fed931ff640 (LWP 287600)]<o:p></o:p><=
/p>
<p class=3D"MsoNormal">EAL: Multi-process socket /var/run/dpdk/rte/mp_socke=
t<o:p></o:p></p>
<p class=3D"MsoNormal">[New Thread 0x7fed929fe640 (LWP 287601)]<o:p></o:p><=
/p>
<p class=3D"MsoNormal">EAL: PCI driver net_iavf for device 0000:00:05.0 wan=
ts IOVA as 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: PCI driver net_ice_dcf for device 0000:00:05.0 =
wants IOVA as 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: PCI driver net_iavf for device 0000:00:06.0 wan=
ts IOVA as 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: PCI driver net_ice_dcf for device 0000:00:06.0 =
wants IOVA as 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Bus pci wants IOVA as 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Bus vdev wants IOVA as 'DC'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Selected IOVA mode 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Probing VFIO support...<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Module /sys/module/vfio not found! error 2 (No =
such file or directory)<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: VFIO modules not loaded, skipping VFIO support.=
..<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Ask a virtual area of 0x2e000 bytes<o:p></o:p><=
/p>
<p class=3D"MsoNormal">EAL: Virtual area found at 0x100002000 (size =3D 0x2=
e000)<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Setting up physically contiguous memory...<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Setting maximum number of open files to 1024<o:=
p></o:p></p>
<p class=3D"MsoNormal">EAL: Detected memory type: socket_id:0 hugepage_sz:1=
073741824<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Detected memory type: socket_id:0 hugepage_sz:2=
097152<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Creating 1 segment lists: n_segs:2 socket_id:0 =
hugepage_sz:1073741824<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Ask a virtual area of 0x1000 bytes<o:p></o:p></=
p>
<p class=3D"MsoNormal">EAL: Virtual area found at 0x100030000 (size =3D 0x1=
000)<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Memseg list allocated at socket 0, page size 0x=
100000kB<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Ask a virtual area of 0x80000000 bytes<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Virtual area found at 0x140000000 (size =3D 0x8=
0000000)<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: VA reserved for memseg list at 0x140000000, siz=
e 80000000<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Creating 1 segment lists: n_segs:1024 socket_id=
:0 hugepage_sz:2097152<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Ask a virtual area of 0xd000 bytes<o:p></o:p></=
p>
<p class=3D"MsoNormal">EAL: Virtual area found at 0x1c0000000 (size =3D 0xd=
000)<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Memseg list allocated at socket 0, page size 0x=
800kB<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Ask a virtual area of 0x80000000 bytes<o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Virtual area found at 0x1c0200000 (size =3D 0x8=
0000000)<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: VA reserved for memseg list at 0x1c0200000, siz=
e 80000000<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Trying to obtain current memory policy.<o:p></o=
:p></p>
<p class=3D"MsoNormal">EAL: Setting policy MPOL_PREFERRED for socket 0<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Setting policy MPOL_PREFERRED for socket 0<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Restoring previous memory policy: 0<o:p></o:p><=
/p>
<p class=3D"MsoNormal">EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Requesting 2 pages of size 1024MB from socket 0=
<b><<<< Same on good and bad</b><o:p></o:p><=
/p>
<p class=3D"MsoNormal">EAL: Attempting to map 1024M on socket 0 =
<b><<<< here, on good VM it states Attemptin=
g to map 2048M on socket 0, we have one numa node or 1 socket.</b><o:p></o:=
p></p>
<p class=3D"MsoNormal">EAL: Allocated 1024M on socket 0 &n=
bsp;  =
; <b>
<<<< here, it allocated the 1024M on socket 0.<o:p></o:p></b></=
p>
<p class=3D"MsoNormal">EAL: Attempting to map 1024M on socket 0 =
<b><<<< here, attempts to map last 1G to soc=
ket 0.</b><o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Could not find space for memseg. Please increas=
e 1024 and/or 2048 in configuration.
<b><<<<o:p></o:p></b></p>
<p class=3D"MsoNormal">EAL: Couldn't remap hugepage files into memseg lists=
<b><<<<</b>
<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: FATAL: Cannot init memory<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Cannot init memory<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">//good<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0<o:p>=
</o:p></p>
<p class=3D"MsoNormal">EAL: Requesting 2 pages of size 1024MB from socket 0=
<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Attempting to map 2048M on socket 0<o:p></o:p><=
/p>
<p class=3D"MsoNormal">EAL: Allocated 2048M on socket 0<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Added 2048M to heap on socket 0<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Could it be that the hugpages are not contiguous and=
reboot clears this issue, not able to confirm.<o:p></o:p></p>
<p class=3D"MsoNormal">I tried rebooting the VM 10 times and could not get =
it to fail.<o:p></o:p></p>
<p class=3D"MsoNormal">Tried multiple VMs and sometimes fails.<o:p></o:p></=
p>
<p class=3D"MsoNormal">Seen on VMWare VM and openStack VMs.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Few months back you helped me reduce the VIRT memory=
of our application.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">I added the following before building the dpdk stati=
c libraries that are used in our application build.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">#define DPDK_REDUCE_VIRT_8G // is used t=
o select the reduced MSL, etc reductions.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">#if defined(DPDK_ORIGINAL) // original, VIRT: 36.6 G=
B<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEMSEG_LISTS 128<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEMSEG_PER_LIST 8192<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEM_MB_PER_LIST 32768<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEMSEG_PER_TYPE 32768<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEM_MB_PER_TYPE 65536<o:p></o:p></p>
<p class=3D"MsoNormal">#endif<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">#if defined(DPDK_REDUCE_VIRT_8G) // VIRT: 5.9 =
GB<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEMSEG_LISTS 2<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEMSEG_PER_LIST 1024<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEM_MB_PER_LIST 2048<o:p></o:p></p>
<p class=3D"MsoNormal">#define RTE_MAX_MEMSEG_PER_TYPE 1024<o:p></o:p></p>
<p class=3D"MsoNormal">#defin<o:p></o:p></p>
<p class=3D"MsoNormal">e RTE_MAX_MEM_MB_PER_TYPE 2048<o:p></o:p></p>
<p class=3D"MsoNormal">#endif<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">We provide to rte_eal_init() the following arguments=
:<o:p></o:p></p>
<p class=3D"MsoNormal">‘app_name, -c0x2, -n4, --socket-mem=3D2048, --=
legacy-mem, --no-telemetry’<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">What do you suggest to eliminate this intermittent m=
ap to memseg list issue?<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Thanks,<o:p></o:p></p>
<p class=3D"MsoNormal">Ed<o:p></o:p></p>
</div>
</body>
</html>
--_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_--