From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2DDAA44191 for ; Thu, 5 Sep 2024 00:23:12 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A255F4025C; Thu, 5 Sep 2024 00:23:11 +0200 (CEST) Received: from mx0a-00196b01.pphosted.com (mx0b-00196b01.pphosted.com [67.231.157.166]) by mails.dpdk.org (Postfix) with ESMTP id A2C1B40151 for ; Thu, 5 Sep 2024 00:23:10 +0200 (CEST) Received: from pps.filterd (m0096262.ppops.net [127.0.0.1]) by mx0b-00196b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 484LIk5T011638; Wed, 4 Sep 2024 18:23:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h= cc:content-type:date:from:message-id:mime-version:subject:to; s= netscout.com.09.24.2020; bh=2P7SgI94O+wtH6wcpFb/4BWCYc1/0D2fB0kI TUbI1MY=; b=vK/VVsMsCWzy7RDFGe2LQOR7jOL+syEaXUwfLt0NeSUgEJFLQY9Z PDI9l/pNKiNbZoN7vZNDj9VxfNwpNxY/Vwr1plpxQmFn3byrQV9v/Z07VyiNv1Hs 8z4X4mYDcCgASU0980NLTSN9O5BhHsH57Cp0Ia1v3Q2FMufMNXlEhmqhZknqpGG9 Cg7Oiebe2W/kgytfQg3/+SQAsjeKpN0xZ09Ch+rZSfsDzb3P3kJEz9GOEVM4bkWI RM9vJJofdNCRBv859g4rjdmWM4u8Z2g9NxzChMAaYyaeGLu17xC94xliDEUNpweG WkMY4MtD5+G7IkYkloUxHRLTNjRzpcnL7w== Received: from nam04-mw2-obe.outbound.protection.outlook.com (mail-mw2nam04lp2177.outbound.protection.outlook.com [104.47.73.177]) by mx0b-00196b01.pphosted.com (PPS) with ESMTPS id 41eyb5r26q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 04 Sep 2024 18:23:09 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BzeuiyWfgwLjRnlp+nkVBjoAFhnMT2iGNVVoatG+3BtNFlNU60fF5oqqmFhlUM2BPEG6VF0v2TSDROG9ysgRdUBUgBrCHSorioIScAeJHK8dLpG1etAJyhamRTgXmW2eYksVelfnW3Q8zPeqetQsILdtc4rlekuAQWK7NohKeWri/AYuGfRV1A0jWZx3bhg00FurZYjVGIctT/EYzUdyzia24QA0CAeljg9Y1l9zTAhb6ytpvSloQ7xzC53Ht3dKzGKRZSL8gvwknr3fEHnVmcwo9PtYiwIDQGNxFP3zSYFbrqAXzBbnZ8lLMAg8+pDEk7SstKpqfGzKR2cSGNHlbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2P7SgI94O+wtH6wcpFb/4BWCYc1/0D2fB0kITUbI1MY=; b=RsZXWD5AJUjQd8MxDbkYrQxYV8O2+gU9OqmEsXyHwtpKTzhrdExJizj7Z9EnNrmM2TlCDQW7o/6zXXqZfpA/VGjzahvV3jUsVSpcXNnjsr++LfvMX5kSs2Lrbw5yuTAJj48/QC//MfR/kUJddQSqy5G8vjG7aJJnK48D3qs2k8ihGwSD9R12mpP28guVP7oU8RoQfjFR8QVT8Jg3lDAg/CovjOJQRnOlPWK3z36k+33nVHrfV4p9HBw+ufBxKxrstARNdYq3xTSxf/EIcE12rNF7E8sM2rX23m/KlYGLnnOnPfaec58P+iXZzeS27bcc4I2d80kgnEwgn8MovXjD1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com; dkim=pass header.d=netscout.com; arc=none Received: from CH3PR01MB8470.prod.exchangelabs.com (2603:10b6:610:1a4::21) by CO1PR01MB8939.prod.exchangelabs.com (2603:10b6:303:275::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Wed, 4 Sep 2024 22:23:07 +0000 Received: from CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd]) by CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd%4]) with mapi id 15.20.7918.024; Wed, 4 Sep 2024 22:23:06 +0000 From: "Lombardo, Ed" To: Dmitry Kozlyuk CC: "users@dpdk.org" Subject: hugepage allocation mapping failure Thread-Topic: hugepage allocation mapping failure Thread-Index: Adr/FMQTwzM00r+tRkmT2Kzap3l71g== Date: Wed, 4 Sep 2024 22:23:06 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR01MB8470:EE_|CO1PR01MB8939:EE_ x-ms-office365-filtering-correlation-id: e84cd805-2bbd-4f2b-da11-08dccd30267b x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|1800799024|376014|366016|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?03RXWJgSAOuQYoWiINwvR+rLX5R60m58AjnFdQV/of+YRkMB237GPc1HBM/I?= =?us-ascii?Q?0wBdSp6Voi0KCnqrfkfgasyJk2QX8QR7x4L2bYO+I154jGkWR4pu1ywzTqa9?= =?us-ascii?Q?yPjr4McXBo3/Gh2hdmap/ScXZDI5gs94IdnNo0njVoRyhrnJdSi/kkDYxYjf?= =?us-ascii?Q?+oAD/0TG/F5cfTQcD/HSNEPoLLGLaR7atpBtt5UUwPTCv/djm+elg4s0OoTl?= =?us-ascii?Q?fbvULaiMFDfhZRvmAfzvst7ew0K3X67i1y5MrVFxh4//G18QZTtuWS5ayJMX?= =?us-ascii?Q?OUDJ5Zb89IhZfXXtyIMdRCV/MvnWiw0WX0JepINfA0Urr7rOPkI6xAwdoeqF?= =?us-ascii?Q?iqBzGozuGYAP6uhUei2wzYmxQlTc7PbwZH2lEt+IUEGVk0B7WrvLl7QWzknA?= =?us-ascii?Q?33UkG9EoOeEZuTcbn6YRKdXQAfLqO7l9d8jp8xshm3tNJz7mb0uxR4MA01np?= =?us-ascii?Q?FyQwq29BLbg2oTiAGXf3F6DzFOXvacnHhO80zst4q5EGFFFFy/1aQvJnmCn0?= =?us-ascii?Q?WfRZEpf+mqndzk0bdpslbR1KLmIT1STKmYbrpSZz+DC1iBhCONjcyXm0Bury?= =?us-ascii?Q?XOMnIIgBxcgfk39ZwK0ktZDeOCDJnZUWN3njRbTmhlD15i7PPQEQeRhjVxmW?= =?us-ascii?Q?a3qnSddwqNEbz4byW1n7dq3EwA/X5DWab9F+GIUqfvq0HOZ1mD7zRxAIvi0i?= =?us-ascii?Q?vH1mXoL2udusrTNiZyF0aZEL7sS5Py1hVj68YlZDj11mmAyQ4OME7Zv+Gx62?= =?us-ascii?Q?w6wqXnKJgeEcwOV58ZSk8+YBE80aEc1YXEILTuaczysZc5yk//QsPAPaBeDE?= =?us-ascii?Q?YwAaf6guQu66C8QYTKzEWPg25GFVOP2XGY60wmr901sdx5VyWMH9WCijMabu?= =?us-ascii?Q?u6uMEpS8hpUd/vNPc2XJxoYtwAk1Azf1Wg3NerYwi4XxnLoS7KPsXXo4NXY1?= =?us-ascii?Q?VHoHBOCZOBKawVbcbDUuNZQeR+Z+lT/xxPLdnlBZGYqtChYhC+sjxeAfLy2E?= =?us-ascii?Q?aLErFL4cHQ4cr5naFWaYsCAYx7Yxf+2DK+r0CrBnWgCZB7PEfj/DN9FQWVnt?= =?us-ascii?Q?WeVHUCATg3sUJiAeByogfttAZ1DaBxjBY33+8FxQ3oI1Htu6P5cDxLg6rjgw?= =?us-ascii?Q?5U3RofeY056VlcRdcLQ7Hsk92TbdMMwgc6G+Js1g06khGZOl5J6zj1C1pSWs?= =?us-ascii?Q?YC+4bEwgZfFgVi76Y06ixsxL/xi4TTaYy3HsmAqJ++ecqkRgnoAox0qh3dlO?= =?us-ascii?Q?/9KvmV3DdXye9PPRkh2QPILEc3/wL64RQqLRMxCbXEKWlnhLiWLg0D6FsGgS?= =?us-ascii?Q?SElL+upBO4QJ3jzss+t9YUdWSThtCBK4rL33YXbfU47TxDgxrBhEKiYwHO3t?= =?us-ascii?Q?Qsl3jsRRCND43bWj85xVwAjU31rPSyFcsOQ96yEjgewl5sPoqA=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR01MB8470.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016)(38070700018); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?t0oG1iqLwWwACusfPf2n8ZNzjSJNmohrrYVLoTuJlBtkumMJ3zRYoJIQyehM?= =?us-ascii?Q?Pwb5SxErVstXNRTn7OHVf5nsWlI6n5FNu/HME3qYEm1Aue6O/ezSw3Ofg03P?= =?us-ascii?Q?euJHoNAbMhjkg2QB/NO/tXjlUOkirr3tGbRcsLTrTbUWhMXmtmBy04skJy2Y?= =?us-ascii?Q?+O04+Pl1IUtoij8t8+U5UvTt30Blvn5J/Ki80OC1c9Kvr2UtviWE8sDDj0FV?= =?us-ascii?Q?slWfNsDgQU69SWjB4PZcUCBn2LOWvGm123fphIQKZiJBvmuS2ifK5QOFqRZI?= =?us-ascii?Q?v7zjEwjS//XoYwtXirqmt32S5ZWpagKSrq+2w3kjFNQyDKsJQfXyivH/V71K?= =?us-ascii?Q?Rnkcfi+lbbXFYS3wbQiPNXFHbZaTKtzH0QEZUoAeJjki8TZINPl3mJFYbq6g?= =?us-ascii?Q?3u4aq3u1Xto2HUaAEmn/pa6VhIbNKiQgkMXdsjjMPrl9jcZOKXpkNv7QFA4r?= =?us-ascii?Q?K1GR/a/MvfE2pMMhtH9L2iWFENpkZI1q6Wnfla1E8ihv9CatM/3epF3r4fn8?= =?us-ascii?Q?Ok/vOQDsnWa5bL5HOEO+Z0dF2hRM74uor7FBl03wK69kNaWLktvcahH5IFr5?= =?us-ascii?Q?Qih1711aEt9LXpVVx1O+Lpahj0Z8c093IKvJKKMBjAMxpgKFPzmM5ImHy5Cl?= =?us-ascii?Q?EoemBfKV9QBmRyikYILCYwlPAg8Vf6oyRPGh2fxIRpbxhDEw8Zb53tBt/Bgk?= =?us-ascii?Q?RzHJDHctdStB9X2XqsLzXyhsPS7hSrr4Eco4y37hwnge/yS7x2xokHgMkofa?= =?us-ascii?Q?4tu+aFDsXOIKbJJae7Kp7B4ChFZP/pm4BXB+GTpRnd5a0vjitQKWKrqI+FUN?= =?us-ascii?Q?Yro83/xgebO7ph3octvjqjngQ1e2fstX4p4IyNMBf8c60kfwF4ukypxJbomN?= =?us-ascii?Q?0Cjagw/CY6ug4TTvl4SIYkqy4eUtsc+g/6G7cupI1FW4W544KW8khGiGr7sR?= =?us-ascii?Q?IaNnJLuAOTAk57PRGvEIRFIDjwyteWkuoOR5h9YwBKZ62PQy68CgeynV2uJ0?= =?us-ascii?Q?ENxdjzQoW1lU8NmHhuyt1NP96Q//y37g8g2KmRjM5C7YNaVLJ+6kWXiHW5yb?= =?us-ascii?Q?UOM6kG2lHBT88NuLteZUqi38JHO7rsqlTDNjIg9WJtYbbb70Xfo/V1iTia+o?= =?us-ascii?Q?oY5q7jzn9bQCRpPTGlbZy6sWlm7M4M1/bYaU7B/BlLS/38uo7elIlaj1WNny?= =?us-ascii?Q?VH1FRwbNwbwjr5jCQKSCC4hoyr/y9JhVXXVSD350q05ryTbEVydu+0jkan7M?= =?us-ascii?Q?8spndUDHfRs9rglMHAZX6HZo3iK5RGxUEMtBBUZwwkGMJjvOhGdu17vziBoM?= =?us-ascii?Q?5bjgqarGYI3iXY8bLRbbMT4ityCOdogjMXGq6jNGyD75FI97dAEMjy/OhmhD?= =?us-ascii?Q?mJFw03pjFqz+wM0EcjEP4vbl6MDcv9HUCUpmFhPeaz/tyfXOYklm4zF+M/c1?= =?us-ascii?Q?FtC9MT8nkJ+8z+ZqsU5QsRO5opgpGxfvJ7qHIIg8t9RXmXYUHDWlXQArlKWY?= =?us-ascii?Q?nBs9ejBz0/5/urbBsRuPqr2j5JrGtu1hTybP13+dFEjjkcEKiZcxg9su5s8+?= =?us-ascii?Q?tMjeFzm4qxmI0JwUalInMB6CDgP/+OdJ8rKaXTR+?= Content-Type: multipart/alternative; boundary="_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_" MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: W50s6heP3VOUruHs/uxOxhFrtSQPrbZLpJkCKxMfkhnswoUV4w/EBbL/QxI0NEW5q51PZKRLssXjVaW3Ve6+X29/894tz/JoXAPVaoANHaVby0wio6tYSNstP5SVvKNXStBbGXPn+8DYnjJwcVOsaih0h7+KpHbKdo2WNyGVGLFFNDvV05tKpdKkRrBOCcI+IDm3yJ/o4JOiK43m8uzENpEiyxyOh5jNdwb+LnGx1vMsG1oEHzpobzqt5IHuU2q9AXpDFMOn4hNZA9xuNYOJHVeoymYAWtSr905fueWXbK/rou8tdSQ8vSFypNTFZaN7BMy7isJxjn2CyxbOMy2lBpebiZuq0Kt1D+0m6wF9wAfb05H1UkSCjcKTn4nyYt0LcHSetrBR1194a2+vpfpRcpsrBPQ6uBsqJPSro0cvw1P0oO0GVpGvXvjg3NAyCnsSogUD8VENujXOytUrt1qhXFsX3eraflZyhhmtmdX2EhF7SF2QYBqccYLENa6ye8933VqDh9gI9pIyEamTa92+kAeFnPaRuun88Ta64e54j2iZ8UL59lxAM+OIMNN0o6JX+ZX/F4BI9rjEEKYVuNwlnCPy+5J4syL/ZjhNmz8CCPSNXLXI1xcx+fMjEFwMuxrN X-OriginatorOrg: netscout.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR01MB8470.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: e84cd805-2bbd-4f2b-da11-08dccd30267b X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Sep 2024 22:23:06.7262 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: g87px1tm0TYyibqNOBHZQBZB8X5kzOa35vD9+420YK1WUsShp87xsJjGgLpmwyvjP4o4QHNyctgrgJva5tKang== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB8939 X-Proofpoint-ORIG-GUID: XI0fZ0SmSffBc2qnhpdbw0eWdaqcIfwa X-Proofpoint-GUID: XI0fZ0SmSffBc2qnhpdbw0eWdaqcIfwa X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 phishscore=0 suspectscore=0 clxscore=1011 mlxlogscore=999 lowpriorityscore=0 malwarescore=0 mlxscore=0 spamscore=0 impostorscore=0 adultscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.21.0-2407110000 definitions=main-2409040170 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Dmitry, I hope you don't mind if I reach out to you for hugepage memory mapping to = memseg list issue that intermittently occurs. We are seeing on occasion the DPDK allocation of hugepages fail. DPDK version 22.11.2 Oracle 91 OS with kernel 5.14.0-284 The VM is configured with 32GB memory and 8 vCPU cores. Setup for 2 x 1GB =3D 2GB hugepage total We dynamically allocate hugepages before our application starts, is not don= e in grub but done in a bash script. I turned on EAL debug in our application, which shows debug messages during= EAL init. Enable dpdk log EAL in nsprobe. EAL: lib.eal log level changed from info to debug EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Detected lcore 4 as core 0 on socket 0 EAL: Detected lcore 5 as core 0 on socket 0 EAL: Detected lcore 6 as core 0 on socket 0 EAL: Detected lcore 7 as core 0 on socket 0 EAL: Maximum logical cores by configuration: 128 EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Checking presence of .so 'librte_eal.so.23.0' EAL: Checking presence of .so 'librte_eal.so.23' EAL: Checking presence of .so 'librte_eal.so' EAL: Detected static linkage of DPDK EAL: Ask a virtual area of 0x2000 bytes EAL: Virtual area found at 0x100000000 (size =3D 0x2000) [New Thread 0x7fed931ff640 (LWP 287600)] EAL: Multi-process socket /var/run/dpdk/rte/mp_socket [New Thread 0x7fed929fe640 (LWP 287601)] EAL: PCI driver net_iavf for device 0000:00:05.0 wants IOVA as 'PA' EAL: PCI driver net_ice_dcf for device 0000:00:05.0 wants IOVA as 'PA' EAL: PCI driver net_iavf for device 0000:00:06.0 wants IOVA as 'PA' EAL: PCI driver net_ice_dcf for device 0000:00:06.0 wants IOVA as 'PA' EAL: Bus pci wants IOVA as 'PA' EAL: Bus vdev wants IOVA as 'DC' EAL: Selected IOVA mode 'PA' EAL: Probing VFIO support... EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) EAL: VFIO modules not loaded, skipping VFIO support... EAL: Ask a virtual area of 0x2e000 bytes EAL: Virtual area found at 0x100002000 (size =3D 0x2e000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 1024 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 EAL: Detected memory type: socket_id:0 hugepage_sz:2097152 EAL: Creating 1 segment lists: n_segs:2 socket_id:0 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x1000 bytes EAL: Virtual area found at 0x100030000 (size =3D 0x1000) EAL: Memseg list allocated at socket 0, page size 0x100000kB EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x140000000 (size =3D 0x80000000) EAL: VA reserved for memseg list at 0x140000000, size 80000000 EAL: Creating 1 segment lists: n_segs:1024 socket_id:0 hugepage_sz:2097152 EAL: Ask a virtual area of 0xd000 bytes EAL: Virtual area found at 0x1c0000000 (size =3D 0xd000) EAL: Memseg list allocated at socket 0, page size 0x800kB EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x1c0200000 (size =3D 0x80000000) EAL: VA reserved for memseg list at 0x1c0200000, size 80000000 EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0 EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0 EAL: Requesting 2 pages of size 1024MB from socket 0 <<<< Same on good a= nd bad EAL: Attempting to map 1024M on socket 0 <<<< here, on good VM it stat= es Attempting to map 2048M on socket 0, we have one numa node or 1 socket. EAL: Allocated 1024M on socket 0 <<<< here, it allo= cated the 1024M on socket 0. EAL: Attempting to map 1024M on socket 0 <<<< here, attempts to map la= st 1G to socket 0. EAL: Could not find space for memseg. Please increase 1024 and/or 2048 in c= onfiguration. <<< EAL: Couldn't remap hugepage files into memseg lists <<<< EAL: FATAL: Cannot init memory EAL: Cannot init memory //good EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0 EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0 EAL: Requesting 2 pages of size 1024MB from socket 0 EAL: Attempting to map 2048M on socket 0 EAL: Allocated 2048M on socket 0 EAL: Added 2048M to heap on socket 0 Could it be that the hugpages are not contiguous and reboot clears this iss= ue, not able to confirm. I tried rebooting the VM 10 times and could not get it to fail. Tried multiple VMs and sometimes fails. Seen on VMWare VM and openStack VMs. Few months back you helped me reduce the VIRT memory of our application. I added the following before building the dpdk static libraries that are us= ed in our application build. #define DPDK_REDUCE_VIRT_8G // is used to select the reduced MSL, etc red= uctions. #if defined(DPDK_ORIGINAL) // original, VIRT: 36.6 GB #define RTE_MAX_MEMSEG_LISTS 128 #define RTE_MAX_MEMSEG_PER_LIST 8192 #define RTE_MAX_MEM_MB_PER_LIST 32768 #define RTE_MAX_MEMSEG_PER_TYPE 32768 #define RTE_MAX_MEM_MB_PER_TYPE 65536 #endif #if defined(DPDK_REDUCE_VIRT_8G) // VIRT: 5.9 GB #define RTE_MAX_MEMSEG_LISTS 2 #define RTE_MAX_MEMSEG_PER_LIST 1024 #define RTE_MAX_MEM_MB_PER_LIST 2048 #define RTE_MAX_MEMSEG_PER_TYPE 1024 #defin e RTE_MAX_MEM_MB_PER_TYPE 2048 #endif We provide to rte_eal_init() the following arguments: 'app_name, -c0x2, -n4, --socket-mem=3D2048, --legacy-mem, --no-telemetry' What do you suggest to eliminate this intermittent map to memseg list issue= ? Thanks, Ed --_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Dmitry,

I hope you don’t mind if I reach out to you fo= r hugepage memory mapping to memseg list issue that intermittently occurs.<= o:p>

 

We are seeing on occasion the DPDK allocation of hug= epages fail.

DPDK version 22.11.2

Oracle 91 OS with kernel 5.14.0-284

The VM is configured with 32GB memory and 8 vCPU cor= es.

Setup for 2 x 1GB =3D 2GB hugepage total<= /p>

We dynamically allocate hugepages before our applica= tion starts, is not done in grub but done in a bash script.

 

I turned on EAL debug in our application, which show= s debug messages during EAL init.

 

 

Enable dpdk log EAL in nsprobe.

EAL: lib.eal log level changed from info to debug

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 0 on socket 0

EAL: Detected lcore 2 as core 0 on socket 0

EAL: Detected lcore 3 as core 0 on socket 0

EAL: Detected lcore 4 as core 0 on socket 0

EAL: Detected lcore 5 as core 0 on socket 0

EAL: Detected lcore 6 as core 0 on socket 0

EAL: Detected lcore 7 as core 0 on socket 0

EAL: Maximum logical cores by configuration: 128

EAL: Detected CPU lcores: 8

EAL: Detected NUMA nodes: 1

EAL: Checking presence of .so 'librte_eal.so.23.0'

EAL: Checking presence of .so 'librte_eal.so.23'

EAL: Checking presence of .so 'librte_eal.so'

EAL: Detected static linkage of DPDK

EAL: Ask a virtual area of 0x2000 bytes

EAL: Virtual area found at 0x100000000 (size =3D 0x2= 000)

[New Thread 0x7fed931ff640 (LWP 287600)]<= /p>

EAL: Multi-process socket /var/run/dpdk/rte/mp_socke= t

[New Thread 0x7fed929fe640 (LWP 287601)]<= /p>

EAL: PCI driver net_iavf for device 0000:00:05.0 wan= ts IOVA as 'PA'

EAL: PCI driver net_ice_dcf for device 0000:00:05.0 = wants IOVA as 'PA'

EAL: PCI driver net_iavf for device 0000:00:06.0 wan= ts IOVA as 'PA'

EAL: PCI driver net_ice_dcf for device 0000:00:06.0 = wants IOVA as 'PA'

EAL: Bus pci wants IOVA as 'PA'

EAL: Bus vdev wants IOVA as 'DC'

EAL: Selected IOVA mode 'PA'

EAL: Probing VFIO support...

EAL: Module /sys/module/vfio not found! error 2 (No = such file or directory)

EAL: VFIO modules not loaded, skipping VFIO support.= ..

EAL: Ask a virtual area of 0x2e000 bytes<= /p>

EAL: Virtual area found at 0x100002000 (size =3D 0x2= e000)

EAL: Setting up physically contiguous memory...=

EAL: Setting maximum number of open files to 1024

EAL: Detected memory type: socket_id:0 hugepage_sz:1= 073741824

EAL: Detected memory type: socket_id:0 hugepage_sz:2= 097152

EAL: Creating 1 segment lists: n_segs:2 socket_id:0 = hugepage_sz:1073741824

EAL: Ask a virtual area of 0x1000 bytes

EAL: Virtual area found at 0x100030000 (size =3D 0x1= 000)

EAL: Memseg list allocated at socket 0, page size 0x= 100000kB

EAL: Ask a virtual area of 0x80000000 bytes

EAL: Virtual area found at 0x140000000 (size =3D 0x8= 0000000)

EAL: VA reserved for memseg list at 0x140000000, siz= e 80000000

EAL: Creating 1 segment lists: n_segs:1024 socket_id= :0 hugepage_sz:2097152

EAL: Ask a virtual area of 0xd000 bytes

EAL: Virtual area found at 0x1c0000000 (size =3D 0xd= 000)

EAL: Memseg list allocated at socket 0, page size 0x= 800kB

EAL: Ask a virtual area of 0x80000000 bytes

EAL: Virtual area found at 0x1c0200000 (size =3D 0x8= 0000000)

EAL: VA reserved for memseg list at 0x1c0200000, siz= e 80000000

EAL: Trying to obtain current memory policy.

EAL: Setting policy MPOL_PREFERRED for socket 0=

EAL: Setting policy MPOL_PREFERRED for socket 0=

EAL: Restoring previous memory policy: 0<= /p>

EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0=

EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0=

EAL: Requesting 2 pages of size 1024MB from socket 0=     <<<< Same on good and bad<= /p>

EAL: Attempting to map 1024M on socket 0  =     <<<< here, on good VM it states Attemptin= g to map 2048M on socket 0, we have one numa node or 1 socket.

EAL: Allocated 1024M on socket 0   &n= bsp;            = ;         <<<< here, it allocated the 1024M on socket 0.

EAL: Attempting to map 1024M on socket 0  =     <<<< here, attempts to map last 1G to soc= ket 0.

EAL: Could not find space for memseg. Please increas= e 1024 and/or 2048 in configuration.   <<<

EAL: Couldn't remap hugepage files into memseg lists=       <<<<

EAL: FATAL: Cannot init memory

EAL: Cannot init memory

 

 

//good

EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0=

EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0=

EAL: Requesting 2 pages of size 1024MB from socket 0=

EAL: Attempting to map 2048M on socket 0<= /p>

EAL: Allocated 2048M on socket 0

EAL: Added 2048M to heap on socket 0

 

Could it be that the hugpages are not contiguous and= reboot clears this issue, not able to confirm.

I tried rebooting the VM 10 times and could not get = it to fail.

Tried multiple VMs and sometimes fails.

Seen on VMWare VM and openStack VMs.

 

Few months back you helped me reduce the VIRT memory= of our application.

 

I added the following before building the dpdk stati= c libraries that are used in our application build.

 

#define DPDK_REDUCE_VIRT_8G   // is used t= o select the reduced MSL, etc reductions.

 

#if defined(DPDK_ORIGINAL) // original, VIRT: 36.6 G= B

#define RTE_MAX_MEMSEG_LISTS 128

#define RTE_MAX_MEMSEG_PER_LIST 8192

#define RTE_MAX_MEM_MB_PER_LIST 32768

#define RTE_MAX_MEMSEG_PER_TYPE 32768

#define RTE_MAX_MEM_MB_PER_TYPE 65536

#endif

 

#if defined(DPDK_REDUCE_VIRT_8G)  // VIRT: 5.9 = GB

#define RTE_MAX_MEMSEG_LISTS 2

#define RTE_MAX_MEMSEG_PER_LIST 1024

#define RTE_MAX_MEM_MB_PER_LIST 2048

#define RTE_MAX_MEMSEG_PER_TYPE 1024

#defin

e RTE_MAX_MEM_MB_PER_TYPE 2048

#endif

 

We provide to rte_eal_init() the following arguments= :

‘app_name, -c0x2, -n4, --socket-mem=3D2048, --= legacy-mem, --no-telemetry’

 

What do you suggest to eliminate this intermittent m= ap to memseg list issue?

 

Thanks,

Ed

--_000_CH3PR01MB847096E58EAF53571CE9115B8F9C2CH3PR01MB8470prod_--