From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 00D084591E for ; Fri, 6 Sep 2024 15:42:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CDAFC42E75; Fri, 6 Sep 2024 15:42:55 +0200 (CEST) Received: from mx0a-00196b01.pphosted.com (mx0a-00196b01.pphosted.com [67.231.149.170]) by mails.dpdk.org (Postfix) with ESMTP id 835A94025D for ; Fri, 6 Sep 2024 15:42:53 +0200 (CEST) Received: from pps.filterd (m0096263.ppops.net [127.0.0.1]) by mx0a-00196b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 486Cf0Sd023771 for ; Fri, 6 Sep 2024 09:42:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h= content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to; s=netscout.com.09.24.2020; bh=QxyLlmvbhE MIm3vDG1SpuhHsmKvq1Lxs+2C908H/j+I=; b=vqX8yDuWlbw7CJvUpj2t0gZ/F0 Mak8HV3uW7US1dy7wDjxbNC8sexC6kydAUk4qIArEcQY5qAooJynur4gIrMlVUMz /vxMhShtsoGzX1H2l+AT30MpGtoQaaL3XKXO6rVvSnAiGIKqsbqHsaTKcEUZdkEa rJCqby1fJtSGkAtwXw8nSlIE3uARDZSk033w5SnGGxR33UbcW/jOQOO+Yt1xv3o/ Hhjg1QxSB04isPUXuDAGzkH7XfIB7Rvg/DEKvJcybPjKTo8UMcvmkfvUXNY1cBFj SQgQVsAOfYIy5p0Af5a5tZSAKSGhfItqYO1xlWgAngyc0w93FQpGQNQW1a+g== Received: from dm1pr04cu001.outbound.protection.outlook.com (mail-centralusazlp17010003.outbound.protection.outlook.com [40.93.13.3]) by mx0a-00196b01.pphosted.com (PPS) with ESMTPS id 41fhwh0ypm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 06 Sep 2024 09:42:52 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kDr/p9XlVT6/Jfdr7UtierckJ0sUevF08M7bxlDcCr9fk/LQbttWVsqr9bPyascDBxX1fgZG9+9UUYv/saVBmVwqizp39/X879dj2wkmvUoF1TTUMa1dxL/qeT4ERcQ6GsjRA+F7le4VivrSgRWovlXrC9EPSiJ97j3LxFYVZ15VCjv2mYXcbntEL/1/XmIbwUOzd7+KrHyM+ftO23yJJqo6wcKtZ1jQ9d5Om3lMuu4g0W4xHl4Y64rwX/RH4rtgIZiicm/QGLNJZKRTKNLdyFTvvZMG934UIqwbu9RJZEAvnN0HffR6WTrxphW+gKKwAUUj10XJPIK+C+rs/q0qDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QxyLlmvbhEMIm3vDG1SpuhHsmKvq1Lxs+2C908H/j+I=; b=N6/Lk7uV41xBXo5qUWxLM4vYEivCZJb8hqqW9w7NW9gzaWHZjOB0D+/gD6hFt2KsuvHZ7d/BGTsQ+F9P4Qg8pQ5S4eTBj+lFFcBZEeyuuI/45Ws1op83gckr0oS/HHjovTAK/6jkzOjxzPygL5duf4fBY5qUNl7vvWEzQdPso5epCQ73rjsUtLQxh9dx4WKheyFt0r/rQFMsxlNYaMc0ljx5AdmqPf6jEGchNGz+NBhnf3+AP/r9/fs/5dWCGcq2C8+8uUNTZa33yqIGiH9nX2CfeaQu2Y6g6S3bILOCtPDifeURzvY+Q8S6ESAV+fr7+oYKJ2yfoWmlOb2Wc0ml3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com; dkim=pass header.d=netscout.com; arc=none Received: from CH3PR01MB8470.prod.exchangelabs.com (2603:10b6:610:1a4::21) by MN0PR01MB7634.prod.exchangelabs.com (2603:10b6:208:376::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Fri, 6 Sep 2024 13:42:45 +0000 Received: from CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd]) by CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd%4]) with mapi id 15.20.7918.024; Fri, 6 Sep 2024 13:42:44 +0000 From: "Lombardo, Ed" To: "users@dpdk.org" Subject: hugepage mapping to memseg failure Thread-Topic: hugepage mapping to memseg failure Thread-Index: AQHbAGKn/2ECToe57EGVE/xXUIj48A== Date: Fri, 6 Sep 2024 13:42:44 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR01MB8470:EE_|MN0PR01MB7634:EE_ x-ms-office365-filtering-correlation-id: a9f28b48-ab15-42f3-7a99-08dcce79c982 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|376014|1800799024|366016|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?Q9MV2KjgnHp3O/6YwsqUniuNQlEQ+gN5wvteFqGpMbUhdevBvTUOtcngL6bI?= =?us-ascii?Q?888+IuXFM7oSBPmPuXTWG6bE6/8WwAW3MnvHO2VeVzWI9iizs01r/SiOg1Bp?= =?us-ascii?Q?Iz14ou4iN0JRXKiV/jQaA2DcwAAiQF2z2Mp6qOkq5zCMtK1fOMGKyBGcjP33?= =?us-ascii?Q?HjkzIGSbXJREnD9kvZf53X25mzoXxC/IA8JtW7dwNcgBqBTo8AQ1EBv7yDDr?= =?us-ascii?Q?ApksQPs1TylMoLEj0/m6M1cp7fMA9vIXYRyedMRvfNbIRB3PeUsBljXP/wbb?= =?us-ascii?Q?adOVcDPVdQWeq/J9mytvpF101SnSTgTTAqDRk7wPKmoaAEjXtp4lghY83SMI?= =?us-ascii?Q?BTP9FPE6GWN2m0eJidniRTMmicZBrUbXz/3xjgLbpIUaOBJTPKegzQ7GdUjT?= =?us-ascii?Q?ll2GQXwazs3quN8A4pjt8JdRPYU8ZuTT3lKCxuJ1/8MKFSm6/NKQrS7RPzki?= =?us-ascii?Q?s7vT+MAGyL4/fao5dELRJwzPO8gg49yb9VuhpXBAVPORQfmwXMYv58AIHGWb?= =?us-ascii?Q?RgSNOxDYxFMIz3+JkWkHBeO5ozzuEtXHgtPHr4H2erCEKoVIiA9kJKxCsrzH?= =?us-ascii?Q?7BSMRCBSOc/CMcg1DkNRVCg5kGtUBXv/Nc3UwLFr0/CEuIA/gVnpwufr7Rtj?= =?us-ascii?Q?qHXOG1MfUjJIhEFTy/Nvr0L7aSZoqiK+FZyrvsU/l3KzzVRNw5bsWy/6OAGM?= =?us-ascii?Q?MKDEGk02IEr6+pr21q8Yj3IJI2tabU8wRtC5Z66qPEpMVK1OcLr/l+AWIQQl?= =?us-ascii?Q?zvt5sqyVtJtBsF1MPlzutktIaIlybI9aUBCdj2julIf5eegpvp99YW6JzHDp?= =?us-ascii?Q?Tl6xkkJ2COlNnvStj7QabpkVttXDOMUbTJucUjaceGqBE6qgYZYfCSzKjG/j?= =?us-ascii?Q?clYGZXMUTUdA5N0Thk/G98t0XhfdtMCddjUQAxBFRoWbDa/fTzQ+zn2d0pCb?= =?us-ascii?Q?cAECaNJhmpfCdQeGMhTya+xz6reTMa5I7HdrNDiOJlGICL05/6Hnvjh67dE8?= =?us-ascii?Q?Emrq1MW4jrWBbAy0ezA4vjexoDdMJu9cgnLqSUCUMM0ZRU7YcrsBSZwOVMVc?= =?us-ascii?Q?IgCJJXc7SJK72F8FhkgGcWMT0qrpglMepDK55w6phOBhLYB7oK6j3As/gWhc?= =?us-ascii?Q?brL1jFAMH8bOeDjrbZWZ41Esfx5brO/NgXuezyuHeyF6zT0X0ZOewFQC7g1Q?= =?us-ascii?Q?e1lX8wH/BfKUCjSFRLrQtcZO4iqxjTIm1vftp1dWvtQE+g4RpMx5iG2r5pFU?= =?us-ascii?Q?AbRMICci0P7XcIlTNfWkSSKyP/TKhyVSvA+HV9O/M2kOEcloYcGUDlBPpnlC?= =?us-ascii?Q?t5uGOI3H7h9FDNnhW29Mya5mfIFfR0JUvf3lDdYtL/C2vMiyHhLWUNnEPdwk?= =?us-ascii?Q?iBeSJ48Y6EuGKapFkBUBeTvXZZnU4MiSuETuFEZyfCWtijIlJA=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR01MB8470.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(366016)(38070700018); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?PBjtXdOGt4lA32U0LC7HA+JbMIajRe0caSsWOSf1RzOHpDBigHhdHcCW3rvQ?= =?us-ascii?Q?MibVAGMgJ/66Tw1Z1WJ4ZWvBTRH+ah2hrJ9Q9gHaI+MRhpo+VyWrtMONZWg/?= =?us-ascii?Q?VF7lGDPirPujVMVNPyia33Mi9H3ZUJNeldNet5vRFpwbYCKNMhK4LSfptoBM?= =?us-ascii?Q?IPZbK4qYLdTnfVAZCwyZeyqLK4BoIzvUKUOtbmSDdd0Av67Z9UQ7sercsGkW?= =?us-ascii?Q?ZvuGtEo5LP8/xYDRKD4gmmSToB5pt+eLnudIg57uuQMfcMfwFeYmoWSUTLWP?= =?us-ascii?Q?+zdrsqhO2modbeHllAi8FIqvacX2ZPxrbXjoU1qpOsW2LAXAWUZbotiqcOxv?= =?us-ascii?Q?iyJTxWAiBAUSqurClXEsruYcVJJAy07wVL99HTdUITXcwjX/T6r9amPRU9Vh?= =?us-ascii?Q?yokr0eoW+Lm2FO/lCdnNn65KuBwEN3/KGacd1LGtqdLNPtPu3jjqTTwAfpwj?= =?us-ascii?Q?LU3A3kXmJk8pvyQ/py+uWgKEXZ0n+UkAFJkkvOf95OFP0NhWo/FNhEfi0I/x?= =?us-ascii?Q?4mk1KuCuaHR2o4GWFkS97I58XrSJYFbUplA9NdLQV3VuReM/aoceP2K2i+M3?= =?us-ascii?Q?NDWSYSWAgwfC8K7ZuMBHFPpmL4Qa8KQzD+m8u0MSEGdXYbAGW7Q8MpswOCsb?= =?us-ascii?Q?rFzcXiq6PE1A92rpCp0FXjQ15bON1jUj0vb7dOU1WAi4eSklsRmU9axs2A80?= =?us-ascii?Q?Dg4+Ttip0EwzA/Rfm+AnoW87egaD5syRse8cJF3rsoIGxVWTj6tsZZNSJEqG?= =?us-ascii?Q?sPxppnmyMd5JKc/7nZRdaEeBX2ShpufZxqbedxSPS2A1QYkVjv9DqMEo+1RU?= =?us-ascii?Q?YTSimv15/ZMFTl+pW4tivfKB1mkAKKKLsVq9Sv/AQVloPlCZ0DJPtVZtU1cR?= =?us-ascii?Q?Z39apFYOg2tt0G/l2bv4PYvhRQ9BUShEeGDlDmPNup0qYesSDJIL5kGllSgs?= =?us-ascii?Q?4XFIO4brOZlXjSsVvJOSolRXBszIn4hjO+4JrtbifK8UyfhXknZdRfdxwO+z?= =?us-ascii?Q?AynWKIvklIyAiD/pi6inm73VOFZt+ilDs8G501J+XUM9BSipWDBXtv5suQmC?= =?us-ascii?Q?grkNMGa1qBOTgXxEmxtbxr6+v/7fnAsHBQCRULu3l7MoXF5WDINz51K2W8kW?= =?us-ascii?Q?DHAbc3b9LSCsjTwkwKFIJRgFNXQ1rX0JOzdRzECXBsA/BAG5ocF7F+0eDp7w?= =?us-ascii?Q?jPkoADH889GXl42H9qDRIKzvRIdyQSAQNpNQD9PDbyoSegn/4Y8qfejPZUsn?= =?us-ascii?Q?dQmUQnmfaW4BV5Wn9ZNfo3APLCpMbNblh7kxA41MFfq/Hhza5/ouXtYm9tCC?= =?us-ascii?Q?ho2BItBawiGtv8DMB+i+lafKO9sdcWjuJHivrpDaHWFdUbmDWuAw3nxVEihJ?= =?us-ascii?Q?MyOladuh+xKhiD+tDbMTBapYfdbg+VasiY6wA7+d5t7oxicUylCRf/rb1ryY?= =?us-ascii?Q?/sa/keYbIaKgOV9LfaPGE2R7m9hJzcjNbRx6PophNIDYmBdIIQF3ixEQ6TyL?= =?us-ascii?Q?yDlo8JR79SXZdm47xqtJsi36TndI7g6BEdiH5I5zdStnhK8+J5ivm60EJ3Si?= =?us-ascii?Q?sKqsj5nZzl3g+B7vkkRsh5DV1IDg9iQ03gS9t1NU?= Content-Type: multipart/alternative; boundary="_000_CH3PR01MB8470B651CE8BDCB4F0D32F7D8F9E2CH3PR01MB8470prod_" MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: OsHtjyKzVH6dqG8JPvh88aNRJG64qCGZ5PDmm7/YfiPmK4xlG3WgdLSlAYhHaxq0jFl3Po88exVpHUiCjuUB3Ah/x8L0O8HTzrIMCMpziLfDRglha5ua/EUM8KKs+J2JBS+K2K4k5fhmgykXrLbvctA+sQKcvjSE5nuS6P07+qg4uUcviwd1h3g22U8yW5iELm/WPHWZQBsuHKsRfHmnCFGKQ92LD2KVnwp4uufFODoAYS5rJbv+nAyupLpHixr48oSAspssHOjLJMwYpeDJqF/nOaN0MkYMT8CmXe1HwMkrzLUWesZcin39KsRTvFG2mDXBUiPVy5JFN+0QAulr5Y+BvKFP/SU7phm8rD/KsGq2MtwgGxLxEqr1kdQB7ZFCfvHyfO6S/QYgLRD2o18Tv/IEpDY1ari+dPfbXrFBWSjTLr/3/ZHOqiFQB9etXHGt+NWaPTfLPps2NhJNF+ZgXxAFXuRdh29vilif+Ix0b0UmToSH6ELm4VgZ/1zQz/CDI5fXA+JDnzowsyPeGRur8h+ZBWKiJzRNQU9uAfIxLtOAuF59Ch+9LxZj9JkwrePZAurHvaAKItVJuVdgblFEkqVrb0t6eiQrS8lGpYVivuspVcGninOwfZ5ltFKgMaU0 X-OriginatorOrg: netscout.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR01MB8470.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: a9f28b48-ab15-42f3-7a99-08dcce79c982 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2024 13:42:44.6739 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yUPzDZ/QfOzrae2i6te8BEaXDktciY6fwOROri9lSZ50erXf/81W0P/2m4lSBEZmJNEHi9H6m9qIk85OqKnxgw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR01MB7634 X-Proofpoint-ORIG-GUID: qdgtkNQeFnNCW1qFeV-pRFy8kuV4DC6j X-Proofpoint-GUID: qdgtkNQeFnNCW1qFeV-pRFy8kuV4DC6j X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 mlxscore=0 suspectscore=0 malwarescore=0 impostorscore=0 mlxlogscore=999 bulkscore=0 lowpriorityscore=0 adultscore=0 phishscore=0 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.21.0-2408220000 definitions=main-2409060099 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --_000_CH3PR01MB8470B651CE8BDCB4F0D32F7D8F9E2CH3PR01MB8470prod_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, I hope someone can help, I am seeing DPDK EAL initialization intermittently= fail when hugepage memory is mapped to memseg. DPDK version 22.11.2 Oracle 91 OS with kernel 5.14.0-284 The VM is configured with 32GB memory and 8 vCPU cores. Setup for 2 x 1GB =3D 2GB hugepage total We dynamically allocate hugepages before our application starts, is not don= e in grub but done in a bash script. I turned on EAL debug in our application, which shows debug messages during= EAL init. Enable dpdk log EAL in nsprobe. EAL: lib.eal log level changed from info to debug EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Detected lcore 4 as core 0 on socket 0 EAL: Detected lcore 5 as core 0 on socket 0 EAL: Detected lcore 6 as core 0 on socket 0 EAL: Detected lcore 7 as core 0 on socket 0 EAL: Maximum logical cores by configuration: 128 EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Checking presence of .so 'librte_eal.so.23.0' EAL: Checking presence of .so 'librte_eal.so.23' EAL: Checking presence of .so 'librte_eal.so' EAL: Detected static linkage of DPDK EAL: Ask a virtual area of 0x2000 bytes EAL: Virtual area found at 0x100000000 (size =3D 0x2000) [New Thread 0x7fed931ff640 (LWP 287600)] EAL: Multi-process socket /var/run/dpdk/rte/mp_socket [New Thread 0x7fed929fe640 (LWP 287601)] EAL: PCI driver net_iavf for device 0000:00:05.0 wants IOVA as 'PA' EAL: PCI driver net_ice_dcf for device 0000:00:05.0 wants IOVA as 'PA' EAL: PCI driver net_iavf for device 0000:00:06.0 wants IOVA as 'PA' EAL: PCI driver net_ice_dcf for device 0000:00:06.0 wants IOVA as 'PA' EAL: Bus pci wants IOVA as 'PA' EAL: Bus vdev wants IOVA as 'DC' EAL: Selected IOVA mode 'PA' EAL: Probing VFIO support... EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) EAL: VFIO modules not loaded, skipping VFIO support... EAL: Ask a virtual area of 0x2e000 bytes EAL: Virtual area found at 0x100002000 (size =3D 0x2e000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 1024 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824 EAL: Detected memory type: socket_id:0 hugepage_sz:2097152 EAL: Creating 1 segment lists: n_segs:2 socket_id:0 hugepage_sz:1073741824 EAL: Ask a virtual area of 0x1000 bytes EAL: Virtual area found at 0x100030000 (size =3D 0x1000) EAL: Memseg list allocated at socket 0, page size 0x100000kB EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x140000000 (size =3D 0x80000000) EAL: VA reserved for memseg list at 0x140000000, size 80000000 EAL: Creating 1 segment lists: n_segs:1024 socket_id:0 hugepage_sz:2097152 EAL: Ask a virtual area of 0xd000 bytes EAL: Virtual area found at 0x1c0000000 (size =3D 0xd000) EAL: Memseg list allocated at socket 0, page size 0x800kB EAL: Ask a virtual area of 0x80000000 bytes EAL: Virtual area found at 0x1c0200000 (size =3D 0x80000000) EAL: VA reserved for memseg list at 0x1c0200000, size 80000000 EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 // bad case EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0 EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0 EAL: Requesting 2 pages of size 1024MB from socket 0 <<<< Same on good a= nd bad EAL: Attempting to map 1024M on socket 0 <<<< here, on good VM it stat= es Attempting to map 2048M on socket 0, we have one numa node or 1 socket. EAL: Allocated 1024M on socket 0 <<<< here, it allo= cated the 1024M on socket 0. EAL: Attempting to map 1024M on socket 0 <<<< here, attempts to map la= st 1G to socket 0. EAL: Could not find space for memseg. Please increase 1024 and/or 2048 in c= onfiguration. EAL: Couldn't remap hugepage files into memseg lists EAL: FATAL: Cannot init memory EAL: Cannot init memory //good case EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0 EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0 EAL: Requesting 2 pages of size 1024MB from socket 0 EAL: Attempting to map 2048M on socket 0 EAL: Allocated 2048M on socket 0 EAL: Added 2048M to heap on socket 0 I tried rebooting the VM 10 times and could not get it to fail. Tried multiple VMs and sometimes fails. Stopping the application and restarting does not clear the error. Seen on VMWare VM and openStack VMs. I modified the DPDK #defines to reduce the VIRT memory of our application. I added the following before building the dpdk static libraries that are us= ed in our application build. #define DPDK_REDUCE_VIRT_8G // is used to select the reduced MSL, etc red= uctions. #if defined(DPDK_ORIGINAL) // original, VIRT: 36.6 GB #define RTE_MAX_MEMSEG_LISTS 128 #define RTE_MAX_MEMSEG_PER_LIST 8192 #define RTE_MAX_MEM_MB_PER_LIST 32768 #define RTE_MAX_MEMSEG_PER_TYPE 32768 #define RTE_MAX_MEM_MB_PER_TYPE 65536 #endif #if defined(DPDK_REDUCE_VIRT_8G) // VIRT: 5.9 GB #define RTE_MAX_MEMSEG_LISTS 2 #define RTE_MAX_MEMSEG_PER_LIST 1024 #define RTE_MAX_MEM_MB_PER_LIST 2048 #define RTE_MAX_MEMSEG_PER_TYPE 1024 #defin #define RTE_MAX_MEM_MB_PER_TYPE 2048 #endif The rte_eal_init() arguments are: 'app_name, -c0x2, -n4, --socket-mem=3D2048, --legacy-mem, --no-telemetry' Could it be that the hugpages are not contiguous and reboot clears this iss= ue, not able to confirm. Need help to eliminate this intermittent map to memseg list issue? Thanks, Ed --_000_CH3PR01MB8470B651CE8BDCB4F0D32F7D8F9E2CH3PR01MB8470prod_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

I hope someone can help, I am seeing DPDK EAL initia= lization intermittently fail when hugepage memory is mapped to memseg.=

 

DPDK version 22.11.2

Oracle 91 OS with kernel 5.14.0-284

The VM is configured with 32GB memory and 8 vCPU cor= es.

Setup for 2 x 1GB =3D 2GB hugepage total<= /p>

We dynamically allocate hugepages before our applica= tion starts, is not done in grub but done in a bash script.

 

I turned on EAL debug in our application, which show= s debug messages during EAL init.

 

 

Enable dpdk log EAL in nsprobe.

EAL: lib.eal log level changed from info to debug

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 0 on socket 0

EAL: Detected lcore 2 as core 0 on socket 0

EAL: Detected lcore 3 as core 0 on socket 0

EAL: Detected lcore 4 as core 0 on socket 0

EAL: Detected lcore 5 as core 0 on socket 0

EAL: Detected lcore 6 as core 0 on socket 0

EAL: Detected lcore 7 as core 0 on socket 0

EAL: Maximum logical cores by configuration: 128

EAL: Detected CPU lcores: 8

EAL: Detected NUMA nodes: 1

EAL: Checking presence of .so 'librte_eal.so.23.0'

EAL: Checking presence of .so 'librte_eal.so.23'

EAL: Checking presence of .so 'librte_eal.so'

EAL: Detected static linkage of DPDK

EAL: Ask a virtual area of 0x2000 bytes

EAL: Virtual area found at 0x100000000 (size =3D 0x2= 000)

[New Thread 0x7fed931ff640 (LWP 287600)]<= /p>

EAL: Multi-process socket /var/run/dpdk/rte/mp_socke= t

[New Thread 0x7fed929fe640 (LWP 287601)]<= /p>

EAL: PCI driver net_iavf for device 0000:00:05.0 wan= ts IOVA as 'PA'

EAL: PCI driver net_ice_dcf for device 0000:00:05.0 = wants IOVA as 'PA'

EAL: PCI driver net_iavf for device 0000:00:06.0 wan= ts IOVA as 'PA'

EAL: PCI driver net_ice_dcf for device 0000:00:06.0 = wants IOVA as 'PA'

EAL: Bus pci wants IOVA as 'PA'

EAL: Bus vdev wants IOVA as 'DC'

EAL: Selected IOVA mode 'PA'

EAL: Probing VFIO support...

EAL: Module /sys/module/vfio not found! error 2 (No = such file or directory)

EAL: VFIO modules not loaded, skipping VFIO support.= ..

EAL: Ask a virtual area of 0x2e000 bytes<= /p>

EAL: Virtual area found at 0x100002000 (size =3D 0x2= e000)

EAL: Setting up physically contiguous memory...=

EAL: Setting maximum number of open files to 1024

EAL: Detected memory type: socket_id:0 hugepage_sz:1= 073741824

EAL: Detected memory type: socket_id:0 hugepage_sz:2= 097152

EAL: Creating 1 segment lists: n_segs:2 socket_id:0 = hugepage_sz:1073741824

EAL: Ask a virtual area of 0x1000 bytes

EAL: Virtual area found at 0x100030000 (size =3D 0x1= 000)

EAL: Memseg list allocated at socket 0, page size 0x= 100000kB

EAL: Ask a virtual area of 0x80000000 bytes

EAL: Virtual area found at 0x140000000 (size =3D 0x8= 0000000)

EAL: VA reserved for memseg list at 0x140000000, siz= e 80000000

EAL: Creating 1 segment lists: n_segs:1024 socket_id= :0 hugepage_sz:2097152

EAL: Ask a virtual area of 0xd000 bytes

EAL: Virtual area found at 0x1c0000000 (size =3D 0xd= 000)

EAL: Memseg list allocated at socket 0, page size 0x= 800kB

EAL: Ask a virtual area of 0x80000000 bytes

EAL: Virtual area found at 0x1c0200000 (size =3D 0x8= 0000000)

EAL: VA reserved for memseg list at 0x1c0200000, siz= e 80000000

EAL: Trying to obtain current memory policy.

EAL: Setting policy MPOL_PREFERRED for socket 0=

EAL: Setting policy MPOL_PREFERRED for socket 0=

EAL: Restoring previous memory policy: 0<= /p>

 

 

// bad case

EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0=

EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0=

EAL: Requesting 2 pages of size 1024MB from socket 0=     <<<< Same on good and bad<= /p>

EAL: Attempting to map 1024M on socket 0  =     <<<< here, on good VM it states Attemptin= g to map 2048M on socket 0, we have one numa node or 1 socket.

EAL: Allocated 1024M on socket 0   &n= bsp;            = ;         <<<< here, it allocated the 1024M on socket 0.

EAL: Attempting to map 1024M on socket 0  =     <<<< here, attempts to map last 1G to soc= ket 0.

EAL: Could not find space for memseg. Please increas= e 1024 and/or 2048 in configuration. 

EAL: Couldn't remap hugepage files into memseg lists=  

EAL: FATAL: Cannot init memory

EAL: Cannot init memory

 

//good case

EAL: Hugepage /mnt/huge/rtemap_1 is on socket 0=

EAL: Hugepage /mnt/huge/rtemap_0 is on socket 0=

EAL: Requesting 2 pages of size 1024MB from socket 0=

EAL: Attempting to map 2048M on socket 0<= /p>

EAL: Allocated 2048M on socket 0

EAL: Added 2048M to heap on socket 0

 

I tried rebooting the VM 10 times and could not get = it to fail.

Tried multiple VMs and sometimes fails.

Stopping the application and restarting does not cle= ar the error.

Seen on VMWare VM and openStack VMs.

 

I modified the DPDK #defines to reduce the VIRT memo= ry of our application.

 

I added the following before building the dpdk stati= c libraries that are used in our application build.

 

#define DPDK_REDUCE_VIRT_8G   // is used t= o select the reduced MSL, etc reductions.

 

#if defined(DPDK_ORIGINAL) // original, VIRT: 36.6 G= B

#define RTE_MAX_MEMSEG_LISTS 128

#define RTE_MAX_MEMSEG_PER_LIST 8192

#define RTE_MAX_MEM_MB_PER_LIST 32768

#define RTE_MAX_MEMSEG_PER_TYPE 32768

#define RTE_MAX_MEM_MB_PER_TYPE 65536

#endif

 

#if defined(DPDK_REDUCE_VIRT_8G)  // VIRT: 5.9 = GB

#define RTE_MAX_MEMSEG_LISTS 2

#define RTE_MAX_MEMSEG_PER_LIST 1024

#define RTE_MAX_MEM_MB_PER_LIST 2048

#define RTE_MAX_MEMSEG_PER_TYPE 1024

#defin

#define RTE_MAX_MEM_MB_PER_TYPE 2048

#endif

 

The rte_eal_init() arguments are:

‘app_name, -c0x2, -n4, --socket-mem=3D2048, --= legacy-mem, --no-telemetry’

 

Could it be that the hugpages are not contiguous and= reboot clears this issue, not able to confirm.

 

Need help to eliminate this intermittent map to mems= eg list issue?

 

 

 

Thanks,

Ed

--_000_CH3PR01MB8470B651CE8BDCB4F0D32F7D8F9E2CH3PR01MB8470prod_--