From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from BAY004-OMC2S3.hotmail.com (bay004-omc2s3.hotmail.com [65.54.190.78]) by dpdk.org (Postfix) with ESMTP id 46CED2BFA for ; Wed, 17 Aug 2016 15:10:47 +0200 (CEST) Received: from IND01-MA1-obe.outbound.protection.outlook.com ([65.54.190.123]) by BAY004-OMC2S3.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 17 Aug 2016 06:10:46 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=EyuxkUgnxrbhLXtyqVtKv3Nl0A7Us08B9YJM81dtTk4=; b=ZIr6fWctM2cAvhK1ZRca3pW/xpSNgbv3Y6ovUU20KhsGRvtVKy1XjpN41q9h5QY/AQsptjsvxDRZgctHhRUdAVuZnN4FkyPV3mUOyXqHPded9y5gG8A14qQQfiMDKidn+wXALqj9gSOoT1T7CjcjkZLfR+4P+ntj4b5S19Q0T8kV0BI/0mSwsOlnX6OYTHTwNcs4mVam+z4pe2p+seD18SoaSzAqRWnUvD+hQjxx9fyRS6jC57kUfOel3gZhHk1mzYW9avzOmtedDFOmjUgwBmFbapS+Jdi/C9C5iYt6v1gbF6RQLeGE+05p9ZdxLhnY0HE8T+qV5epNhM7dk10GJw== Received: from MA1IND01FT004.eop-IND01.prod.protection.outlook.com (10.152.200.59) by MA1IND01HT013.eop-IND01.prod.protection.outlook.com (10.152.200.65) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.567.7; Wed, 17 Aug 2016 13:10:37 +0000 Received: from MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM (10.152.200.53) by MA1IND01FT004.mail.protection.outlook.com (10.152.200.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.567.7 via Frontend Transport; Wed, 17 Aug 2016 13:10:36 +0000 Received: from MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM ([10.164.150.135]) by MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM ([10.164.150.135]) with mapi id 15.01.0549.027; Wed, 17 Aug 2016 13:10:35 +0000 From: Sarthak Ray To: "users@dpdk.org" Thread-Topic: Mempool allocation fails on first boot; but succeeds after system reboot Thread-Index: AQHR+IgCb1ohkt73GEOBylDCKgMeMA== Date: Wed, 17 Aug 2016 13:10:35 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.200.53) smtp.mailfrom=outlook.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=fail action=none header.from=outlook.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning outlook.com discourages use of 10.152.200.53 as permitted sender) x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [eevKHRpwupCiUwHAkGmW4zrejN3fgxPM] x-eopattributedmessage: 0 x-forefront-antispam-report: CIP:10.152.200.53; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:MA1IND01HT013; H:MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; x-microsoft-exchange-diagnostics: 1; MA1IND01HT013; 6:+nTMf7rG1zZBpVTps3VASdKQvNDDt0gKVuBUzJxtZuirQUdh6v/C3ukOpSV58FF2tuuArmyWFr+6E2ztNfQnPmVN9VZwt7vgi8A7vxOmilL6f6mDVNaQ1849/UevOQ8KTUmeClRx2doQJgp2u4PBUn5tkIB32DUW/34YiFqGZ/Q66tfpNXdDIcM0A+lA3/hU1/52o6qK98XAL0MqjnZZkehv6R04ar92Dd6sRsWTcwADmWpWiD6oz6D3qltl9XLJZFfkzgOI9e25bK0kFDeMsJUgoekjnLQThPBSj/btGWbSy7z0KfOSGkKTQRh68MgY; 5:waVWLW9VpJCnpTw1KOn1kbYZl8pGjPGvFSMjf46XaNCWN3dLQ3HiVKYpHgj7IDV1LCj3MKROUM/hQ2Uzo1qygpSp4gXZoZD2tAmgOoSrOFnifNWacHGW1G4h4feHbjhhddjKgaTf0iUaik9G2RhnHA==; 24:CicSQwttq2tab4Sm/iPDmIubIJVVccIVr26d9XeVj2HTdtGsXY85n4sP8lMdD79yPPSr13dSHWm+XflbheNxRS29mHJXwNKA7oQ73/Z2470=; 7:4hon5LgW+8gDoHCBThBEMFoVuC9TJyN7m7S+G9h5jNZTlcpBRzIYjCCCBVw6iGi/u1exmDf+Z8hAgB6aamaBtMlIVdF3IKXcqeFBAzBn3b9BxWArpCa+LAFM6T0lApT19+DUX0L09Dza+Jdd+mEGt1LdV0Wuskhzj/xmOf74wWXbgc1K3L03PsLmwrtPg2luw2ZP1pIhg1Jd2CGj9/7ZOV4mTy8MP3gmpIe9UDBDVn9c0+g38eM/LFSzL2YTWwIC x-ms-office365-filtering-correlation-id: d4929df7-d8e6-4b89-971b-08d3c69fe05b x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1601125047); SRVR:MA1IND01HT013; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:MA1IND01HT013; BCL:0; PCL:0; RULEID:; SRVR:MA1IND01HT013; x-forefront-prvs: 0037FD6480 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Aug 2016 13:10:35.3708 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: MA1IND01HT013 X-OriginalArrivalTime: 17 Aug 2016 13:10:46.0515 (UTC) FILETIME=[C46CA430:01D1F888] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] Mempool allocation fails on first boot; but succeeds after system reboot X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Aug 2016 13:10:48 -0000 Hi, I am using dpdk-2.1.0 for a platform appliance, where I am facing issue wit= h mempool allocation. On the firstboot of the newly installed appliance, my dpdk application is n= ot coming up saying failure in mbuf allocation on socket 0. But once I rebo= ot the system, it comes up without any issues. I tried "rte_malloc_dump_stats" api to check the heap statistics right befo= re allocating mbuf pools. Heap Statistics on first boot (with --socket-mem=3D128,128) Socket:0 Heap_size:134215808, Free_size:127706432, Alloc_size:6509376, Greatest_free_size:8388544, // This value is very less than the "contig= uous memory block" that my app is trying to allocate Alloc_count:29, Free_count:31, Please Note: Increasing --socket-mem value from 128 to 192 has no impact on= Greatest_free_size value and I don't see this fragmentation on socket 1. Heap Statistics after reboot (with --socket-mem=3D128,128) Socket:0 Heap_size:134217600, Free_size:127708224, Alloc_size:6509376, Greatest_free_size:125982080, Alloc_count:29, Free_count:3, After reboot, the largest free block size is increased drastically resultin= g successful mbuf pool allocation. So looks like this is heap fragmentation= issue on Socket 0. Output of "numactl -H" on my sytem available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 36 37 38 39 40 41 = 42 43 44 45 46 47 48 49 50 51 52 53 node 0 size: 65170 MB node 0 free: 49476 MB node 1 cpus: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 54 55 56= 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 node 1 size: 65536 MB node 1 free: 50759 MB node distances: node 0 1 0: 10 21 1: 21 10 Kernel Boot Arguments for hugepage setting hugepagesz=3D1g hugepages=3D24 Can anyone please comment on how to address this issue? Is there any way to= reserve hugepages that can't be fragmented? Thanks in advance for the valuable suggestion. Regards, Sarthak