From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from BLU004-OMC1S13.hotmail.com (blu004-omc1s13.hotmail.com [65.55.116.24]) by dpdk.org (Postfix) with ESMTP id 229F537A6 for ; Thu, 1 Sep 2016 08:47:10 +0200 (CEST) Received: from IND01-BO1-obe.outbound.protection.outlook.com ([65.55.116.7]) by BLU004-OMC1S13.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 31 Aug 2016 23:47:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=py3W7Dh5Sz3Px8ard9ggMbxrpH7WtxkNtgMhDTIHB+E=; b=bwdITT8ybpXUDEFMlwTlvieWR8DrI66uoQagSkQOHai1sPaYhSI5GH9l+DkJc+BEy2Dlr5VnW1TQ1foUGri7ba4Rtock1s2whhGLATBjAzmnJkb24l1S0W5ASpV1O6mryoRnNcWq/8yEge7jJInGmbQVpiPLD6hckR7ISBU3whbYLonLKd03/cOCjbIdbmYpMRryNLOk9/0I+gYQ1B40Mntw21aCJbkZ9+9AKAhMdkfh7wKoBRJOdaB09BoqA/6vDNav6dcz7hT9aY73ptO1adT51lq7PtblCc+OIkaTm4WC86LImnr0HTmINVDjX1tmTDFaelwIOtVgl7/6+RyMzQ== Received: from BO1IND01FT009.eop-IND01.prod.protection.outlook.com (10.152.202.54) by BO1IND01HT005.eop-IND01.prod.protection.outlook.com (10.152.202.104) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.6; Thu, 1 Sep 2016 06:46:57 +0000 Received: from MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM (10.152.202.53) by BO1IND01FT009.mail.protection.outlook.com (10.152.202.94) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.6 via Frontend Transport; Thu, 1 Sep 2016 06:46:54 +0000 Received: from MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM ([10.164.150.135]) by MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM ([10.164.150.135]) with mapi id 15.01.0587.013; Thu, 1 Sep 2016 06:46:54 +0000 From: Sarthak Ray To: "Wiles, Keith" CC: "users@dpdk.org" Thread-Topic: [dpdk-users] rte_eal_init fails with --socket-mem set to more than 192MB per socket Thread-Index: AQHSA7sl8b/+rXx5hESEAPuv9lK7+qBj8gAAgAA8tHE= Date: Thu, 1 Sep 2016 06:46:53 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=softfail (sender IP is 10.152.202.53) smtp.mailfrom=outlook.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=fail action=none header.from=outlook.com; received-spf: SoftFail (protection.outlook.com: domain of transitioning outlook.com discourages use of 10.152.202.53 as permitted sender) x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [ZvOa5ZtxiMwmtU2+W26iQ3SBB+luKPMC] x-eopattributedmessage: 0 x-forefront-antispam-report: CIP:10.152.202.53; IPV:NLI; CTRY:; EFV:NLI; SFV:NSPM; SFS:(10019020)(98900003); DIR:OUT; SFP:1102; SCL:1; SRVR:BO1IND01HT005; H:MAXPR01MB0315.INDPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; x-microsoft-exchange-diagnostics: 1; BO1IND01HT005; 6:JoiD3RLZHo4HGs6amjS70rDemU/ndWJYJi5PvIXjRg85jXeN3Xz+CZZFEpMM8oxqOSvdRGsH6cKgg6H+qOtJ934V5R4SGhJibW++H2k1CVnzMNWC3AHH22mpT8B+Ri8PgByXze5f4mtFvltxYd7G16QvYOL4FXT/PKSKs1n13+OAoc6CNG6Mz7nUpF+ZboUzwojhN/9gCNp5lM6t2OjHZ88O7LSbiaSaeiNMBkdat33RPYEoLDNN8HIaQoMXokfiet7vU/ZHEqGDP1wz3HPmyooV8qNxeIwD+m/EviGSR4RZO/USCrQgKmWbLkFhFFnz; 5:Ff+seg4lnwGaSjtgXNTFnUeZHTybsFlYkX9/DJmdQ3cD6FJmzuMIgJKkMGavQQvZaqd/iwFE8BNksyhgNhCRt5wDUpRBDhYOGETVS+EkbXBvLjgLrIV4Z7rCIcAGNED3qtfE3iFgFGhENvb1kbAmGA==; 24:YiAP2ockumoW6ssIoMSiJrAeT1ndJ9HE4NEcwWLlJzOqCmXRGUgNf6TQElIkHnO4ZWTfqABl2lPlWl9OlfHhfO94CxGDmtPuVYcdlX/anS0=; 7:c8U9ZTLOpWKHs0iJ9GaXeGlR1R7ZmZe2vkqwP0IwRzMdYjTZ36eWPzJAQrGu89NOeep62zt8I9tC2Y4JWRj1gwakZKJpJpDDKIUZUT3L/IjKpr572cD0ln+Up47LeqrtI6MKeDJZzv0Xi86pYzrY8l5JwC2OHztvTQaISZ50NeYF9ZF+XUODrxVHTHgA0zabeoni67N9nN7UNvWx98yi6AwQHXfGG5tHJ/YZF4OOXXtuJPIO9UvK4BWg5aJuUs51 x-ms-office365-filtering-correlation-id: 754b402d-7210-4572-d42e-08d3d233c31c x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(1601124038)(1603103081)(1601125047); SRVR:BO1IND01HT005; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(432015012)(82015046); SRVR:BO1IND01HT005; BCL:0; PCL:0; RULEID:; SRVR:BO1IND01HT005; x-forefront-prvs: 0052308DC6 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Sep 2016 06:46:53.9916 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BO1IND01HT005 X-OriginalArrivalTime: 01 Sep 2016 06:47:09.0686 (UTC) FILETIME=[A9858160:01D2041C] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] rte_eal_init fails with --socket-mem set to more than 192MB per socket X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Sep 2016 06:47:10 -0000 Hi Keith, Thanks for your response; but I am reserving enough hugepages during boot t= ime so that user space can't fragment those memories. System boot arguments related to hugepages hugepagesz=3D1g hugepages=3D24 hugepagesz=3D2m hugepages=3D1024 Memory status before my application starts. # cat /proc/meminfo MemTotal: 131990816 kB MemFree: 103888408 kB Buffers: 220 kB Cached: 326596 kB SwapCached: 0 kB Active: 619104 kB Inactive: 309796 kB Active(anon): 602108 kB Inactive(anon): 952 kB Active(file): 16996 kB Inactive(file): 308844 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 2252 kB Writeback: 0 kB AnonPages: 601976 kB Mapped: 41560 kB Shmem: 976 kB Slab: 130140 kB SReclaimable: 69672 kB SUnreclaim: 60468 kB KernelStack: 6104 kB PageTables: 10320 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 52888208 kB Committed_AS: 1771680 kB VmallocTotal: 34359738367 kB VmallocUsed: 1310168 kB VmallocChunk: 34291387912 kB HugePages_Total: 512 HugePages_Free: 512 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 12480 kB DirectMap2M: 1906688 kB DirectMap1G: 134217728 kB Memory status after my application exits with rte_eal_init failure. # cat /proc/meminfo MemTotal: 131990816 kB MemFree: 99747864 kB Buffers: 240 kB Cached: 440156 kB SwapCached: 0 kB Active: 839728 kB Inactive: 327164 kB Active(anon): 820448 kB Inactive(anon): 1920 kB Active(file): 19280 kB Inactive(file): 325244 kB Unevictable: 3704100 kB Mlocked: 3704100 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 7832 kB Writeback: 0 kB AnonPages: 4433016 kB Mapped: 135536 kB Shmem: 9088 kB Slab: 133932 kB SReclaimable: 71568 kB SUnreclaim: 62364 kB KernelStack: 7344 kB PageTables: 20876 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 52798096 kB Committed_AS: 5902824 kB VmallocTotal: 34359738367 kB VmallocUsed: 1310168 kB VmallocChunk: 34291387912 kB HugePages_Total: 600 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 12480 kB DirectMap2M: 1906688 kB DirectMap1G: 134217728 kB Thanks, Sarthak ________________________________ From: Wiles, Keith Sent: Thursday, September 1, 2016 1:28:36 AM To: Sarthak Ray Cc: users@dpdk.org Subject: Re: [dpdk-users] rte_eal_init fails with --socket-mem set to more = than 192MB per socket Regards, Keith > On Aug 31, 2016, at 2:15 PM, Sarthak Ray wrote: > > Hi, > > I am using dpdk-2.1.0 and I am not able to reserve memory beyond 192MB pe= r socket by using --socket-mem option. I see below error logs, though my sy= stem has enough free memory. > > EAL: Not enough memory available on socket 0! Requested: 256MB, available= : 192MB > PANIC in rte_eal_init(): \Most of the time this means that contiguous memory is not available and yo= u have fragmented huge pages. The normal fix is to make sure you allocate t= he huge pages early in boot up, which to me means making sure you have the = /etc/sysctl.conf file setup with the number of huge pages. vm.nr_hugepages=3DNNN > > # numactl -H > available: 2 nodes (0-1) > node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 36 37 38 39 40 4= 1 42 43 44 45 46 47 48 49 50 51 52 53 > node 0 size: 65170 MB > node 0 free: 47433 MB > node 1 cpus: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 54 55 = 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 > node 1 size: 65536 MB > node 1 free: 49999 MB > node distances: > node 0 1 > 0: 10 21 > 1: 21 10 > > Is there any max limit for reserving memory per socket? If yes, then plea= se suggest me how to increase that limit. > > Thanks in advance, > Sarthak