From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by dpdk.org (Postfix) with ESMTP id 6BDE5AD95 for ; Thu, 5 Feb 2015 14:20:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=1390; q=dns/txt; s=iport; t=1423142405; x=1424352005; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=9ObkPn2KSnMDogSHDqw5CI9JJ/GbtXEhPESBCLlnx7Y=; b=mUgoV0b1eKKxnbGWhx92uHEweMXxqmb7GzatHK7QgfmF11SWQuujPww5 vX/HJjeujq3PqAnpGku0mShiOiKt3SRJbOiFT7ZX5vR7qg/edNafhTdRz WOAtIhR80Z0qF6YjOkXEV82uE5J98dTIZSNk9vZnxjoyPUgSWcmzhrY9P U=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ak8FAJRt01StJV2T/2dsb2JhbABagmQigSsEyDkCgSVDAQEBAQF9hAwBAQEDATo/BQsCAQgOCh4QMiUCBA4FiCUIAdYHAQEBAQEBAQEBAQEBAQEBAQEBAQEBF48hJDMHgxaBEwEEjxSJKoEXjhWDPiKDbm+BA0F+AQEB X-IronPort-AV: E=Sophos;i="5.09,524,1418083200"; d="scan'208";a="390564859" Received: from rcdn-core-11.cisco.com ([173.37.93.147]) by rcdn-iport-9.cisco.com with ESMTP; 05 Feb 2015 13:20:04 +0000 Received: from xhc-aln-x15.cisco.com (xhc-aln-x15.cisco.com [173.36.12.89]) by rcdn-core-11.cisco.com (8.14.5/8.14.5) with ESMTP id t15DK3ab003427 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 5 Feb 2015 13:20:03 GMT Received: from xmb-aln-x01.cisco.com ([fe80::747b:83e1:9755:d453]) by xhc-aln-x15.cisco.com ([173.36.12.89]) with mapi id 14.03.0195.001; Thu, 5 Feb 2015 07:20:02 -0600 From: "Damjan Marion (damarion)" To: Neil Horman Thread-Topic: [dpdk-dev] mmap fails with more than 40000 hugepages Thread-Index: AQHQQTthrW63bl6xlkWr3i+nt5poHJziaVwAgAAFpoA= Date: Thu, 5 Feb 2015 13:20:01 +0000 Message-ID: <2F27D954-A245-45DC-A1BE-0CA3E17AAD3B@cisco.com> References: <736BD68D-C5DF-4883-A720-DAD8A2A866BE@cisco.com> <20150205125948.GC28355@hmsreliant.think-freely.org> In-Reply-To: <20150205125948.GC28355@hmsreliant.think-freely.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.55.15.242] Content-Type: text/plain; charset="us-ascii" Content-ID: <6AF6E6AC4A747845ABDDE9EE72D08939@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] mmap fails with more than 40000 hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Feb 2015 13:20:04 -0000 > On 05 Feb 2015, at 13:59, Neil Horman wrote: >=20 > On Thu, Feb 05, 2015 at 12:00:48PM +0000, Damjan Marion (damarion) wrote: >> Hi, >>=20 >> I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK = crashes in rte_eal_init() >> when number of available hugepages is around 40000 or above. >> Everything works fine with lower values (i.e. 30000). >>=20 >> I also tried with allocating 40000 on node0 and 0 on node1, same crash h= appens. >>=20 >>=20 >> Any idea what might be causing this? >>=20 >> Thanks, >>=20 >> Damjan >>=20 >>=20 >> $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hu= gepages >> 20000 >> 20000 >>=20 >> $ grep -i huge /proc/meminfo >> AnonHugePages: 706560 kB >> HugePages_Total: 40000 >> HugePages_Free: 40000 >> HugePages_Rsvd: 0 >> HugePages_Surp: 0 >> Hugepagesize: 2048 kB >>=20 > Whats your shmmax value set to? 40000 2MB hugepages is way above the defa= ult > setting for how much shared ram a system will allow. I've not done the m= ath on > your logs below, but judging by the size of some of the mapped segments, = I'm > betting your hitting the default limit of 4GB. $ cat /proc/sys/kernel/shmmax 33554432 $ sysctl -w kernel.shmmax=3D8589934592 kernel.shmmax =3D 8589934592 same crash :( Thanks, Damjan=