From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rcdn-iport-8.cisco.com (rcdn-iport-8.cisco.com [173.37.86.79]) by dpdk.org (Postfix) with ESMTP id 1C261B406 for ; Thu, 5 Feb 2015 13:00:49 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=16844; q=dns/txt; s=iport; t=1423137650; x=1424347250; h=from:to:subject:date:message-id:content-id: content-transfer-encoding:mime-version; bh=xNzL/c9BXKMRgYnnu5xFGJ4a5W4TdOuZ0XZKwyD9SqY=; b=VExb6KNTfe5M8sN3QVHFtFbZlOTxG4US/OCMwVaTmUnuknXORBCe8xX6 SCSWukfMaVMUyjs5+cSZFsw5efJdXbPPC+VsK+1kGpKtZEnIkP/6gIGkA EWEKLSrFaRtX46aszZE+zvuV5fDFHychZVLQJCRJLcUjL4OeLMtv6n+R2 s=; X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0AtBQAuW9NU/4YNJK1agmQigS/FaoN2QwEBAQEBfYQTOlEBPkInBIhAAa92pgMBAQEBAQEEAQEBAQEBAQEBGZMVgRMFjxSJKpJqIoF/H4FQgjN+AQEB X-IronPort-AV: E=Sophos;i="5.09,523,1418083200"; d="scan'208";a="393565259" Received: from alln-core-12.cisco.com ([173.36.13.134]) by rcdn-iport-8.cisco.com with ESMTP; 05 Feb 2015 12:00:48 +0000 Received: from xhc-rcd-x07.cisco.com (xhc-rcd-x07.cisco.com [173.37.183.81]) by alln-core-12.cisco.com (8.14.5/8.14.5) with ESMTP id t15C0mF9027267 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL) for ; Thu, 5 Feb 2015 12:00:48 GMT Received: from xmb-aln-x01.cisco.com ([fe80::747b:83e1:9755:d453]) by xhc-rcd-x07.cisco.com ([173.37.183.81]) with mapi id 14.03.0195.001; Thu, 5 Feb 2015 06:00:48 -0600 From: "Damjan Marion (damarion)" To: "dev@dpdk.org" Thread-Topic: mmap fails with more than 40000 hugepages Thread-Index: AQHQQTthrW63bl6xlkWr3i+nt5poHA== Date: Thu, 5 Feb 2015 12:00:48 +0000 Message-ID: <736BD68D-C5DF-4883-A720-DAD8A2A866BE@cisco.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.55.15.242] Content-Type: text/plain; charset="us-ascii" Content-ID: <79AB35F807C1204A90392D65B313D4CE@emea.cisco.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [dpdk-dev] mmap fails with more than 40000 hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Feb 2015 12:00:50 -0000 Hi, I have system with 2 NUMA nodes and 256G RAM total. I noticed that DPDK cra= shes in rte_eal_init() when number of available hugepages is around 40000 or above. Everything works fine with lower values (i.e. 30000). I also tried with allocating 40000 on node0 and 0 on node1, same crash happ= ens. Any idea what might be causing this? Thanks, Damjan $ cat /sys/devices/system/node/node[01]/hugepages/hugepages-2048kB/nr_hugep= ages 20000 20000 $ grep -i huge /proc/meminfo AnonHugePages: 706560 kB HugePages_Total: 40000 HugePages_Free: 40000 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB $ sudo ~/src/dpdk/x86_64-native-linuxapp-gcc/app/testpmd -l 5-7 -n 3 --sock= et-mem 512,512 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Detected lcore 8 as core 0 on socket 1 EAL: Detected lcore 9 as core 1 on socket 1 EAL: Detected lcore 10 as core 2 on socket 1 EAL: Detected lcore 11 as core 3 on socket 1 EAL: Detected lcore 12 as core 4 on socket 1 EAL: Detected lcore 13 as core 5 on socket 1 EAL: Detected lcore 14 as core 6 on socket 1 EAL: Detected lcore 15 as core 7 on socket 1 EAL: Detected lcore 16 as core 0 on socket 0 EAL: Detected lcore 17 as core 1 on socket 0 EAL: Detected lcore 18 as core 2 on socket 0 EAL: Detected lcore 19 as core 3 on socket 0 EAL: Detected lcore 20 as core 4 on socket 0 EAL: Detected lcore 21 as core 5 on socket 0 EAL: Detected lcore 22 as core 6 on socket 0 EAL: Detected lcore 23 as core 7 on socket 0 EAL: Detected lcore 24 as core 0 on socket 1 EAL: Detected lcore 25 as core 1 on socket 1 EAL: Detected lcore 26 as core 2 on socket 1 EAL: Detected lcore 27 as core 3 on socket 1 EAL: Detected lcore 28 as core 4 on socket 1 EAL: Detected lcore 29 as core 5 on socket 1 EAL: Detected lcore 30 as core 6 on socket 1 EAL: Detected lcore 31 as core 7 on socket 1 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 32 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x800000 bytes EAL: Virtual area found at 0x7fae2a200000 (size =3D 0x800000) EAL: Ask a virtual area of 0x7600000 bytes EAL: Virtual area found at 0x7fae22a00000 (size =3D 0x7600000) EAL: Ask a virtual area of 0x1400000 bytes EAL: Virtual area found at 0x7fae21400000 (size =3D 0x1400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fae21000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fae20a00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fae20600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fae20200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fae1fe00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x5800000 bytes EAL: Virtual area found at 0x7fae1a400000 (size =3D 0x5800000) EAL: Ask a virtual area of 0x3b200000 bytes EAL: Virtual area found at 0x7faddf000000 (size =3D 0x3b200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7faddea00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x7c00000 bytes EAL: Virtual area found at 0x7fadd6c00000 (size =3D 0x7c00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fadd6800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fadd6400000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fadd6000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fadd5c00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fadd5800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x9800000 bytes EAL: Virtual area found at 0x7fadcbe00000 (size =3D 0x9800000) EAL: Ask a virtual area of 0x19000000 bytes EAL: Virtual area found at 0x7fadb2c00000 (size =3D 0x19000000) EAL: Ask a virtual area of 0x24400000 bytes EAL: Virtual area found at 0x7fad8e600000 (size =3D 0x24400000) EAL: Ask a virtual area of 0xc800000 bytes EAL: Virtual area found at 0x7fad81c00000 (size =3D 0xc800000) EAL: Ask a virtual area of 0x32000000 bytes EAL: Virtual area found at 0x7fad4fa00000 (size =3D 0x32000000) EAL: Ask a virtual area of 0x3db800000 bytes EAL: Virtual area found at 0x7fa974000000 (size =3D 0x3db800000) EAL: Ask a virtual area of 0xfa000000 bytes EAL: Virtual area found at 0x7fa879e00000 (size =3D 0xfa000000) EAL: Ask a virtual area of 0x168400000 bytes EAL: Virtual area found at 0x7fa711800000 (size =3D 0x168400000) EAL: Ask a virtual area of 0x7d000000 bytes EAL: Virtual area found at 0x7fa694600000 (size =3D 0x7d000000) EAL: Ask a virtual area of 0x1f4000000 bytes EAL: Virtual area found at 0x7fa4a0400000 (size =3D 0x1f4000000) EAL: Ask a virtual area of 0x2e400000 bytes EAL: Virtual area found at 0x7fa471e00000 (size =3D 0x2e400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa471a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa471600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa471200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa470e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa470a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa470600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa470200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46fe00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46fa00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa46f400000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46f000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa46ea00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46e600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46e200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46de00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46da00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46d600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa46d000000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46cc00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46c800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46c400000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46c000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46bc00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46b800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46b400000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46b000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa46aa00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46a600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa46a200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa469e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa469a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa469600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa469200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa468e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa468a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa468600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa468200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa467e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa467800000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa467400000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa467000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa466c00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa466600000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa466200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa465e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa465a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa465600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa465000000 (size =3D 0x400000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa464a00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa464600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa464200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa463e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa463a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa463600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa463200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa462e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa462a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa462600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa462200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa461c00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa461800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa461400000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa461000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa460c00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa460800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa460400000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa460000000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45fc00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45f800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa45f200000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45ee00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45ea00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45e600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45e200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45de00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45da00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45d600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45d200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45ce00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45ca00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45c600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45c200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45be00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45ba00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45b600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa45b000000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45ac00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa45a800000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa45a200000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa459e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa459a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa459600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa459200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa458c00000 (size =3D 0x400000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa458600000 (size =3D 0x400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa458200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa457e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa457a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa457600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa457200000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa456e00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa456a00000 (size =3D 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa456600000 (size =3D 0x200000) EAL: Ask a virtual area of 0x139400000 bytes EAL: Virtual area found at 0x7fa31d000000 (size =3D 0x139400000) EAL: Ask a virtual area of 0xe9000000 bytes EAL: Virtual area found at 0x7fa233e00000 (size =3D 0xe9000000) EAL: Ask a virtual area of 0x188000000 bytes EAL: Virtual area found at 0x7fa0abc00000 (size =3D 0x188000000) EAL: map_all_hugepages(): mmap failed: Cannot allocate memory EAL: Failed to remap 2 MB pages PANIC in rte_eal_init(): Cannot init memory Aborted (core dumped)