From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 3E3C42C2B for ; Thu, 23 Feb 2017 03:46:48 +0100 (CET) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Feb 2017 18:46:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,197,1484035200"; d="scan'208";a="51578703" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga002.jf.intel.com with ESMTP; 22 Feb 2017 18:46:48 -0800 Received: from fmsmsx121.amr.corp.intel.com (10.18.125.36) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.248.2; Wed, 22 Feb 2017 18:46:47 -0800 Received: from fmsmsx113.amr.corp.intel.com ([169.254.13.17]) by fmsmsx121.amr.corp.intel.com ([169.254.6.23]) with mapi id 14.03.0248.002; Wed, 22 Feb 2017 18:46:46 -0800 From: "Wiles, Keith" To: Sushil Adhikari CC: "users@dpdk.org" Thread-Topic: [dpdk-users] Larger number of hugepages causes bus error. Thread-Index: AQHSjWBD8SKJIsMU3UanuEUvvEb4haF2XfkAgAAHAICAAAOjAIAAAOOA Date: Thu, 23 Feb 2017 02:46:46 +0000 Message-ID: <986FF8E4-1D37-4275-B9E3-8C7E19996461@intel.com> References: <18013811-F9E1-42C6-A87B-D087018AF0B9@intel.com> <6649EDAE-E9E4-4C49-855C-56E37BD4FA88@intel.com> In-Reply-To: <6649EDAE-E9E4-4C49-855C-56E37BD4FA88@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.254.190.182] Content-Type: text/plain; charset="us-ascii" Content-ID: <335ACF67F7195A46A7443E84BD7CBCEC@intel.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-users] Larger number of hugepages causes bus error. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2017 02:46:49 -0000 > On Feb 22, 2017, at 6:43 PM, Wiles, Keith wrote: >=20 >>=20 >> On Feb 22, 2017, at 6:30 PM, Sushil Adhikari wrote= : >>=20 >> I used the basic command line option "dpdkTimer -c 0xf -n 4" >> And to update on my findings so far I have narrowed down to this line(14= 05) >> memset(hugepage, 0, nr_hugefiles * sizeof(struct hugepage_file)); >> of function rte_eal_hugepage_init() in file dpdk\lib\librte_eal\linuxapp= \eal\eal_memory.c >=20 > What version of DPDK are you using? I was looking at the file at 1405 and= I do not see a memset() call. I found the memset call at 1142 in my 17.05-rc0 code. Please try the latest= version and see if you get the same problem. >>=20 >> Yes I have the hugepages of size 2MB(2048) and when I calculate the memo= ry this memset function is trying to set, it comes out to 512(nr_hugefiles)= * 4144 ( sizeof(struct hugepage_file) ) =3D 2121728 which larger than 2MB,= so my doubt is that the hugepages I have allocated(512*2MB) is not contigu= ous 1GB memory its trying to access memory thats not part of hugepage, is t= hat a possibility, even though I am setting up hugepages during boot time b= y providing it through kernel option. >>=20 >>=20 >> On Wed, Feb 22, 2017 at 8:05 PM, Wiles, Keith wr= ote: >>=20 >>> On Feb 22, 2017, at 3:05 PM, Sushil Adhikari wrot= e: >>>=20 >>> Hi, >>>=20 >>> I was trying to run dpdk timer app by setting 512 2MB hugepages but the >>> application crashed with following error >>> EAL: Detected 4 lcore(s) >>> EAL: Probing VFIO support... >>> Bus error (core dumped) >>>=20 >>> If I reduce the number of hugepages to 256 it works fine. I wondering w= hat >>> could be the problem here. Here's my cpu info >>=20 >> I normally run with 2048 x 2 or 2048 per socket on my machine. What is t= he command line you are using to start the application? >>=20 >>>=20 >>> processor : 0 >>> vendor_id : GenuineIntel >>> cpu family : 6 >>> model : 26 >>> model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz >>> stepping : 5 >>> microcode : 0x11 >>> cpu MHz : 2794.000 >>> cache size : 8192 KB >>> physical id : 0 >>> siblings : 4 >>> core id : 0 >>> cpu cores : 4 >>> apicid : 0 >>> initial apicid : 0 >>> fpu : yes >>> fpu_exception : yes >>> cpuid level : 11 >>> wp : yes >>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge = mca >>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall = nx >>> rdtscp lm constant_tsc arch_ >>> perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dte= s64 >>> monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt >>> lahf_lm ida dtherm tpr_shadow vnm >>> i flexpriority ept vpid >>> bugs : >>> bogomips : 5600.00 >>> clflush size : 64 >>> cache_alignment : 64 >>> address sizes : 36 bits physical, 48 bits virtual >>> power management: >>>=20 >>> processor : 1 >>> vendor_id : GenuineIntel >>> cpu family : 6 >>> model : 26 >>> model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz >>> stepping : 5 >>> microcode : 0x11 >>> cpu MHz : 2794.000 >>> cache size : 8192 KB >>> physical id : 0 >>> siblings : 4 >>> core id : 1 >>> cpu cores : 4 >>> apicid : 2 >>> initial apicid : 2 >>> fpu : yes >>> fpu_exception : yes >>> cpuid level : 11 >>> wp : yes >>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge = mca >>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall = nx >>> rdtscp lm constant_tsc arch_ >>> perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dte= s64 >>> monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt >>> lahf_lm ida dtherm tpr_shadow vnm >>> i flexpriority ept vpid >>> bugs : >>> bogomips : 5600.00 >>> clflush size : 64 >>> cache_alignment : 64 >>> address sizes : 36 bits physical, 48 bits virtual >>> power management:...... >>>=20 >>> And Here's my meminfo >>>=20 >>> MemTotal: 24679608 kB >>> MemFree: 24014156 kB >>> MemAvailable: 23950600 kB >>> Buffers: 3540 kB >>> Cached: 31436 kB >>> SwapCached: 0 kB >>> Active: 21980 kB >>> Inactive: 22256 kB >>> Active(anon): 10760 kB >>> Inactive(anon): 2940 kB >>> Active(file): 11220 kB >>> Inactive(file): 19316 kB >>> Unevictable: 0 kB >>> Mlocked: 0 kB >>> SwapTotal: 0 kB >>> SwapFree: 0 kB >>> Dirty: 32 kB >>> Writeback: 0 kB >>> AnonPages: 9252 kB >>> Mapped: 11912 kB >>> Shmem: 4448 kB >>> Slab: 27712 kB >>> SReclaimable: 11276 kB >>> SUnreclaim: 16436 kB >>> KernelStack: 2672 kB >>> PageTables: 1000 kB >>> NFS_Unstable: 0 kB >>> Bounce: 0 kB >>> WritebackTmp: 0 kB >>> CommitLimit: 12077660 kB >>> Committed_AS: 137792 kB >>> VmallocTotal: 34359738367 kB >>> VmallocUsed: 0 kB >>> VmallocChunk: 0 kB >>> HardwareCorrupted: 0 kB >>> AnonHugePages: 2048 kB >>> CmaTotal: 0 kB >>> CmaFree: 0 kB >>> HugePages_Total: 256 >>> HugePages_Free: 0 >>> HugePages_Rsvd: 0 >>> HugePages_Surp: 0 >>> Hugepagesize: 2048 kB >>> DirectMap4k: 22000 kB >>> DirectMap2M: 25133056 kB >>=20 >> Regards, >> Keith >>=20 >>=20 >=20 > Regards, > Keith Regards, Keith