From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot0-f195.google.com (mail-ot0-f195.google.com [74.125.82.195]) by dpdk.org (Postfix) with ESMTP id DF136316B for ; Thu, 23 Feb 2017 17:19:29 +0100 (CET) Received: by mail-ot0-f195.google.com with SMTP id w44so3926570otw.1 for ; Thu, 23 Feb 2017 08:19:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=w8HDKoLkmDIdjp9j9h78XxpA54ArA6mmUXmw/LZTLlw=; b=k4dnAgEXRB/Imvs4Stzzs4Rq+7zCu+oddifMmnN8tNunAtfAy+pL7lRlxTFkE5R9v/ iWDnGMmdfBx4FsQIagIy9YocsJj2WEGBpHVtL20lgQEzgkd7OpBoE4ep7+Q/Xhtvv3er fGsehaqj51mRbhLlHUQDxPh4xGb3+IQDb0v+x69iexvuPieuXlimeuorRVGOEuF+rOeo ldjKcMinzhO3wZf+Gm/2tTsgqXhK3zIL+3a9RlqYGvXL4VqWnSqz+cUim1ULSHO8f1MD QySK8yHGsXC40LKAYBe9q+hJXmPCzSof8VMOTpB/QE4PLIjw6YQHJapHvv0MC12pBmRI HOcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=w8HDKoLkmDIdjp9j9h78XxpA54ArA6mmUXmw/LZTLlw=; b=s/3GKXmbsfMCuCj/UNR3grVUi9+YTFkiLd8Z3qLnBOt6zGapSElAEZ8f7CoGcc/aY5 CFp2r74V+aClKXjqjQ1kOAxWQykh1fBWSlnADEh+9gLKUdGP0uR0PDYYfKpfqepR/oTd ELznDHcg2WkXchFtkix5MQuurVVOwPJX+C/tWwJKRuB427OiLJ4DnWTlqD+Lvad7Sz0m /ad3yJQDew3vSqIjZK7/jOJoKB6/g1vh9CYSZQzpW5Xit36BA3mrf8IPtzM2Y64EWq21 i0boGLSEng0t8Yh+qr1clKFEYb3hts50ecG7VEFzFKcwfJf8/SkzGrN9c/+47XpY3a9n +5dQ== X-Gm-Message-State: AMke39nruCcR9gWs2cPWWhciR5ZnlNvNl6dOoisqUj2WJNz/LEwvjHeQ5A97sXTQ/xC3GlmX2fHfUP002Ek0jw== X-Received: by 10.157.73.148 with SMTP id g20mr15359105otf.56.1487866769030; Thu, 23 Feb 2017 08:19:29 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.110.38 with HTTP; Thu, 23 Feb 2017 08:19:08 -0800 (PST) In-Reply-To: References: <18013811-F9E1-42C6-A87B-D087018AF0B9@intel.com> <6649EDAE-E9E4-4C49-855C-56E37BD4FA88@intel.com> <986FF8E4-1D37-4275-B9E3-8C7E19996461@intel.com> From: Sushil Adhikari Date: Thu, 23 Feb 2017 10:19:08 -0600 Message-ID: To: "Wiles, Keith" Cc: "users@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Larger number of hugepages causes bus error. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2017 16:19:30 -0000 I didn't understand what you mean by hugepage value, if you mean number of hugepages here's what it looks like [~]$ grep -ri hugepages /proc/meminfo AnonHugePages: 0 kB HugePages_Total: 512 HugePages_Free: 512 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB And the linux version is 4.4.20. On Thu, Feb 23, 2017 at 9:17 AM, Wiles, Keith wrote= : > > > On Feb 22, 2017, at 7:18 PM, Sushil Adhikari > wrote: > > > > Thank you Keith for the response, > > > > Yes it should be line 1142 not 1405, I was using 16.11 and now I'm usin= g > 17.02 and still getting the same error. > > Not sure what to say here, it looks like some type of system configuratio= n > issue as I do not see it on my machine. > > Can you tell if the hugepage has a value and is it sane? The next thing i= s > to see where in that memory is it failing start, end or middle someplace. > Use GDB and compile the code with =E2=80=98make install > T=3Dx86_64-native-lunixapp-gcc EXTRA_CFLAGS=3D=E2=80=9C-g -O0=E2=80=9D' t= hen set a break point > on =E2=80=98b eal_memory.c:1142=E2=80=99 and inspect the memory pointer h= ugepage. I do not > think it is overrun error meaning the size for memset is different then > what was allocated and just stepping off the end. > > Also you did not tell me the linux version you are using? > > > > > On Wed, Feb 22, 2017 at 8:46 PM, Wiles, Keith > wrote: > > > > > On Feb 22, 2017, at 6:43 PM, Wiles, Keith > wrote: > > > > > >> > > >> On Feb 22, 2017, at 6:30 PM, Sushil Adhikari > wrote: > > >> > > >> I used the basic command line option "dpdkTimer -c 0xf -n 4" > > >> And to update on my findings so far I have narrowed down to this > line(1405) > > >> memset(hugepage, 0, nr_hugefiles * sizeof(struct hugepage_file)); > > >> of function rte_eal_hugepage_init() in file > dpdk\lib\librte_eal\linuxapp\eal\eal_memory.c > > > > > > What version of DPDK are you using? I was looking at the file at 1405 > and I do not see a memset() call. > > > > I found the memset call at 1142 in my 17.05-rc0 code. Please try the > latest version and see if you get the same problem. > > > > >> > > >> Yes I have the hugepages of size 2MB(2048) and when I calculate the > memory this memset function is trying to set, it comes out to > 512(nr_hugefiles) * 4144 ( sizeof(struct hugepage_file) ) =3D 2121728 whi= ch > larger than 2MB, so my doubt is that the hugepages I have > allocated(512*2MB) is not contiguous 1GB memory its trying to access memo= ry > thats not part of hugepage, is that a possibility, even though I am setti= ng > up hugepages during boot time by providing it through kernel option. > > >> > > >> > > >> On Wed, Feb 22, 2017 at 8:05 PM, Wiles, Keith > wrote: > > >> > > >>> On Feb 22, 2017, at 3:05 PM, Sushil Adhikari > wrote: > > >>> > > >>> Hi, > > >>> > > >>> I was trying to run dpdk timer app by setting 512 2MB hugepages but > the > > >>> application crashed with following error > > >>> EAL: Detected 4 lcore(s) > > >>> EAL: Probing VFIO support... > > >>> Bus error (core dumped) > > >>> > > >>> If I reduce the number of hugepages to 256 it works fine. I > wondering what > > >>> could be the problem here. Here's my cpu info > > >> > > >> I normally run with 2048 x 2 or 2048 per socket on my machine. What > is the command line you are using to start the application? > > >> > > >>> > > >>> processor : 0 > > >>> vendor_id : GenuineIntel > > >>> cpu family : 6 > > >>> model : 26 > > >>> model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz > > >>> stepping : 5 > > >>> microcode : 0x11 > > >>> cpu MHz : 2794.000 > > >>> cache size : 8192 KB > > >>> physical id : 0 > > >>> siblings : 4 > > >>> core id : 0 > > >>> cpu cores : 4 > > >>> apicid : 0 > > >>> initial apicid : 0 > > >>> fpu : yes > > >>> fpu_exception : yes > > >>> cpuid level : 11 > > >>> wp : yes > > >>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca > > >>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe > syscall nx > > >>> rdtscp lm constant_tsc arch_ > > >>> perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni > dtes64 > > >>> monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcn= t > > >>> lahf_lm ida dtherm tpr_shadow vnm > > >>> i flexpriority ept vpid > > >>> bugs : > > >>> bogomips : 5600.00 > > >>> clflush size : 64 > > >>> cache_alignment : 64 > > >>> address sizes : 36 bits physical, 48 bits virtual > > >>> power management: > > >>> > > >>> processor : 1 > > >>> vendor_id : GenuineIntel > > >>> cpu family : 6 > > >>> model : 26 > > >>> model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz > > >>> stepping : 5 > > >>> microcode : 0x11 > > >>> cpu MHz : 2794.000 > > >>> cache size : 8192 KB > > >>> physical id : 0 > > >>> siblings : 4 > > >>> core id : 1 > > >>> cpu cores : 4 > > >>> apicid : 2 > > >>> initial apicid : 2 > > >>> fpu : yes > > >>> fpu_exception : yes > > >>> cpuid level : 11 > > >>> wp : yes > > >>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr > pge mca > > >>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe > syscall nx > > >>> rdtscp lm constant_tsc arch_ > > >>> perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni > dtes64 > > >>> monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcn= t > > >>> lahf_lm ida dtherm tpr_shadow vnm > > >>> i flexpriority ept vpid > > >>> bugs : > > >>> bogomips : 5600.00 > > >>> clflush size : 64 > > >>> cache_alignment : 64 > > >>> address sizes : 36 bits physical, 48 bits virtual > > >>> power management:...... > > >>> > > >>> And Here's my meminfo > > >>> > > >>> MemTotal: 24679608 kB > > >>> MemFree: 24014156 kB > > >>> MemAvailable: 23950600 kB > > >>> Buffers: 3540 kB > > >>> Cached: 31436 kB > > >>> SwapCached: 0 kB > > >>> Active: 21980 kB > > >>> Inactive: 22256 kB > > >>> Active(anon): 10760 kB > > >>> Inactive(anon): 2940 kB > > >>> Active(file): 11220 kB > > >>> Inactive(file): 19316 kB > > >>> Unevictable: 0 kB > > >>> Mlocked: 0 kB > > >>> SwapTotal: 0 kB > > >>> SwapFree: 0 kB > > >>> Dirty: 32 kB > > >>> Writeback: 0 kB > > >>> AnonPages: 9252 kB > > >>> Mapped: 11912 kB > > >>> Shmem: 4448 kB > > >>> Slab: 27712 kB > > >>> SReclaimable: 11276 kB > > >>> SUnreclaim: 16436 kB > > >>> KernelStack: 2672 kB > > >>> PageTables: 1000 kB > > >>> NFS_Unstable: 0 kB > > >>> Bounce: 0 kB > > >>> WritebackTmp: 0 kB > > >>> CommitLimit: 12077660 kB > > >>> Committed_AS: 137792 kB > > >>> VmallocTotal: 34359738367 kB > > >>> VmallocUsed: 0 kB > > >>> VmallocChunk: 0 kB > > >>> HardwareCorrupted: 0 kB > > >>> AnonHugePages: 2048 kB > > >>> CmaTotal: 0 kB > > >>> CmaFree: 0 kB > > >>> HugePages_Total: 256 > > >>> HugePages_Free: 0 > > >>> HugePages_Rsvd: 0 > > >>> HugePages_Surp: 0 > > >>> Hugepagesize: 2048 kB > > >>> DirectMap4k: 22000 kB > > >>> DirectMap2M: 25133056 kB > > >> > > >> Regards, > > >> Keith > > >> > > >> > > > > > > Regards, > > > Keith > > > > Regards, > > Keith > > > > > > Regards, > Keith > >