From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f66.google.com (mail-oi0-f66.google.com [209.85.218.66]) by dpdk.org (Postfix) with ESMTP id 0D53B316B for ; Thu, 23 Feb 2017 17:10:56 +0100 (CET) Received: by mail-oi0-f66.google.com with SMTP id f204so1623224oia.1 for ; Thu, 23 Feb 2017 08:10:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mJs2zHH5HU9Z8eGI8CDOtjDCC2drjwzKAgt1qAivY8o=; b=X/XsB4IZV4HiXdXpLrHKn8IoEc+oq08Ev5ybN82OIeP2RDuK7b1IQYnht1mgomOqlf QFu4xi2wE5iyr3iAwbmY6VUzHwOkie6sCFafEcEV4MbqsfbKm/Ho4kgAjDBwcjOuwq98 js80mfaRHSh1tz89qZYeTTMgzhmAFEdoAbTQU/NyegYQatVAe/MXyhGMZzoG32cNYTYh fprF/diCUWyTripOykD6M+bijLKX4RT9/qu6J57jUraIHKRBYndqw3J2ucGUv0ZZVo4y Np55+Xp9ATzxjIoH512g2nzxxAfR631+MN3nv/jVGg5+RfPDQgbQtKecfSWNUshQLwdk 7XQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mJs2zHH5HU9Z8eGI8CDOtjDCC2drjwzKAgt1qAivY8o=; b=h4nh6lPT9Iu7hUccryvkhk+1Vn1e6l4fGXTikd0VoRd+7japnPKqTC2V+yRJsxnhUz 1UinxXoOWTqBxSA4gJ6J3eWYCn6HDxUnYFlNZAMVfqXN50/kYoab2/gUTci+NhEN1HnR BdGZ/ic+UHw47avzsHm9cEU27QW3cNRJLXgtpc9gJ2U3VH4sAQOAnsoIG2nTcSb7j1Je /8KypbpJsDKTSz7KCmHDhRLnh/ZKYA2OGvez05v9su/dikdfGcqc8P+Mxy6gp/CAwTH6 K8YU3pysZSOhvVdBBNn11/MWivvBWGws6tqd4XG5Gm0SAPSc5mbSx0dtyO2qd+dKn/lU OCgg== X-Gm-Message-State: AMke39kFACWdc6Rr5O+VkhBGfssmlRj+YR05BwVyEDsHtTCdYc6AqI1THenT1hMO1zr8ISA1nfjgdKwtFn7EaQ== X-Received: by 10.202.72.2 with SMTP id v2mr7584180oia.179.1487866256055; Thu, 23 Feb 2017 08:10:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.182.110.38 with HTTP; Thu, 23 Feb 2017 08:10:35 -0800 (PST) In-Reply-To: References: <18013811-F9E1-42C6-A87B-D087018AF0B9@intel.com> <6649EDAE-E9E4-4C49-855C-56E37BD4FA88@intel.com> <986FF8E4-1D37-4275-B9E3-8C7E19996461@intel.com> <516ac630-f787-9bae-88b3-2ef5b35cda9d@intel.com> From: Sushil Adhikari Date: Thu, 23 Feb 2017 10:10:35 -0600 Message-ID: To: Sergio Gonzalez Monroy Cc: "Wiles, Keith" , "users@dpdk.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Larger number of hugepages causes bus error. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Feb 2017 16:10:57 -0000 [~]$ ulimit -aHS -f: file size (blocks) unlimited -t: cpu time (seconds) unlimited -d: data seg size (kb) unlimited -s: stack size (kb) 8192 -c: core file size (blocks) 0 -m: resident set size (kb) unlimited -l: locked memory (kb) 64 -p: processes 96391 -n: file descriptors 1024 -v: address space (kb) unlimited -w: locks unlimited -e: scheduling priority 0 -r: real-time priority 0 On Thu, Feb 23, 2017 at 9:20 AM, Sergio Gonzalez Monroy < sergio.gonzalez.monroy@intel.com> wrote: > Nevermind the 'root' question, given that it works with 256 hugepages :) > > > On 23/02/2017 15:18, Sergio Gonzalez Monroy wrote: > >> That line is just setting to 0 a shared memory area where we store >> hugepage information. >> >> Are you running as privileged user (ie. root)? >> What is the result of 'ulimit -aHS'? >> >> Thanks, >> Sergio >> >> On 23/02/2017 03:18, Sushil Adhikari wrote: >> >>> Thank you Keith for the response, >>> >>> Yes it should be line 1142 not 1405, I was using 16.11 and now I'm using >>> 17.02 and still getting the same error. >>> >>> On Wed, Feb 22, 2017 at 8:46 PM, Wiles, Keith >>> wrote: >>> >>> On Feb 22, 2017, at 6:43 PM, Wiles, Keith wrote: >>>>> >>>>> On Feb 22, 2017, at 6:30 PM, Sushil Adhikari >>>>>> >>>>> wrote: >>>> >>>>> I used the basic command line option "dpdkTimer -c 0xf -n 4" >>>>>> And to update on my findings so far I have narrowed down to this >>>>>> >>>>> line(1405) >>>> >>>>> memset(hugepage, 0, nr_hugefiles * sizeof(struct hugepage_file)); >>>>>> of function rte_eal_hugepage_init() in file >>>>>> >>>>> dpdk\lib\librte_eal\linuxapp\eal\eal_memory.c >>>> >>>>> What version of DPDK are you using? I was looking at the file at 1405 >>>>> >>>> and I do not see a memset() call. >>>> >>>> I found the memset call at 1142 in my 17.05-rc0 code. Please try the >>>> latest version and see if you get the same problem. >>>> >>>> Yes I have the hugepages of size 2MB(2048) and when I calculate the >>>>>> >>>>> memory this memset function is trying to set, it comes out to >>>> 512(nr_hugefiles) * 4144 ( sizeof(struct hugepage_file) ) = 2121728 >>>> which >>>> larger than 2MB, so my doubt is that the hugepages I have >>>> allocated(512*2MB) is not contiguous 1GB memory its trying to access >>>> memory >>>> thats not part of hugepage, is that a possibility, even though I am >>>> setting >>>> up hugepages during boot time by providing it through kernel option. >>>> >>>>> >>>>>> On Wed, Feb 22, 2017 at 8:05 PM, Wiles, Keith >>>>>> >>>>> wrote: >>>> >>>>> On Feb 22, 2017, at 3:05 PM, Sushil Adhikari >>>>>>> >>>>>> wrote: >>>> >>>>> Hi, >>>>>>> >>>>>>> I was trying to run dpdk timer app by setting 512 2MB hugepages but >>>>>>> the >>>>>>> application crashed with following error >>>>>>> EAL: Detected 4 lcore(s) >>>>>>> EAL: Probing VFIO support... >>>>>>> Bus error (core dumped) >>>>>>> >>>>>>> If I reduce the number of hugepages to 256 it works fine. I wondering >>>>>>> >>>>>> what >>>> >>>>> could be the problem here. Here's my cpu info >>>>>>> >>>>>> I normally run with 2048 x 2 or 2048 per socket on my machine. What is >>>>>> >>>>> the command line you are using to start the application? >>>> >>>>> processor : 0 >>>>>>> vendor_id : GenuineIntel >>>>>>> cpu family : 6 >>>>>>> model : 26 >>>>>>> model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz >>>>>>> stepping : 5 >>>>>>> microcode : 0x11 >>>>>>> cpu MHz : 2794.000 >>>>>>> cache size : 8192 KB >>>>>>> physical id : 0 >>>>>>> siblings : 4 >>>>>>> core id : 0 >>>>>>> cpu cores : 4 >>>>>>> apicid : 0 >>>>>>> initial apicid : 0 >>>>>>> fpu : yes >>>>>>> fpu_exception : yes >>>>>>> cpuid level : 11 >>>>>>> wp : yes >>>>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr >>>>>>> pge >>>>>>> >>>>>> mca >>>> >>>>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall >>>>>>> >>>>>> nx >>>> >>>>> rdtscp lm constant_tsc arch_ >>>>>>> perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni >>>>>>> >>>>>> dtes64 >>>> >>>>> monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt >>>>>>> lahf_lm ida dtherm tpr_shadow vnm >>>>>>> i flexpriority ept vpid >>>>>>> bugs : >>>>>>> bogomips : 5600.00 >>>>>>> clflush size : 64 >>>>>>> cache_alignment : 64 >>>>>>> address sizes : 36 bits physical, 48 bits virtual >>>>>>> power management: >>>>>>> >>>>>>> processor : 1 >>>>>>> vendor_id : GenuineIntel >>>>>>> cpu family : 6 >>>>>>> model : 26 >>>>>>> model name : Intel(R) Core(TM) i7 CPU 950 @ 3.07GHz >>>>>>> stepping : 5 >>>>>>> microcode : 0x11 >>>>>>> cpu MHz : 2794.000 >>>>>>> cache size : 8192 KB >>>>>>> physical id : 0 >>>>>>> siblings : 4 >>>>>>> core id : 1 >>>>>>> cpu cores : 4 >>>>>>> apicid : 2 >>>>>>> initial apicid : 2 >>>>>>> fpu : yes >>>>>>> fpu_exception : yes >>>>>>> cpuid level : 11 >>>>>>> wp : yes >>>>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr >>>>>>> pge >>>>>>> >>>>>> mca >>>> >>>>> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall >>>>>>> >>>>>> nx >>>> >>>>> rdtscp lm constant_tsc arch_ >>>>>>> perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni >>>>>>> >>>>>> dtes64 >>>> >>>>> monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt >>>>>>> lahf_lm ida dtherm tpr_shadow vnm >>>>>>> i flexpriority ept vpid >>>>>>> bugs : >>>>>>> bogomips : 5600.00 >>>>>>> clflush size : 64 >>>>>>> cache_alignment : 64 >>>>>>> address sizes : 36 bits physical, 48 bits virtual >>>>>>> power management:...... >>>>>>> >>>>>>> And Here's my meminfo >>>>>>> >>>>>>> MemTotal: 24679608 kB >>>>>>> MemFree: 24014156 kB >>>>>>> MemAvailable: 23950600 kB >>>>>>> Buffers: 3540 kB >>>>>>> Cached: 31436 kB >>>>>>> SwapCached: 0 kB >>>>>>> Active: 21980 kB >>>>>>> Inactive: 22256 kB >>>>>>> Active(anon): 10760 kB >>>>>>> Inactive(anon): 2940 kB >>>>>>> Active(file): 11220 kB >>>>>>> Inactive(file): 19316 kB >>>>>>> Unevictable: 0 kB >>>>>>> Mlocked: 0 kB >>>>>>> SwapTotal: 0 kB >>>>>>> SwapFree: 0 kB >>>>>>> Dirty: 32 kB >>>>>>> Writeback: 0 kB >>>>>>> AnonPages: 9252 kB >>>>>>> Mapped: 11912 kB >>>>>>> Shmem: 4448 kB >>>>>>> Slab: 27712 kB >>>>>>> SReclaimable: 11276 kB >>>>>>> SUnreclaim: 16436 kB >>>>>>> KernelStack: 2672 kB >>>>>>> PageTables: 1000 kB >>>>>>> NFS_Unstable: 0 kB >>>>>>> Bounce: 0 kB >>>>>>> WritebackTmp: 0 kB >>>>>>> CommitLimit: 12077660 kB >>>>>>> Committed_AS: 137792 kB >>>>>>> VmallocTotal: 34359738367 kB >>>>>>> VmallocUsed: 0 kB >>>>>>> VmallocChunk: 0 kB >>>>>>> HardwareCorrupted: 0 kB >>>>>>> AnonHugePages: 2048 kB >>>>>>> CmaTotal: 0 kB >>>>>>> CmaFree: 0 kB >>>>>>> HugePages_Total: 256 >>>>>>> HugePages_Free: 0 >>>>>>> HugePages_Rsvd: 0 >>>>>>> HugePages_Surp: 0 >>>>>>> Hugepagesize: 2048 kB >>>>>>> DirectMap4k: 22000 kB >>>>>>> DirectMap2M: 25133056 kB >>>>>>> >>>>>> Regards, >>>>>> Keith >>>>>> >>>>>> >>>>>> Regards, >>>>> Keith >>>>> >>>> Regards, >>>> Keith >>>> >>>> >>>> >> >