From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0113.outbound.protection.outlook.com [157.56.110.113]) by dpdk.org (Postfix) with ESMTP id 052EF567A for ; Mon, 25 Jan 2016 21:59:52 +0100 (CET) Received: from BY2PR07MB057.namprd07.prod.outlook.com (10.255.240.149) by BY2PR07MB060.namprd07.prod.outlook.com (10.255.240.152) with Microsoft SMTP Server (TLS) id 15.1.365.19; Mon, 25 Jan 2016 20:59:48 +0000 Received: from BY2PR07MB057.namprd07.prod.outlook.com ([169.254.11.152]) by BY2PR07MB057.namprd07.prod.outlook.com ([169.254.11.152]) with mapi id 15.01.0365.024; Mon, 25 Jan 2016 20:59:47 +0000 From: Alain Gautherot To: Sergio Gonzalez Monroy , "users@dpdk.org" Thread-Topic: [dpdk-users] Using DPDK for contiguous physical memory allocation Thread-Index: AdFVcytLoG1gyaOuSxWKRzvG0vqHRACBCuIAAA6awIA= Date: Mon, 25 Jan 2016 20:59:46 +0000 Message-ID: References: <56A62820.4090101@intel.com> In-Reply-To: <56A62820.4090101@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=alain@edicogenome.com; x-originating-ip: [70.167.8.242] x-microsoft-exchange-diagnostics: 1; BY2PR07MB060; 5:4n5BnPkiZXE3XgnJt/tEQo6VtD2LBpa2CVWk8xuPlauxdevKtfIvX5NRBMTFR/1+qab74mpzSQGgUPrCBzw/M3WyIyxbsKC40QZVYGKUx45gjGF/C4DITSDxiFYiO92Bja/JkShQEpfm/U7FKZlrAA==; 24:oPPUkOiNT8v8oaVMFupRB3dS2MV19YxBrjt3pI/Lvcfoi92oKJH11RDbo56S5fO2GIVyft96WSmQZHCnz9898q1gqsw7pUX40h5cz4w0EM0= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY2PR07MB060; x-ms-office365-filtering-correlation-id: a53997d3-559c-468b-647a-08d325ca75b7 x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(102615245)(601004)(2401047)(5005006)(520078)(8121501046)(3002001)(10201501046); SRVR:BY2PR07MB060; BCL:0; PCL:0; RULEID:; SRVR:BY2PR07MB060; x-forefront-prvs: 083289FD26 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(377454003)(189002)(164054003)(24454002)(13464003)(199003)(24434003)(479174004)(10400500002)(87936001)(19580405001)(122556002)(2906002)(575784001)(5890100001)(92566002)(86362001)(40100003)(2501003)(15395725005)(5004730100002)(1220700001)(102836003)(1096002)(76576001)(2900100001)(107886002)(5001960100002)(74316001)(15975445007)(54356999)(2950100001)(3846002)(1720100001)(99936001)(81156007)(6116002)(97736004)(33656002)(5001770100001)(19580395003)(189998001)(101416001)(99286002)(66066001)(106356001)(50986999)(105586002)(5008740100001)(5003600100002)(586003)(76176999)(3280700002)(5002640100001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY2PR07MB060; H:BY2PR07MB057.namprd07.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: edicogenome.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: edicogenome.com X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Jan 2016 20:59:46.9990 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 715cf729-d8ae-46d7-a6f6-5d36941a1b6c X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR07MB060 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory allocation X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2016 20:59:52 -0000 Hello Sergio, I'm running the following command $ ./build/helloworld -c fff -n 1 And get the attached log (hope it goes through). Using "-n 2" (I'm not sure how many channels) gives the same SIGSEGV error. Here's the configuration: $ numactl -H available: 1 nodes (0) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 node 0 size: 65431 MB node 0 free: 62040 MB node distances: node 0 0: 10 $ cat /proc/meminfo MemTotal: 65867360 kB MemFree: 63529276 kB Buffers: 93996 kB Cached: 562160 kB SwapCached: 0 kB Active: 314816 kB Inactive: 483752 kB Active(anon): 144372 kB Inactive(anon): 28 kB Active(file): 170444 kB Inactive(file): 483724 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 12 kB Writeback: 0 kB AnonPages: 144184 kB Mapped: 49004 kB Shmem: 280 kB Slab: 77572 kB SReclaimable: 31580 kB SUnreclaim: 45992 kB KernelStack: 2904 kB PageTables: 7744 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 32421680 kB Committed_AS: 383316 kB VmallocTotal: 34359738367 kB VmallocUsed: 378992 kB VmallocChunk: 34359352736 kB HardwareCorrupted: 0 kB AnonHugePages: 73728 kB HugePa ges_Total: 500 HugePages_Free: 9 HugePages_Rsvd: 9 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 4096 kB DirectMap2M: 2027520 kB DirectMap1G: 65011712 kB Thanks, Alain -----Original Message----- From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.monroy@intel.com] Sent: Monday, January 25, 2016 5:50 AM To: Alain Gautherot ; users@dpdk.org Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory allocation On 23/01/2016 00:20, Alain Gautherot wrote: > Hello, > > I came across DPDK in a thread @ http://stackoverflow.com/questions/4401912/linux-contiguous-physical-memory-from-userspace (bottom reply from mrsmith) and wanted to see if I can use rte_malloc() to allocate large blocks of contiguous physical memory (16GB or even 32GB at some point). > > The platform I'm working on has an FPGA that shares host memory with the x86_64 cores via a QPI link. > The FPGA crunches data directly from host memory and uses physical addresses (mostly a QPI limitation, but it is also dictated by performance considerations and the ability to make the best possible use of multiple memory controllers). > The data shared is 16GB or up to 32GB and could be provided as multiple descriptors to the FPGA, but that still means that each descriptor is in the order of several GBytes each. > I understand that allocation may fail, but is ok for now, since I'm still in the proof-of-concept stage, trying to rule things out. > > My sample application attempts to allocate memory by chunks of 100MB like so: > > int main(int argc, char **argv) > { > int ret; > > ret = rte_eal_init(argc, argv); > if (ret < 0) { > rte_panic("Cannot init EAL\n"); > } > > int i; > for (i = 1; i <= 100; ++i) { > size_t allocsize = i * 100*1000*1000; > > printf(" Allocating %3.1fGB: ", ((float )i)/10.0f); > fflush(stdout); > void* ptr = rte_malloc(NULL, allocsize, 0U); > if (ptr != NULL) { > printf("PASS\n"); > rte_free(ptr); > } else { > printf("fail\n"); > } > } > > printf("Done\n"); > return 0; > } > > I get a consistent crash @ the 2.2GB mark: > (gdb) r -c f -n 4 > ... > EAL: PCI device 0000:06:00.1 on NUMA socket 0 > EAL: probe driver: 8086:1521 rte_igb_pmd > EAL: Not managed by a supported kernel driver, skipped > Allocating 0.1GB: fail > Allocating 0.2GB: fail > ... > Allocating 2.0GB: fail > Allocating 2.1GB: fail > Allocating 2.2GB: > Program received signal SIGSEGV, Segmentation fault. > 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880, heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064) > at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61 > 61 elem->heap = heap; > Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.el6_6.5.x86_64 > (gdb) bt > ... > #0 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880, heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064) > at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61 > #1 0x00000000004c694e in split_elem (elem=0x7ffff3e00000, split_pt=0x800070eaa880) at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:121 > #2 0x00000000004c6bda in malloc_elem_alloc (elem=0x7ffff3e00000, size=18446744071614584320, align=64) > at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:223 > #3 0x00000000004c736e in malloc_heap_alloc (heap=0x7ffff7fe561c, type=0x0, size=18446744071614584320, align=64) > at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_heap.c:167 > #4 0x00000000004c0aa1 in rte_malloc_socket (type=0x0, size=18446744071614584320, align=0, socket_arg=-1) > at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:89 > #5 0x00000000004c0b5b in rte_malloc (type=0x0, size=18446744071614584320, align=0) at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:115 > #6 0x000000000041ca6e in main (argc=5, argv=0x7fffffffdd48) at /home/alaing/INTEL/dpdk-2.0.0/examples/hugephymem/main.c:66 > > > Has anybody seen such an issue? > Could I be misusing RTE somehow? > What options are you running your DPDK app with? Can you also provide the full initialization log and hugepage info? Sergio > Thanks for your time, > Alain > > > -- > Alain Gautherot > Edico Genome > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dpdk_log.txt URL: