From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-outbound-2.vmware.com (smtp-outbound-2.vmware.com [208.91.2.13]) by dpdk.org (Postfix) with ESMTP id 453FC2E8B for ; Thu, 11 Sep 2014 00:35:52 +0200 (CEST) Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-2.vmware.com (Postfix) with ESMTP id F34BCA0748 for ; Wed, 10 Sep 2014 15:40:55 -0700 (PDT) Received: from EX13-CAS-006.vmware.com (EX13-CAS-006.vmware.com [10.113.191.56]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id D912F1A1D4 for ; Wed, 10 Sep 2014 15:40:55 -0700 (PDT) Received: from EX13-MBX-010.vmware.com (10.113.191.30) by EX13-MBX-006.vmware.com (10.113.191.26) with Microsoft SMTP Server (TLS) id 15.0.775.38; Wed, 10 Sep 2014 15:40:55 -0700 Received: from EX13-MBX-010.vmware.com ([fe80::c937:743c:749c:829b]) by EX13-MBX-010.vmware.com ([fe80::c937:743c:749c:829b%15]) with mapi id 15.00.0775.031; Wed, 10 Sep 2014 15:40:37 -0700 From: "Michael Hu (NSBU)" To: "dev@dpdk.org" Thread-Topic: dpdk starting issue with descending virtual address allocation in new kernel Thread-Index: AQHPzUg9qbpltOS5kEGRYY5uwZL+hw== Date: Wed, 10 Sep 2014 22:40:36 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.113.160.246] MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] dpdk starting issue with descending virtual address allocation in new kernel X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 22:35:53 -0000 Hi All, We have a kernel config question to consult you. DPDK failed to start due to mbuf creation issue with new kernel 3.14.17 + g= rsecurity patches. We tries to trace down the issue, it seems that the virtual address of hug= e page is allocated from high address to low address by kernel where dpdk e= xpects it to be low to high to think it is as consecutive. See dumped virt = address bellow. It is first 0x710421400000, then 0x710421200000. Where prev= iously it would be 0x710421200000 first , then 0x710421400000. But they are= still consecutive. ---- Initialize Port 0 -- TxQ 1, RxQ 1, Src MAC 00:0c:29:b3:30:db Create: Default RX 0:0 - Memory used (MBUFs 4096 x (size 1984 + Hdr 6= 4)) + 790720 =3D 8965 KB Zone 0: name:, phys:0x6ac00000, len:0x2080, virt:0x71042= 1400000, socket_id:0, flags:0 Zone 1: name:, phys:0x6ac02080, len:0x1d10c0, virt:0x710421= 402080, socket_id:0, flags:0 Zone 2: name:, phys:0x6ae00000, len:0x160000, virt:0x7104= 21200000, socket_id:0, flags:0 Zone 3: name:, phys:0x6add3140, len:0x11a00, virt:0x71042= 15d3140, socket_id:0, flags:0 Zone 4: name:, phys:0x6ade4b40, len:0x300, virt:0= x7104215e4b40, socket_id:0, flags:0 Zone 5: name:, phys:0x6ade4e80, len:0x200, vir= t:0x7104215e4e80, socket_id:0, flags:0 Zone 6: name:, phys:0x6ade5080, len:0x10080, virt:0x= 7104215e5080, socket_id:0, flags:0 Segment 0: phys:0x6ac00000, len:2097152, virt:0x710421400000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 1: phys:0x6ae00000, len:2097152, virt:0x710421200000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 2: phys:0x6b000000, len:2097152, virt:0x710421000000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 3: phys:0x6b200000, len:2097152, virt:0x710420e00000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 4: phys:0x6b400000, len:2097152, virt:0x710420c00000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 5: phys:0x6b600000, len:2097152, virt:0x710420a00000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 6: phys:0x6b800000, len:2097152, virt:0x710420800000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 7: phys:0x6ba00000, len:2097152, virt:0x710420600000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 8: phys:0x6bc00000, len:2097152, virt:0x710420400000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 Segment 9: phys:0x6be00000, len:2097152, virt:0x710420200000, socket_id:0, = hugepage_sz:2097152, nchannel:0, nrank:0 --- Related dpdk code is in dpdk/lib/librte_eal/linuxapp/eal/eal_memory.c :: rte_eal_hugepage_init() for (i =3D 0; i < nr_hugefiles; i++) { new_memseg =3D 0; /* if this is a new section, create a new memseg */ if (i =3D=3D 0) new_memseg =3D 1; else if (hugepage[i].socket_id !=3D hugepage[i-1].socket_id) new_memseg =3D 1; else if (hugepage[i].size !=3D hugepage[i-1].size) new_memseg =3D 1; else if ((hugepage[i].physaddr - hugepage[i-1].physaddr) !=3D hugepage[i].size) new_memseg =3D 1; else if (((unsigned long)hugepage[i].final_va - (unsigned long)hugepage[i-1].final_va) !=3D hugepage[i].size) { new_memseg =3D 1; } Is this a known issue? Is there any workaround? Or Could you advise which k= ernel config may relate this this kernel behavior change? Thanks, Michael