From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 93CEBB126 for ; Fri, 13 Jun 2014 23:31:06 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP; 13 Jun 2014 14:31:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,473,1400050800"; d="scan'208";a="555300850" Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159]) by fmsmga002.fm.intel.com with ESMTP; 13 Jun 2014 14:30:57 -0700 Received: from irsmsx105.ger.corp.intel.com (163.33.3.28) by IRSMSX104.ger.corp.intel.com (163.33.3.159) with Microsoft SMTP Server (TLS) id 14.3.123.3; Fri, 13 Jun 2014 22:27:56 +0100 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.58]) by IRSMSX105.ger.corp.intel.com ([169.254.7.239]) with mapi id 14.03.0123.003; Fri, 13 Jun 2014 22:27:56 +0100 From: "Richardson, Bruce" To: "Gooch, Stephen (Wind River)" , "dev@dpdk.org" Thread-Topic: mmap() hint address Thread-Index: Ac+HSjModLMx+SPRQcmV/p2lSa3M/AAA1WOw Date: Fri, 13 Jun 2014 21:27:55 +0000 Message-ID: <59AF69C657FD0841A61C55336867B5B01AA361A6@IRSMSX103.ger.corp.intel.com> References: <9205DC19ECCD944CA2FAC59508A772BABCEFF60C@ALA-MBA.corp.ad.wrs.com> In-Reply-To: <9205DC19ECCD944CA2FAC59508A772BABCEFF60C@ALA-MBA.corp.ad.wrs.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] mmap() hint address X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 21:31:07 -0000 > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Gooch, Stephen > Sent: Friday, June 13, 2014 2:03 PM > To: dev@dpdk.org > Subject: [dpdk-dev] mmap() hint address >=20 > Hello, >=20 > I have seen a case where a secondary DPDK process tries to map uio resour= ce in > which mmap() normally sends the corresponding virtual address as a hint > address. However on some instances mmap() returns a virtual address that= is > not the hint address, and it result in rte_panic() and the secondary proc= ess goes > defunct. >=20 > This happens from time to time on an embedded device when nr_hugepages is > set to 128, but never when nr_hugepage is set to 256 on the same device. = My > question is, if mmap() can find the correct memory regions when hugepages= is > set to 256, would it not require less resources (and therefore be more li= kely to > pass) at a lower value such as 128? >=20 > Any ideas what would cause this mmap() behavior at a lower nr_hugepage > value? >=20 > - Stephen Hi Stephen, That's a strange one! I don't know for definite why this is happening, but here is one possible t= heory. :-) It could be due to the size of the memory blocks that are getting mmapped. = When you use 256 pages, the blocks of memory getting mapped may well be lar= ger (depending on how fragmented in memory the 2MB pages are), and so may b= e getting mapped at a higher set of address ranges where there is more free= memory. This set of address ranges is then free in the secondary process a= nd it is similarly able to map the memory. With the 128 hugepages, you may be looking for smaller amounts of memory an= d so the addresses get mapped in at a different spot in the virtual address= space, one that may be more heavily used. Then when the secondary process = tries to duplicate the mappings, it already has memory in that region in us= e and the mapping fails. In short - one theory is that having bigger blocks to map causes the memory= to be mapped to a different location in memory which is free from conflict= s in the secondary process. So, how to confirm or refute this, and generally debug this issue? Well, in general we would need to look at the messages printed out at star= tup in the primary process to see how big of blocks it is trying to map in = each case, and where they end up in the virtual address-space. Regards, /Bruce