From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 851853257 for ; Wed, 12 Jul 2017 04:46:04 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Jul 2017 19:46:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,347,1496127600"; d="scan'208";a="110246450" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga002.jf.intel.com with ESMTP; 11 Jul 2017 19:46:03 -0700 Received: from fmsmsx122.amr.corp.intel.com (10.18.125.37) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 11 Jul 2017 19:46:03 -0700 Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by fmsmsx122.amr.corp.intel.com (10.18.125.37) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 11 Jul 2017 19:46:02 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.116]) by shsmsx102.ccr.corp.intel.com ([169.254.2.146]) with mapi id 14.03.0319.002; Wed, 12 Jul 2017 10:45:59 +0800 From: "Tan, Jianfeng" To: "Gonzalez Monroy, Sergio" , "Stephen Hemminger" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [RFC] pci: force address of mappings in secondary process Thread-Index: AQHS+eLgyG6dcA8FKUmPJC0hVH7QXaJN3UYwgAAce4CAAYJAwA== Date: Wed, 12 Jul 2017 02:45:57 +0000 Message-ID: References: <20170711011242.4606-1-stephen@networkplumber.org> <3f48cbd3-1e60-5a92-3929-7ba06e3cc69c@intel.com> In-Reply-To: <3f48cbd3-1e60-5a92-3929-7ba06e3cc69c@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jul 2017 02:46:05 -0000 > -----Original Message----- > From: Gonzalez Monroy, Sergio > Sent: Tuesday, July 11, 2017 7:36 PM > To: Tan, Jianfeng; Stephen Hemminger; dev@dpdk.org > Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary > process >=20 > On 11/07/2017 02:56, Tan, Jianfeng wrote: > >> -----Original Message----- > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen > >> Hemminger > >> Sent: Tuesday, July 11, 2017 9:13 AM > >> To: dev@dpdk.org > >> Cc: Stephen Hemminger > >> Subject: [dpdk-dev] [RFC] pci: force address of mappings in secondary > >> process > >> > >> The PCI memory resources in the secondary process should be in > >> the exact same location as the primary process. Otherwise > >> there is a risk of a stray pointer. > >> > >> Not sure if this is right, but it looks like a potential > >> problem. > >> > >> --- > >> lib/librte_eal/common/eal_common_pci_uio.c | 2 +- > >> 1 file changed, 1 insertion(+), 1 deletion(-) > >> > >> diff --git a/lib/librte_eal/common/eal_common_pci_uio.c > >> b/lib/librte_eal/common/eal_common_pci_uio.c > >> index 367a6816dcb8..2156b1a436c4 100644 > >> --- a/lib/librte_eal/common/eal_common_pci_uio.c > >> +++ b/lib/librte_eal/common/eal_common_pci_uio.c > >> @@ -77,7 +77,7 @@ pci_uio_map_secondary(struct rte_pci_device *dev) > >> > >> void *mapaddr =3D pci_map_resource(uio_res- > >>> maps[i].addr, > >> fd, (off_t)uio_res->maps[i].offset, > >> - (size_t)uio_res->maps[i].size, 0); > >> + (size_t)uio_res->maps[i].size, > >> MAP_FIXED); > >> /* fd is not needed in slave process, close it */ > >> close(fd); > >> if (mapaddr !=3D uio_res->maps[i].addr) { > >> -- > >> 2.11.0 > > +1 for this RFC. I also once encounter such problem, and I use the same > way to solve it. The addr parameter of mmap() syscall is only a hint inst= ead of > a must even the VMA is not occupied yet. > > > > Thanks, > > Jianfeng >=20 > How do you know the VMA is not occupied? I did by check /proc/self/maps. >=20 > I think the risk here is that the dynamic linker loaded some shared > library in that VMA, and forcing MAP_FIXED is not a safe solution. > What I have observed is that Linux will return a different VMA than the > one hinted when there is already a mapping in the requested/hinted VMA. IMO, that's not the target of this RFC. The target is to solve the situatio= n (in current primary/secondary model) that kernel will not use the addr ev= en there's no VMA on that addr. This is my understanding, Stephen, please c= orrect me if I'm wrong. >=20 > I reckon this is a similar issue as we have with the multi-process model > when we do not get the VMA requested for the huge-pages. > AFAIK we do not have a robust solution for this issue other than restart > the program and hope the dynamic linker does not map anything in the VMA > ranges that we need to map from the primary. This is also assuming that > the application does not allocate memory and maps things before calling > eal_init as it could potentially use VMA ranges that we need in the > secondary process. This is another problem. >=20 > The proposal for new secondary process model would solve these issues: > http://dpdk.org/ml/archives/dev/2017-May/066147.html And yes, this might happen to solve the targeted issue in this RFC. But bef= ore the new model is out, this patch seems a workable way for the original = issue. Thanks, Jianfeng >=20 > Thanks, > Sergio