From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id E781C2E8B for ; Wed, 12 Jul 2017 10:58:39 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga104.jf.intel.com with ESMTP; 12 Jul 2017 01:58:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,349,1496127600"; d="scan'208";a="1171489453" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by fmsmga001.fm.intel.com with ESMTP; 12 Jul 2017 01:58:38 -0700 Received: from fmsmsx151.amr.corp.intel.com (10.18.125.4) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 12 Jul 2017 01:58:38 -0700 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by FMSMSX151.amr.corp.intel.com (10.18.125.4) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 12 Jul 2017 01:58:30 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.116]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.122]) with mapi id 14.03.0319.002; Wed, 12 Jul 2017 16:58:29 +0800 From: "Tan, Jianfeng" To: "Gonzalez Monroy, Sergio" , "Stephen Hemminger" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [RFC] pci: force address of mappings in secondary process Thread-Index: AQHS+eLgyG6dcA8FKUmPJC0hVH7QXaJN3UYwgAAce4CAAYJAwP//y+oAgACc1zA= Date: Wed, 12 Jul 2017 08:58:28 +0000 Message-ID: References: <20170711011242.4606-1-stephen@networkplumber.org> <3f48cbd3-1e60-5a92-3929-7ba06e3cc69c@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jul 2017 08:58:40 -0000 > -----Original Message----- > From: Gonzalez Monroy, Sergio > Sent: Wednesday, July 12, 2017 3:32 PM > To: Tan, Jianfeng; Stephen Hemminger; dev@dpdk.org > Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary > process >=20 > On 12/07/2017 03:45, Tan, Jianfeng wrote: > > > >> -----Original Message----- > >> From: Gonzalez Monroy, Sergio > >> Sent: Tuesday, July 11, 2017 7:36 PM > >> To: Tan, Jianfeng; Stephen Hemminger; dev@dpdk.org > >> Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in second= ary > >> process > >> > >> On 11/07/2017 02:56, Tan, Jianfeng wrote: > >>>> -----Original Message----- > >>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Stephen > >>>> Hemminger > >>>> Sent: Tuesday, July 11, 2017 9:13 AM > >>>> To: dev@dpdk.org > >>>> Cc: Stephen Hemminger > >>>> Subject: [dpdk-dev] [RFC] pci: force address of mappings in secondar= y > >>>> process > >>>> > >>>> The PCI memory resources in the secondary process should be in > >>>> the exact same location as the primary process. Otherwise > >>>> there is a risk of a stray pointer. > >>>> > >>>> Not sure if this is right, but it looks like a potential > >>>> problem. > >>>> > >>>> --- > >>>> lib/librte_eal/common/eal_common_pci_uio.c | 2 +- > >>>> 1 file changed, 1 insertion(+), 1 deletion(-) > >>>> > >>>> diff --git a/lib/librte_eal/common/eal_common_pci_uio.c > >>>> b/lib/librte_eal/common/eal_common_pci_uio.c > >>>> index 367a6816dcb8..2156b1a436c4 100644 > >>>> --- a/lib/librte_eal/common/eal_common_pci_uio.c > >>>> +++ b/lib/librte_eal/common/eal_common_pci_uio.c > >>>> @@ -77,7 +77,7 @@ pci_uio_map_secondary(struct rte_pci_device > *dev) > >>>> > >>>> void *mapaddr =3D pci_map_resource(uio_res- > >>>>> maps[i].addr, > >>>> fd, (off_t)uio_res->maps[i].offset, > >>>> - (size_t)uio_res->maps[i].size, 0); > >>>> + (size_t)uio_res->maps[i].size, > >>>> MAP_FIXED); > >>>> /* fd is not needed in slave process, close it */ > >>>> close(fd); > >>>> if (mapaddr !=3D uio_res->maps[i].addr) { > >>>> -- > >>>> 2.11.0 > >>> +1 for this RFC. I also once encounter such problem, and I use the sa= me > >> way to solve it. The addr parameter of mmap() syscall is only a hint i= nstead > of > >> a must even the VMA is not occupied yet. > >>> Thanks, > >>> Jianfeng > >> How do you know the VMA is not occupied? > > I did by check /proc/self/maps. > > > >> I think the risk here is that the dynamic linker loaded some shared > >> library in that VMA, and forcing MAP_FIXED is not a safe solution. > >> What I have observed is that Linux will return a different VMA than th= e > >> one hinted when there is already a mapping in the requested/hinted > VMA. > > IMO, that's not the target of this RFC. The target is to solve the situ= ation (in > current primary/secondary model) that kernel will not use the addr even > there's no VMA on that addr. This is my understanding, Stephen, please > correct me if I'm wrong. >=20 > The point I was trying to make is, that you do not know if there is a > mapping or not in that address, and by using MAP_FIXED it will unmap > whatever was in there before. Oh, I missed that if there's conflict, the existing VMA will be unmapped. T= hat's a bad effect. >=20 > So unless you parse /proc/self/maps and check that the VMA range is not > being used, forcing MAP_FIXED is not safe. >=20 > >> I reckon this is a similar issue as we have with the multi-process mod= el > >> when we do not get the VMA requested for the huge-pages. > >> AFAIK we do not have a robust solution for this issue other than resta= rt > >> the program and hope the dynamic linker does not map anything in the > VMA > >> ranges that we need to map from the primary. This is also assuming tha= t > >> the application does not allocate memory and maps things before callin= g > >> eal_init as it could potentially use VMA ranges that we need in the > >> secondary process. > > This is another problem. >=20 > It is the same problem, VMA ranges that we need to map being already used= . Still two problems from my side: (1) A VMA already exists on that addr/len range; conflict happens. (2) Kernel will not allocate the VMA to DPDK even there is no VMA on that r= anges; there's no conflict. Thanks, Jianfeng >=20 > Thanks, > Sergio >=20 > >> The proposal for new secondary process model would solve these issues: > >> http://dpdk.org/ml/archives/dev/2017-May/066147.html > > And yes, this might happen to solve the targeted issue in this RFC. But > before the new model is out, this patch seems a workable way for the > original issue. > > > > Thanks, > > Jianfeng > > > >> Thanks, > >> Sergio >=20