From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 438701150 for ; Wed, 23 Jan 2019 20:21:06 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 Jan 2019 11:21:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,512,1539673200"; d="scan'208";a="118856510" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.22]) ([10.237.221.22]) by fmsmga008.fm.intel.com with ESMTP; 23 Jan 2019 11:21:04 -0800 To: Stephen Hemminger References: <20170711011242.4606-1-stephen@networkplumber.org> <3f48cbd3-1e60-5a92-3929-7ba06e3cc69c@intel.com> Cc: dpdk-dev , Thomas Monjalon , Anatoly Burakov From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJVBBMBAgA/AhsDBgsJCAcDAgYVCAIJCgsE FgIDAQIeAQIXgBYhBNI2U4dCLsKE45mBx/kz60PfE2EfBQJbughWBQkHwjOGAAoJEPkz60Pf E2Eft84QAIbKWqhgqRfoiw/BbXbA1+qm2o4UgkCRQ0yJgt9QsnbpOmPKydHH0ixCliNz1J8e mRXCkMini1bTpnzp7spOjQGLeAFkNFz6BMq8YF2mVWbGEDE9WgnAxZdi0eLY7ZQnHbE6AxKL SXmpe9INb6z3ztseFt7mqje/W/6DWYIMnH3Yz9KzxujFWDcq8UCAvPkxVQXLTMpauhFgYeEx Nub5HbvhxTfUkapLwRQsSd/HbywzqZ3s/bbYMjj5JO3tgMiM9g9HOjv1G2f1dQjHi5YQiTZl 1eIIqQ3pTic6ROaiZqNmQFXPsoOOFfXF8nN2zg8kl/sSdoXWHhama5hbwwtl1vdaygQYlmdK H2ueiFh/UvT3WG3waNv2eZiEbHV8Rk52Xyn2w1G90lV0fYC6Ket1Xjoch7kjwbx793Kz/RfQ rmBY8/S4DTGn3oq3dMdQY+b6+7VMUeLMMh2CXYO9ErkOq+qNTD1IY+cBAkXnaDbQfz0zbste ZGWH74FAZ9nCpDOqbRTrBL42aMGhfOWEyeA1x7+hl6JZfabBWAuf4nnCXuorKHzBXTrf7u7p fXsKQClWRW77PF1VmzrtKNVSytQAmlCWApQIw20AarFipXmVdIjHmJPU611WoyxZPb4JTOxx 5cv9B+nr/RIB+v5dcStyHCCwO1be7nBDdCgd4F6kTQPLuQINBFfWTL4BEACnNA29e8TarUsB L5n6eLZHXcFvVwNLVlirWOClHXf44o2KnN3ww+eBEmKVfEFo9MSuGDNHS8Zw1NiGMYxLIUgd U6gGrVVs/VrQWL82pbMk6jCj98N+BXIri+6K1z+AImz7ax7iF1kDgRAnFWU0znWWBgM2mM8Y gDjcxfXk4sCKnvf6Gjo08Ey5zmqx7dekAKU2EEp8Q1EJY3jbymLdZWRP4AFFMTS1rGMk0/tt v71NBg1GobCcbNfn9chK/jhqxYhAJqq86RdJQkt3/9x1U1Oq0vXCt4JVVHmkxePtUiuWTTt+ aYlUAsKYZsWvncExvw77x2ArYDmaK0yfjh37wp0lY7DOJHFxoyT8tyWZlLci/VMRG2Ja33xj 0CN4C1yBg+QDeV3QFxQo42iA/ykdXPUR3ezmsND3XKvVLTC4DNb3V/EZQ7jBj64+bEK0VW4G B31VP00ApNQvSoczsIOAKdk97RNbpmPw6q10ILIB+9T1xbnFYzshzGF17oC0/GENIHATx8vZ masOZoDiOZQpeneLgnFE9JfzhLTxv6wNZcc/HLXRQVTkDsQr8ERtkAoHCf1E5+b5Yr7pfnE4 YuhET746o25S53ELUYPIs49qoJsEJL34/oexMfPGyPIlrbufiNyty5jc/1MRwUlhJlJ5IOHy ZUa+6CLR7GdImusFkPJUJwARAQABiQI8BBgBAgAmAhsMFiEE0jZTh0IuwoTjmYHH+TPrQ98T YR8FAlu6CHAFCQXE7zIACgkQ+TPrQ98TYR9nXxAAqNBgkYNyGuWUuy0GwDQCbu3iiMyH1+D7 llafPcK4NYy1Z4AYuVwC9nmLaoj+ozdqS3ncRo57ncRsKEJC46nDJJZYZ5LSJVn63Y3NBF86 lxQAgjj2oyZEwaLKtKbAFsXL43jv1pUGgSvWwYtDwHITXXFQto9rZEuUDRFSx4sg9OR+Q6/6 LY+nQQ3OdHlBkflzYMPcWgDcvcTAO6yasLEUf7UcYoSWTyMYjLB4QuNlXzTswzGVMssJF/vo V8lD1eqqaSUWG3STF6GVLQOr1NLvN5+kUBiEStHFxBpgSCvYY9sNV8FS6N24CAWMBl+10W+D 2h1yiiP5dOdPcBDYKsgqDD91/sP0WdyMJkwdQJtD49f9f+lYloxHnSAxMleOpyscg1pldw+i mPaUY1bmIknLhhkqfMmjywQOXpac5LRMibAAYkcB8v7y3kwELnt8mhqqZy6LUsqcWygNbH/W K3GGt5tRpeIXeJ25x8gg5EBQ0Jnvp/IbBYQfPLtXH0Myq2QuAhk/1q2yEIbVjS+7iowEZNyE 56K63WBJxsJPB2mvmLgn98GqB4G6GufP1ndS0XDti/2K0o8rep9xoY/JDGi0n0L0tk9BHyoP Y7kaEpu7UyY3nVdRLe5H1/MnFG8hdJ97WqnPS0buYZlrbTV0nRFL/NI2VABl18vEEXvNQiO+ vM8= Message-ID: <6f4d8c45-4215-e258-49a0-93235806d00d@intel.com> Date: Wed, 23 Jan 2019 19:21:03 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 19:21:06 -0000 On 7/12/2017 9:58 AM, jianfeng.tan at intel.com (Tan, Jianfeng) wrote: >> -----Original Message----- >> From: Gonzalez Monroy, Sergio >> Sent: Wednesday, July 12, 2017 3:32 PM >> To: Tan, Jianfeng; Stephen Hemminger; dev at dpdk.org >> Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary >> process >> >> On 12/07/2017 03:45, Tan, Jianfeng wrote: >>> >>>> -----Original Message----- >>>> From: Gonzalez Monroy, Sergio >>>> Sent: Tuesday, July 11, 2017 7:36 PM >>>> To: Tan, Jianfeng; Stephen Hemminger; dev at dpdk.org >>>> Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary >>>> process >>>> >>>> On 11/07/2017 02:56, Tan, Jianfeng wrote: >>>>>> -----Original Message----- >>>>>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen >>>>>> Hemminger >>>>>> Sent: Tuesday, July 11, 2017 9:13 AM >>>>>> To: dev at dpdk.org >>>>>> Cc: Stephen Hemminger >>>>>> Subject: [dpdk-dev] [RFC] pci: force address of mappings in secondary >>>>>> process >>>>>> >>>>>> The PCI memory resources in the secondary process should be in >>>>>> the exact same location as the primary process. Otherwise >>>>>> there is a risk of a stray pointer. >>>>>> >>>>>> Not sure if this is right, but it looks like a potential >>>>>> problem. >>>>>> >>>>>> --- >>>>>> lib/librte_eal/common/eal_common_pci_uio.c | 2 +- >>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/lib/librte_eal/common/eal_common_pci_uio.c >>>>>> b/lib/librte_eal/common/eal_common_pci_uio.c >>>>>> index 367a6816dcb8..2156b1a436c4 100644 >>>>>> --- a/lib/librte_eal/common/eal_common_pci_uio.c >>>>>> +++ b/lib/librte_eal/common/eal_common_pci_uio.c >>>>>> @@ -77,7 +77,7 @@ pci_uio_map_secondary(struct rte_pci_device >> *dev) >>>>>> >>>>>> void *mapaddr = pci_map_resource(uio_res- >>>>>>> maps[i].addr, >>>>>> fd, (off_t)uio_res->maps[i].offset, >>>>>> - (size_t)uio_res->maps[i].size, 0); >>>>>> + (size_t)uio_res->maps[i].size, >>>>>> MAP_FIXED); >>>>>> /* fd is not needed in slave process, close it */ >>>>>> close(fd); >>>>>> if (mapaddr != uio_res->maps[i].addr) { >>>>>> -- >>>>>> 2.11.0 >>>>> +1 for this RFC. I also once encounter such problem, and I use the same >>>> way to solve it. The addr parameter of mmap() syscall is only a hint instead >> of >>>> a must even the VMA is not occupied yet. >>>>> Thanks, >>>>> Jianfeng >>>> How do you know the VMA is not occupied? >>> I did by check /proc/self/maps. >>> >>>> I think the risk here is that the dynamic linker loaded some shared >>>> library in that VMA, and forcing MAP_FIXED is not a safe solution. >>>> What I have observed is that Linux will return a different VMA than the >>>> one hinted when there is already a mapping in the requested/hinted >> VMA. >>> IMO, that's not the target of this RFC. The target is to solve the situation (in >> current primary/secondary model) that kernel will not use the addr even >> there's no VMA on that addr. This is my understanding, Stephen, please >> correct me if I'm wrong. >> >> The point I was trying to make is, that you do not know if there is a >> mapping or not in that address, and by using MAP_FIXED it will unmap >> whatever was in there before. > > Oh, I missed that if there's conflict, the existing VMA will be unmapped. That's a bad effect. > >> >> So unless you parse /proc/self/maps and check that the VMA range is not >> being used, forcing MAP_FIXED is not safe. >> >>>> I reckon this is a similar issue as we have with the multi-process model >>>> when we do not get the VMA requested for the huge-pages. >>>> AFAIK we do not have a robust solution for this issue other than restart >>>> the program and hope the dynamic linker does not map anything in the >> VMA >>>> ranges that we need to map from the primary. This is also assuming that >>>> the application does not allocate memory and maps things before calling >>>> eal_init as it could potentially use VMA ranges that we need in the >>>> secondary process. >>> This is another problem. >> >> It is the same problem, VMA ranges that we need to map being already used. > > Still two problems from my side: > (1) A VMA already exists on that addr/len range; conflict happens. > (2) Kernel will not allocate the VMA to DPDK even there is no VMA on that ranges; there's no conflict. Hi Stephen, Both Sergio & Jianfeng are not active anymore in DPDK, which the discussion seems was going on with. cc'ed Anatoly. Is this RFC still valid? Should we expect an update on it? Thanks, ferruh > > Thanks, > Jianfeng > >> >> Thanks, >> Sergio >> >>>> The proposal for new secondary process model would solve these issues: >>>> http://dpdk.org/ml/archives/dev/2017-May/066147.html >>> And yes, this might happen to solve the targeted issue in this RFC. But >> before the new model is out, this patch seems a workable way for the >> original issue. >>> >>> Thanks, >>> Jianfeng >>> >>>> Thanks, >>>> Sergio >> > >