From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40043.outbound.protection.outlook.com [40.107.4.43]) by dpdk.org (Postfix) with ESMTP id 8BCBE1041 for ; Tue, 18 Sep 2018 14:30:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yKQGPoIehCByTD77NeSXXI0sT1Vs/F2TY2cwGsIpAzE=; b=S18qpyD0wAYBbdTWlY32uRBpwUNhlJexUV4yrEPC2VRKyY+RdY7dY4W0HrL5UEIz+Dv1MPai63+/14f7X2ihMMzeG1MYh0tZWindhoKsSYNanASW3Jj+YnhpO8hBclDiOD6DvhMD6U83BYOx51Wtdnb6eCfeVe32kk7CWScWDQ0= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=shreyansh.jain@nxp.com; Received: from [10.232.14.39] (14.142.187.166) by AM0PR04MB4674.eurprd04.prod.outlook.com (2603:10a6:208:75::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.18; Tue, 18 Sep 2018 12:30:19 +0000 To: "Burakov, Anatoly" , Shahaf Shuler Cc: "dev@dpdk.org" , "laszlo.madarassy@ericsson.com" , "laszlo.vadkerti@ericsson.com" , "andras.kovacs@ericsson.com" , "winnie.tian@ericsson.com" , "daniel.andrasi@ericsson.com" , "janos.kobor@ericsson.com" , "srinath.mannam@broadcom.com" , "scott.branden@broadcom.com" , "ajit.khaparde@broadcom.com" , "keith.wiles@intel.com" , "bruce.richardson@intel.com" , Thomas Monjalon References: <9c763791-2c10-fd36-7f50-3df5efbc20ab@intel.com> From: Shreyansh Jain Message-ID: Date: Tue, 18 Sep 2018 17:59:56 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: SG2PR04CA0196.apcprd04.prod.outlook.com (2603:1096:4:14::34) To AM0PR04MB4674.eurprd04.prod.outlook.com (2603:10a6:208:75::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c93c5c5e-d617-47fb-f730-08d61d62824c X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR04MB4674; X-Microsoft-Exchange-Diagnostics: 1; AM0PR04MB4674; 3:dQUqTmjOMFzI1SiurAHK1fdiIG5I46hsJiASHa/c4oTvPAHvGPHQl2xqXLcvq3JB3Ky3OcRY67Hlpa2UHannnk5gYvbVACm4+oJxhNsSskHmd36b8G0QhWR+mxjE82DnWp/sGUT30bklqe8oP+tRDgXdYludMl2aOTVQnpXLj1vHqkkSYF/Lc1F5Y7sqpcyMRH15evsDiXpy4NcQ3XiGcM79Ae0XoqL2I6PGF21G3fy+sAxq8jMQneW3ypItYqoH; 25:zHpq9NBhqvuPpYSwnYw+ub9RgoMl0Gfht3GNz7Z6ZcZXDEJTsdsMbVYYHJHxIqOG1PuUfn9pHMRca8Q72bXCEAJDwjMxdfRi5rXKWyOT0qVZ1Z3DxZuZnI8/Cqwti3z2H420wlfhT0phUeANap62XSDgyPfDKLZh0PRsbAA3gGAfc2Znrp6ztntF+v+nfmRgbv4JIUTsFFJCxdx+l/c0T3iWA1nZqv2OyfYibTS9WVQYkAlHsc77ZnXePOFq0QDKTpVp5aMhj87enpncK7aCZymh6O25UV2ynLpzhWeFxSDzE3LNtPRDRy2fuTBz7AI+6B17uBUfMYuvSeFFKgQ33A==; 31:zTzNlXQu+rFi9mu5yP/iHKYLwbwMp8wQYMTlPSGkzPZTHdW0GzJthC7YDsmRLYaO7z4Jkox6Upfx9dHbkc5V+cwqNfxTcM9YGo94jgYU0wb5/xlwQmXgCJq7vTq4ETgLE0Wj04/VC+fbEc+DcGPcs6h2gbcbl+cz6B9dlYgLfSDSDD5875DNEXSqL2lyqEnyADkkxR1/kU4q7EXlWP2IclZZlMH9NPzK/uWzHtqDhz8= X-MS-TrafficTypeDiagnostic: AM0PR04MB4674: X-Microsoft-Exchange-Diagnostics: 1; AM0PR04MB4674; 20:KfBQMF+Py4Z5vh7x/hLlesALFKlvhoKlVRcnZjBv1wJmnwSXVM4Jo+QHqBZ5LoB+CmA/avF/hBOZ7Iw4KoRfGTcD1kZjB3s9XUIzbRZZEXUBQfJY3m2s/V3sksB9c0GqMK0g9vyRddeVLICFih2pjqOy/qfaL8Crtg24P5vjnjwDdLl+jO9OnmQEbYBrnoIOhrh/QpaQQctbSHTeOgrotIIZDVOmcVDLLjzATuCuRU2zZaEeMITePuBoBUUtpejTzhMp77WZLBRk1YaBFg0md3VGimsru36yknC+GvIgZ+h60bQZ7TpFI5G5PDgtNIpZecLN/QyWeRGZ5rR1lebsQ506u+fXoKhC4ZXrrqtS5GpPjU6WGd0qrx/hc/eGSU0YKrLKDP3hLdUhOL3xZ2krpMUuYGec5/OGjfCvRDftn+rGriFmj/6Qzox3VRdkhg9rkCtXe4UWqKGPZ/72lRFQqytK14d2BQ4yxfwHTxgfK9+FCSbCdII6dXlFdV0vmOWn; 4:Qgkk2DkaoaJQczZiQ3DIZv26GHLTdtMG2Zc3bgqg7VLWYx7m0MAVotedMCfECQW9QBflmga8sqUE7Jwmsr5j6hZgYW3pFW9/29WavuXz9ftrLIrKOZrrcExs3bnFiYF720bJg+fDKm8qhk72nbzmcFniBuTILATYWyAiAEagp/6OW3+Y+WtqBhKHajJt8AjSv0xCXvgBjkB6A7sSQgERAlotuQM6oCfyFwtsrGb9nnWVS8u83WEL24phFSXAdjUnUwUGIfE5Hs6TILkq4tdSxNwxHm1gxBi00fvdRnuBGtAyWo333fN1H7dGXPj94Aj7 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(17755550239193); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(3002001)(3231355)(944501410)(52105095)(93006095)(93001095)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699050); SRVR:AM0PR04MB4674; BCL:0; PCL:0; RULEID:; SRVR:AM0PR04MB4674; X-Forefront-PRVS: 0799B1B2D7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6049001)(376002)(396003)(39860400002)(136003)(346002)(366004)(189003)(199004)(31696002)(76176011)(7736002)(8936002)(305945005)(8676002)(446003)(67846002)(44832011)(25786009)(81156014)(81166006)(86362001)(36756003)(478600001)(386003)(93886005)(55236004)(3846002)(6116002)(53546011)(5660300001)(50466002)(65806001)(65956001)(66066001)(64126003)(6666003)(7416002)(68736007)(2906002)(47776003)(65826007)(230700001)(316002)(16576012)(6486002)(229853002)(110136005)(54906003)(5009440100003)(956004)(52146003)(2616005)(2486003)(26005)(6246003)(14444005)(5024004)(58126008)(77096007)(52116002)(97736004)(4326008)(53936002)(186003)(16526019)(31686004)(106356001)(23676004)(11346002)(476003)(486006)(105586002)(110426005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR04MB4674; H:[10.232.14.39]; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTTBQUjA0TUI0Njc0OzIzOkRYbTZQT1hjc2w0MW44eVJnUnB4dXpwNE9k?= =?utf-8?B?SXB3ZkpnZ3lwUi9BSjk0YmwxNEE0NXhQUGlISDBBTGZCU3VoczJMc1NmUFFp?= =?utf-8?B?UlNBa1Y2VmZrN0dMT2twbEtaWnJzMUphS3NDWnk4NDNnc01ONnlmWGliS1A4?= =?utf-8?B?SFo2Mks3R2RoUUlXTlZ0cjErdTZTTWJScTZmc1BRSm1JUEdkdnEzVGltQys5?= =?utf-8?B?bnhkKzRtOU4wNjZrR2JmQnZjZ2NTb2E1TTg2TjQ1WTgvZ1ZFK3FQOUtkYWpw?= =?utf-8?B?VnYzT0FPQ1h4OTRoKzlVa2JPdnNTT2JtQmRjZmZlUitVY0pkTkRqUUI3S3dE?= =?utf-8?B?VW5xS3NDN2wyb0RsSmNBamxaa3VjTFVlRU5lR3QwMVNtaGpEbjdsaVVIcE5C?= =?utf-8?B?b3YxM2JYcks0N3JCY3I1cFZxTUdnTUpRUWp0RlBUd01XTDc5d0dhdTRYak9O?= =?utf-8?B?V0RybzdteUJPS3pBcEhhVlRXcEx2Qk95RGlyanJQK0UxNXFLcXM1NnVxUkxr?= =?utf-8?B?THJ2Y3N1QUE1eWc2a1pkZ0hSblhRMU1sekR6T1cwZGx1ZDRNUVc1aUFjWFAx?= =?utf-8?B?ZU9IcEdFWUQyNGZHcXRmRWNGYmRsR3BteWlabzBrMm9ScDczcmVBeTVlaFBk?= =?utf-8?B?QzQ2Tk0vVEsrSHJFdmdDc2I3TElHalFrTUZrSzNnLzNjVm0zNXZaeHVrTXpB?= =?utf-8?B?Y1llYjU0dm9PNWxQVHIwaDBMUjBRaTNUSFA4aExNTmdFYTRtU1h6aGpncGhr?= =?utf-8?B?ZXdyQmNDKyt2cWdIMk1seWJYa1UwYlFQR2hnS1VnU0VMelhkN2tuTVpjRVNn?= =?utf-8?B?NmU3QkZNUXVrcEtlbElkcGdnM21YZUswQjl1SWxYNk9EdXVxdWs3RExqREdq?= =?utf-8?B?a0EwL3FZYWpiaHUxUkMxT0ZVcElNQko3UUdmUW9qOUh0V3lVRVlqZG1Dc2Fk?= =?utf-8?B?dVovUmdZWGRRbWt0UjVaZTR1Tjc0UnZhd0pvL1BCR0VwdzNyb0lwa3g3RGlG?= =?utf-8?B?S0FtMkdzUllwRzJOaG0zRTVHTFQrUTJWejRZbWhML2hWK0R6U1VBSHB6ck5j?= =?utf-8?B?a0wwSzhHVlJOMFZ3NHVKQlcrU0hjSFYzRmd1QWUzbkNiam9vTGdNSDRybVlZ?= =?utf-8?B?aG41ZlI2RWgzc0cyMktEZmNaU2pYUHpYaFpHWVA3cXhzRWs3MDN2aEdKR2hN?= =?utf-8?B?aXJaNFV4QnYyKytJU0FaRUZOZXgyQk9Xb0FaWmxXVUY3cEFkeHBkUWNHNEt4?= =?utf-8?B?RHY5UHNLOWcrS2dmcWVOczVZZFhuMUhWTGRqYXd0VkVycW1HU29LQ25pc0g1?= =?utf-8?B?WERoSG02ai9Id3Q1L01BSjFTYjNUdlFBUHZWeDlyTm01YWpBS3pKcEtlQzg4?= =?utf-8?B?T3BJQU85TThUKzFWN1ZTWEF6cXZQR1NpNlJiVGtwdXEydGF2L2NLTmprcE1k?= =?utf-8?B?QUE4eEI4RW1PbFhyUjhsWHlqOG4vZUw2RDZUcEh3aGlUd21sa2JUVkU0Y0ZY?= =?utf-8?B?cWQvOVZNVXI0S2dPSWZRQ3lHK3NtZWFneHdLTFg0aVcxb3dPaHJ0dE5jOFh4?= =?utf-8?B?a3h0VHJQUDJhSDBUcm0rMWNHem9PSmtUYjFTR3V3NEtnLzAzdThpOXp0TjhP?= =?utf-8?B?S0JBUlJLemZtMUo4MkpxOFVVeUVPQ1V1Mjk4RnlwcGxHb3htUlFGbnB5dGNh?= =?utf-8?B?MC8xREpPTEFHYkY1UWNQUEVubU5yb3RCeEFBUzVzdXVKWmRGVGowTm9hdFhZ?= =?utf-8?B?WGpaczFnVUFTZGZiWlkyRk9uQTQxOTZWaEVBeHM3QjlvUUMyeThDWVU4L0k0?= =?utf-8?B?aUtua1hqbml1NUplSzlCVzAveU5MVllLWjBsZWlKczVRSkozYUFId3ZacjdQ?= =?utf-8?B?azVoYkFXYjB0T1czMlBNL2JEbU1IejJuN0JZRUxBei95dDVCWkhOdHZwQnJv?= =?utf-8?B?ZzJOSXdVdEcxd2RTdFUyWmF2NllaVGEwUFkzdm55aUVjNG54dWhkcjRzbXFU?= =?utf-8?B?bkZzSVNlYk5YaVVZSDV4TnpuaE9yRVJXYzhqaVNGSFYvbG43cjluUGc1YWVl?= =?utf-8?B?RlBvc2NTSGJ4RC8vbFExVmVOT2Jadjl4a0hkZFUxdGpCTXBZY0hEZHZLQytl?= =?utf-8?B?dTRNd09oeW1QWHZXRDFRR2xwTGV4Ny9SSmJNK1JqY210UkFYVkFxNWMzbGF0?= =?utf-8?B?eHpNclE3MUxoV0hSZ25NbWhLUUlBPT0=?= X-Microsoft-Antispam-Message-Info: OTQdg8mmFepscjBhIHaIUuPorhFpeF9oKS70y40BqvhKc8v9ZSMNp3WMN1jchm9G8QKH1YUDNfyqbg0d9OpNML44vTrDyMj820NR9DgAVZhNU3t5Q237GT2H/AzIvFPYjjVFl9KdijhWiV0Stf5cM+NFgYNSnxTgRlLEiO88RCGGPJWMxvZWowvL3QKNZv+9cJOjPLCfKjFeI5UjBWjHJJfXLAXpUv9wlFyy3vGrOfnYwgkIwt0e/3OYS678YdbySH7rrxrvUazektDOzvBsY7Ls38xaueeZ40jKpTsFc0UDllSAKFqAq5YCOHs+QpR4RGjCId0aIbvkz73l8Ywvy5mTm7uXNIxQnqjj0YticQo= X-Microsoft-Exchange-Diagnostics: 1; AM0PR04MB4674; 6:XBoAghrN7UHxRm113iLOhOFwdc8oqa3xrZoNdcJ5Ku/ZBedD/dazlKeLixenJGPuiNzjAcbEeOG74A3sGhE0Rd6PaR8Oix6MOPsTsXWCNkqkdJ2uemkMbciGhez5M92dPrLfyY2qDccKm7bIf3JSjM5TlXXTaR9GWRTjszZqhh55Y6jC+5JJbtRh7pt5/l3ZKGOVqyWFZmcSET9VkF1vxWLb6yqcW62KcDlsTcNK2uG60SuFYGCKKEumOARWTyn2+H78gfb4ghLoXfr/A6kOpA0euSTH2SHBTKM7UuySW288dtB270ZLTDIKz/sPzzt4FxVkRvpmZ2XOUQrDxge3M7rxbLo//3znvCr8mlEGTTnXRTeJx/fQHF+gJxmV1z0xsoLpnmEmZ4sSrMWJRT8iOm0D0coyqaXHnVLgbV/SVTOb7LrYu+ciQ0pGg9Ab+GoTUVHLwEg4xUUwA3hTq5qJ8g==; 5:OsBu9VwX3FFCkY8JiiSt3kY6Iil3Szusc7/7hmkej8zWC7u+YRe5gEBG41s22jZFPv2Wrac/KmdqfUQhZk7O2Ih8UntOCKf9zdUle6x36MJ/qA/6yN6ZmdWmaY1WMX8R23gyvIFl5P8e2WR2bttRWKgFNoEXzQLuDQ4n0uDhl7A=; 7:Xvdke1y7YOZ7uebgcNtoTmOihHU+TWkyGD9g3GsUhm2+XBZAmv58Lg76vfivhNphW6C5UBVvPfovIC6uAWm8jqPfNMsMI1M/gbL8+l0iAvgiVFDmR/9vMAisa283znAuzTmHXqAyt3owWg3Mzpug3xWDiO1F+syD1xpQLsMq/jVPX8iVdZnNuobTNmF8VNQuBJ14azOjcgjZvD0AGLWXsT/D/VEEwT7Jb7o/eLaI1a/Cvmlh7sZaYfpHWUow4Yzl SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Sep 2018 12:30:19.6188 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c93c5c5e-d617-47fb-f730-08d61d62824c X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB4674 Subject: Re: [dpdk-dev] [PATCH 00/16] Support externally allocated memory in DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2018 12:30:26 -0000 On Monday 17 September 2018 06:30 PM, Burakov, Anatoly wrote: > On 17-Sep-18 1:16 PM, Shahaf Shuler wrote: >> Monday, September 17, 2018 1:07 PM, Burakov, Anatoly: >>> Subject: Re: [dpdk-dev] [PATCH 00/16] Support externally allocated >>> memory >>> in DPDK >>> >>> On 13-Sep-18 8:44 AM, Shahaf Shuler wrote: >> >> [...] >> >>>>> The responsibility to ensure memory is accessible before using it is >>>>> on the shoulders of the user - there is no checking done with regards >>>>> to validity of the memory (nor could there be...). >>>> >>>> That makes sense. However who should be in-charge of mapping this >>> memory for dma access? >>>> The user or internally be the PMD when encounter the first packet or >>>> while >>> traversing the existing mempools? >>>> >>> Hi Shahaf, >>> >>> There are two ways this can be solved. The first way is to perform VFIO >>> mapping automatically on adding/attaching memory. The second is to force >>> user to do it manually. For now, the latter is chosen because user >>> knows best >>> if they intend to do DMA on that memory, but i'm open to suggestions. >> >> I agree with that approach, and will add not only if the mempool is >> for dma or not but also which ports will use this mempool (this can >> effect on the mapping). > > That is perhaps too hardware-specific - this should probably be handled > inside the driver callbacks. > >> However I don't think this is generic enough to use only VFIO. As you >> said, there are some devices not using VFIO for mapping rather some >> proprietary driver utility. >> IMO DPDK should introduce generic and device agnostic APIs to the user. >> >> My suggestion is instead of doing vfio_dma_map that or vfio_dma_unmap >> that have a generic dma_map(uint8_t port, address, len). Each driver >> will register with its own mapping callback (can be vfio_dma_map). >> It can be outside of this series, just wondering the people opinion on >> such approach. > > I don't disagree. I don't like bus/net/etc drivers doing their own thing > with regards to mapping, and i would by far prefer generic way to set up > DMA maps, to which VFIO will be a subscriber. > >> >>> >>> There is an issue with some devices and buses (i.e. bus/fslmc) >>> bypassing EAL >>> VFIO infrastructure and performing their own VFIO/DMA mapping magic, but >>> solving that problem is outside the scope of this patchset. Those >>> devices/buses should fix themselves :) DMA mapping is a very common principle and can be easily be a candidate for lets-make-generic-movement, but, being close to hardware (or hardware specific), it does require the driver to have some flexibility in terms of its eventual implementation. I maintain one of those drivers (bus/fslmc) in DPDK which needs to have special VFIO layer - and from that experience, I can say that VFIO mapping does require some flexibility. SoC semantics are sometimes too complex to pin to general-universally-agreed-standard concept. (or, one can easily call it a 'bug', while it is a 'feature' for others :D) In fact, NXP has another driver (bus/dpaa) which doesn't even work with VFIO - loves to work directly with Phys_addr. And, it is not at a lower priority than one with VFIO. Thus, I really don't think a strongly controlled VFIO mapping should be EAL's responsibility. Failure because of lack of mapping is a driver's problem. >>> >>> When not using VFIO, it's out of our hands anyway. >> >> Why? >> VFIO is not a must requirement for devices in DPDK. > > When i say "out of our hands", what i mean to say is, currently as far > as EAL API is concerned, there is no DMA mapping outside of VFIO. > >> >>> >>> -- >>> Thanks, >>> Anatoly > >