From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31D44A3168 for ; Wed, 16 Oct 2019 18:22:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8B35F1E95B; Wed, 16 Oct 2019 18:22:03 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 0126C1E95A for ; Wed, 16 Oct 2019 18:22:01 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Oct 2019 09:22:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,304,1566889200"; d="scan'208";a="347472895" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.10]) ([10.237.221.10]) by orsmga004.jf.intel.com with ESMTP; 16 Oct 2019 09:21:59 -0700 To: Vamsi Krishna Attunuru , "Yigit, Ferruh" Cc: "thomas@monjalon.net" , Jerin Jacob Kollanukkaran , "olivier.matz@6wind.com" , "anatoly.burakov@intel.com" , "arybchenko@solarflare.com" , Kiran Kumar Kokkilagadda , "dev@dpdk.org" References: <20190729121313.30639-2-vattunuru@marvell.com> <20190816061252.17214-1-vattunuru@marvell.com> From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJUBBMBCgA+AhsDAh4BAheABQsJCAcDBRUK CQgLBRYCAwEAFiEE0jZTh0IuwoTjmYHH+TPrQ98TYR8FAl1meboFCQlupOoACgkQ+TPrQ98T YR9ACBAAv2tomhyxY0Tp9Up7mNGLfEdBu/7joB/vIdqMRv63ojkwr9orQq5V16V/25+JEAD0 60cKodBDM6HdUvqLHatS8fooWRueSXHKYwJ3vxyB2tWDyZrLzLI1jxEvunGodoIzUOtum0Ce gPynnfQCelXBja0BwLXJMplM6TY1wXX22ap0ZViC0m714U5U4LQpzjabtFtjT8qOUR6L7hfy YQ72PBuktGb00UR/N5UrR6GqB0x4W41aZBHXfUQnvWIMmmCrRUJX36hOTYBzh+x86ULgg7H2 1499tA4o6rvE13FiGccplBNWCAIroAe/G11rdoN5NBgYVXu++38gTa/MBmIt6zRi6ch15oLA Ln2vHOdqhrgDuxjhMpG2bpNE36DG/V9WWyWdIRlz3NYPCDM/S3anbHlhjStXHOz1uHOnerXM 1jEjcsvmj1vSyYoQMyRcRJmBZLrekvgZeh7nJzbPHxtth8M7AoqiZ/o/BpYU+0xZ+J5/szWZ aYxxmIRu5ejFf+Wn9s5eXNHmyqxBidpCWvcbKYDBnkw2+Y9E5YTpL0mS0dCCOlrO7gca27ux ybtbj84aaW1g0CfIlUnOtHgMCmz6zPXThb+A8H8j3O6qmPoVqT3qnq3Uhy6GOoH8Fdu2Vchh TWiF5yo+pvUagQP6LpslffufSnu+RKAagkj7/RSuZV25Ag0EV9ZMvgEQAKc0Db17xNqtSwEv mfp4tkddwW9XA0tWWKtY4KUdd/jijYqc3fDD54ESYpV8QWj0xK4YM0dLxnDU2IYxjEshSB1T qAatVWz9WtBYvzalsyTqMKP3w34FciuL7orXP4AibPtrHuIXWQOBECcVZTTOdZYGAzaYzxiA ONzF9eTiwIqe9/oaOjTwTLnOarHt16QApTYQSnxDUQljeNvKYt1lZE/gAUUxNLWsYyTT+22/ vU0GDUahsJxs1+f1yEr+OGrFiEAmqrzpF0lCS3f/3HVTU6rS9cK3glVUeaTF4+1SK5ZNO35p iVQCwphmxa+dwTG/DvvHYCtgOZorTJ+OHfvCnSVjsM4kcXGjJPy3JZmUtyL9UxEbYlrffGPQ I3gLXIGD5AN5XdAXFCjjaID/KR1c9RHd7Oaw0Pdcq9UtMLgM1vdX8RlDuMGPrj5sQrRVbgYH fVU/TQCk1C9KhzOwg4Ap2T3tE1umY/DqrXQgsgH71PXFucVjOyHMYXXugLT8YQ0gcBPHy9mZ qw5mgOI5lCl6d4uCcUT0l/OEtPG/rA1lxz8ctdFBVOQOxCvwRG2QCgcJ/UTn5vlivul+cThi 6ERPvjqjblLncQtRg8izj2qgmwQkvfj+h7Ex88bI8iWtu5+I3K3LmNz/UxHBSWEmUnkg4fJl Rr7oItHsZ0ia6wWQ8lQnABEBAAGJAjwEGAEKACYCGwwWIQTSNlOHQi7ChOOZgcf5M+tD3xNh HwUCXWZ5wAUJB3FgggAKCRD5M+tD3xNhH2O+D/9OEz62YuJQLuIuOfL67eFTIB5/1+0j8Tsu o2psca1PUQ61SZJZOMl6VwNxpdvEaolVdrpnSxUF31kPEvR0Igy8HysQ11pj8AcgH0a9FrvU /8k2Roccd2ZIdpNLkirGFZR7LtRw41Kt1Jg+lafI0efkiHKMT/6D/P1EUp1RxOBNtWGV2hrd 0Yg9ds+VMphHHU69fDH02SwgpvXwG8Qm14Zi5WQ66R4CtTkHuYtA63sS17vMl8fDuTCtvfPF HzvdJLIhDYN3Mm1oMjKLlq4PUdYh68Fiwm+boJoBUFGuregJFlO3hM7uHBDhSEnXQr5mqpPM 6R/7Q5BjAxrwVBisH0yQGjsWlnysRWNfExAE2sRePSl0or9q19ddkRYltl6X4FDUXy2DTXa9 a+Fw4e1EvmcF3PjmTYs9IE3Vc64CRQXkhujcN4ZZh5lvOpU8WgyDxFq7bavFnSS6kx7Tk29/ wNJBp+cf9qsQxLbqhW5kfORuZGecus0TLcmpZEFKKjTJBK9gELRBB/zoN3j41hlEl7uTUXTI JQFLhpsFlEdKLujyvT/aCwP3XWT+B2uZDKrMAElF6ltpTxI53JYi22WO7NH7MR16Fhi4R6vh FHNBOkiAhUpoXRZXaCR6+X4qwA8CwHGqHRBfYFSU/Ulq1ZLR+S3hNj2mbnSx0lBs1eEqe2vh cA== Message-ID: <165d7042-8276-1897-13a4-8f4d78b35fe6@intel.com> Date: Wed, 16 Oct 2019 17:21:58 +0100 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH v10 0/5] kni: add IOVA=VA support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 10/16/2019 1:17 PM, Vamsi Krishna Attunuru wrote: > > >> -----Original Message----- >> From: dev On Behalf Of Yigit, Ferruh >> Sent: Tuesday, October 15, 2019 9:05 PM >> To: Vamsi Krishna Attunuru ; dev@dpdk.org >> Cc: thomas@monjalon.net; Jerin Jacob Kollanukkaran ; >> olivier.matz@6wind.com; ferruh.yigit@intel.com; anatoly.burakov@intel.com; >> arybchenko@solarflare.com; Kiran Kumar Kokkilagadda >> >> Subject: Re: [dpdk-dev] [PATCH v10 0/5] kni: add IOVA=VA support >> >> On 8/16/2019 7:12 AM, vattunuru@marvell.com wrote: >>> From: Vamsi Attunuru >>> >>> --- >>> V10 Changes: >>> * Fixed function return code on failure when min_chunk_size > pg_sz. >>> * Marked new mempool populate routine as EXPERIMENTAL. >>> >>> V9 Changes: >>> * Used rte_mempool_ops_calc_mem_size() instead of default handler in >>> the new mempool populate routine. >>> * Check min_chunk_size and return values. >>> * Removed ethdev_info memset to '0' and moved pci dev_info populate >>> into >>> kni_dev_pci_addr_get() routine. >>> * Addressed misc. review comments. >>> >>> V8 Changes: >>> * Remove default mempool populate() routine changes. >>> * Add kni app specific mempool create & free routines. >>> * Add new mempool populate routine to allocate page-aligned memzones >>> with page size to make sure all mempool objects reside on a page. >>> * Update release notes and map files. >>> >>> V7 Changes: >>> * Removed previously proposed mempool flag and made those page >>> boundary checks default in mempool populate() except for the objects >>> size bigger than the size of page. >>> * Removed KNI example application related changes since pool related >>> requirement is taken care in mempool lib. >>> * All PCI dev related info is moved under rte_eal_iova_mode() == VA check. >>> * Added wrapper functions in KNI module to hide IOVA checks and make >>> address translation routines more readable. >>> * Updated IOVA mode checks that enforcing IOVA=PA mode when IOVA=VA >>> mode is enabled. >>> >>> V6 Changes: >>> * Added new mempool flag to ensure mbuf memory is not scattered across >>> page boundaries. >>> * Added KNI kernel module required PCI device information. >>> * Modified KNI example application to create mempool with new mempool >>> flag. >>> >>> V5 changes: >>> * Fixed build issue with 32b build >>> >>> V4 changes: >>> * Fixed build issues with older kernel versions >>> * This approach will only work with kernel above 4.4.0 >>> >>> V3 Changes: >>> * Add new approach to work kni with IOVA=VA mode using >>> iommu_iova_to_phys API. >>> >>> Kiran Kumar K (1): >>> kni: add IOVA=VA support in KNI module >>> >>> Vamsi Attunuru (4): >>> mempool: populate mempool with the page sized chunks >>> kni: add IOVA=VA support in KNI lib >>> kni: add app specific mempool create and free routines >>> kni: modify IOVA mode checks to support VA >> >> Hi Vamsi, >> >> I am aware that this patchset is around for a long time, and I have seen your >> request to merge in 19.11, but as you can understand the concern I have is to >> break KNI or existing KNI applications while trying to add this new feature. >> >> In high level, there are two issues, >> >> 1) kernel modules updates expect there will be a backed device of the KNI which >> is not always true: >> >> if (dev_info.iova_mode) { >> #ifdef HAVE_IOVA_AS_VA_SUPPORT >> pci = pci_get_device(dev_info.vendor_id, >> dev_info.device_id, NULL); >> if (pci == NULL) { >> pr_err("pci dev does not exist\n"); >> return -ENODEV; >> } >> >> For example this breaks: >> ./build/app/testpmd -w0:0.0 --vdev net_kni0 --vdev net_kni1 -- -i > > Vamsi> Yes, these can be fixed by forcing iommu_mode to PA for > vdev or vdev&pdev based KNI usecases. > >> >> >> 2) Applications will have to change the API to allocate the mempool. >> If the user upgraded to new version of DPDK, now it is possible to have iova=va >> mode and application should use new KNI API 'rte_kni_pktmbuf_pool_create()' >> to allocate mempool. And most probably application will have datapath and will >> use the KNI only for exception path, will there be any affect using KNI version of >> mempool alloc? > > Vamsi> There would not be any affect in using KNI version of mempool. If you were developing a product, would you rely on a KNI API for pkt_mbuf allocation? What if there is a problem, will it be as easy as mempool/mbuf API to figure out and fix? I am not sure about pushing users to this direction? > >> >> >> I would like to see KNI is enabled via iova=va mode, but can we have it limited to >> a specific command line argument or config option? This increases the test >> surface but at least old application can continue to work by default, what do you >> think? > > Vamsi> Yes, it's appropriate to control the mode to ensure old apps work by default. > We are fine with having a command line arg or config option to enable KNI in iova=va mode. > Earlier we thought of having similar approach that also controls mempool allocation using > a newer mempool flag. After multiple reviews, flag has been discard and added a separate > mempool populate routine for these usecase. I didn't like the idea of having a flag in mempool library, but perhaps we can have it in KNI scope. > > When command line arg/config option is introduced, functionality will be as below. > Please correct me if any cases are missed or not considered. > Without command: > Existing KNI is intact, iommu mode will be PA. +1 > With command: > Pdev/vdev's iommu mode is considered and accordingly iova=va/pa is enabled. Application is > supposed to use KNI version of mempool alloc. +1 > I think these mempool quirk will go away when > Olivier's mempool patchset(RFC) is merged. > >> >> And I will put a few minor comments to the patches... >> >