From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 87EA8A04AB; Thu, 7 Nov 2019 20:53:22 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D109F1BF71; Thu, 7 Nov 2019 20:53:21 +0100 (CET) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 2570B1B948 for ; Thu, 7 Nov 2019 20:53:19 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Nov 2019 11:53:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.68,279,1569308400"; d="scan'208";a="233377801" Received: from fyigit-mobl.ger.corp.intel.com (HELO [10.237.221.96]) ([10.237.221.96]) by fmsmga002.fm.intel.com with ESMTP; 07 Nov 2019 11:53:16 -0800 To: vattunuru@marvell.com, dev@dpdk.org Cc: thomas@monjalon.net, jerinj@marvell.com, kirankumark@marvell.com, olivier.matz@6wind.com, anatoly.burakov@intel.com, arybchenko@solarflare.com, stephen@networkplumber.org References: <20191021080324.10659-1-vattunuru@marvell.com> <20191105110416.8955-1-vattunuru@marvell.com> From: Ferruh Yigit Openpgp: preference=signencrypt Autocrypt: addr=ferruh.yigit@intel.com; prefer-encrypt=mutual; keydata= mQINBFXZCFABEADCujshBOAaqPZpwShdkzkyGpJ15lmxiSr3jVMqOtQS/sB3FYLT0/d3+bvy qbL9YnlbPyRvZfnP3pXiKwkRoR1RJwEo2BOf6hxdzTmLRtGtwWzI9MwrUPj6n/ldiD58VAGQ +iR1I/z9UBUN/ZMksElA2D7Jgg7vZ78iKwNnd+vLBD6I61kVrZ45Vjo3r+pPOByUBXOUlxp9 GWEKKIrJ4eogqkVNSixN16VYK7xR+5OUkBYUO+sE6etSxCr7BahMPKxH+XPlZZjKrxciaWQb +dElz3Ab4Opl+ZT/bK2huX+W+NJBEBVzjTkhjSTjcyRdxvS1gwWRuXqAml/sh+KQjPV1PPHF YK5LcqLkle+OKTCa82OvUb7cr+ALxATIZXQkgmn+zFT8UzSS3aiBBohg3BtbTIWy51jNlYdy ezUZ4UxKSsFuUTPt+JjHQBvF7WKbmNGS3fCid5Iag4tWOfZoqiCNzxApkVugltxoc6rG2TyX CmI2rP0mQ0GOsGXA3+3c1MCdQFzdIn/5tLBZyKy4F54UFo35eOX8/g7OaE+xrgY/4bZjpxC1 1pd66AAtKb3aNXpHvIfkVV6NYloo52H+FUE5ZDPNCGD0/btFGPWmWRmkPybzColTy7fmPaGz cBcEEqHK4T0aY4UJmE7Ylvg255Kz7s6wGZe6IR3N0cKNv++O7QARAQABtCVGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBpbnRlbC5jb20+iQJUBBMBCgA+AhsDAh4BAheABQsJCAcDBRUK CQgLBRYCAwEAFiEE0jZTh0IuwoTjmYHH+TPrQ98TYR8FAl1meboFCQlupOoACgkQ+TPrQ98T YR9ACBAAv2tomhyxY0Tp9Up7mNGLfEdBu/7joB/vIdqMRv63ojkwr9orQq5V16V/25+JEAD0 60cKodBDM6HdUvqLHatS8fooWRueSXHKYwJ3vxyB2tWDyZrLzLI1jxEvunGodoIzUOtum0Ce gPynnfQCelXBja0BwLXJMplM6TY1wXX22ap0ZViC0m714U5U4LQpzjabtFtjT8qOUR6L7hfy YQ72PBuktGb00UR/N5UrR6GqB0x4W41aZBHXfUQnvWIMmmCrRUJX36hOTYBzh+x86ULgg7H2 1499tA4o6rvE13FiGccplBNWCAIroAe/G11rdoN5NBgYVXu++38gTa/MBmIt6zRi6ch15oLA Ln2vHOdqhrgDuxjhMpG2bpNE36DG/V9WWyWdIRlz3NYPCDM/S3anbHlhjStXHOz1uHOnerXM 1jEjcsvmj1vSyYoQMyRcRJmBZLrekvgZeh7nJzbPHxtth8M7AoqiZ/o/BpYU+0xZ+J5/szWZ aYxxmIRu5ejFf+Wn9s5eXNHmyqxBidpCWvcbKYDBnkw2+Y9E5YTpL0mS0dCCOlrO7gca27ux ybtbj84aaW1g0CfIlUnOtHgMCmz6zPXThb+A8H8j3O6qmPoVqT3qnq3Uhy6GOoH8Fdu2Vchh TWiF5yo+pvUagQP6LpslffufSnu+RKAagkj7/RSuZV25Ag0EV9ZMvgEQAKc0Db17xNqtSwEv mfp4tkddwW9XA0tWWKtY4KUdd/jijYqc3fDD54ESYpV8QWj0xK4YM0dLxnDU2IYxjEshSB1T qAatVWz9WtBYvzalsyTqMKP3w34FciuL7orXP4AibPtrHuIXWQOBECcVZTTOdZYGAzaYzxiA ONzF9eTiwIqe9/oaOjTwTLnOarHt16QApTYQSnxDUQljeNvKYt1lZE/gAUUxNLWsYyTT+22/ vU0GDUahsJxs1+f1yEr+OGrFiEAmqrzpF0lCS3f/3HVTU6rS9cK3glVUeaTF4+1SK5ZNO35p iVQCwphmxa+dwTG/DvvHYCtgOZorTJ+OHfvCnSVjsM4kcXGjJPy3JZmUtyL9UxEbYlrffGPQ I3gLXIGD5AN5XdAXFCjjaID/KR1c9RHd7Oaw0Pdcq9UtMLgM1vdX8RlDuMGPrj5sQrRVbgYH fVU/TQCk1C9KhzOwg4Ap2T3tE1umY/DqrXQgsgH71PXFucVjOyHMYXXugLT8YQ0gcBPHy9mZ qw5mgOI5lCl6d4uCcUT0l/OEtPG/rA1lxz8ctdFBVOQOxCvwRG2QCgcJ/UTn5vlivul+cThi 6ERPvjqjblLncQtRg8izj2qgmwQkvfj+h7Ex88bI8iWtu5+I3K3LmNz/UxHBSWEmUnkg4fJl Rr7oItHsZ0ia6wWQ8lQnABEBAAGJAjwEGAEKACYCGwwWIQTSNlOHQi7ChOOZgcf5M+tD3xNh HwUCXWZ5wAUJB3FgggAKCRD5M+tD3xNhH2O+D/9OEz62YuJQLuIuOfL67eFTIB5/1+0j8Tsu o2psca1PUQ61SZJZOMl6VwNxpdvEaolVdrpnSxUF31kPEvR0Igy8HysQ11pj8AcgH0a9FrvU /8k2Roccd2ZIdpNLkirGFZR7LtRw41Kt1Jg+lafI0efkiHKMT/6D/P1EUp1RxOBNtWGV2hrd 0Yg9ds+VMphHHU69fDH02SwgpvXwG8Qm14Zi5WQ66R4CtTkHuYtA63sS17vMl8fDuTCtvfPF HzvdJLIhDYN3Mm1oMjKLlq4PUdYh68Fiwm+boJoBUFGuregJFlO3hM7uHBDhSEnXQr5mqpPM 6R/7Q5BjAxrwVBisH0yQGjsWlnysRWNfExAE2sRePSl0or9q19ddkRYltl6X4FDUXy2DTXa9 a+Fw4e1EvmcF3PjmTYs9IE3Vc64CRQXkhujcN4ZZh5lvOpU8WgyDxFq7bavFnSS6kx7Tk29/ wNJBp+cf9qsQxLbqhW5kfORuZGecus0TLcmpZEFKKjTJBK9gELRBB/zoN3j41hlEl7uTUXTI JQFLhpsFlEdKLujyvT/aCwP3XWT+B2uZDKrMAElF6ltpTxI53JYi22WO7NH7MR16Fhi4R6vh FHNBOkiAhUpoXRZXaCR6+X4qwA8CwHGqHRBfYFSU/Ulq1ZLR+S3hNj2mbnSx0lBs1eEqe2vh cA== Message-ID: Date: Thu, 7 Nov 2019 19:53:15 +0000 MIME-Version: 1.0 In-Reply-To: <20191105110416.8955-1-vattunuru@marvell.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH v12 0/2] add IOVA=VA mode support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 11/5/2019 11:04 AM, vattunuru@marvell.com wrote: > From: Vamsi Attunuru > > --- > V12 Changes: > * Removed previously added `--legacy-kni` eal option. > * Removed previously added kni specific mempool create routines > and mempool populate routines. > > This patch set(V12) is dependent on following patch set, since the mempool > related support to enable KNI in IOVA=VA mode is taken care in below > patchset. > > https://patchwork.dpdk.org/cover/62376/ Hi Vasim, Jerin, Overall looks good and I not getting any functional error but I am observing a huge performance drop with this update, 3.8Mpps to 0.7Mpps [1]. I don't know really what to do, I think we need to give a decision as community, and even we go with the patch we should document this performance drop clearly and document how to mitigate it. [1] This is with kni sample application, a) IOVA=VA mode selected ./examples/kni/build/kni -l0,40-47 --log-level=*:debug -- -p 0x3 -P --config "(0,44,45,40),(1,46,47,41)" forwarding performance is around 0.7Mpps and 'kni_single' kernel thread consumes all cpu. b) IOVA=PA mode forced ./examples/kni/build/kni -l0,40-47 --log-level=*:debug --iova=pa -- -p 0x3 -P --config "(0,44,45,40),(1,46,47,41)" forwarding performance is around 3.8Mpps and 'kni_single' core utilization is ~80%. I am on 5.1.20-300.fc30.x86_64 kernel. kni module inserted as: "insmod ./build/kmod/rte_kni.ko lo_mode=lo_mode_fifo" > > V11 Changes: > * Added iova to kva address translation routines in kernel module to > make it work in iova=va mode which enables DPDK to create kni devices > on any kind of backed device/memory. > * Added ``--legacy-kni`` eal option to make existing KNI applications > work with DPDK 19.11 and later versions. > * Removed previously added pci device info from kni device info struct. > > V10 Changes: > * Fixed function return code on failure when min_chunk_size > pg_sz. > * Marked new mempool populate routine as EXPERIMENTAL. > > V9 Changes: > * Used rte_mempool_ops_calc_mem_size() instead of default handler in the > new mempool populate routine. > * Check min_chunk_size and return values. > * Removed ethdev_info memset to '0' and moved pci dev_info populate into > kni_dev_pci_addr_get() routine. > * Addressed misc. review comments. > > V8 Changes: > * Remove default mempool populate() routine changes. > * Add kni app specific mempool create & free routines. > * Add new mempool populate routine to allocate page-aligned memzones > with page size to make sure all mempool objects reside on a page. > * Update release notes and map files. > > V7 Changes: > * Removed previously proposed mempool flag and made those page > boundary checks default in mempool populate() except for the objects size > bigger than the size of page. > * Removed KNI example application related changes since pool related > requirement is taken care in mempool lib. > * All PCI dev related info is moved under rte_eal_iova_mode() == VA check. > * Added wrapper functions in KNI module to hide IOVA checks and make > address translation routines more readable. > * Updated IOVA mode checks that enforcing IOVA=PA mode when IOVA=VA > mode is enabled. > > V6 Changes: > * Added new mempool flag to ensure mbuf memory is not scattered across > page boundaries. > * Added KNI kernel module required PCI device information. > * Modified KNI example application to create mempool with new mempool > flag. > > V5 changes: > * Fixed build issue with 32b build > > V4 changes: > * Fixed build issues with older kernel versions > * This approach will only work with kernel above 4.4.0 > > V3 Changes: > * Add new approach to work kni with IOVA=VA mode using > iommu_iova_to_phys API. > > Vamsi Attunuru (2): > kni: add IOVA=VA mode support > kni: add IOVA=VA support in kernel module > > doc/guides/prog_guide/kernel_nic_interface.rst | 9 ++++ > doc/guides/rel_notes/release_19_11.rst | 5 ++ > kernel/linux/kni/compat.h | 15 ++++++ > kernel/linux/kni/kni_dev.h | 42 +++++++++++++++ > kernel/linux/kni/kni_misc.c | 39 ++++++++++---- > kernel/linux/kni/kni_net.c | 62 ++++++++++++++++++----- > lib/librte_eal/linux/eal/eal.c | 29 ++++++----- > lib/librte_eal/linux/eal/include/rte_kni_common.h | 1 + > lib/librte_kni/rte_kni.c | 7 +-- > 9 files changed, 170 insertions(+), 39 deletions(-) >