From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A998FA04CA; Fri, 15 Nov 2019 12:19:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 80F91374E; Fri, 15 Nov 2019 12:19:11 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id A788B2C12 for ; Fri, 15 Nov 2019 12:19:09 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xAFBFfAt013525; Fri, 15 Nov 2019 03:19:09 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=RbOTmXnLBFkml6s01MvBqBImjxFeKuF+TaTYW3qfa9g=; b=XLlwcMz0TFEiGYskEgZM+yl+Hv8zTTpWrIZVQM2TRlkOzcmPLajRyNW2pMaIenCLI0ZF jO+f2KY8tu0vA3zX/ucNBZcB0fiJmjlxhty1l8gJ4W8FxxPIRo8K8DBE3EfayaLY578F 3be1c41ER1AdHQim+k5uzmpQg6OVZHcGa+MHJklUx8r5Veoo5krfPuRpe9u5IQY5B2ow K0x3bszfRewarCwDP3Tp3j21nXVaAmRPaKxtqRzGHmm7jFem/SxSlEtWDnNGpwXI9J+i eSI/x8em0OpgR6NFm6FwkbyIBhxQygWd7P93JcJ6l/IDl/htb4reAnDWUcwNE6VvDoKH 5g== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2w8whqygss-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 15 Nov 2019 03:19:08 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 15 Nov 2019 03:19:06 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 15 Nov 2019 03:19:06 -0800 Received: from hyd1vattunuru-dt.caveonetworks.com (unknown [10.29.52.72]) by maili.marvell.com (Postfix) with ESMTP id 7454A3F704C; Fri, 15 Nov 2019 03:19:02 -0800 (PST) From: To: CC: , , , , , , , , , Vamsi Attunuru Date: Fri, 15 Nov 2019 16:48:07 +0530 Message-ID: <20191115111807.20935-3-vattunuru@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20191115111807.20935-1-vattunuru@marvell.com> References: <20191105110416.8955-1-vattunuru@marvell.com> <20191115111807.20935-1-vattunuru@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-11-15_03:2019-11-15,2019-11-15 signatures=0 Subject: [dpdk-dev] [PATCH v13 2/2] kni: support IOVA mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Current KNI implementation only operates in IOVA_PA mode patch adds required functionality to enable KNI in IOVA_VA mode. KNI loopback mode tests will have performance impact in this mode due to IOVA to KVA address translations. However, In KNI real world use cases, the performace impact will be based on Linux kernel stack and scheduler latencies. Performance varies based on the KNI use case. If bus iommu scheme is IOVA_DC and KNI module is loaded, DPDK chooses IOVA as PA as existing behaviour. During KNI creation, app's iova_mode details are passed to the KNI kernel module, accordingly kernel module translates PA/IOVA addresses to KVA and vice-versa. Signed-off-by: Vamsi Attunuru Signed-off-by: Kiran Kumar K Suggested-by: Ferruh Yigit --- doc/guides/prog_guide/kernel_nic_interface.rst | 15 +++++++++++++++ doc/guides/rel_notes/release_19_11.rst | 15 ++++++++++++++- lib/librte_eal/linux/eal/eal.c | 23 ++++++++++++++++------- lib/librte_kni/rte_kni.c | 6 ++++++ 4 files changed, 51 insertions(+), 8 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 2fd58e1..3bed18f 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -300,6 +300,21 @@ The sk_buff is then freed and the mbuf sent in the tx_q FIFO. The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. +IOVA = VA: Support +------------------ + +KNI operates in IOVA_VA scheme when + +- LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0) and +- eal option `iova-mode=va` is passed or bus IOVA scheme in the DPDK is selected + as RTE_IOVA_VA. + +KNI loopback mode tests will have performance impact in this mode due to IOVA to +KVA address translations. However, In KNI real world use cases, the performace +impact will be based on Linux kernel stack and scheduler latencies. Performance +varies based on the KNI use case. If bus iommu scheme is IOVA_DC and KNI module +is loaded, DPDK chooses IOVA as PA as existing behaviour. + Ethtool ------- diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst index 682c1bd..c207c93 100644 --- a/doc/guides/rel_notes/release_19_11.rst +++ b/doc/guides/rel_notes/release_19_11.rst @@ -292,8 +292,21 @@ New Features compilers store their internal representation of the source code that the linker uses at the final stage of compilation process. - See :doc:`../prog_guide/lto` for more information: +* **Added IOVA as VA support for KNI.** + + * Added IOVA = VA support for KNI, KNI can operate in IOVA = VA mode when + `iova-mode=va` eal option is passed to the application or when bus IOVA + scheme is selected as RTE_IOVA_VA. This mode only works on Linux Kernel + versions 4.6.0 and above. + * KNI loopback mode tests will have performance impact in this mode due + to IOVA to KVA address translations. However, In KNI real world use cases, + the performace impact will be based on Linux kernel stack and scheduler + latencies. Performance varies based on the KNI use case. If bus iommu + scheme is IOVA_DC and KNI module is loaded, DPDK chooses IOVA as PA as + existing behaviour. + + See :doc:`../prog_guide/lto` for more information: Removed Items diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 9e2d50c..53ca84b 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -1086,14 +1086,23 @@ rte_eal_init(int argc, char **argv) } } #ifdef RTE_LIBRTE_KNI - /* Workaround for KNI which requires physical address to work */ - if (iova_mode == RTE_IOVA_VA && - rte_eal_check_module("rte_kni") == 1) { - if (phys_addrs) { + if (rte_eal_check_module("rte_kni") == 1) { +#if KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE + if (iova_mode == RTE_IOVA_VA) { iova_mode = RTE_IOVA_PA; - RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI module is loaded\n"); - } else { - RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); + RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because " + "Kernel version supports only 'PA' mode for KNI module\n"); + } +#endif + if (rte_bus_get_iommu_class() == RTE_IOVA_DC) + iova_mode = RTE_IOVA_PA; + + if (iova_mode == RTE_IOVA_PA) { + if (phys_addrs && is_iommu_enabled()) + RTE_LOG(WARNING, EAL, "Forced IOVA as 'PA' because KNI module is loaded\n"); + + if (!phys_addrs) + RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); } } #endif diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 7fbcf22..a609d2f 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -6,6 +6,8 @@ #error "KNI is not supported" #endif +#include + #include #include #include @@ -97,10 +99,12 @@ static volatile int kni_fd = -1; int rte_kni_init(unsigned int max_kni_ifaces __rte_unused) { +#if KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE if (rte_eal_iova_mode() != RTE_IOVA_PA) { RTE_LOG(ERR, KNI, "KNI requires IOVA as PA\n"); return -1; } +#endif /* Check FD and open */ if (kni_fd < 0) { @@ -302,6 +306,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size; + dev_info.iova_mode = (rte_eal_iova_mode() == RTE_IOVA_VA) ? 1 : 0; + ret = ioctl(kni_fd, RTE_KNI_IOCTL_CREATE, &dev_info); if (ret < 0) goto ioctl_fail; -- 2.8.4