From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA724A04CC; Fri, 15 Nov 2019 18:08:27 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A089D37AF; Fri, 15 Nov 2019 18:08:27 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 1F971316B for ; Fri, 15 Nov 2019 18:08:26 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xAFH5xqU004572; Fri, 15 Nov 2019 09:08:25 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=e0w4dzRKgC208hiG857d7u0NYpn2cQWA1+WmTzhUK/8=; b=K4j2smGQxalK+ol7F6hypeRb15/Lyrogrzz8YKiV1Z32jbbVchD6NpvAvKlzYDZUaI9P Th0G+yKBCZssU3WMt5vYocK5Aym4W90rG3R/CvJCWYhz28iMVe4TZafZd3aqotHPtmlj wnUXnlaMrTGxecsGeAShmsTjSB18Ek8tM6CWrFKOn7sk3tedQWjWmvyP+JUQwrYVRN8L hWYHyhht0DMd/EGblSwoUmaVJ6p1So0KAeA0SDLnyw4iCVOLAM8R2rK2zUrYjWzP+ASw 9vcnxfAzpffDp+zu9VIpMB9Ephd0RZecYTpc9Jjfra6jIqdIf6mguEFDiPvjhUqDPBUL 9g== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2w8whr0ubr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 15 Nov 2019 09:08:25 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 15 Nov 2019 09:08:23 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 15 Nov 2019 09:08:23 -0800 Received: from hyd1vattunuru-dt.caveonetworks.com (unknown [10.29.52.72]) by maili.marvell.com (Postfix) with ESMTP id B644F3F703F; Fri, 15 Nov 2019 09:08:19 -0800 (PST) From: To: CC: , , , , , , , , , Vamsi Attunuru Date: Fri, 15 Nov 2019 22:37:52 +0530 Message-ID: <20191115170752.9488-3-vattunuru@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20191115170752.9488-1-vattunuru@marvell.com> References: <20191115111807.20935-1-vattunuru@marvell.com> <20191115170752.9488-1-vattunuru@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-11-15_05:2019-11-15,2019-11-15 signatures=0 Subject: [dpdk-dev] [PATCH v14 2/2] kni: support IOVA mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: David Marchand Current KNI implementation only operates in IOVA_PA mode patch adds required functionality to enable KNI in IOVA_VA mode. Due to IOVA to KVA address translations, based on the KNI use case there can be a performance impact. To mitigate the impact on performance,forcing IOVA to PA via eal "--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also result in IOVA as PA. During KNI creation, app's iova_mode details are passed to the KNI kernel module, accordingly kernel module translates PA/IOVA addresses to KVA and vice-versa. Signed-off-by: David Marchand Signed-off-by: Vamsi Attunuru Signed-off-by: Kiran Kumar K --- doc/guides/prog_guide/kernel_nic_interface.rst | 14 ++++++++++++++ doc/guides/rel_notes/release_19_11.rst | 13 ++++++++++++- lib/librte_eal/linux/eal/eal.c | 11 +++++++++-- lib/librte_kni/rte_kni.c | 5 +++++ 4 files changed, 40 insertions(+), 3 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 2fd58e1..e688efc 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -300,6 +300,20 @@ The sk_buff is then freed and the mbuf sent in the tx_q FIFO. The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. +IOVA = VA: Support +------------------ + +KNI operates in IOVA_VA scheme when + +- LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0) and +- eal option `iova-mode=va` is passed or bus IOVA scheme in the DPDK is selected + as RTE_IOVA_VA. + +Due to IOVA to KVA address translations, based on the KNI use case there +can be a performance impact. For mitigation, forcing IOVA to PA via eal +"--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also +result in IOVA as PA. + Ethtool ------- diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst index 682c1bd..bed3344 100644 --- a/doc/guides/rel_notes/release_19_11.rst +++ b/doc/guides/rel_notes/release_19_11.rst @@ -292,8 +292,19 @@ New Features compilers store their internal representation of the source code that the linker uses at the final stage of compilation process. - See :doc:`../prog_guide/lto` for more information: +* **Added IOVA as VA support for KNI.** + + * Added IOVA = VA support for KNI, KNI can operate in IOVA = VA mode when + `iova-mode=va` eal option is passed to the application or when bus IOVA + scheme is selected as RTE_IOVA_VA. This mode only works on Linux Kernel + versions 4.6.0 and above. + * Due to IOVA to KVA address translations, based on the KNI use case there + can be a performance impact. For mitigation, forcing IOVA to PA via eal + "--iova-mode=pa" option can be used, IOVA_DC bus iommu scheme can also + result in IOVA as PA. + + See :doc:`../prog_guide/lto` for more information: Removed Items diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 9e2d50c..b5b7150 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -1073,6 +1073,11 @@ rte_eal_init(int argc, char **argv) */ iova_mode = RTE_IOVA_VA; RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n"); +#if defined(RTE_LIBRTE_KNI) && LINUX_VERSION_CODE >= KERNEL_VERSION(4, 6, 0) + } else if (rte_eal_check_module("rte_kni") == 1) { + iova_mode = RTE_IOVA_PA; + RTE_LOG(DEBUG, EAL, "KNI is loaded, selecting IOVA as PA mode for better KNI perfomance.\n"); +#endif } else if (is_iommu_enabled()) { /* we have an IOMMU, pick IOVA as VA mode */ iova_mode = RTE_IOVA_VA; @@ -1085,8 +1090,10 @@ rte_eal_init(int argc, char **argv) RTE_LOG(DEBUG, EAL, "IOMMU is not available, selecting IOVA as PA mode.\n"); } } -#ifdef RTE_LIBRTE_KNI - /* Workaround for KNI which requires physical address to work */ +#if defined(RTE_LIBRTE_KNI) && LINUX_VERSION_CODE < KERNEL_VERSION(4, 6, 0) + /* Workaround for KNI which requires physical address to work + * in kernels < 4.6 + */ if (iova_mode == RTE_IOVA_VA && rte_eal_check_module("rte_kni") == 1) { if (phys_addrs) { diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 7fbcf22..0507a48 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -97,10 +98,12 @@ static volatile int kni_fd = -1; int rte_kni_init(unsigned int max_kni_ifaces __rte_unused) { +#if KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE if (rte_eal_iova_mode() != RTE_IOVA_PA) { RTE_LOG(ERR, KNI, "KNI requires IOVA as PA\n"); return -1; } +#endif /* Check FD and open */ if (kni_fd < 0) { @@ -302,6 +305,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size; + dev_info.iova_mode = (rte_eal_iova_mode() == RTE_IOVA_VA) ? 1 : 0; + ret = ioctl(kni_fd, RTE_KNI_IOCTL_CREATE, &dev_info); if (ret < 0) goto ioctl_fail; -- 2.8.4