From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C6BC2A04C2; Fri, 15 Nov 2019 14:40:54 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 89B142C18; Fri, 15 Nov 2019 14:40:54 +0100 (CET) Received: from mail-io1-f68.google.com (mail-io1-f68.google.com [209.85.166.68]) by dpdk.org (Postfix) with ESMTP id 2A8F82C08 for ; Fri, 15 Nov 2019 14:40:53 +0100 (CET) Received: by mail-io1-f68.google.com with SMTP id v17so10424225iol.12 for ; Fri, 15 Nov 2019 05:40:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WFZbf/etmCmkecOBNCOQN9GpsXLRimG+ObZvF3Wt+lI=; b=pZBnFu1ih7hjkFM1PVUowV9GFWFQlu433aQ1U4toW7SxXesw4urJwm537AlYs7ioNv lNoTeUtHeKSF7wfFTGmWlKRBTSAiTIYQLjra8asyfwrK0MatyHmL9PNHEs4xjl7fLqFZ Rmi61iV7M/iQkCC/f0XBWUO28EFWGuZr+rznVdmigjyJ5qSPU9O0bNxYm+1GtZzyT/rJ oHwMjYrJEIt1MxpqCjUA8fcAW2OZEo5LpI7NpgME8A/YqZe/R/JmiRCjVhsN8iCXdTuf xUxeztp+4BaAITfSEZ8AoMNk2uUJ2UsVq1E1BCltbX7ZVFJ3I48k0cC/GDzEg94SYk8f 67ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WFZbf/etmCmkecOBNCOQN9GpsXLRimG+ObZvF3Wt+lI=; b=mpnVBMcdZ2nLlIl4670AOcZjhfLZjYFpZyH97tka6ofyi1+R910YV96XxJpQDdax2C JwV5yJwVuLidDpJAUxA+YoJC2UqJc/KcXTrp6o0JHyKRDWa5okV1J42gpLaW4pOCwLVu hrer2du3XCbK6JkROk2tshf0p/OfOjRI0afD2Y0Vq/z6Z9B/HlKfB8qJlaVshkE1MEEh Mvn0B0+3FNtRUPBtVwf+9yrOzH8rOa2yXzaf9Ehlrf7F47eAKGgRe36NrYoLTwQ6PxC5 UFbaXQEDN0P1Nl7oEw/2IOH7ScbMPyWlVK3x80K5UlfXgEE9JFQKOHtAQzgNjmTE/cDJ 8xGg== X-Gm-Message-State: APjAAAXmPOo9G8OH9iWpAfh82uSuAhIKGYJnHlsCFZTDLkE3esCtd6Ze loHzXBdyhJUwxhQ0bUXdHv3ZQYIJYvtR3cb2BUE= X-Google-Smtp-Source: APXvYqx7LY2YD9Lf2gn63gFt9sq8F3j1RRrxyojP3TPCaQwvEZj41YywL3syA79HqyoBz5LKlUgdYX/TTcY4TImRKeE= X-Received: by 2002:a5d:8056:: with SMTP id b22mr678726ior.60.1573825252090; Fri, 15 Nov 2019 05:40:52 -0800 (PST) MIME-Version: 1.0 References: <20191105110416.8955-1-vattunuru@marvell.com> <20191115111807.20935-1-vattunuru@marvell.com> <20191115111807.20935-3-vattunuru@marvell.com> In-Reply-To: From: Jerin Jacob Date: Fri, 15 Nov 2019 19:10:34 +0530 Message-ID: To: David Marchand Cc: Vamsi Attunuru , "Yigit, Ferruh" , dev , Thomas Monjalon , Jerin Jacob Kollanukkaran , Kiran Kumar Kokkilagadda , Olivier Matz , "Burakov, Anatoly" , Andrew Rybchenko , Stephen Hemminger Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v13 2/2] kni: support IOVA mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Fri, Nov 15, 2019 at 6:29 PM David Marchand wrote: > > I can't see an interest in splitting this patch from the kmod update. > Ferruh, what do you think? > > > On Fri, Nov 15, 2019 at 12:19 PM wrote: > > > > From: Vamsi Attunuru > > > > Current KNI implementation only operates in IOVA_PA mode > > patch adds required functionality to enable KNI in > > IOVA_VA mode. > > > > KNI loopback mode tests will have performance impact in > > this mode due to IOVA to KVA address translations. > > However, In KNI real world use cases, the performace > > performance > > > impact will be based on Linux kernel stack and scheduler > > latencies. Performance varies based on the KNI use case. > > If bus iommu scheme is IOVA_DC and KNI module is loaded, > > DPDK chooses IOVA as PA as existing behaviour. > > > > During KNI creation, app's iova_mode details are passed to > > the KNI kernel module, accordingly kernel module translates > > PA/IOVA addresses to KVA and vice-versa. > > > > Signed-off-by: Vamsi Attunuru > > Signed-off-by: Kiran Kumar K > > Suggested-by: Ferruh Yigit > > --- > > doc/guides/prog_guide/kernel_nic_interface.rst | 15 +++++++++++++++ > > doc/guides/rel_notes/release_19_11.rst | 15 ++++++++++++++- > > lib/librte_eal/linux/eal/eal.c | 23 ++++++++++++++++------- > > lib/librte_kni/rte_kni.c | 6 ++++++ > > 4 files changed, 51 insertions(+), 8 deletions(-) > > > > [snip] > > > diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c > > index 9e2d50c..53ca84b 100644 > > --- a/lib/librte_eal/linux/eal/eal.c > > +++ b/lib/librte_eal/linux/eal/eal.c > > @@ -1086,14 +1086,23 @@ rte_eal_init(int argc, char **argv) > > } > > } > > #ifdef RTE_LIBRTE_KNI > > - /* Workaround for KNI which requires physical address to work */ > > - if (iova_mode == RTE_IOVA_VA && > > - rte_eal_check_module("rte_kni") == 1) { > > - if (phys_addrs) { > > + if (rte_eal_check_module("rte_kni") == 1) { > > +#if KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE > > + if (iova_mode == RTE_IOVA_VA) { > > iova_mode = RTE_IOVA_PA; > > - RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI module is loaded\n"); > > - } else { > > - RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); > > + RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because " > > + "Kernel version supports only 'PA' mode for KNI module\n"); > > + } > > +#endif > > + if (rte_bus_get_iommu_class() == RTE_IOVA_DC) > > + iova_mode = RTE_IOVA_PA; > > If physical addresses are unavailable, this code forces PA anyway. > > > > + > > + if (iova_mode == RTE_IOVA_PA) { > > + if (phys_addrs && is_iommu_enabled()) > > + RTE_LOG(WARNING, EAL, "Forced IOVA as 'PA' because KNI module is loaded\n"); > > + > > + if (!phys_addrs) > > + RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); > > } > > Checking physical addresses availability after, and having a log is not enough. > > > So far, KNI could not work with IOVA as VA. > Your patchset adds support for IOVA as VA if kernel is >= 4.6. > Repeating my proposal (as far as eal.c is concerned) of just changing: We need achive the following. IOVA as PA has performance implication on KNI case. So we need to make IOVA as PA when KNI module is loaded. Your suggestion makes IOVA as PA when bus return IOVA as VA due the fact the KNI check is latter. We need to move that up. How about the following [master]dell[dpdk.org] $ git diff diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 9e2d50cfb..085fde767 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -1064,6 +1064,21 @@ rte_eal_init(int argc, char **argv) /* autodetect the IOVA mapping mode */ enum rte_iova_mode iova_mode = rte_bus_get_iommu_class(); +#if defined(RTE_LIBRTE_KNI) && KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE + /* IOVA as PA gives better performance for KNI. Choose IOVA as + * PA when bus returns RTE_IOVA_DC and KNI module is present. + */ + if (iova_mode == RTE_IOVA_DC && + rte_eal_check_module("rte_kni") == 1) { + if (phys_addrs) { + iova_mode = RTE_IOVA_PA; + RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI module is loaded\n"); + } else { + RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); + } + } +#endif + if (iova_mode == RTE_IOVA_DC) { RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode.\n"); @@ -1085,18 +1100,6 @@ rte_eal_init(int argc, char **argv) RTE_LOG(DEBUG, EAL, "IOMMU is not available, selecting IOVA as PA mode.\n"); } } -#ifdef RTE_LIBRTE_KNI - /* Workaround for KNI which requires physical address to work */ - if (iova_mode == RTE_IOVA_VA && - rte_eal_check_module("rte_kni") == 1) { - if (phys_addrs) { - iova_mode = RTE_IOVA_PA; - RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI module is loaded\n"); - } else { - RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); - } - } -#endif rte_eal_get_configuration()->iova_mode = iova_mode; } else { rte_eal_get_configuration()->iova_mode = [master]dell[dpdk.org] $ > > @@ -1085,7 +1085,7 @@ rte_eal_init(int argc, char **argv) > RTE_LOG(DEBUG, EAL, "IOMMU is not > available, selecting IOVA as PA mode.\n"); > } > } > -#ifdef RTE_LIBRTE_KNI > +#if defined(RTE_LIBRTE_KNI) && KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE > /* Workaround for KNI which requires physical address to work */ > if (iova_mode == RTE_IOVA_VA && > > > > -- > David Marchand >