From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4101EA04C2; Thu, 14 Nov 2019 11:57:38 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 932DD2AB; Thu, 14 Nov 2019 11:57:36 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 0576127D for ; Thu, 14 Nov 2019 11:57:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573729054; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QSLXz4fv8DT85kAEx+waI88C23WjTo15Ac/snoxwOaU=; b=Mz3PNhmMp8+hFGGuopWx2eFqq1fMAHfgJMZNXg+n0FN8NKNCVhH2qqsk9TiUCoBjiSrCYB y/FQ5pCaXXPCSfoiWLAqf6T1GSBMXw4MOZH/RrMSA+gQ+0PCo+kIerjVBuLfTMKA7AC2tU 4nwTM4YO1fO8uoNWT2LaTgRouH0UPS4= Received: from mail-vk1-f200.google.com (mail-vk1-f200.google.com [209.85.221.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-La35aRuDM3ivbUY_889cRg-1; Thu, 14 Nov 2019 05:57:31 -0500 Received: by mail-vk1-f200.google.com with SMTP id i124so2463901vkc.2 for ; Thu, 14 Nov 2019 02:57:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=DQ1RvuN1T+wudVDonGxFuF4mcEl2JO+H6vLqO/sf15Y=; b=d0ohXeo25/8M/cQeP6bCWFGH8RyvttDKzipguk93QN3PLDHNcsaDo5IpBnCUGeemn2 a0mpCgwRkuv2hc2uGJbhzwD2i4tSCPEs1jIWUeFKvfz5PdXb3wtjnDHPmBv8bF2KtTob DvasuUtBtmcRxWo8ZQXI2wo8UWuJP22zlQCygp+T70lFy7YXMePOnWZgcxwJRFWGDfwA pef3nerdU5UPOUIn9WIziKWoOpCUwl/ODFl/l5Tx4vDBM2F3dVRip4Yqjpjj4wbiXhY1 AmiuI3Nput3F449IOkNiIq+4vTgBegRxaEB0NOZZYChvyJvZK0pV2vErrOrL41vYMens sKpw== X-Gm-Message-State: APjAAAWUBoimhho2p2SERPSRUusiGnni7Ec9ZcK645Qoq9tNzxAvrxd1 yrft7UnUukZjwhPJPxLpgkipBN/n9Tv8RNBHFr5AaqIxdVY/5yQS6+pHlu2O7AcMYjAdNeTBS8Y Mcer+8QVtIJz1ZsSVvxU= X-Received: by 2002:a1f:60ce:: with SMTP id u197mr4743129vkb.80.1573729050746; Thu, 14 Nov 2019 02:57:30 -0800 (PST) X-Google-Smtp-Source: APXvYqy5n4OMYQhFA1f0idfT7haHn1O3/oxczRPYUCwvcYacv8Ak076Tkxz1qFQ6WuxYo+VDzxav8NY7Bsic60tqQWE= X-Received: by 2002:a1f:60ce:: with SMTP id u197mr4743112vkb.80.1573729050317; Thu, 14 Nov 2019 02:57:30 -0800 (PST) MIME-Version: 1.0 References: <20191021080324.10659-1-vattunuru@marvell.com> <20191105110416.8955-1-vattunuru@marvell.com> <20191105110416.8955-2-vattunuru@marvell.com> In-Reply-To: <20191105110416.8955-2-vattunuru@marvell.com> From: David Marchand Date: Thu, 14 Nov 2019 11:57:18 +0100 Message-ID: To: Vamsi Attunuru Cc: dev , Thomas Monjalon , Jerin Jacob Kollanukkaran , kirankumark@marvell.com, Olivier Matz , "Yigit, Ferruh" , "Burakov, Anatoly" , Andrew Rybchenko , Stephen Hemminger X-MC-Unique: La35aRuDM3ivbUY_889cRg-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH v12 1/2] kni: add IOVA=VA mode support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Nov 5, 2019 at 12:05 PM wrote: > > From: Vamsi Attunuru > > Current KNI implementation only operates in IOVA_PA mode > patch adds required functionality to enable KNI in > IOVA_VA mode. > > KNI can operate in IOVA_VA mode when this mode is requested > using eal option (`iova-mode=3Dva`) or when bus iommu scheme is > selected as RTE_IOVA_VA. > > During KNI creation, app's iova_mode details are passed to > the KNI kernel module, accordingly kernel module translates > PA/IOVA addresses to KVA and vice-versa. > > Signed-off-by: Vamsi Attunuru > Signed-off-by: Kiran Kumar K > Suggested-by: Ferruh Yigit > --- > doc/guides/prog_guide/kernel_nic_interface.rst | 9 +++++++ > doc/guides/rel_notes/release_19_11.rst | 5 ++++ > lib/librte_eal/linux/eal/eal.c | 29 +++++++++++++----= ------ > lib/librte_eal/linux/eal/include/rte_kni_common.h | 1 + > lib/librte_kni/rte_kni.c | 7 ++---- > 5 files changed, 34 insertions(+), 17 deletions(-) > > diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/= prog_guide/kernel_nic_interface.rst > index 2fd58e1..f869493 100644 > --- a/doc/guides/prog_guide/kernel_nic_interface.rst > +++ b/doc/guides/prog_guide/kernel_nic_interface.rst > @@ -300,6 +300,15 @@ The sk_buff is then freed and the mbuf sent in the t= x_q FIFO. > The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_e= th_tx_burst()``. > It then puts the mbuf back in the cache. > > +IOVA =3D VA: Support > +------------------ > + > +KNI operates in IOVA_VA scheme when > + > +- LINUX_VERSION_CODE >=3D KERNEL_VERSION(4, 6, 0) and > +- eal option `iova-mode=3Dva` is passed or bus IOVA scheme in the DPDK i= s selected > + as RTE_IOVA_VA. > + > Ethtool > ------- > > diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_note= s/release_19_11.rst > index ae8e7b2..e5d2996 100644 > --- a/doc/guides/rel_notes/release_19_11.rst > +++ b/doc/guides/rel_notes/release_19_11.rst > @@ -231,6 +231,11 @@ New Features > * Added a console command to testpmd app, ``show port (port_id) ptypes= `` which > gives ability to print port supported ptypes in different protocol l= ayers. > > +* **Added IOVA as VA support for KNI.** > + > + Added IOVA =3D VA support for KNI, KNI can operate in IOVA =3D VA mode= when > + `iova-mode=3Dva` eal option is passed to the application or when bus I= OVA > + scheme is selected as RTE_IOVA_VA. Add a mention about the kernel requirement here too please. Missing newline. > > Removed Items > ------------- > diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/ea= l.c > index 9e2d50c..a1c5bf6 100644 > --- a/lib/librte_eal/linux/eal/eal.c > +++ b/lib/librte_eal/linux/eal/eal.c > @@ -922,6 +922,19 @@ static int rte_eal_vfio_setup(void) > } > #endif > > +static enum rte_iova_mode > +rte_eal_kni_get_iova_mode(enum rte_iova_mode iova_mode) > +{ > + if (iova_mode =3D=3D RTE_IOVA_VA) { > +#if KERNEL_VERSION(4, 6, 0) > LINUX_VERSION_CODE I understood that IOVA as VA is possible with kernel >=3D 4.6. Should it be < ? > + iova_mode =3D RTE_IOVA_PA; > + RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI m= odule does not support VA\n"); > +#endif > + } > + > + return iova_mode; > +} > + > static void rte_eal_init_alert(const char *msg) > { > fprintf(stderr, "EAL: FATAL: %s\n", msg); > @@ -1085,24 +1098,16 @@ rte_eal_init(int argc, char **argv) > RTE_LOG(DEBUG, EAL, "IOMMU is not availab= le, selecting IOVA as PA mode.\n"); > } > } > -#ifdef RTE_LIBRTE_KNI You removed this part, but we don't have to consider kni things if RTE_LIBRTE_KNI is not compiled in. > - /* Workaround for KNI which requires physical address to = work */ > - if (iova_mode =3D=3D RTE_IOVA_VA && > - rte_eal_check_module("rte_kni") =3D=3D 1)= { > - if (phys_addrs) { The check on physical address availability has been lost. > - iova_mode =3D RTE_IOVA_PA; > - RTE_LOG(WARNING, EAL, "Forcing IOVA as 'P= A' because KNI module is loaded\n"); > - } else { > - RTE_LOG(DEBUG, EAL, "KNI can not work sin= ce physical addresses are unavailable\n"); > - } > - } > -#endif > rte_eal_get_configuration()->iova_mode =3D iova_mode; > } else { > rte_eal_get_configuration()->iova_mode =3D > internal_config.iova_mode; > } > > + if (rte_eal_check_module("rte_kni") =3D=3D 1) > + rte_eal_get_configuration()->iova_mode =3D > + rte_eal_kni_get_iova_mode(rte_eal_iova_mo= de()); > + You are forcing this even if the user specified a iova mode. In the end, can't we just change the check on the workaround? Something like: @@ -1085,7 +1085,7 @@ rte_eal_init(int argc, char **argv) RTE_LOG(DEBUG, EAL, "IOMMU is not available, selecting IOVA as PA mode.\n"); } } -#ifdef RTE_LIBRTE_KNI +#if defined(RTE_LIBRTE_KNI) && KERNEL_VERSION(4, 6, 0) < LINUX_VERSION_COD= E /* Workaround for KNI which requires physical address to wo= rk */ if (iova_mode =3D=3D RTE_IOVA_VA && rte_eal_check_module("rte_kni") =3D=3D 1) { It would be the only change in the whole file lib/librte_eal/linux/eal/eal.= c > if (rte_eal_iova_mode() =3D=3D RTE_IOVA_PA && !phys_addrs) { > rte_eal_init_alert("Cannot use IOVA as 'PA' since physica= l addresses are not available"); > rte_errno =3D EINVAL; > diff --git a/lib/librte_eal/linux/eal/include/rte_kni_common.h b/lib/libr= te_eal/linux/eal/include/rte_kni_common.h > index 46f75a7..2427a96 100644 > --- a/lib/librte_eal/linux/eal/include/rte_kni_common.h > +++ b/lib/librte_eal/linux/eal/include/rte_kni_common.h > @@ -125,6 +125,7 @@ struct rte_kni_device_info { > unsigned int min_mtu; > unsigned int max_mtu; > uint8_t mac_addr[6]; > + uint8_t iova_mode; > }; > > #define KNI_DEVICE "kni" > diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c > index 7fbcf22..7221280 100644 > --- a/lib/librte_kni/rte_kni.c > +++ b/lib/librte_kni/rte_kni.c > @@ -97,11 +97,6 @@ static volatile int kni_fd =3D -1; > int > rte_kni_init(unsigned int max_kni_ifaces __rte_unused) > { Should this check still apply with kernel < 4.6? Like: + #if KERNEL_VERSION(4, 6, 0) < LINUX_VERSION_CODE > - if (rte_eal_iova_mode() !=3D RTE_IOVA_PA) { > - RTE_LOG(ERR, KNI, "KNI requires IOVA as PA\n"); > - return -1; > - } > - + #endif > /* Check FD and open */ > if (kni_fd < 0) { > kni_fd =3D open("/dev/" KNI_DEVICE, O_RDWR); > @@ -302,6 +297,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, > kni->group_id =3D conf->group_id; > kni->mbuf_size =3D conf->mbuf_size; > > + dev_info.iova_mode =3D (rte_eal_iova_mode() =3D=3D RTE_IOVA_VA) ?= 1 : 0; > + > ret =3D ioctl(kni_fd, RTE_KNI_IOCTL_CREATE, &dev_info); > if (ret < 0) > goto ioctl_fail; > -- > 2.8.4 > --=20 David Marchand