From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 25589A00E6 for ; Fri, 12 Jul 2019 14:27:34 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EFBAD1B9B5; Fri, 12 Jul 2019 14:27:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id E4BED1B9B2 for ; Fri, 12 Jul 2019 14:27:30 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x6CCPFKs012384; Fri, 12 Jul 2019 05:27:29 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=pfpt0818; bh=HvtVz9772lACdK7iCRFWTt8owZujRfwdpsHcKV024/s=; b=og4Fqx99WeRrxLxSrqHaZHDWMaZZwszKSmE2lSh5Xq6aI8+ycYZ12uRJaobfAmiuwVcr zh3sJb0ClLDgcOTd+ITolFT+EENqe1KKtlYNxNSziLi00FHF7/9Ete9+kUD+CODulbmj H+U27E7Cy0Vu0qQpnv/2VSsueQgGA7FhLhjjmhKQvU5GBEvyEHwT65iBin344pAYPfUm vzT85fdYdI9+wvgwC5fm55Y7ZRqlqd+7xyFEr2Xm1nR/G2B5JVMUelK2QrUaJYQN4XRM csIe3BETenwTkKFFAk6DWLabrdH/u2ihyqA+dDNncWC+tDYF12QOAt4nFqh7ndgHjpxR BQ== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2tpr4j0dnu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 12 Jul 2019 05:27:29 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 12 Jul 2019 05:27:28 -0700 Received: from NAM04-CO1-obe.outbound.protection.outlook.com (104.47.45.55) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Fri, 12 Jul 2019 05:27:28 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EdDVWL0EBt+IdyYUgjr546LCMJPCeIy8WiyxJzS/jbhqJgB5d5RKERod6EgKv/O34Zs0ID7hVgJJHiqf+l2dXwgEdgVY8f2myCCahdhibaPGTTlE2vsH06GGb9klwJYKqxFJ75NnSzwcbzZi5EZUToVDg33HD3y4Ter5qrHS4SmLSLxH3hKJK+jUeY+zMyoqnpSQs1a6opoj4CDCuMI0hDU8Q4aX0TwhsO39jpO4KqHfL8Y860l4fP5NvvC9scIFXfqSJiJb7mmoMMtdQ8T82Ou9OVuNCl/dGpcOXJx9rUNj2YhN+N13E4/Tv/oc0Zs/9eFk67xhb328qbEjaryvOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HvtVz9772lACdK7iCRFWTt8owZujRfwdpsHcKV024/s=; b=ez5NVg/Gd+YUt4tZCC0/z1sFBk8zslIE6rglL2/Of5VjAtLIlm9ZAWqWgOSvaOkcfSwXMhq3Hm1Gxe2LuCK4g6wRFLKonQ0eIp2yS8A/SwAOf75wQJudYlVi24fFb2vcA4D44SyJspdqbZtmxNqmhrnGDrbe3lsOCKmarBqPaieuCtL2oz1TsJ3EjDK61PSRPN2EKk8za+rMPppg+LNk688ZNpuBIYhrref6FfDvpp9gJe6k7ubCSQwuxDgRUDCnW6301zaMSR1zZuwBxJsLl/z/ojKaZe50XwYvQ78qQSYzejFiu+1tG85eBkIf4OAE3P4VvONdJ8tBcuiUJY2nLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=marvell.com;dmarc=pass action=none header.from=marvell.com;dkim=pass header.d=marvell.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HvtVz9772lACdK7iCRFWTt8owZujRfwdpsHcKV024/s=; b=tbrxEmFlBSuPNJzloShxocIDGaF29Q5FTCTr2hXzqsx7/pF4huKmGrlsSC/6I7MDwGJOjbFfhXa2fVskOwrbYaK0cpWrQDiN3JlSalMTVDvbz8OuDGNPEdgvLFaDfUKQD+rdGmL5mCR0qz7B6EuCTQGWygaIuRwAYa6szA73pE4= Received: from CH2PR18MB3381.namprd18.prod.outlook.com (52.132.246.204) by CH2PR18MB3383.namprd18.prod.outlook.com (52.132.247.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.14; Fri, 12 Jul 2019 12:27:25 +0000 Received: from CH2PR18MB3381.namprd18.prod.outlook.com ([fe80::189c:3889:b207:8922]) by CH2PR18MB3381.namprd18.prod.outlook.com ([fe80::189c:3889:b207:8922%5]) with mapi id 15.20.2052.019; Fri, 12 Jul 2019 12:27:25 +0000 From: Vamsi Krishna Attunuru To: Ferruh Yigit , "dev@dpdk.org" , Jerin Jacob Kollanukkaran CC: "olivier.matz@6wind.com" , "arybchenko@solarflare.com" , "Kiran Kumar Kokkilagadda" Thread-Topic: [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module Thread-Index: AQHVKwoY02zw7QZOhk6Qx0GGn6LtY6bFtcEAgAEscCyAAAxvgIAAFDLH Date: Fri, 12 Jul 2019 12:27:25 +0000 Message-ID: References: <20190422061533.17538-1-kirankumark@marvell.com> <20190625035700.2953-1-vattunuru@marvell.com> <20190625035700.2953-5-vattunuru@marvell.com> <98bf2103-f48f-4baa-0d4a-f03f9e538519@intel.com> , <285145b7-a829-b614-7971-2df171800466@intel.com> In-Reply-To: <285145b7-a829-b614-7971-2df171800466@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [103.227.98.232] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e7fd9266-bc9a-47bc-6787-08d706c44c3b x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:CH2PR18MB3383; x-ms-traffictypediagnostic: CH2PR18MB3383: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-forefront-prvs: 00963989E5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(346002)(136003)(376002)(396003)(39860400002)(199004)(189003)(19627405001)(53936002)(52536014)(229853002)(68736007)(6636002)(6606003)(14444005)(66066001)(7736002)(86362001)(7696005)(6436002)(74316002)(54896002)(99286004)(256004)(9686003)(5660300002)(6246003)(107886003)(2501003)(4326008)(186003)(76176011)(55016002)(71190400001)(6506007)(66946007)(478600001)(8936002)(33656002)(110136005)(316002)(486006)(81156014)(66476007)(26005)(14454004)(102836004)(54906003)(81166006)(76116006)(66446008)(64756008)(25786009)(476003)(446003)(53546011)(11346002)(71200400001)(6116002)(66556008)(3846002)(2906002)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:CH2PR18MB3383; H:CH2PR18MB3381.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: AHMN5Ydesa58UT6lkA5Sr/t6Wbf06L/p1Njkmtw9sWN3egeFirbxCac6jooE1XVYWnMsI2/qaILJuT2DG24hUYRpxsZoXViDZz/EdAJZ2Oul7p4vAA9sascmrMEMBaj69Yev98ho6BZ7XKSA3PA+wE3+/ccgfOlBuAF7l1NcHrlQ9jU0uukhK4TdAJPv2Bu4r8zYVeQB5zeSNRP79QZ4D/Wnp1tFKLralvW7/0YPcY2UGvZpTJcb02f9p2CrKb1QrNU3GC9uDYfPMaSm63XxqRqvgAtVKVq5M+byKqgxIo9IYAzKU9MQNQVNel+gJd+Cpfi+w/AwNlM0sP8Z5RDxxp+4pYu9ztYsQjz9NvZiNbRf8YxRfJV2tdxjMPU6hKsDftfAkBKNWJVseVpfL9SuZvFgYqiu4boP2JWSKquyTVg= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: e7fd9266-bc9a-47bc-6787-08d706c44c3b X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jul 2019 12:27:25.7643 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: vattunuru@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR18MB3383 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-12_04:, , signatures=0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" ________________________________ From: Ferruh Yigit Sent: Friday, July 12, 2019 4:40 PM To: Vamsi Krishna Attunuru; dev@dpdk.org Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Kiran Kumar Kokkilag= adda Subject: Re: [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in= kni module On 7/12/2019 11:38 AM, Vamsi Krishna Attunuru wrote: > > > > -------------------------------------------------------------------------= ------- > *From:* Ferruh Yigit > *Sent:* Thursday, July 11, 2019 10:00 PM > *To:* Vamsi Krishna Attunuru; dev@dpdk.org > *Cc:* olivier.matz@6wind.com; arybchenko@solarflare.com; Kiran Kumar Kokk= ilagadda > *Subject:* [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in= kni > module > > External Email > > ---------------------------------------------------------------------- > On 6/25/2019 4:57 AM, vattunuru@marvell.com wrote: >> From: Kiran Kumar K >> >> Patch adds support for kernel module to work in IOVA =3D VA mode, >> the idea is to get physical address from iova address using >> iommu_iova_to_phys API and later use phys_to_virt API to >> convert the physical address to kernel virtual address. >> >> When compared with IOVA =3D PA mode, there is no performance >> drop with this approach. >> >> This approach does not work with the kernel versions less >> than 4.4.0 because of API compatibility issues. >> >> Signed-off-by: Kiran Kumar K >> Signed-off-by: Vamsi Attunuru > > <...> > >> @@ -351,15 +354,56 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_n= um, >> strncpy(kni->name, dev_info.name, RTE_KNI_NAMESIZE); >> >> /* Translate user space info into kernel space info */ >> - kni->tx_q =3D phys_to_virt(dev_info.tx_phys); >> - kni->rx_q =3D phys_to_virt(dev_info.rx_phys); >> - kni->alloc_q =3D phys_to_virt(dev_info.alloc_phys); >> - kni->free_q =3D phys_to_virt(dev_info.free_phys); >> - >> - kni->req_q =3D phys_to_virt(dev_info.req_phys); >> - kni->resp_q =3D phys_to_virt(dev_info.resp_phys); >> - kni->sync_va =3D dev_info.sync_va; >> - kni->sync_kva =3D phys_to_virt(dev_info.sync_phys); >> + if (dev_info.iova_mode) { >> +#if KERNEL_VERSION(4, 4, 0) > LINUX_VERSION_CODE > > We have "kni/compat.h" to put the version checks, please use abstracted f= eature > checks only in the code. > From experience this goes ugly quickly with the addition to distro kernel= s and > their specific versioning, so better to hide these all from the source co= de. > > And this version requirement needs to be documented in kni doc. > > ack > >> + (void)pci; >> + pr_err("Kernel version is not supported\n"); > > Can you please include 'iova_mode' condition into the message log, becaus= e this > kernel version is supported if user wants to use via 'iova_mode =3D=3D 0'= condition. > > ack > >> + return -EINVAL; >> +#else >> + pci =3D pci_get_device(dev_info.vendor_id, >> + dev_info.device_id, NULL); >> + while (pci) { >> + if ((pci->bus->number =3D=3D dev_info.bus) && >> + (PCI_SLOT(pci->devfn) =3D=3D dev_info.devid) &= & >> + (PCI_FUNC(pci->devfn) =3D=3D dev_info.function= )) { >> + domain =3D iommu_get_domain_for_dev(&pci->= dev); >> + break; >> + } >> + pci =3D pci_get_device(dev_info.vendor_id, >> + dev_info.device_id, pci); >> + } > > What if 'pci' is NULL here? > In kni it is not required to provide a device at all. > > Ack, will add a NULL check. > other point is not clear to me, device info is absolutely required at lea= st > for IOVA=3DVA mode, since it requires to procure iommu domain details. "device info is absolutely required" *only* for IOVA=3DVA mode, so user may= skip to provide it. > Any thoughts or ways to address this without device.? Return error if 'iova_mode' requested but device info not? But you didn't replied to passing 'iova_mode' from application, I would lik= e hear what you are thinking about it.. There is a query on that regard in the cover letter mail chain. Anyways bel= ow is that """ > I support the idea to remove 'kni' forcing to the IOVA=3DPA mode, but als= o not > sure about forcing all KNI users to update their code to allocate mempool= in a > very specific way. > > What about giving more control to the user on this? > > Any user want to use IOVA=3DVA and KNI together can update application to > justify memory allocation of the KNI and give an explicit "kni iova_mode= =3D1" > config. Jerin > Where this config comes, eal or kni sample app or KNI public API? """ > > <...> > >> @@ -186,7 +202,10 @@ kni_fifo_trans_pa2va(struct kni_dev *kni, >> return; >> >> for (i =3D 0; i < num_rx; i++) { >> - kva =3D pa2kva(kni->pa[i]); >> + if (likely(kni->iova_mode =3D=3D 1)) >> + kva =3D iova2kva(kni, kni->pa[i]); >> + else >> + kva =3D pa2kva(kni->pa[i]); > > To reduce the churn, what about updating the 'pa2kva()' and put the > "(kni->iova_mode =3D=3D 1)" check there? Does it help? (not only 'pa2kva(= )' but its > friends also, and if it makes more sense agree to rename the functions) > > No, in VA mode, kni->pa[i] points to iova address, pa2kva() of iova addre= ss might > crash, hence the if..else check is added. I understand that part. What I am suggestion is something like this: kva =3D common_function(kni, kni->pa[i]); --- common_function() { if (unlikely(kni->iova_mode =3D=3D 1)) return iova2kva(kni, kni->pa[i]); return pa2kva(kni->pa[i]); } To hide the check in the function and make code more readable > > And btw, why 'likely' case is "kni->iova_mode =3D=3D 1"? > > no specific case other than branch predict, will remove this if it's real= ly > harmful to PA mode.