From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 80DCFA00E6 for ; Fri, 12 Jul 2019 12:38:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 62B511B9E7; Fri, 12 Jul 2019 12:38:26 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 1A3701B9DF for ; Fri, 12 Jul 2019 12:38:23 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x6CAartq022515; Fri, 12 Jul 2019 03:38:22 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=pfpt0818; bh=3XM1W1SchoWzPu0lRMJixhgHeK76PtDQ7CaAWG0AkB0=; b=p50BgXi0075K5dF7IaQqO1O2GFCwG1YfeCd8yY4xzW+XeZSM1p969Ggyl1iRbsbCqkYG 4Mj91CE9HhUnfLlwMz1ygyjflyMe+8JeBclkajHDdns8rYcCQGRBzDDZHKmpIhre+zwJ sxPUGIDeKhVmDdY/YiyJxd+jFUeRoijEzTJbHXvQ39pubFFeO6Sg4JQNtCJfBegT2FS3 aIvmM9IMC/SnzIH+FdQSqOGqD45f2iERpk3iO7Q2Zzq3BXBviEOBB9KoWfxLJx6GMVRQ O5i5MpGUmY1a546YptSUT+m3y/psTK2s8d+IO3W8arwA9YRVkuiwnroBbGMHLFH2QM8r ZQ== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2tpr4j03g5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 12 Jul 2019 03:38:22 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 12 Jul 2019 03:38:21 -0700 Received: from NAM04-SN1-obe.outbound.protection.outlook.com (104.47.44.51) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Fri, 12 Jul 2019 03:38:20 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NYPr58m85y49zaySu4aF6h1w6ADhYDdYT6X+C1NOfaqR8qLauJKz70H7L4FSYYnj9vop3igeGdz0DFtur5BT1uKzN9q2I+d5E4+pvhSJvY92QMhtSWeRGmUzVdtvx4XGxSVoJnpsBJr7G6nwSvH8jR+klwZ2HBMFGbScjLzQ7Af0dTssSiTM+B1aZqCchBgT/Z2d6GeU+kuKctt/+uwepRTh6UtRhrvA65/47Z00ANS32pFnRRVxmXYGvZZHfo1JKSIL1J96gg34TEZVC1Ex8Rk64Rlhql3JR3qPSmc7cmTqxrr4pXj+WCBwwS0c33uJIGu7SE1DJTRhn6oYsTgWyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3XM1W1SchoWzPu0lRMJixhgHeK76PtDQ7CaAWG0AkB0=; b=hZ3sP7b14RFZCkvU+kbnCHd35iULt4NSH4UZMaOQJelEFH2pbZnqgP9xoktCOe8yKoabugmr655QXL17u79pRP1674C0dvZ3tw3UQumH/Z4XH1faNDeQG+kLDDKZC1Tw9FYOo4a+JsrYZQ+R/z/Ryd+OczrM2knS22pID8PCiIoT7evlRc9rVJOiY7/qUzyUU/0YlmRsXlvufZGqYyEDBk4Fu2Znwpcdmw6GTMq+7YxvZzXbQ0DPYprtrqt3jKoZ3GWaSPu2EwK+8nFmfRdIgvtrKwn9wor+F0+H4b0T3QyFdRGdLMqZnBJs/g1Ocf5R3H/xUy8uLGupFj2+q4rQJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=marvell.com;dmarc=pass action=none header.from=marvell.com;dkim=pass header.d=marvell.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3XM1W1SchoWzPu0lRMJixhgHeK76PtDQ7CaAWG0AkB0=; b=o/Ez9z3jhksCiABH04W8+TDgOBVuvteqX+OzCGMX6JG7a4JMZtKiWdFSEFHjVFGc1/v3JKcoKEFNknuxWmjJD+Q+f8h98qtAHKG7/cAMp804ZbFByQmSQs6ra6/tzfYnNe0RClTT+HkzjcHjIaUqf/FcxjqSo2JrZTE6QwBsd24= Received: from CH2PR18MB3381.namprd18.prod.outlook.com (52.132.246.204) by CH2PR18MB3414.namprd18.prod.outlook.com (52.132.247.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.10; Fri, 12 Jul 2019 10:38:19 +0000 Received: from CH2PR18MB3381.namprd18.prod.outlook.com ([fe80::189c:3889:b207:8922]) by CH2PR18MB3381.namprd18.prod.outlook.com ([fe80::189c:3889:b207:8922%5]) with mapi id 15.20.2052.019; Fri, 12 Jul 2019 10:38:19 +0000 From: Vamsi Krishna Attunuru To: Ferruh Yigit , "dev@dpdk.org" CC: "olivier.matz@6wind.com" , "arybchenko@solarflare.com" , "Kiran Kumar Kokkilagadda" Thread-Topic: [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module Thread-Index: AQHVKwoY02zw7QZOhk6Qx0GGn6LtY6bFtcEAgAEscCw= Date: Fri, 12 Jul 2019 10:38:19 +0000 Message-ID: References: <20190422061533.17538-1-kirankumark@marvell.com> <20190625035700.2953-1-vattunuru@marvell.com> <20190625035700.2953-5-vattunuru@marvell.com>, <98bf2103-f48f-4baa-0d4a-f03f9e538519@intel.com> In-Reply-To: <98bf2103-f48f-4baa-0d4a-f03f9e538519@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.140.231.66] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a302092c-20af-49f8-65d8-08d706b50e5a x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:CH2PR18MB3414; x-ms-traffictypediagnostic: CH2PR18MB3414: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3276; x-forefront-prvs: 00963989E5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(346002)(376002)(39860400002)(366004)(396003)(199004)(189003)(86362001)(6606003)(55236004)(8936002)(99286004)(256004)(3846002)(54896002)(9686003)(6116002)(55016002)(2501003)(76176011)(7736002)(6436002)(6246003)(53936002)(74316002)(14454004)(54906003)(446003)(8676002)(33656002)(476003)(11346002)(486006)(81166006)(107886003)(68736007)(229853002)(102836004)(2906002)(19627405001)(4326008)(64756008)(110136005)(6506007)(26005)(7696005)(186003)(66066001)(25786009)(66556008)(66446008)(66946007)(71190400001)(71200400001)(52536014)(76116006)(478600001)(66476007)(53546011)(81156014)(5660300002)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:CH2PR18MB3414; H:CH2PR18MB3381.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 8CU+kg/5L+VNgVSC3YKc+jwmyLU+BbAEPtfAPTd76rSEnvAfnLVhIBqxQ4uc7lmXWhWvpEhEyBQCiCT2YImZ9PDpDH5hlwuRJmOttIvPy/qu481K9/ne5+8xnIOOwrE5V/rUDxUUAURkAwDTXOab/Pgj4mLoLBtY8+gxOFVcvgfX9Ogdl21ROsj/F1Da6ProbyOClLHyVrFAJ4gGv0KGG+aae4z9dO4uiO6d6dVv+BijmWOF77Lrx1IbP8JdhGDH2Ub6OxPxLBuK1FbXTFerHj5hBXb2uyvGQWPBAjihtZCeAN3BUJXILPxpK+Vv2ALa+QjbRlWrpCMohwwS83CtVIqVDVmWVtRVQxMZm/AsnX9UGOC+bLfalhTxbNcD7xg8neJl4Nqr5Oj7093o4SplwRaIfH+n3xCGygCbOdTR1JU= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: a302092c-20af-49f8-65d8-08d706b50e5a X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Jul 2019 10:38:19.4597 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: vattunuru@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR18MB3414 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-12_03:, , signatures=0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni module X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" ________________________________ From: Ferruh Yigit Sent: Thursday, July 11, 2019 10:00 PM To: Vamsi Krishna Attunuru; dev@dpdk.org Cc: olivier.matz@6wind.com; arybchenko@solarflare.com; Kiran Kumar Kokkilag= adda Subject: [EXT] Re: [PATCH v6 4/4] kernel/linux/kni: add IOVA support in kni= module External Email ---------------------------------------------------------------------- On 6/25/2019 4:57 AM, vattunuru@marvell.com wrote: > From: Kiran Kumar K > > Patch adds support for kernel module to work in IOVA =3D VA mode, > the idea is to get physical address from iova address using > iommu_iova_to_phys API and later use phys_to_virt API to > convert the physical address to kernel virtual address. > > When compared with IOVA =3D PA mode, there is no performance > drop with this approach. > > This approach does not work with the kernel versions less > than 4.4.0 because of API compatibility issues. > > Signed-off-by: Kiran Kumar K > Signed-off-by: Vamsi Attunuru <...> > @@ -351,15 +354,56 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_nu= m, > strncpy(kni->name, dev_info.name, RTE_KNI_NAMESIZE); > > /* Translate user space info into kernel space info */ > - kni->tx_q =3D phys_to_virt(dev_info.tx_phys); > - kni->rx_q =3D phys_to_virt(dev_info.rx_phys); > - kni->alloc_q =3D phys_to_virt(dev_info.alloc_phys); > - kni->free_q =3D phys_to_virt(dev_info.free_phys); > - > - kni->req_q =3D phys_to_virt(dev_info.req_phys); > - kni->resp_q =3D phys_to_virt(dev_info.resp_phys); > - kni->sync_va =3D dev_info.sync_va; > - kni->sync_kva =3D phys_to_virt(dev_info.sync_phys); > + if (dev_info.iova_mode) { > +#if KERNEL_VERSION(4, 4, 0) > LINUX_VERSION_CODE We have "kni/compat.h" to put the version checks, please use abstracted fea= ture checks only in the code. >From experience this goes ugly quickly with the addition to distro kernels = and their specific versioning, so better to hide these all from the source code= . And this version requirement needs to be documented in kni doc. ack > + (void)pci; > + pr_err("Kernel version is not supported\n"); Can you please include 'iova_mode' condition into the message log, because = this kernel version is supported if user wants to use via 'iova_mode =3D=3D 0' c= ondition. ack > + return -EINVAL; > +#else > + pci =3D pci_get_device(dev_info.vendor_id, > + dev_info.device_id, NULL); > + while (pci) { > + if ((pci->bus->number =3D=3D dev_info.bus) && > + (PCI_SLOT(pci->devfn) =3D=3D dev_info.devid) && > + (PCI_FUNC(pci->devfn) =3D=3D dev_info.function)= ) { > + domain =3D iommu_get_domain_for_dev(&pci->d= ev); > + break; > + } > + pci =3D pci_get_device(dev_info.vendor_id, > + dev_info.device_id, pci); > + } What if 'pci' is NULL here? In kni it is not required to provide a device at all. Ack, will add a NULL check. other point is not clear to me, device info is absolutely required at least for IOVA=3DVA mode, since it requires to procure iommu domain details. Any thoughts or ways to address this without device.? <...> > @@ -186,7 +202,10 @@ kni_fifo_trans_pa2va(struct kni_dev *kni, > return; > > for (i =3D 0; i < num_rx; i++) { > - kva =3D pa2kva(kni->pa[i]); > + if (likely(kni->iova_mode =3D=3D 1)) > + kva =3D iova2kva(kni, kni->pa[i]); > + else > + kva =3D pa2kva(kni->pa[i]); To reduce the churn, what about updating the 'pa2kva()' and put the "(kni->iova_mode =3D=3D 1)" check there? Does it help? (not only 'pa2kva()'= but its friends also, and if it makes more sense agree to rename the functions) No, in VA mode, kni->pa[i] points to iova address, pa2kva() of iova address= might crash, hence the if..else check is added. And btw, why 'likely' case is "kni->iova_mode =3D=3D 1"? no specific case other than branch predict, will remove this if it's really= harmful to PA mode.