From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 92A82A0471 for ; Mon, 15 Jul 2019 17:36:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26E9B2C18; Mon, 15 Jul 2019 17:36:04 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 304A82BA8 for ; Mon, 15 Jul 2019 17:36:03 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6FFUPHK027455; Mon, 15 Jul 2019 08:35:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=eMHlLICRmCWRQNNT4SPEpCDiIauMtW3qf5GMu0qQBBI=; b=CSMUJwV441dT0mILHs4rCTQFUn8UA7XbSOgrbS5na5jz04g6k8R4K5Eue+CNRUNiW/AT RMbasWvhK0TFzUpm9w2CX/yAVQaoNqNFtyR80bBVFjQG04ynIoBIELvTv4EQIFuceTuP 9j/vlO4RV3pW7iN5U3Ib2NW+8zAFta8VlbcOVVzoaw8/WrLBux56yfOjZKvQTArFMcnk Cl0lHMr/YQ5r+KcUgZj2e6sozYZifwvqkN2X//J+dpTYN9StCWu6RwRMZerOozooZr3C EbKBPl67V2UMtjJjiYjagG0L8Q9uu9413vf5E4WUrVyrIBIS8ZEIC5tjg8NVcc6Lotsv 6w== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2tqetmq7wt-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 15 Jul 2019 08:35:58 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 15 Jul 2019 08:35:56 -0700 Received: from NAM04-CO1-obe.outbound.protection.outlook.com (104.47.45.52) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Mon, 15 Jul 2019 08:35:56 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Nb/pm2mGZzdtCPQO/5nB0ZB8V0K1qKKR/mIe5OzTYpipPpjD0ESeniuenS7tBea/Nia6RJPKFh8DHOtF2r/DeCzfzxKrSFeWGjXONW2ptKO/2cyTowbtnIT7Idu4oVa1wOMXERxbM35hP5tUALs0/WEuV7Xo2GCgPqq8fc1RGM1QzxHsX+dHq6xoLAGAn+sojCHAeVK0JZWAUWlVnhwzvjYV6pH/MbdpjolQNR3KchlirdQwj6RYEh6BDV11pGRYpw/8J6yT8eUgXoILTSg5gz/7P2nVmXav6pNuxHajdusecjulwxL3ywTOoTezs30RX6UPf8EVa02dIWJ3SiI0IQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eMHlLICRmCWRQNNT4SPEpCDiIauMtW3qf5GMu0qQBBI=; b=RbUoD4Qo0fCZ7Lnk72VyYuJdTu+0GmFqGAZwCcU+Ewopgi+bf3eCQ3xvdLO4byfGYZMnEvyQY9BAS7fCaL3x61WvToTbv9FrPju8abvhvbXTZrtFVngo34d06nTGS+sKyQNR6FbBCLY1buXwavQXhmCgZ0pFXqKFD1ayMO56EwH13uChsX+WskhtyFSUVpgQRwyT7q/cqj2MAx/mo/SV+tC3TsUoN2WF1FvJp8bAsETv/2rGtp+XKjyQ6y/l6oPisjD8VOi+hg9ZhNqrRRipfKZeRWhaCvVSPHC7j+axWhKdwgrB+rXcs/qfDhKVsfQYDB+ot3r7n6GUQArmzn9KNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=marvell.com;dmarc=pass action=none header.from=marvell.com;dkim=pass header.d=marvell.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eMHlLICRmCWRQNNT4SPEpCDiIauMtW3qf5GMu0qQBBI=; b=mMVHXGkHww25D8OEFQN68WDuUh6197hiHb1qF7odH29INdb+56AbmGrV54Xxuf/2dHbDn/xWShlqyUB7HSQWET0+POFUdZ2V2i2UylJvLvFBCbLCfOvD800cYBliaMNuLS4b+gvgOvEexUTBbRc7wI6x1Ns+qV/86jOo4cWezxQ= Received: from BYAPR18MB2424.namprd18.prod.outlook.com (20.179.91.149) by BYAPR18MB2792.namprd18.prod.outlook.com (20.179.57.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.14; Mon, 15 Jul 2019 15:35:50 +0000 Received: from BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::2d42:12b6:aa2e:2862]) by BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::2d42:12b6:aa2e:2862%4]) with mapi id 15.20.2073.012; Mon, 15 Jul 2019 15:35:50 +0000 From: Jerin Jacob Kollanukkaran To: Thomas Monjalon CC: "Burakov, Anatoly" , David Marchand , "dev@dpdk.org" , John McNamara , Marko Kovacevic , "Igor Russkikh" , Pavel Belous , Ajit Khaparde , Somnath Kotur , Wenzhuo Lu , John Daley , Hyong Youb Kim , Qi Zhang , Xiao Wang , Beilei Xing , Jingjing Wu , Qiming Yang , "Konstantin Ananyev" , Matan Azrad , Shahaf Shuler , Yongseok Koh , Viacheslav Ovsiienko , Alejandro Lucero , Nithin Kumar Dabilpuram , Kiran Kumar Kokkilagadda , Rasesh Mody , Shahed Shaikh , Bruce Richardson , "alialnu@mellanox.com" , "aconole@redhat.com" Thread-Topic: [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers Thread-Index: AQHVN2lUvS4FC9UUg0eo3J1cfmDleKbG1AEAgAAcB4CABNF8gIAADJgAgAABjkA= Date: Mon, 15 Jul 2019 15:35:50 +0000 Message-ID: References: <1562795329-16652-1-git-send-email-david.marchand@redhat.com> <2927698.45TxNz31xh@xps> <71077296.xTBSyl0q4B@xps> In-Reply-To: <71077296.xTBSyl0q4B@xps> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [106.200.248.176] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: eb5fdd66-a2bd-4fb6-4ba8-08d7093a1dd7 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BYAPR18MB2792; x-ms-traffictypediagnostic: BYAPR18MB2792: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 00997889E7 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(199004)(43544003)(13464003)(52314003)(189003)(66946007)(476003)(66476007)(86362001)(66556008)(64756008)(66446008)(229853002)(446003)(71190400001)(11346002)(71200400001)(102836004)(26005)(52536014)(8936002)(53546011)(54906003)(81166006)(81156014)(76116006)(6506007)(4326008)(5660300002)(186003)(7696005)(14444005)(6916009)(256004)(99286004)(14454004)(76176011)(25786009)(68736007)(8676002)(561944003)(966005)(305945005)(74316002)(7416002)(498600001)(7736002)(55016002)(3846002)(9686003)(6116002)(53936002)(6306002)(33656002)(6246003)(486006)(66066001)(2906002)(6436002); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR18MB2792; H:BYAPR18MB2424.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: sbjjy30HAwlbccMXHdoCgAMAr/8V3EbNPGecxjCktQrGLX1xBnUpJgBCBmaxrbHE4iliCqIHP6FtEb+sOmiIw1XjYvGdOUXmyIzJ8bHsrXsmkrG5J7ZS8uMVVVTsfNP+FxWvTDoW/bIMurpHHiacTKHxALBajunt6zRVBjDd87WolsB2iHmo5eijfIuOwDreDihIPrCdUL4kRWQPisBSsQRu6De39a+kgFKIrjJ9ixPJXcfBtRf7g4pJ1nC1whaXAUG95jcCvuFmn3LBm/9oZtm/Xi8zZWZd8QJKpJG71o8zxvUPimB6Ud273/AmZ28Mp/gWEoa755ydmE8SmSey7eKGCLkEjq00oZ/BJXvkbKNKnj5cJ6C0Ie+H8BBsGeP7zhkLVMPBLw/xg55O4f6xNeCL9mMw/pzCUvJW2F0SzO4= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: eb5fdd66-a2bd-4fb6-4ba8-08d7093a1dd7 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jul 2019 15:35:50.5938 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jerinj@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR18MB2792 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-15_04:2019-07-15,2019-07-15 signatures=0 Subject: Re: [dpdk-dev] [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Thomas Monjalon > Sent: Monday, July 15, 2019 8:34 PM > To: Jerin Jacob Kollanukkaran > Cc: Burakov, Anatoly ; David Marchand > ; dev@dpdk.org; John McNamara > ; Marko Kovacevic > ; Igor Russkikh > ; Pavel Belous ; > Ajit Khaparde ; Somnath Kotur > ; Wenzhuo Lu ; > John Daley ; Hyong Youb Kim ; > Qi Zhang ; Xiao Wang ; > Beilei Xing ; Jingjing Wu ; > Qiming Yang ; Konstantin Ananyev > ; Matan Azrad ; > Shahaf Shuler ; Yongseok Koh > ; Viacheslav Ovsiienko > ; Alejandro Lucero > ; Nithin Kumar Dabilpuram > ; Kiran Kumar Kokkilagadda > ; Rasesh Mody ; Shahed > Shaikh ; Bruce Richardson > ; alialnu@mellanox.com; > aconole@redhat.com > Subject: Re: [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drive= rs >=20 > 15/07/2019 16:26, Jerin Jacob Kollanukkaran: > > > > > + > > > > > +IOVA Mode is selected by considering what the current usable > > > > > +Devices on the system requires and/or supports. > > > > > + > > > > > +Below is the 2-step heuristic for this choice. > > > > > + > > > > > +For the first step, EAL asks each bus its requirement in terms > > > > > +of IOVA mode and decides on a preferred IOVA mode. > > > > > + > > > > > +- if all buses report RTE_IOVA_PA, then the preferred IOVA mode > > > > > +is RTE_IOVA_PA, > > > > > +- if all buses report RTE_IOVA_VA, then the preferred IOVA mode > > > > > +is RTE_IOVA_VA, > > > > > +- if all buses report RTE_IOVA_DC, no bus expressed a > > > > > +preferrence, then the > > > > > + preferred mode is RTE_IOVA_DC, > > > > > +- if the buses disagree (at least one wants RTE_IOVA_PA and at > > > > > +least one wants > > > > > + RTE_IOVA_VA), then the preferred IOVA mode is RTE_IOVA_DC > > > > > +(see below with the > > > > > + check on Physical Addresses availability), > > > > > + > > > > > +The second step is checking if the preferred mode complies with > > > > > +the Physical Addresses availability since those are only > > > > > +available to root user in recent kernels. > > > > > + > > > > > +- if the preferred mode is RTE_IOVA_PA but there is no access > > > > > +to Physical > > > > > + Addresses, then EAL init will fail early, since later probing > > > > > +of the devices > > > > > + would fail anyway, > > > > > +- if the preferred mode is RTE_IOVA_DC then based on the > > > > > +Physical Addresses > > > > > + availability, the preferred mode is adjusted to RTE_IOVA_PA > > > > > +or > > > RTE_IOVA_VA. > > > > > + In the case when the buses had disagreed on the IOVA Mode at > > > > > +the first step, > > > > > + part of the buses won't work because of this decision. > > > > > > > > Is there any specific reason why we always prefer PA if physical > > > > addresses are available? Since we're already assuming that all > > > > devices support PA and VA anyway, what's the harm in enabling VA by > default? > > > > > > If PA is available, it means we are running as root. > > > We can assume that using root is a choice, probably related to a > > > preference for PA. > > > > # Even if we are running as root, Why to choose PA in case of DC? > > ie. Following logic is not need > > if (iova_mode =3D=3D RTE_IOVA_DC) { > > iova_mode =3D phys_addrs ? RTE_IOVA_PA : RTE_IO= VA_VA; > > RTE_LOG(DEBUG, EAL, > > "Buses did not request a specific IOVA = mode, using '%s' > based on physical addresses availability.\n", > > phys_addrs ? "PA" : "VA"); > > } >=20 > Why running as root if using VA anyway? > We can assume the user knows what he is doing, so it is a user choice. > We want to allow the user choosing, right? The user can override iova=3Dpa/va as eal argument if user needs to run a s= pecific mode. Running as root for various other reason(just be lazy) etc. it is not or it= should not be connected to set the mode as PA. >=20 > > # When DPDK running on guest, Anyway it can not access the real PA, It = will > be IPA. >=20 > What is IPA? Isn't it a beer? There may a beer with that name. In this context, it is "Intermediate physi= cal address" >=20 > > So I don't understand logic behind choose PA when DC. > > To me, it make sense to choose PA when DC. >=20 > You probably mean "choose VA". Yup. >=20 > > # To align with RTE_PCI_DRV_NEED_MAPPING flag and reflect it "need" > > rather than support, I think, flag can be changed to > > RTE_PCI_DRV_NEED_IOVA_AS_VA >=20 > I think the most important is to have a good documentation of this flag (= it > was not done properly when Cavium introduced it initially). > If you want to rename the flag, you can do it in a separate patch. > If renaming, I really would like to get an answer to an old question: > Why IO adress is called IOVA? The name "IOVA_AS_VA" looks strange. IOVA =3D IO virtual address Since IOVA can be PA or VA, the name IOVA_AS_VA as chosen=20 > For reference, one description of addressing: > https://lists.linuxfoundation.org/pipermail/iommu/2018-May/027686.html >=20 > About the naming, do you remember how I insisted to have a correct naming > of all related stuff in DPDK? It was hard to get it accepted, the discuss= ion was > not nice and I stopped insisting to get all details fine because I just g= ot bored. > It was a really bad experience. I agree. To me that bad experience was due to mostly not having enough technical com= ments On the proposal. Though I am not the author/owner of it. > You can ask why I remind this now? Because we must take care of all detai= ls, > make sure our messages are well understood, and be cooperative. No disagreement. If we see the history the meaning got changed/updated in this commit By adding intel drivers to it. I would nt say it is big ideal, It just C c= ode, It can be changed based on the need. I think, what really import is, maintain the the feature and commitment towards fixing any issue. commit f37dfab21c988d2d0ecb3c82be4ba9738c7e51c7 Author: Jianfeng Tan Date: Wed Oct 11 10:33:48 2017 +0000 drivers/net: enable IOVA mode for Intel PMDs If we want to enable IOVA mode, introduced by commit 93878cf0255e ("eal: introduce helper API for IOVA mode"), we need PMDs (for PCI devices) to expose this flag. Signed-off-by: Jianfeng Tan Acked-by: Anatoly Burakov Reviewed-by: Santosh Shukla >=20 > > Other than above points, > > Reviewed this patch and tested on octeontx2, It looks good to me. >=20 >=20