From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 093E8A0613 for ; Mon, 5 Aug 2019 05:58:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 237341BE2B; Mon, 5 Aug 2019 05:58:33 +0200 (CEST) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by dpdk.org (Postfix) with ESMTP id 7A9251BE24 for ; Mon, 5 Aug 2019 05:58:31 +0200 (CEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x753wT5N076261 for ; Sun, 4 Aug 2019 23:58:30 -0400 Received: from smtp.notes.na.collabserv.com (smtp.notes.na.collabserv.com [192.155.248.81]) by mx0a-001b2d01.pphosted.com with ESMTP id 2u65r8w5gu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 04 Aug 2019 23:58:29 -0400 Received: from localhost by smtp.notes.na.collabserv.com with smtp.notes.na.collabserv.com ESMTP for from ; Mon, 5 Aug 2019 03:57:40 -0000 Received: from us1a3-smtp03.a3.dal06.isc4sb.com (10.106.154.98) by smtp.notes.na.collabserv.com (10.106.227.88) with smtp.notes.na.collabserv.com ESMTP; Mon, 5 Aug 2019 03:57:36 -0000 Received: from us1a3-mail157.a3.dal06.isc4sb.com ([10.146.71.179]) by us1a3-smtp03.a3.dal06.isc4sb.com with ESMTP id 2019080503573551-24006 ; Mon, 5 Aug 2019 03:57:35 +0000 In-Reply-To: <1564740872-27225-1-git-send-email-david.marchand@redhat.com> From: "Takeshi T Yoshimura" To: David Marchand Cc: dev@dpdk.org, anatoly.burakov@intel.com, drc@linux.vnet.ibm.com Date: Mon, 5 Aug 2019 03:57:36 +0000 MIME-Version: 1.0 Sensitivity: Importance: Normal X-Priority: 3 (Normal) References: <1564740872-27225-1-git-send-email-david.marchand@redhat.com> X-Mailer: IBM iNotes ($HaikuForm 1054) | IBM Domino Build SCN1812108_20180501T0841_FP55 May 22, 2019 at 11:09 X-LLNOutbound: False X-Disclaimed: 51515 X-TNEFEvaluated: 1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 x-cbid: 19080503-3067-0000-0000-00000040B302 X-IBM-SpamModules-Scores: BY=0; FL=0; FP=0; FZ=0; HX=0; KW=0; PH=0; SC=0.40962; ST=0; TS=0; UL=0; ISC=; MB=0.000004 X-IBM-SpamModules-Versions: BY=3.00011553; HX=3.00000242; KW=3.00000007; PH=3.00000004; SC=3.00000287; SDB=6.01242248; UDB=6.00655222; IPR=6.01023705; MB=3.00028044; MTD=3.00000008; XFM=3.00000015; UTC=2019-08-05 03:57:40 X-IBM-AV-DETECTION: SAVI=unsuspicious REMOTE=unsuspicious XFE=unused X-IBM-AV-VERSION: SAVI=2019-08-05 01:40:52 - 6.00010249 x-cbparentid: 19080503-3068-0000-0000-00000062BB89 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-08-05_02:, , signatures=0 X-Proofpoint-Spam-Reason: safe Subject: Re: [dpdk-dev] [PATCH] bus/pci: always check IOMMU capabilities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" -----David Marchand wrote: ----- >To: dev@dpdk.org >From: David Marchand >Date: 08/02/2019 07:14PM >Cc: anatoly.burakov@intel.com, tyos@jp.ibm.com, >drc@linux.vnet.ibm.com >Subject: [PATCH] bus/pci: always check IOMMU capabilities > >IOMMU capabilities won't change and must be checked even if no PCI >device >seem to be supported yet when EAL initialised. > >This is to accommodate with SPDK that registers its drivers after >rte=5Feal=5Finit(), especially on PPC platform where the IOMMU does not >support VA. > >Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA >mode") > >Signed-off-by: David Marchand >--- > drivers/bus/pci/bsd/pci.c | 6 ++++++ > drivers/bus/pci/linux/pci.c | 25 ++++++------------------- > drivers/bus/pci/pci=5Fcommon.c | 16 +++++++++++++++- > drivers/bus/pci/private.h | 5 ++++- > 4 files changed, 31 insertions(+), 21 deletions(-) > >diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c >index a2de709..8f07ed9 100644 >--- a/drivers/bus/pci/bsd/pci.c >+++ b/drivers/bus/pci/bsd/pci.c >@@ -376,6 +376,12 @@ error: > return -1; > } >=20 >+bool >+pci=5Fdevice=5Fiommu=5Fsupport=5Fva(=5F=5Frte=5Funused const struct rte= =5Fpci=5Fdevice >*dev) >+{ >+ return false; >+} >+ > enum rte=5Fiova=5Fmode > pci=5Fdevice=5Fiova=5Fmode(const struct rte=5Fpci=5Fdriver *pdrv =5F=5Frt= e=5Funused, > const struct rte=5Fpci=5Fdevice *pdev) >diff --git a/drivers/bus/pci/linux/pci.c >b/drivers/bus/pci/linux/pci.c >index f4fb742..43debaa 100644 >--- a/drivers/bus/pci/linux/pci.c >+++ b/drivers/bus/pci/linux/pci.c >@@ -498,8 +498,8 @@ error: > } >=20 > #if defined(RTE=5FARCH=5FX86) >-static bool >-pci=5Fone=5Fdevice=5Fiommu=5Fsupport=5Fva(const struct rte=5Fpci=5Fdevice= *dev) >+bool >+pci=5Fdevice=5Fiommu=5Fsupport=5Fva(const struct rte=5Fpci=5Fdevice *dev) > { > #define VTD=5FCAP=5FMGAW=5FSHIFT 16 > #define VTD=5FCAP=5FMGAW=5FMASK (0x3fULL << VTD=5FCAP=5FMGAW=5FSHIFT) >@@ -546,14 +546,14 @@ pci=5Fone=5Fdevice=5Fiommu=5Fsupport=5Fva(const stru= ct >rte=5Fpci=5Fdevice *dev) > return true; > } > #elif defined(RTE=5FARCH=5FPPC=5F64) >-static bool >-pci=5Fone=5Fdevice=5Fiommu=5Fsupport=5Fva(=5F=5Frte=5Funused const struct >rte=5Fpci=5Fdevice *dev) >+bool >+pci=5Fdevice=5Fiommu=5Fsupport=5Fva(=5F=5Frte=5Funused const struct rte= =5Fpci=5Fdevice >*dev) > { > return false; > } > #else >-static bool >-pci=5Fone=5Fdevice=5Fiommu=5Fsupport=5Fva(=5F=5Frte=5Funused const struct >rte=5Fpci=5Fdevice *dev) >+bool >+pci=5Fdevice=5Fiommu=5Fsupport=5Fva(=5F=5Frte=5Funused const struct rte= =5Fpci=5Fdevice >*dev) > { > return true; > } >@@ -564,7 +564,6 @@ pci=5Fdevice=5Fiova=5Fmode(const struct rte=5Fpci=5Fdr= iver >*pdrv, > const struct rte=5Fpci=5Fdevice *pdev) > { > enum rte=5Fiova=5Fmode iova=5Fmode =3D RTE=5FIOVA=5FDC; >- static int iommu=5Fno=5Fva =3D -1; >=20 > switch (pdev->kdrv) { > case RTE=5FKDRV=5FVFIO: { >@@ -595,18 +594,6 @@ pci=5Fdevice=5Fiova=5Fmode(const struct rte=5Fpci=5Fd= river >*pdrv, > iova=5Fmode =3D RTE=5FIOVA=5FVA; > break; > } >- >- if (iova=5Fmode !=3D RTE=5FIOVA=5FPA) { >- /* >- * We can check this only once, because the IOMMU hardware is >- * the same for all of them. >- */ >- if (iommu=5Fno=5Fva =3D=3D -1) >- iommu=5Fno=5Fva =3D pci=5Fone=5Fdevice=5Fiommu=5Fsupport=5Fva(pdev) >- ? 0 : 1; >- if (iommu=5Fno=5Fva !=3D 0) >- iova=5Fmode =3D RTE=5FIOVA=5FPA; >- } > return iova=5Fmode; > } >=20 >diff --git a/drivers/bus/pci/pci=5Fcommon.c >b/drivers/bus/pci/pci=5Fcommon.c >index 9794552..8d1d6ab 100644 >--- a/drivers/bus/pci/pci=5Fcommon.c >+++ b/drivers/bus/pci/pci=5Fcommon.c >@@ -616,8 +616,16 @@ rte=5Fpci=5Fget=5Fiommu=5Fclass(void) > const struct rte=5Fpci=5Fdriver *drv; > bool devices=5Fwant=5Fva =3D false; > bool devices=5Fwant=5Fpa =3D false; >+ static int iommu=5Fno=5Fva =3D -1; >=20 > FOREACH=5FDEVICE=5FON=5FPCIBUS(dev) { >+ /* >+ * We can check this only once, because the IOMMU hardware is >+ * the same for all of them. >+ */ >+ if (iommu=5Fno=5Fva =3D=3D -1) >+ iommu=5Fno=5Fva =3D pci=5Fdevice=5Fiommu=5Fsupport=5Fva(dev) >+ ? 0 : 1; > if (pci=5Fignore=5Fdevice(dev)) > continue; > if (dev->kdrv =3D=3D RTE=5FKDRV=5FUNKNOWN || >@@ -643,7 +651,13 @@ rte=5Fpci=5Fget=5Fiommu=5Fclass(void) > devices=5Fwant=5Fva =3D true; > } > } >- if (devices=5Fwant=5Fva && !devices=5Fwant=5Fpa) { >+ if (iommu=5Fno=5Fva =3D=3D 1) { >+ iova=5Fmode =3D RTE=5FIOVA=5FPA; >+ if (devices=5Fwant=5Fva) { >+ RTE=5FLOG(WARNING, EAL, "Some devices want 'VA' but because IOMMU >does not support 'VA'.\n"); >+ RTE=5FLOG(WARNING, EAL, "The devices that want 'VA' won't >initialize.\n"); >+ } >+ } else if (devices=5Fwant=5Fva && !devices=5Fwant=5Fpa) { > iova=5Fmode =3D RTE=5FIOVA=5FVA; > } else if (devices=5Fwant=5Fpa && !devices=5Fwant=5Fva) { > iova=5Fmode =3D RTE=5FIOVA=5FPA; >diff --git a/drivers/bus/pci/private.h b/drivers/bus/pci/private.h >index 8a55240..a205d4d 100644 >--- a/drivers/bus/pci/private.h >+++ b/drivers/bus/pci/private.h >@@ -173,9 +173,12 @@ rte=5Fpci=5Fmatch(const struct rte=5Fpci=5Fdriver >*pci=5Fdrv, > const struct rte=5Fpci=5Fdevice *pci=5Fdev); >=20 > /** >- * OS specific callback for rte=5Fpci=5Fget=5Fiommu=5Fclass >+ * OS specific callbacks for rte=5Fpci=5Fget=5Fiommu=5Fclass > * > */ >+bool >+pci=5Fdevice=5Fiommu=5Fsupport=5Fva(const struct rte=5Fpci=5Fdevice *dev); >+ > enum rte=5Fiova=5Fmode > pci=5Fdevice=5Fiova=5Fmode(const struct rte=5Fpci=5Fdriver *pci=5Fdrv, > const struct rte=5Fpci=5Fdevice *pci=5Fdev); >--=20 >1.8.3.1 > > Tested-by: Takeshi Yoshimura