From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by dpdk.org (Postfix) with ESMTP id 2375A2BC7 for ; Tue, 8 Aug 2017 11:29:50 +0200 (CEST) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v789SiU6021732 for ; Tue, 8 Aug 2017 05:29:50 -0400 Received: from smtp.notes.na.collabserv.com (smtp.notes.na.collabserv.com [192.155.248.66]) by mx0b-001b2d01.pphosted.com with ESMTP id 2c7a6c1pcg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 08 Aug 2017 05:29:50 -0400 Received: from localhost by smtp.notes.na.collabserv.com with smtp.notes.na.collabserv.com ESMTP for from ; Tue, 8 Aug 2017 09:29:49 -0000 Received: from us1a3-smtp08.a3.dal06.isc4sb.com (10.146.103.57) by smtp.notes.na.collabserv.com (10.106.227.127) with smtp.notes.na.collabserv.com ESMTP; Tue, 8 Aug 2017 09:29:46 -0000 Received: from us1a3-mail173.a3.dal06.isc4sb.com ([10.146.71.126]) by us1a3-smtp08.a3.dal06.isc4sb.com with ESMTP id 2017080809294651-245208 ; Tue, 8 Aug 2017 09:29:46 +0000 MIME-Version: 1.0 In-Reply-To: To: "Burakov, Anatoly" Cc: "aik@ozlabs.ru" , "dev@dpdk.org" From: "Jonas Pfefferle1" Date: Tue, 8 Aug 2017 11:29:45 +0200 References: <1502181667-17949-1-git-send-email-jpf@zurich.ibm.com> X-KeepSent: 030B56D6:394D9B6F-C1258176:00335A94; type=4; name=$KeepSent X-Mailer: IBM Notes Release 9.0.1 October 14, 2013 X-LLNOutbound: False X-Disclaimed: 15247 X-TNEFEvaluated: 1 x-cbid: 17080809-6357-0000-0000-000002A46551 X-IBM-SpamModules-Scores: BY=0; FL=0; FP=0; FZ=0; HX=0; KW=0; PH=0; SC=0.38988; ST=0; TS=0; UL=0; ISC=; MB=0.000040 X-IBM-SpamModules-Versions: BY=3.00007506; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000217; SDB=6.00899263; UDB=6.00450094; IPR=6.00679468; BA=6.00005518; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016589; XFM=3.00000015; UTC=2017-08-08 09:29:47 X-IBM-AV-DETECTION: SAVI=unsuspicious REMOTE=unsuspicious XFE=unused X-IBM-AV-VERSION: SAVI=2017-08-08 07:27:34 - 6.00007147 x-cbparentid: 17080809-6358-0000-0000-0000D606A79A Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-08_04:, , signatures=0 X-Proofpoint-Spam-Reason: safe Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v3] vfio: fix sPAPR IOMMU DMA window size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 09:29:51 -0000 "Burakov, Anatoly" wrote on 08/08/2017 11:15:24 AM: > From: "Burakov, Anatoly" > To: Jonas Pfefferle > Cc: "dev@dpdk.org" , "aik@ozlabs.ru" > Date: 08/08/2017 11:18 AM > Subject: RE: [PATCH v3] vfio: fix sPAPR IOMMU DMA window size > > From: Jonas Pfefferle [mailto:jpf@zurich.ibm.com] > > Sent: Tuesday, August 8, 2017 9:41 AM > > To: Burakov, Anatoly > > Cc: dev@dpdk.org; aik@ozlabs.ru; Jonas Pfefferle > > Subject: [PATCH v3] vfio: fix sPAPR IOMMU DMA window size > > > > DMA window size needs to be big enough to span all memory segment's > > physical addresses. We do not need multiple levels of IOMMU tables > > as we already span ~70TB of physical memory with 16MB hugepages. > > > > Signed-off-by: Jonas Pfefferle > > --- > > v2: > > * roundup to next power 2 function without loop. > > > > v3: > > * Replace roundup=5Fnext=5Fpow2 with rte=5Falign64pow2 > > > > lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c | 13 ++++++++++--- > > 1 file changed, 10 insertions(+), 3 deletions(-) > > > > diff --git a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > b/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > index 946df7e..550c41c 100644 > > --- a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > +++ b/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > @@ -759,10 +759,12 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ff= d) > > return -1; > > } > > > > - /* calculate window size based on number of hugepages configured > > */ > > - create.window=5Fsize =3D rte=5Feal=5Fget=5Fphysmem=5Fsize(); > > + /* physicaly pages are sorted descending i.e. ms[0].phys=5Faddr is max > > */ > > Do we always expect that to be the case in the future? Maybe it > would be safer to walk the memsegs list. > > Thanks, > Anatoly I had this loop in before but removed it in favor of simplicity. If we believe that the ordering is going to change in the future I'm happy to bring back the loop. Is there other code which is relying on the fact that the memsegs are sorted by their physical addresses? > > > + /* create DMA window from 0 to max(phys=5Faddr + len) */ > > + /* sPAPR requires window size to be a power of 2 */ > > + create.window=5Fsize =3D rte=5Falign64pow2(ms[0].phys=5Faddr + > > ms[0].len); > > create.page=5Fshift =3D =5F=5Fbuiltin=5Fctzll(ms->hugepage=5Fsz); > > - create.levels =3D 2; > > + create.levels =3D 1; > > > > ret =3D ioctl(vfio=5Fcontainer=5Ffd, VFIO=5FIOMMU=5FSPAPR=5FTCE=5FC= REATE, > > &create); > > if (ret) { > > @@ -771,6 +773,11 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > > return -1; > > } > > > > + if (create.start=5Faddr !=3D 0) { > > + RTE=5FLOG(ERR, EAL, " DMA window start address !=3D 0\n"); > > + return -1; > > + } > > + > > /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ > > for (i =3D 0; i < RTE=5FMAX=5FMEMSEG; i++) { > > struct vfio=5Fiommu=5Ftype1=5Fdma=5Fmap dma=5Fmap; > > -- > > 2.7.4 >