From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by dpdk.org (Postfix) with ESMTP id C790F2BE9 for ; Tue, 8 Aug 2017 09:56:10 +0200 (CEST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v787sCro049869 for ; Tue, 8 Aug 2017 03:56:10 -0400 Received: from smtp.notes.na.collabserv.com (smtp.notes.na.collabserv.com [192.155.248.82]) by mx0b-001b2d01.pphosted.com with ESMTP id 2c734gp39n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 08 Aug 2017 03:56:09 -0400 Received: from localhost by smtp.notes.na.collabserv.com with smtp.notes.na.collabserv.com ESMTP for from ; Tue, 8 Aug 2017 07:56:09 -0000 Received: from us1a3-smtp03.a3.dal06.isc4sb.com (10.106.154.98) by smtp.notes.na.collabserv.com (10.106.227.105) with smtp.notes.na.collabserv.com ESMTP; Tue, 8 Aug 2017 07:56:06 -0000 Received: from us1a3-mail173.a3.dal06.isc4sb.com ([10.146.71.126]) by us1a3-smtp03.a3.dal06.isc4sb.com with ESMTP id 2017080807560574-199242 ; Tue, 8 Aug 2017 07:56:05 +0000 MIME-Version: 1.0 In-Reply-To: <33a0b391-0a97-0392-39c1-377af4196c69@ozlabs.ru> To: Alexey Kardashevskiy Cc: anatoly.burakov@intel.com, dev@dpdk.org From: "Jonas Pfefferle1" Date: Tue, 8 Aug 2017 09:56:05 +0200 References: <1502118665-27439-1-git-send-email-jpf@zurich.ibm.com> <33a0b391-0a97-0392-39c1-377af4196c69@ozlabs.ru> X-KeepSent: 5FE7D2C3:7A07FEF2-C1258176:002B5998; type=4; name=$KeepSent X-Mailer: IBM Notes Release 9.0.1 October 14, 2013 X-LLNOutbound: False X-Disclaimed: 36287 X-TNEFEvaluated: 1 x-cbid: 17080807-5101-0000-0000-00000607873D X-IBM-SpamModules-Scores: BY=0.292635; FL=0; FP=0; FZ=0; HX=0; KW=0; PH=0; SC=0.449102; ST=0; TS=0; UL=0; ISC=; MB=0.000004 X-IBM-SpamModules-Versions: BY=3.00007505; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000217; SDB=6.00899232; UDB=6.00450075; IPR=6.00679437; BA=6.00005518; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016587; XFM=3.00000015; UTC=2017-08-08 07:56:07 X-IBM-AV-DETECTION: SAVI=unsuspicious REMOTE=unsuspicious XFE=unused X-IBM-AV-VERSION: SAVI=2017-08-08 07:30:16 - 6.00007147 x-cbparentid: 17080807-5102-0000-0000-0000A79FFFEF Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-08_03:, , signatures=0 X-Proofpoint-Spam-Reason: safe Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] vfio: fix sPAPR IOMMU DMA window size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 07:56:11 -0000 Alexey Kardashevskiy wrote on 08/08/2017 09:38:00 AM: > From: Alexey Kardashevskiy > To: Jonas Pfefferle , anatoly.burakov@intel.com > Cc: dev@dpdk.org > Date: 08/08/2017 09:38 AM > Subject: Re: [PATCH] vfio: fix sPAPR IOMMU DMA window size > > On 08/08/17 01:11, Jonas Pfefferle wrote: > > DMA window size needs to be big enough to span all memory segment's > > physical addresses. We do not need multiple levels of IOMMU tables > > as we already span ~70TB of physical memory with 16MB hugepages. > > > > Signed-off-by: Jonas Pfefferle > > --- > > lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c | 25 ++++++++++++++++++++++= --- > > 1 file changed, 22 insertions(+), 3 deletions(-) > > > > diff --git a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c b/lib/ > librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > index 946df7e..8502216 100644 > > --- a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > +++ b/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > @@ -722,6 +722,18 @@ vfio=5Ftype1=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > > return 0; > > } > > > > +static uint64=5Ft > > +roundup=5Fnext=5Fpow2(uint64=5Ft n) > > +{ > > + uint32=5Ft i; > > + > > + n--; > > + for (i =3D 1; i < sizeof(n) * CHAR=5FBIT; i +=3D i) > > + n |=3D n >> i; > > + > > + return ++n; > > +} > > + > > wow :) > > QEMU does it using =5F=5Fbuiltin=5Fctzll() (used below for the page=5Fshi= ft) > without a loop: > > https://git.qemu.org/gitweb.cgi?p=3Dqemu.git;a=3Dblob;f=3Dinclude/qemu/ > host- > utils.h;h=3D95cf4f4163e50457cdf808263065ca5ef3f935da;hb=3Df22ab6cb0c47bd2a2= 785b7d58130949bd7d8d9af#l382 > > > Anyway, seems working. Ok let me fix that :) > > > Reviewed-by: Alexey Kardashevskiy > > > > > > static int > > vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > > { > > @@ -759,10 +771,12 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ff= d) > > return -1; > > } > > > > - /* calculate window size based on number of hugepages configured */ > > - create.window=5Fsize =3D rte=5Feal=5Fget=5Fphysmem=5Fsize(); > > + /* physicaly pages are sorted descending i.e. ms[0].phys=5Faddr is max */ > > + /* create DMA window from 0 to max(phys=5Faddr + len) */ > > + /* sPAPR requires window size to be a power of 2 */ > > + create.window=5Fsize =3D roundup=5Fnext=5Fpow2(ms[0].phys=5Faddr + = ms [0].len); > > create.page=5Fshift =3D =5F=5Fbuiltin=5Fctzll(ms->hugepage=5Fsz); > > - create.levels =3D 2; > > + create.levels =3D 1; > > > > ret =3D ioctl(vfio=5Fcontainer=5Ffd, VFIO=5FIOMMU=5FSPAPR=5FTCE=5FC= REATE, &create); > > if (ret) { > > @@ -771,6 +785,11 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > > return -1; > > } > > > > + if (create.start=5Faddr !=3D 0) { > > + RTE=5FLOG(ERR, EAL, " DMA window start address !=3D 0\n"); > > + return -1; > > + } > > + > > /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ > > for (i =3D 0; i < RTE=5FMAX=5FMEMSEG; i++) { > > struct vfio=5Fiommu=5Ftype1=5Fdma=5Fmap dma=5Fmap; > > > > > -- > Alexey >