From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by dpdk.org (Postfix) with ESMTP id 84BBF14E8 for ; Tue, 8 Aug 2017 10:47:19 +0200 (CEST) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v788itGf055911 for ; Tue, 8 Aug 2017 04:47:19 -0400 Received: from smtp.notes.na.collabserv.com (smtp.notes.na.collabserv.com [192.155.248.82]) by mx0b-001b2d01.pphosted.com with ESMTP id 2c728rhvyf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 08 Aug 2017 04:47:18 -0400 Received: from localhost by smtp.notes.na.collabserv.com with smtp.notes.na.collabserv.com ESMTP for from ; Tue, 8 Aug 2017 08:47:17 -0000 Received: from us1a3-smtp03.a3.dal06.isc4sb.com (10.106.154.98) by smtp.notes.na.collabserv.com (10.106.227.105) with smtp.notes.na.collabserv.com ESMTP; Tue, 8 Aug 2017 08:47:14 -0000 Received: from us1a3-mail173.a3.dal06.isc4sb.com ([10.146.71.126]) by us1a3-smtp03.a3.dal06.isc4sb.com with ESMTP id 2017080808471305-246972 ; Tue, 8 Aug 2017 08:47:13 +0000 MIME-Version: 1.0 In-Reply-To: <2601191342CEEE43887BDE71AB9772584888BE1D@IRSMSX103.ger.corp.intel.com> To: "Ananyev, Konstantin" Cc: Alexey Kardashevskiy , "Burakov, Anatoly" , "dev@dpdk.org" From: "Jonas Pfefferle1" Date: Tue, 8 Aug 2017 10:47:12 +0200 References: <1502118665-27439-1-git-send-email-jpf@zurich.ibm.com> <33a0b391-0a97-0392-39c1-377af4196c69@ozlabs.ru> <2601191342CEEE43887BDE71AB9772584888BE1D@IRSMSX103.ger.corp.intel.com> X-KeepSent: EFA5A823:06730AE0-C1258176:0030382F; type=4; name=$KeepSent X-Mailer: IBM Notes Release 9.0.1 October 14, 2013 X-LLNOutbound: False X-Disclaimed: 40279 X-TNEFEvaluated: 1 x-cbid: 17080808-5101-0000-0000-000006079883 X-IBM-SpamModules-Scores: BY=0.292635; FL=0; FP=0; FZ=0; HX=0; KW=0; PH=0; SC=0.410717; ST=0; TS=0; UL=0; ISC=; MB=0.000000 X-IBM-SpamModules-Versions: BY=3.00007506; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000217; SDB=6.00899249; UDB=6.00450085; IPR=6.00679454; BA=6.00005518; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016589; XFM=3.00000015; UTC=2017-08-08 08:47:16 X-IBM-AV-DETECTION: SAVI=unsuspicious REMOTE=unsuspicious XFE=unused X-IBM-AV-VERSION: SAVI=2017-08-08 07:30:16 - 6.00007147 x-cbparentid: 17080808-5102-0000-0000-0000A7A01D65 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-08_04:, , signatures=0 X-Proofpoint-Spam-Reason: safe Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] vfio: fix sPAPR IOMMU DMA window size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 08:47:19 -0000 "Ananyev, Konstantin" wrote on 08/08/2017 10:27:28 AM: > From: "Ananyev, Konstantin" > To: Alexey Kardashevskiy , Jonas Pfefferle > , "Burakov, Anatoly" > Cc: "dev@dpdk.org" > Date: 08/08/2017 10:27 AM > Subject: RE: [dpdk-dev] [PATCH] vfio: fix sPAPR IOMMU DMA window size > > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alexey Kardashevskiy > > Sent: Tuesday, August 8, 2017 10:38 AM > > To: Jonas Pfefferle ; Burakov, Anatoly > > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH] vfio: fix sPAPR IOMMU DMA window size > > > > On 08/08/17 01:11, Jonas Pfefferle wrote: > > > DMA window size needs to be big enough to span all memory segment's > > > physical addresses. We do not need multiple levels of IOMMU tables > > > as we already span ~70TB of physical memory with 16MB hugepages. > > > > > > Signed-off-by: Jonas Pfefferle > > > --- > > > lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c | 25 +++++++++++++++++++++ +--- > > > 1 file changed, 22 insertions(+), 3 deletions(-) > > > > > > diff --git a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c b/lib/ > librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > > index 946df7e..8502216 100644 > > > --- a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > > +++ b/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > > > @@ -722,6 +722,18 @@ vfio=5Ftype1=5Fdma=5Fmap(int vfio=5Fcontainer=5F= fd) > > > return 0; > > > } > > > > > > +static uint64=5Ft > > > +roundup=5Fnext=5Fpow2(uint64=5Ft n) > > > +{ > > > + uint32=5Ft i; > > > + > > > + n--; > > > + for (i =3D 1; i < sizeof(n) * CHAR=5FBIT; i +=3D i) > > > + n |=3D n >> i; > > > + > > > + return ++n; > > > +} > > > + > > > > wow :) > > > > QEMU does it using =5F=5Fbuiltin=5Fctzll() (used below for the page=5Fs= hift) > > without a loop: > > > > https://git.qemu.org/gitweb.cgi?p=3Dqemu.git;a=3Dblob;f=3Dinclude/qemu/= host- > > > utils.h;h=3D95cf4f4163e50457cdf808263065ca5ef3f935da;hb=3Df22ab6cb0c47bd2a2= 785b7d58130949bd7d8d9af#l382 > > > > > > Anyway, seems working. > > As I remember, there already exists rte=5Falign64pow2(). > Konstantin Thanks. Fixed it. > > > > > > > Reviewed-by: Alexey Kardashevskiy > > > > > > > > > > > static int > > > vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > > > { > > > @@ -759,10 +771,12 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer= =5Ffd) > > > return -1; > > > } > > > > > > - /* calculate window size based on number of hugepages configured */ > > > - create.window=5Fsize =3D rte=5Feal=5Fget=5Fphysmem=5Fsize(); > > > + /* physicaly pages are sorted descending i.e. ms > [0].phys=5Faddr is max */ > > > + /* create DMA window from 0 to max(phys=5Faddr + len) */ > > > + /* sPAPR requires window size to be a power of 2 */ > > > + create.window=5Fsize =3D roundup=5Fnext=5Fpow2(ms[0].phys=5Faddr = + ms [0].len); > > > create.page=5Fshift =3D =5F=5Fbuiltin=5Fctzll(ms->hugepage=5Fsz); > > > - create.levels =3D 2; > > > + create.levels =3D 1; > > > > > > ret =3D ioctl(vfio=5Fcontainer=5Ffd, VFIO=5FIOMMU=5FSPAPR=5FTCE= =5FCREATE, &create); > > > if (ret) { > > > @@ -771,6 +785,11 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5F= fd) > > > return -1; > > > } > > > > > > + if (create.start=5Faddr !=3D 0) { > > > + RTE=5FLOG(ERR, EAL, " DMA window start address !=3D 0\n"); > > > + return -1; > > > + } > > > + > > > /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ > > > for (i =3D 0; i < RTE=5FMAX=5FMEMSEG; i++) { > > > struct vfio=5Fiommu=5Ftype1=5Fdma=5Fmap dma=5Fmap; > > > > > > > > > -- > > Alexey