From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by dpdk.org (Postfix) with ESMTP id D0AE92BA2 for ; Thu, 20 Apr 2017 11:04:26 +0200 (CEST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v3K941iq132780 for ; Thu, 20 Apr 2017 05:04:25 -0400 Received: from e38.co.us.ibm.com (e38.co.us.ibm.com [32.97.110.159]) by mx0a-001b2d01.pphosted.com with ESMTP id 29x7547tq0-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 20 Apr 2017 05:04:24 -0400 Received: from localhost by e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 20 Apr 2017 03:04:24 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e38.co.us.ibm.com (192.168.1.138) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 20 Apr 2017 03:04:22 -0600 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v3K94LPi10813744 for ; Thu, 20 Apr 2017 02:04:21 -0700 Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A8CB16E035 for ; Thu, 20 Apr 2017 03:04:21 -0600 (MDT) Received: from e39.co.us.ibm.com (unknown [9.17.249.49]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTPS id 91E406E04A for ; Thu, 20 Apr 2017 03:04:21 -0600 (MDT) Received: from localhost by e39.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 20 Apr 2017 03:04:21 -0600 Received: from smtp.notes.na.collabserv.com (192.155.248.75) by e39.co.us.ibm.com (192.168.2.139) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128/128) Thu, 20 Apr 2017 03:04:19 -0600 Received: from localhost by smtp.notes.na.collabserv.com with smtp.notes.na.collabserv.com ESMTP for from ; Thu, 20 Apr 2017 09:04:17 -0000 Received: from us1a3-smtp03.a3.dal06.isc4sb.com (10.106.154.98) by smtp.notes.na.collabserv.com (10.106.227.123) with smtp.notes.na.collabserv.com ESMTP; Thu, 20 Apr 2017 09:04:16 -0000 Received: from us1a3-mail173.a3.dal06.isc4sb.com ([10.146.71.126]) by us1a3-smtp03.a3.dal06.isc4sb.com with ESMTP id 2017042009041659-157740 ; Thu, 20 Apr 2017 09:04:16 +0000 MIME-Version: 1.0 In-Reply-To: <20170420072402.38106-6-aik@ozlabs.ru> To: Alexey Kardashevskiy Cc: dev@dpdk.org, Gowrishankar Muthukrishnan , "Adrian Schuepbach" From: "Jonas Pfefferle1" Date: Thu, 20 Apr 2017 11:04:37 +0200 References: <20170420072402.38106-1-aik@ozlabs.ru> <20170420072402.38106-6-aik@ozlabs.ru> X-KeepSent: 6279168F:35E9F404-C1258108:002AE7FE; type=4; name=$KeepSent X-Mailer: IBM Notes Release 9.0.1 October 14, 2013 X-LLNOutbound: False X-Disclaimed: 35207 X-TNEFEvaluated: 1 x-cbid: 17042009-0028-0000-0000-0000076FFEB4 X-IBM-SpamModules-Scores: BY=0.292235; FL=0; FP=0; FZ=0; HX=0; KW=0; PH=0; SC=0.433748; ST=0; TS=0; UL=0; ISC=; MB=0.038508 X-IBM-SpamModules-Versions: BY=3.00006943; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000208; SDB=6.00850004; UDB=6.00419777; IPR=6.00628634; BA=6.00005304; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015104; XFM=3.00000013; UTC=2017-04-20 09:04:17 X-IBM-AV-DETECTION: SAVI=unsuspicious REMOTE=unsuspicious XFE=unused X-IBM-AV-VERSION: SAVI=2017-04-20 08:30:14 - 6.00006624 x-cbparentid: 17042009-3816-0000-0000-00000F0AA3EA X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused X-TM-AS-GCONF: 00 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006943; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000208; SDB=6.00850004; UDB=6.00419776; IPR=6.00628634; BA=6.00005304; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015104; XFM=3.00000013; UTC=2017-04-20 09:04:23 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-20_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=18 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704200073 X-Mailman-Approved-At: Fri, 21 Apr 2017 03:50:44 +0200 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH dpdk 5/5] RFC: vfio/ppc64/spapr: Use correct bus addresses for DMA map X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Apr 2017 09:04:27 -0000 Alexey Kardashevskiy wrote on 20/04/2017 09:24:02: > From: Alexey Kardashevskiy > To: dev@dpdk.org > Cc: Alexey Kardashevskiy , JPF@zurich.ibm.com, > Gowrishankar Muthukrishnan > Date: 20/04/2017 09:24 > Subject: [PATCH dpdk 5/5] RFC: vfio/ppc64/spapr: Use correct bus > addresses for DMA map > > VFIO=5FIOMMU=5FSPAPR=5FTCE=5FCREATE ioctl() returns the actual bus addres= s for > just created DMA window. It happens to start from zero because the default > window is removed (leaving no windows) and new window starts from zero. > However this is not guaranteed and the new window may start from another > address, this adds an error check. > > Another issue is that IOVA passed to VFIO=5FIOMMU=5FMAP=5FDMA should be a= PCI > bus address while in this case a physical address of a user page is used. > This changes IOVA to start from zero in a hope that the rest of DPDK > expects this. This is not the case. DPDK expects a 1:1 mapping PA=3D=3DIOVA. It will use = the phys=5Faddr of the memory segment it got from /proc/self/pagemap cf. librte=5Feal/linuxapp/eal/eal=5Fmemory.c. We could try setting it here to t= he actual iova which basically makes the whole virtual to phyiscal mapping with pagemap unnecessary which I believe should be the case for VFIO anyway. Pagemap should only be needed when using pci=5Fuio. > > Signed-off-by: Alexey Kardashevskiy > --- > lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c | 12 ++++++++++-- > 1 file changed, 10 insertions(+), 2 deletions(-) > > diff --git a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c b/lib/ > librte=5Feal/linuxapp/eal/eal=5Fvfio.c > index 46f951f4d..8b8e75c4f 100644 > --- a/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > +++ b/lib/librte=5Feal/linuxapp/eal/eal=5Fvfio.c > @@ -658,7 +658,7 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > { > const struct rte=5Fmemseg *ms =3D rte=5Feal=5Fget=5Fphysmem=5Flayout(= ); > int i, ret; > - > + phys=5Faddr=5Ft io=5Foffset; > struct vfio=5Fiommu=5Fspapr=5Fregister=5Fmemory reg =3D { > .argsz =3D sizeof(reg), > .flags =3D 0 > @@ -702,6 +702,13 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > return -1; > } > > + io=5Foffset =3D create.start=5Faddr; > + if (io=5Foffset) { > + RTE=5FLOG(ERR, EAL, " DMA offsets other than zero is not supporte= d, " > + "new window is created at %lx\n", io=5Foffset); > + return -1; > + } > + > /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ > for (i =3D 0; i < RTE=5FMAX=5FMEMSEG; i++) { > struct vfio=5Fiommu=5Ftype1=5Fdma=5Fmap dma=5Fmap; > @@ -723,7 +730,7 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > dma=5Fmap.argsz =3D sizeof(struct vfio=5Fiommu=5Ftype1=5Fdma=5Fmap= ); > dma=5Fmap.vaddr =3D ms[i].addr=5F64; > dma=5Fmap.size =3D ms[i].len; > - dma=5Fmap.iova =3D ms[i].phys=5Faddr; > + dma=5Fmap.iova =3D io=5Foffset; > dma=5Fmap.flags =3D VFIO=5FDMA=5FMAP=5FFLAG=5FREAD | > VFIO=5FDMA=5FMAP=5FFLAG=5FWRITE; > > @@ -735,6 +742,7 @@ vfio=5Fspapr=5Fdma=5Fmap(int vfio=5Fcontainer=5Ffd) > return -1; > } > > + io=5Foffset +=3D dma=5Fmap.size; > } > > return 0; > -- > 2.11.0 >