From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id D26E78E98 for ; Tue, 29 Dec 2015 15:04:39 +0100 (CET) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 1EC2C7AE8B; Tue, 29 Dec 2015 14:04:39 +0000 (UTC) Received: from t450s.home ([10.3.113.4]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id tBTE4cw5018863; Tue, 29 Dec 2015 09:04:38 -0500 Message-ID: <1451397878.18084.4.camel@redhat.com> From: Alex Williamson To: Santosh Shukla , "Burakov, Anatoly" Date: Tue, 29 Dec 2015 07:04:38 -0700 In-Reply-To: References: <2241331.HNmyzf8foi@xps13> <2979402.yeVYlcCDUH@xps13> <20151218053053.GL29571@yliu-dev.sh.intel.com> <20151218082139.GC18863@yliu-dev.sh.intel.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH] eal: map io resources for non x86 architectures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Dec 2015 14:04:40 -0000 On Tue, 2015-12-29 at 16:17 +0530, Santosh Shukla wrote: > On Tue, Dec 29, 2015 at 3:26 PM, Burakov, Anatoly > wrote: > > Hi Santosh, > > > > > On Fri, Dec 18, 2015 at 6:25 PM, Santosh Shukla > > om> > > > wrote: > > > > On Fri, Dec 18, 2015 at 1:51 PM, Yuanhan Liu > > > > wrote: > > > > > On Fri, Dec 18, 2015 at 01:24:41PM +0530, Santosh Shukla > > > > > wrote: > > > > > > > > I guess we have done enough evaluation / investigation > > > > > > > > that > > > > > > > > suggest - so to map iopci region to userspace in arch > > > > > > > > agnostic-way - > > > > > > > > > > > > > > > > # either we need to modify kernel > > > > > > > >                - Make sure all the non-x86 arch to > > > > > > > > support > > > > > > > > mapping for iopci region (i.e. pci_mmap_page_range). I > > > > > > > > don;t > > > > > > > > think its a correct approach though. > > > > > > > >             or > > > > > > > >                - include /dev/ioport char-mem device > > > > > > > > file who > > > > > > > > could do more than byte operation, Note that this > > > > > > > > implementation > > > > > > > > does not exist in kernel.  I could send an RFC to lkml. > > > > > > > > > > > > > > Maybe you could propose the two to lkml, to get some > > > > > > > feedbacks > > > > > > > from those kernel/ARM gurus? Please cc me if you do so. > > > > > > > > > > > > > > > > > > > The latter one I already shared old lkml thread, Pl. > > > > > > revisit my v1 > > > > > > 0/6 patch [1] and in that refer [2]. > > > > > > > > > > Oops, sorry, a bit busy, that I didn't look at it carefully. > > > > > My bad, > > > > > anyway. > > > > > > > > > > > Josh has already proposed to lkml but for some reason > > > > > > thread didn't > > > > > > went far. I can restart that discussion giving dpdk use- > > > > > > case as an > > > > > > example/ requirement. > > > > > > > > > > I had a quick go through of the discussion. Both hpa and Arnd > > > > > seem to > > > > > be fine with the ioctl interface on /dev/port. Have you tried > > > > > that? > > > > > And if you want to restart it, ioctl might be a better option > > > > > than > > > > > /dev/ioport, judging from the discussion. > > > > > > > > > > > > > I tried legacy patch and re-writing with ioctl-way; doing > > > > changes in > > > > dpdk port as well in kernel, had to test on quite other hw not > > > > only > > > > arm64 though! so it will take time for me, I am travelling > > > > tomorrow so > > > > bit delayed, We'll post patch to lkml and share dpdk-virtio > > > > feedback > > > > perhaps by Monday. > > > > > > > > > > I posted a query about /dev/ioports approach in lkml thread [1], > > > and Arnd > > > suggested to use vfio framework but it looks like vfio too does > > > not map > > > ioresource_io region. Same communicated to Arnd and waiting for > > > his reply. > > > > > > In mean time I like to ask general question; > > > - Has anyone tried vfio/non-iommu approach for virtio pmd driver? > > > If not > > > then Is there any plan? Someone planning to take up. > > > [1] https://lkml.org/lkml/2015/12/23/145 > > > > I have submitted a patch to support no-iommu mode, but I'm not > > aware of anyone trying VFIO-noiommu at all. That's probably > > expected since it's Christmas/New Year in a lot of places, and > > everyone is on a break. > > > > That said, I'm not sure I completely understand what is it that > > you're asking about. The code you're referring to (in vfio_pci.c, > > line 854 as of kernel 4.3) is checking if a PCI BAR is available > > for IO (hence checking if the IORESOURCE_MEM > > > Thanks for reply! You comment might help to move this discuss to next > level. > > Look at kernel/resource.c, it exports two symbol ioport_resource and > iomem_resource and sets appropriate flag type i.e.. IORESOURCE_IO and > IORESOURCE_MEM. In virtio-net case; it creates both pci region i.e.. > _io bar and _mem bar. dpdk virtio pmd driver (<= 0.95 virtio spec) > uses pci _io bar region for device initialization as virtio headers > are locate at pci _io bar region. Since it uses pci _iobar region so > likely it update pci_resource.[index].flag = IORESOURCE_IO.  and vfio > mmap function wont handle ioresource_io (i guess). And that is why I > asked same to lkml thread. > > > bit is set). There isn't any "ioresource_mem" region as far as VFIO > is > concerned, there are only BARs, ROM, VGA and PCI > config regions (linux/vfio.h, line 314 as of kernel 4.3). So if > you're > missing some PCI regions for VFIO to map, they would first need to be > added to VFIO kernel implementation before they can be used by DPDK. > That is, unless I'm misunderstanding something :) > > > > Agree. As mentioned above, I guess ioresource_io pci bar > implementation missing in vfio kernel? but before adding code support > in kernel I would like to hear from experts example, You, Alex! > (looping alex) MMIO and I/O port space are simply regions as far as VFIO is concerned.  The userspace API supports the concept of I/O port regions that can be mmap'd by userspace, but the implementation does not since I/O port regions cannot be mmap'd on x86.  Someone needs to propose some code for vfio-pci to support it.  Thanks, Alex