From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) by dpdk.org (Postfix) with ESMTP id D10FB8D9D for ; Tue, 29 Dec 2015 11:47:28 +0100 (CET) Received: by mail-pa0-f44.google.com with SMTP id cy9so122895604pac.0 for ; Tue, 29 Dec 2015 02:47:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mvista-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=J3Edru9e7lUdVoaTGZTK5sTrBdxVxdyPB1Z72ic00xI=; b=vqUngKrAjeeplNsEKfg6r+T4OHux5JmsgCtUM6A5RKd2uLfXwCDWgLuct2IODfPyLu pCuWyNhvEeXRsFrVQQz129OZ5UqFKv3ihOfM1yfIKjtQuQMbzQj5T/mvFdcdrxrSxJPt 4hPyYhwbDSpgMisqDsAwa4UK14hp0xg8P0t7P/1cu3lQNwhOOzePsFvDgCOH6BlSAdVX nXTiBarKi0CcU2cgv9IQAj1dZaf0g0ouj2dK/I9KBQwyUcneS3aiia41dKDHkFMgtx4H 7xQGUxLHfZJOOOfTTQ20IeIKKCJ9neuOy6kQDuB0TmWad9rrsMf3DbeHLktRXkh3VsyA VmNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=J3Edru9e7lUdVoaTGZTK5sTrBdxVxdyPB1Z72ic00xI=; b=VsXH3c+tXW3eWbmMSBXV5wUWKK3up0h+Ips/I0JVKKLOvDLcvdXihgaHOdxN5T+iaw 5LDb0CWvGlALiK/lA+viI0PLaxsEW4a8HiBROSoNPfV+atjoULX+mRj9jFvrQEbRGKQL YKvpQ7a+W9vVox1Dz9/xsQYto4bei2f7H4jEYRNNtpXShncEDUWVRepFgCbRabkDRlU5 h29WbkTshK+v0CQ43JuEaPrX8lgYVyo1WjkpYKEbStPKfsWKdpXNLqyo67ucH4Sx2gBd Sf9YgaH0pqn6WtRd5LWJIEtzYVz7QEixZSTPRHxxNirSSWpRC/XkWWPF5RIjujuvl0Qb 8N8Q== X-Gm-Message-State: ALoCoQlpjyMrrWay+VaRM0qPRvLGRI+AlJ6ZFNYddrLdZFVyqQ8qxxczvyiJTFo2jFsCioJbN0l6OrU8gTi99wHk+UuUssdiaCSuMttNc42q2126976a7YI= MIME-Version: 1.0 X-Received: by 10.66.177.193 with SMTP id cs1mr27325987pac.132.1451386048177; Tue, 29 Dec 2015 02:47:28 -0800 (PST) Received: by 10.66.196.81 with HTTP; Tue, 29 Dec 2015 02:47:28 -0800 (PST) In-Reply-To: References: <2241331.HNmyzf8foi@xps13> <2979402.yeVYlcCDUH@xps13> <20151218053053.GL29571@yliu-dev.sh.intel.com> <20151218082139.GC18863@yliu-dev.sh.intel.com> Date: Tue, 29 Dec 2015 16:17:28 +0530 Message-ID: From: Santosh Shukla To: "Burakov, Anatoly" , alex.williamson@redhat.com Content-Type: text/plain; charset=UTF-8 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH] eal: map io resources for non x86 architectures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Dec 2015 10:47:29 -0000 On Tue, Dec 29, 2015 at 3:26 PM, Burakov, Anatoly wrote: > Hi Santosh, > >> On Fri, Dec 18, 2015 at 6:25 PM, Santosh Shukla >> wrote: >> > On Fri, Dec 18, 2015 at 1:51 PM, Yuanhan Liu >> > wrote: >> >> On Fri, Dec 18, 2015 at 01:24:41PM +0530, Santosh Shukla wrote: >> >>> >> I guess we have done enough evaluation / investigation that >> >>> >> suggest - so to map iopci region to userspace in arch >> >>> >> agnostic-way - >> >>> >> >> >>> >> # either we need to modify kernel >> >>> >> - Make sure all the non-x86 arch to support >> >>> >> mapping for iopci region (i.e. pci_mmap_page_range). I don;t >> >>> >> think its a correct approach though. >> >>> >> or >> >>> >> - include /dev/ioport char-mem device file who >> >>> >> could do more than byte operation, Note that this implementation >> >>> >> does not exist in kernel. I could send an RFC to lkml. >> >>> > >> >>> > Maybe you could propose the two to lkml, to get some feedbacks >> >>> > from those kernel/ARM gurus? Please cc me if you do so. >> >>> > >> >>> >> >>> The latter one I already shared old lkml thread, Pl. revisit my v1 >> >>> 0/6 patch [1] and in that refer [2]. >> >> >> >> Oops, sorry, a bit busy, that I didn't look at it carefully. My bad, >> >> anyway. >> >> >> >>> Josh has already proposed to lkml but for some reason thread didn't >> >>> went far. I can restart that discussion giving dpdk use-case as an >> >>> example/ requirement. >> >> >> >> I had a quick go through of the discussion. Both hpa and Arnd seem to >> >> be fine with the ioctl interface on /dev/port. Have you tried that? >> >> And if you want to restart it, ioctl might be a better option than >> >> /dev/ioport, judging from the discussion. >> >> >> > >> > I tried legacy patch and re-writing with ioctl-way; doing changes in >> > dpdk port as well in kernel, had to test on quite other hw not only >> > arm64 though! so it will take time for me, I am travelling tomorrow so >> > bit delayed, We'll post patch to lkml and share dpdk-virtio feedback >> > perhaps by Monday. >> > >> >> I posted a query about /dev/ioports approach in lkml thread [1], and Arnd >> suggested to use vfio framework but it looks like vfio too does not map >> ioresource_io region. Same communicated to Arnd and waiting for his reply. >> >> In mean time I like to ask general question; >> - Has anyone tried vfio/non-iommu approach for virtio pmd driver? If not >> then Is there any plan? Someone planning to take up. >> [1] https://lkml.org/lkml/2015/12/23/145 > > I have submitted a patch to support no-iommu mode, but I'm not aware of anyone trying VFIO-noiommu at all. That's probably expected since it's Christmas/New Year in a lot of places, and everyone is on a break. > > That said, I'm not sure I completely understand what is it that you're asking about. The code you're referring to (in vfio_pci.c, line 854 as of kernel 4.3) is checking if a PCI BAR is available for IO (hence checking if the IORESOURCE_MEM Thanks for reply! You comment might help to move this discuss to next level. Look at kernel/resource.c, it exports two symbol ioport_resource and iomem_resource and sets appropriate flag type i.e.. IORESOURCE_IO and IORESOURCE_MEM. In virtio-net case; it creates both pci region i.e.. _io bar and _mem bar. dpdk virtio pmd driver (<= 0.95 virtio spec) uses pci _io bar region for device initialization as virtio headers are locate at pci _io bar region. Since it uses pci _iobar region so likely it update pci_resource.[index].flag = IORESOURCE_IO. and vfio mmap function wont handle ioresource_io (i guess). And that is why I asked same to lkml thread. bit is set). There isn't any "ioresource_mem" region as far as VFIO is concerned, there are only BARs, ROM, VGA and PCI config regions (linux/vfio.h, line 314 as of kernel 4.3). So if you're missing some PCI regions for VFIO to map, they would first need to be added to VFIO kernel implementation before they can be used by DPDK. That is, unless I'm misunderstanding something :) > Agree. As mentioned above, I guess ioresource_io pci bar implementation missing in vfio kernel? but before adding code support in kernel I would like to hear from experts example, You, Alex! (looping alex) > Thanks, > Anatoly