From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f194.google.com (mail-io0-f194.google.com [209.85.223.194]) by dpdk.org (Postfix) with ESMTP id 8B9802C08 for ; Thu, 20 Apr 2017 15:25:43 +0200 (CEST) Received: by mail-io0-f194.google.com with SMTP id h41so16437778ioi.1 for ; Thu, 20 Apr 2017 06:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ozlabs-ru.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=0sWiHLooqPpSliZuz+JYyFroEocnyCNQRbaBSnI4cLI=; b=k0daHzWO6Nq6Pu0QTZm5gscJN7rMr+9lGFEwjmVhPimE1DxhIJSe5jRr0fH2x9J4Gi 072vy/Byn5O1mjHlUJh9MS6uBRr5hfuwvo5ht074mCJUcKtYJJ2ndABuLwOl/tBTYj0n 54cDEEYZAbBh2v8wZAKdwWMYc5enaJiRSUhLMHyVzm7M99aavU94CwyDFmDzMa1/YEBb a6n58Res7oH7ZLxxVIdoRweglxgQ/WrzSC5s370XAwfdP8oGuH5EgwwTBbXAMsxKSY4l 34YP970npRnfWRmB5N7vfGtsUYMo1n83jyZGWGEZ4kG6PmAaWu6f1O/ND/auFlHb6eLX 63mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=0sWiHLooqPpSliZuz+JYyFroEocnyCNQRbaBSnI4cLI=; b=kxJ0ojNPDr6q4gavoxuAFPaSjwEDsgfzWdgRC+4J3TBnWvBZ+4TI8y8PAUSIzZNW3A tjw/gUWF0tAqprVLtkfsV5XlYjh36BNku2+o53GR00Hl3UOPbeVU3jEBhij/WJtMOx5G GhFX+nThBzrgnvSaGGSRmhZ4eKSGflf7+4Iz8cfPcY3zRD6Q0Pi2T94ouJAkn1klxWFm plP81r7bBFR4SO0X8CJ5lYwfNExOq1Kae1Ct0TLKWzzpGIc+csM8JJyqmpvc46fJIR2v AWNHzVqmiz1RPi2VhJ/RwnwMmSuN4bKnE1h6hf8TGgzrofNaau/RXEFvV7q4+utqOzVf /Gsg== X-Gm-Message-State: AN3rC/4t/N38wib+1zClMYLSUpnjhd79fzfRDIDilDHW583FvJc75h93 IUi1w/5ZpI30pA== X-Received: by 10.99.111.1 with SMTP id k1mr8020666pgc.194.1492694742675; Thu, 20 Apr 2017 06:25:42 -0700 (PDT) Received: from [192.168.1.21] (124-171-134-5.dyn.iinet.net.au. [124.171.134.5]) by smtp.googlemail.com with ESMTPSA id 65sm10769943pgc.1.2017.04.20.06.25.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Apr 2017 06:25:42 -0700 (PDT) To: Jonas Pfefferle1 Cc: dev@dpdk.org, Gowrishankar Muthukrishnan , Adrian Schuepbach References: <20170420072402.38106-1-aik@ozlabs.ru> <20170420072402.38106-6-aik@ozlabs.ru> From: Alexey Kardashevskiy Message-ID: <12566b0a-8f9a-4040-a37d-2a106e49adcf@ozlabs.ru> Date: Thu, 20 Apr 2017 23:25:36 +1000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=koi8-r Content-Language: en-AU Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH dpdk 5/5] RFC: vfio/ppc64/spapr: Use correct bus addresses for DMA map X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Apr 2017 13:25:43 -0000 On 20/04/17 19:04, Jonas Pfefferle1 wrote: > Alexey Kardashevskiy wrote on 20/04/2017 09:24:02: > >> From: Alexey Kardashevskiy >> To: dev@dpdk.org >> Cc: Alexey Kardashevskiy , JPF@zurich.ibm.com, >> Gowrishankar Muthukrishnan >> Date: 20/04/2017 09:24 >> Subject: [PATCH dpdk 5/5] RFC: vfio/ppc64/spapr: Use correct bus >> addresses for DMA map >> >> VFIO_IOMMU_SPAPR_TCE_CREATE ioctl() returns the actual bus address for >> just created DMA window. It happens to start from zero because the default >> window is removed (leaving no windows) and new window starts from zero. >> However this is not guaranteed and the new window may start from another >> address, this adds an error check. >> >> Another issue is that IOVA passed to VFIO_IOMMU_MAP_DMA should be a PCI >> bus address while in this case a physical address of a user page is used. >> This changes IOVA to start from zero in a hope that the rest of DPDK >> expects this. > > This is not the case. DPDK expects a 1:1 mapping PA==IOVA. It will use the > phys_addr of the memory segment it got from /proc/self/pagemap cf. > librte_eal/linuxapp/eal/eal_memory.c. We could try setting it here to the > actual iova which basically makes the whole virtual to phyiscal mapping > with pagemap unnecessary which I believe should be the case for VFIO > anyway. Pagemap should only be needed when using pci_uio. Ah, ok, makes sense now. But it sure needs a big fat comment there as it is not obvious why host RAM address is used there as DMA window start is not guaranteed. > >> >> Signed-off-by: Alexey Kardashevskiy >> --- >> lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- >> 1 file changed, 10 insertions(+), 2 deletions(-) >> >> diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/ >> librte_eal/linuxapp/eal/eal_vfio.c >> index 46f951f4d..8b8e75c4f 100644 >> --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c >> +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c >> @@ -658,7 +658,7 @@ vfio_spapr_dma_map(int vfio_container_fd) >> { >> const struct rte_memseg *ms = rte_eal_get_physmem_layout(); >> int i, ret; >> - >> + phys_addr_t io_offset; >> struct vfio_iommu_spapr_register_memory reg = { >> .argsz = sizeof(reg), >> .flags = 0 >> @@ -702,6 +702,13 @@ vfio_spapr_dma_map(int vfio_container_fd) >> return -1; >> } >> >> + io_offset = create.start_addr; >> + if (io_offset) { >> + RTE_LOG(ERR, EAL, " DMA offsets other than zero is not supported, " >> + "new window is created at %lx\n", io_offset); >> + return -1; >> + } >> + >> /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ >> for (i = 0; i < RTE_MAX_MEMSEG; i++) { >> struct vfio_iommu_type1_dma_map dma_map; >> @@ -723,7 +730,7 @@ vfio_spapr_dma_map(int vfio_container_fd) >> dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map); >> dma_map.vaddr = ms[i].addr_64; >> dma_map.size = ms[i].len; >> - dma_map.iova = ms[i].phys_addr; >> + dma_map.iova = io_offset; >> dma_map.flags = VFIO_DMA_MAP_FLAG_READ | >> VFIO_DMA_MAP_FLAG_WRITE; >> >> @@ -735,6 +742,7 @@ vfio_spapr_dma_map(int vfio_container_fd) >> return -1; >> } >> >> + io_offset += dma_map.size; >> } >> >> return 0; >> -- >> 2.11.0 >> > -- Alexey