From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f193.google.com (mail-io0-f193.google.com [209.85.223.193]) by dpdk.org (Postfix) with ESMTP id B73C12BA4 for ; Thu, 20 Apr 2017 16:22:08 +0200 (CEST) Received: by mail-io0-f193.google.com with SMTP id d203so16976039iof.2 for ; Thu, 20 Apr 2017 07:22:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ozlabs-ru.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=6sXHt2hhIj37Dpiwss34vqbHEbITBtCN9UCEvJ5bV4k=; b=l/OfUVURLMUMh8P4YZiXxAyyPknrva5q7/eDeBFgNtO25D9bbCdXgDUCAKZX7Hf/cw pXqt5Qgr81rvW8rwoHiZqSj5WyeGK1txNlF6LMAYvEPuaFU/d05Q1F4ir496wXnwP87Y xObTaveRGkLyB8xusRKCdxi8E6pPeT+LjBDCovhqG77fOunAZRqFP7lsPUFUyGvYMRz3 uZtKyDJOIZojqiMcZc2qELKn877O7oMf0UGkUXfSwFUKly5/laht6j5KJP8vZNE72hXA U7lKnqDmTbUKWcodEi5UNAK5t7HA/co+y/ZvFwlK9Ux+5DdFDwDetDu9h59RaMBv8med lXqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=6sXHt2hhIj37Dpiwss34vqbHEbITBtCN9UCEvJ5bV4k=; b=qH2m13caaZficA+knNIpTnRXdhBFgkCYGCEdnyETte/oqC5EdCg2Am5U47qVInPSjz uqYmYjq7TZ7dyDyxQQwnSQphXP6Eq26yFb178nThQwr0QSK90DIcBA8/DUY6nMz4NzGX G+1ZnoVx6hYytHD6tsJoMbMY/Q8fzi6UtQBjMRKOlaDipZPvWoFR9GrCRqPiLzR/gcI1 9GPqSAGAwsi4VsnKW/a2r4fZXN2bwm9XUuio8NiHdN6fYJ6a5O96uEZqW3hBHQu6F09V iS8mETfoQa+iEK7VMorAmdQnR3CVzUZ25SfRuKIPYiiEmhkVOALd1pBAwF2kyhHTJPMi hCGg== X-Gm-Message-State: AN3rC/7up3g4NKQUTJDezXwnffqchuYl4Qg9Z9RKbMm0Hcg6vNlhUKXg pr68wQ5EXBm5K9WT X-Received: by 10.99.115.6 with SMTP id o6mr8194206pgc.216.1492698127683; Thu, 20 Apr 2017 07:22:07 -0700 (PDT) Received: from [192.168.1.22] (124-171-134-5.dyn.iinet.net.au. [124.171.134.5]) by smtp.googlemail.com with ESMTPSA id m19sm10977361pfg.115.2017.04.20.07.22.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Apr 2017 07:22:07 -0700 (PDT) From: Alexey Kardashevskiy To: Jonas Pfefferle1 Cc: dev@dpdk.org, Gowrishankar Muthukrishnan , Adrian Schuepbach References: <20170420072402.38106-1-aik@ozlabs.ru> <20170420072402.38106-6-aik@ozlabs.ru> <12566b0a-8f9a-4040-a37d-2a106e49adcf@ozlabs.ru> Message-ID: <6e669e2b-2cfd-078d-b6b0-5c3819fad796@ozlabs.ru> Date: Fri, 21 Apr 2017 00:22:01 +1000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.0 MIME-Version: 1.0 In-Reply-To: <12566b0a-8f9a-4040-a37d-2a106e49adcf@ozlabs.ru> Content-Type: text/plain; charset=koi8-r Content-Language: en-AU Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH dpdk 5/5] RFC: vfio/ppc64/spapr: Use correct bus addresses for DMA map X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Apr 2017 14:22:09 -0000 On 20/04/17 23:25, Alexey Kardashevskiy wrote: > On 20/04/17 19:04, Jonas Pfefferle1 wrote: >> Alexey Kardashevskiy wrote on 20/04/2017 09:24:02: >> >>> From: Alexey Kardashevskiy >>> To: dev@dpdk.org >>> Cc: Alexey Kardashevskiy , JPF@zurich.ibm.com, >>> Gowrishankar Muthukrishnan >>> Date: 20/04/2017 09:24 >>> Subject: [PATCH dpdk 5/5] RFC: vfio/ppc64/spapr: Use correct bus >>> addresses for DMA map >>> >>> VFIO_IOMMU_SPAPR_TCE_CREATE ioctl() returns the actual bus address for >>> just created DMA window. It happens to start from zero because the default >>> window is removed (leaving no windows) and new window starts from zero. >>> However this is not guaranteed and the new window may start from another >>> address, this adds an error check. >>> >>> Another issue is that IOVA passed to VFIO_IOMMU_MAP_DMA should be a PCI >>> bus address while in this case a physical address of a user page is used. >>> This changes IOVA to start from zero in a hope that the rest of DPDK >>> expects this. >> >> This is not the case. DPDK expects a 1:1 mapping PA==IOVA. It will use the >> phys_addr of the memory segment it got from /proc/self/pagemap cf. >> librte_eal/linuxapp/eal/eal_memory.c. We could try setting it here to the >> actual iova which basically makes the whole virtual to phyiscal mapping >> with pagemap unnecessary which I believe should be the case for VFIO >> anyway. Pagemap should only be needed when using pci_uio. > > > Ah, ok, makes sense now. But it sure needs a big fat comment there as it is > not obvious why host RAM address is used there as DMA window start is not > guaranteed. Well, either way there is some bug - ms[i].phys_addr and ms[i].addr_64 both have exact same value, in my setup it is 3fffb33c0000 which is a userspace address - at least ms[i].phys_addr must be physical address. > > >> >>> >>> Signed-off-by: Alexey Kardashevskiy >>> --- >>> lib/librte_eal/linuxapp/eal/eal_vfio.c | 12 ++++++++++-- >>> 1 file changed, 10 insertions(+), 2 deletions(-) >>> >>> diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/ >>> librte_eal/linuxapp/eal/eal_vfio.c >>> index 46f951f4d..8b8e75c4f 100644 >>> --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c >>> +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c >>> @@ -658,7 +658,7 @@ vfio_spapr_dma_map(int vfio_container_fd) >>> { >>> const struct rte_memseg *ms = rte_eal_get_physmem_layout(); >>> int i, ret; >>> - >>> + phys_addr_t io_offset; >>> struct vfio_iommu_spapr_register_memory reg = { >>> .argsz = sizeof(reg), >>> .flags = 0 >>> @@ -702,6 +702,13 @@ vfio_spapr_dma_map(int vfio_container_fd) >>> return -1; >>> } >>> >>> + io_offset = create.start_addr; >>> + if (io_offset) { >>> + RTE_LOG(ERR, EAL, " DMA offsets other than zero is not supported, " >>> + "new window is created at %lx\n", io_offset); >>> + return -1; >>> + } >>> + >>> /* map all DPDK segments for DMA. use 1:1 PA to IOVA mapping */ >>> for (i = 0; i < RTE_MAX_MEMSEG; i++) { >>> struct vfio_iommu_type1_dma_map dma_map; >>> @@ -723,7 +730,7 @@ vfio_spapr_dma_map(int vfio_container_fd) >>> dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map); >>> dma_map.vaddr = ms[i].addr_64; >>> dma_map.size = ms[i].len; >>> - dma_map.iova = ms[i].phys_addr; >>> + dma_map.iova = io_offset; >>> dma_map.flags = VFIO_DMA_MAP_FLAG_READ | >>> VFIO_DMA_MAP_FLAG_WRITE; >>> >>> @@ -735,6 +742,7 @@ vfio_spapr_dma_map(int vfio_container_fd) >>> return -1; >>> } >>> >>> + io_offset += dma_map.size; >>> } >>> >>> return 0; >>> -- >>> 2.11.0 >>> >> > > -- Alexey