Re: [dpdk-dev] [PATCH] vfio: fix VFIO mapping failures in ppc64le

[David Christensen <drc@linux.vnet.ibm.com>]

> ppc64le failed when using large physical memory. I found problems in my two
> commits in the past.
> 
> In commit e072d16f8920 ("vfio: fix expanding DMA area in ppc64le"), I added
> a sanity check using a mapped address to resolve an issue around expanding
> IOMMU window, but this was not enough, since memory allocation can return
> memory anywhere dependent on memory fragmentation. DPDK may still skip DMA
> mapping and attempts to unmap non-mapped DMA during expanding IOMMU window.
> As a result, SPDK apps using large physical memory frequently failed to
> proceed the communication with NVMe and/or went into an infinite loop.
> 
> The root cause of the bug was in a gap between memory segments managed by
> DPDK and firmware-level DMA mapping. DPDK's memory segments don't contain
> the state of DMA mapping, and so, the memesg_walk cannot determine if an
> iterated memory segment is mapped or not. This resulted in incorrect DMA
> maps and unmaps.
> 
> At this time, I added the code to avoid iterating non-mapped memory
> segments during DMA mapping. The memseg_walk iterates over memory segments
> marked as "used", and so, the code sets memory segments that will be
> mapped or unmapped as "free" transiently.
> 
> The commit db90b4969e2e ("vfio: retry creating sPAPR DMA window") allows
> retring different page levels and sizes to create DMA window. However, this
> allows page sizes different from hugepage sizes. This inconsistency caused
> failures at the time of DMA mapping after the window creation. This patch
> fixes to retry only different page levels.
> 
> Fixes: e072d16f8920 ("vfio: fix expanding DMA area in ppc64le")
> Fixes: db90b4969e2e ("vfio: retry creating sPAPR DMA window")
> 
> Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
> ---

Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>