Re: [dpdk-dev] [PATCH v3 0/6] use IOVAs check based on DMA mask

["Yao, Lei A" <lei.a.yao@intel.com>]

Hi, Lucero, Thomas

This patch set will cause deadlock during memory initialization. 
rte_memseg_walk and try_expand_heap both will lock 
the file &mcfg->memory_hotplug_lock. So dead lock will occur. 

#0       rte_memseg_walk
#1  <-rte_eal_check_dma_mask
#2  <-alloc_pages_on_heap
#3  <-try_expand_heap_primary   
#4  <-try_expand_heap

Log as following:
EAL: TSC frequency is ~2494156 KHz
EAL: Master lcore 0 is ready (tid=7ffff7fe3c00;cpuset=[0])
[New Thread 0x7ffff5e0d700 (LWP 330350)]
EAL: lcore 1 is ready (tid=7ffff5e0d700;cpuset=[1])
EAL: Trying to obtain current memory policy.
EAL: Setting policy MPOL_PREFERRED for socket 0
EAL: Restoring previous memory policy: 0

Could you have a check on this? A lot of test cases in our validation 
team fail because of this. Thanks a lot!

BRs
Lei


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Monday, October 29, 2018 5:04 AM
> To: Alejandro Lucero <alejandro.lucero@netronome.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v3 0/6] use IOVAs check based on DMA
> mask
> 
> 05/10/2018 14:45, Alejandro Lucero:
> > I sent a patchset about this to be applied on 17.11 stable. The memory
> > code has had main changes since that version, so here it is the patchset
> > adjusted to current master repo.
> >
> > This patchset adds, mainly, a check for ensuring IOVAs are within a
> > restricted range due to addressing limitations with some devices. There
> > are two known cases: NFP and IOMMU VT-d emulation.
> >
> > With this check IOVAs out of range are detected and PMDs can abort
> > initialization. For the VT-d case, IOVA VA mode is allowed as long as
> > IOVAs are within the supported range, avoiding to forbid IOVA VA by
> > default.
> >
> > For the addressing limitations known cases, there are just 40(NFP) or
> > 39(VT-d) bits for handling IOVAs. When using IOVA PA, those limitations
> > imply 1TB(NFP) or 512M(VT-d) as upper limits, which is likely enough for
> > most systems. With machines using more memory, the added check will
> > ensure IOVAs within the range.
> >
> > With IOVA VA, and because the way the Linux kernel serves mmap calls
> > in 64 bits systems, 39 or 40 bits are not enough. It is possible to
> > give an address hint with a lower starting address than the default one
> > used by the kernel, and then ensuring the mmap uses that hint or hint plus
> > some offset. With 64 bits systems, the process virtual address space is
> > large enoguh for doing the hugepages mmaping within the supported
> range
> > when those addressing limitations exist. This patchset also adds a change
> > for using such a hint making the use of IOVA VA a more than likely
> > possibility when there are those addressing limitations.
> >
> > The check is not done by default but just when it is required. This
> > patchset adds the check for NFP initialization and for setting the IOVA
> > mode is an emulated VT-d is detected. Also, because the recent patchset
> > adding dynamic memory allocation, the check is also invoked for ensuring
> > the new memsegs are within the required range.
> >
> > This patchset could be applied to stable 18.05.
> 
> Applied, thanks
> 
> 
>