From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by dpdk.org (Postfix) with ESMTP id D270B1F1C for ; Mon, 29 Oct 2018 09:42:39 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 1C2072206D; Mon, 29 Oct 2018 04:42:39 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 29 Oct 2018 04:42:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=VJcb08CSqsQkWcihcOAviFdtdcFuVjRaHCw4RkkNZLE=; b=n4uOd7xFz4Zm UbRGoGnv5NmIy7ugFs6U2y+aB+/3Px+VSyj0bnlLdFybB6ji1ViD3mPyxdHX3gr+ O58sX3F892LxlmIM5gGofEABT3oCRDjYqLuC8O5BDTraP4YYHN+09hvofbjpMypH SpsuREPuKuIz5YTbtz3s0Nsmc53fMrI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=VJcb08CSqsQkWcihcOAviFdtdcFuVjRaHCw4RkkNZ LE=; b=QkHFwxuhY2T42HfNUzYRpHPwzKkGa6kud7ol/WV7JVto99HtNM+j0KREx UVJJAnhatONS3fbm9WxmAw1C0sEfDSi3wsZIK2lPVHpKje14qehBog6CCerm7qjR kps/pv+mqyfHFVmhykCJ5/SND0RiWA2NN2w0vox1bSMiuZV3SJBpXcx+jTw160dS 7Y3WuILadfZa5feG3ewr97k3eNpoFtK198RcZ0Fj1qt5P0kj6OJEPMHufjNaiZar juXmhHsCcO8lkwHI9RTXqD2psnzwlJu52KWdqnnJsQ1oMQkXbecl08oxsv9ag/+c Xo3Cq+Le2MCuy8xZ3C0KGXBbyQCnA== X-ME-Sender: X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 9146EE49FA; Mon, 29 Oct 2018 04:42:37 -0400 (EDT) From: Thomas Monjalon To: "Yao, Lei A" Cc: Alejandro Lucero , "dev@dpdk.org" , "Xu, Qian Q" , "Lin, Xueqin" , anatoly.burakov@intel.com Date: Mon, 29 Oct 2018 09:42:43 +0100 Message-ID: <1593678.TTmrtHRuFR@xps> In-Reply-To: <2DBBFF226F7CF64BAFCA79B681719D954502B75A@shsmsx102.ccr.corp.intel.com> References: <1538743527-8285-1-git-send-email-alejandro.lucero@netronome.com> <4805099.7RpDHruce7@xps> <2DBBFF226F7CF64BAFCA79B681719D954502B75A@shsmsx102.ccr.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v3 0/6] use IOVAs check based on DMA mask X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2018 08:42:40 -0000 29/10/2018 09:23, Yao, Lei A: > Hi, Lucero, Thomas > > This patch set will cause deadlock during memory initialization. > rte_memseg_walk and try_expand_heap both will lock > the file &mcfg->memory_hotplug_lock. So dead lock will occur. > > #0 rte_memseg_walk > #1 <-rte_eal_check_dma_mask > #2 <-alloc_pages_on_heap > #3 <-try_expand_heap_primary > #4 <-try_expand_heap > > Log as following: > EAL: TSC frequency is ~2494156 KHz > EAL: Master lcore 0 is ready (tid=7ffff7fe3c00;cpuset=[0]) > [New Thread 0x7ffff5e0d700 (LWP 330350)] > EAL: lcore 1 is ready (tid=7ffff5e0d700;cpuset=[1]) > EAL: Trying to obtain current memory policy. > EAL: Setting policy MPOL_PREFERRED for socket 0 > EAL: Restoring previous memory policy: 0 > > Could you have a check on this? A lot of test cases in our validation > team fail because of this. Thanks a lot! Can we just call rte_memseg_walk_thread_unsafe()? +Cc Anatoly > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon > > 05/10/2018 14:45, Alejandro Lucero: > > > I sent a patchset about this to be applied on 17.11 stable. The memory > > > code has had main changes since that version, so here it is the patchset > > > adjusted to current master repo. > > > > > > This patchset adds, mainly, a check for ensuring IOVAs are within a > > > restricted range due to addressing limitations with some devices. There > > > are two known cases: NFP and IOMMU VT-d emulation. > > > > > > With this check IOVAs out of range are detected and PMDs can abort > > > initialization. For the VT-d case, IOVA VA mode is allowed as long as > > > IOVAs are within the supported range, avoiding to forbid IOVA VA by > > > default. > > > > > > For the addressing limitations known cases, there are just 40(NFP) or > > > 39(VT-d) bits for handling IOVAs. When using IOVA PA, those limitations > > > imply 1TB(NFP) or 512M(VT-d) as upper limits, which is likely enough for > > > most systems. With machines using more memory, the added check will > > > ensure IOVAs within the range. > > > > > > With IOVA VA, and because the way the Linux kernel serves mmap calls > > > in 64 bits systems, 39 or 40 bits are not enough. It is possible to > > > give an address hint with a lower starting address than the default one > > > used by the kernel, and then ensuring the mmap uses that hint or hint plus > > > some offset. With 64 bits systems, the process virtual address space is > > > large enoguh for doing the hugepages mmaping within the supported > > range > > > when those addressing limitations exist. This patchset also adds a change > > > for using such a hint making the use of IOVA VA a more than likely > > > possibility when there are those addressing limitations. > > > > > > The check is not done by default but just when it is required. This > > > patchset adds the check for NFP initialization and for setting the IOVA > > > mode is an emulated VT-d is detected. Also, because the recent patchset > > > adding dynamic memory allocation, the check is also invoked for ensuring > > > the new memsegs are within the required range. > > > > > > This patchset could be applied to stable 18.05. > > > > Applied, thanks