From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id A803E1B2AA for ; Wed, 1 Nov 2017 07:21:20 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 31 Oct 2017 23:21:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,327,1505804400"; d="scan'208";a="916116740" Received: from fyigit-mobl1.ger.corp.intel.com (HELO [10.254.27.219]) ([10.254.27.219]) by FMSMGA003.fm.intel.com with ESMTP; 31 Oct 2017 23:21:18 -0700 To: Jerin Jacob Cc: Thomas Monjalon , Bruce Richardson , Sergio Gonzalez Monroy , dev@dpdk.org, Jianfeng Tan , Santosh Shukla References: <20171101010726.17781-1-ferruh.yigit@intel.com> <20171101040251.GA13569@jerin> From: Ferruh Yigit Message-ID: <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> Date: Tue, 31 Oct 2017 23:21:18 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20171101040251.GA13569@jerin> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Nov 2017 06:21:21 -0000 On 10/31/2017 9:02 PM, Jerin Jacob wrote: > -----Original Message----- >> Date: Wed, 1 Nov 2017 01:07:26 +0000 >> From: Ferruh Yigit >> To: Thomas Monjalon , Bruce Richardson >> , Sergio Gonzalez Monroy >> >> CC: dev@dpdk.org, Ferruh Yigit , Jianfeng Tan >> , Santosh Shukla >> >> Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default >> X-Mailer: git-send-email 2.13.6 >> >> Fix kernel crash with KNI because KNI requires physical addresses. > > The actual fix would be to make KNI IOMMU aware based on the DPDK mode. > > ie. On slow path, > > /* Get iommu domain for iova to physical addr conversion */ > if (rte_eal_iova_mode() == RTE_IOVA_VA) > kni->iommu_domain = iommu_get_domain_for_dev(dev); > else > kni->iommu_domain = NULL; > > On fast path, > > static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr) > { > /* Translation is installed only when IOMMU is present */ > if (kni->iommu_domain) > return iommu_iova_to_phys(kni->iommu_domain, dma_addr); > return dma_addr; > } > >> >> A config option introduced to disable IOVA mode detection and to set it >> to physical address by default. Disabling config option will enable IOVA >> mode detection. >> >> When there is no intension to use KNI, it is safe to enable detection. >> >> Config option disable IOVA mode detection by default to be sure only who >> is aware of result enable it. >> >> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy") >> >> Signed-off-by: Ferruh Yigit >> --- >> Cc: Jianfeng Tan >> Cc: Santosh Shukla >> Cc: Thomas Monjalon >> --- >> config/common_base | 5 +++++ >> lib/librte_eal/bsdapp/eal/eal.c | 4 ++++ >> lib/librte_eal/linuxapp/eal/eal.c | 4 ++++ >> 3 files changed, 13 insertions(+) >> >> diff --git a/config/common_base b/config/common_base >> index 82ee75456..903e7685b 100644 >> --- a/config/common_base >> +++ b/config/common_base >> @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n >> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n >> >> # >> +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution >> +# >> +CONFIG_RTE_EAL_USE_PHYS_IOVA=y > > Defeat the purpose of all dynamic probing scheme. > Either we can fix the KNI or revert the following patch for this release. > > http://dpdk.org/commit/f37dfab2 This commit just enables IOVA VA mode for Intel drivers, that is how I can able to observe the issue, but it is not the source of the problem. Reverting that commit will not solve KNI crash with any other PMD that enables IOVA VA mode. Related to the KNI, iommu is not involved at all, I am not clear with your above suggestion, physical address is required for kernelspace - userspace communication. KNI uses physical address for two things: 1- Creates a buffer in userspace, a memzone, and shares its physical address to the kernel, so that both kernel and app can access same buffer. The question is, even all devices supports IOVA VA mode, why creating a memzone that has real physical address info is not possible anymore? For KNI case this is not related at all with what devices supports. 2- For each mbuf that will be sent to kernel, dpdk app puts physical address of that mbuf into shared buffer, so that kernel can access it. So that kernel can access to mbuf data that can be coming from any mempool. For this the physical address of the mbuf is required. Overall KNI needs to know physical address of mbufs and memzone. This patch provides a way to have old behavior with a config option, so that KNI can work and anyone who needs to create memzone with physical address can a way to have it, and all PMDs will work fine. If there is no need for these, it is possible to disable config and dynamic probing scheme is already there. Thanks, ferruh