From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 1C45D1B218 for ; Wed, 1 Nov 2017 08:39:19 +0100 (CET) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Nov 2017 00:39:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,327,1505804400"; d="scan'208";a="1031981011" Received: from tanjianf-mobl.ccr.corp.intel.com (HELO [10.67.64.54]) ([10.67.64.54]) by orsmga003.jf.intel.com with ESMTP; 01 Nov 2017 00:39:02 -0700 To: Jerin Jacob , Ferruh Yigit References: <20171101010726.17781-1-ferruh.yigit@intel.com> <20171101040251.GA13569@jerin> <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> <20171101065431.GA25152@jerin> Cc: Thomas Monjalon , Bruce Richardson , Sergio Gonzalez Monroy , dev@dpdk.org, Santosh Shukla From: "Tan, Jianfeng" Message-ID: Date: Wed, 1 Nov 2017 15:39:01 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20171101065431.GA25152@jerin> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Nov 2017 07:39:20 -0000 On 11/1/2017 2:54 PM, Jerin Jacob wrote: > -----Original Message----- >> Date: Tue, 31 Oct 2017 23:21:18 -0700 >> From: Ferruh Yigit >> To: Jerin Jacob >> CC: Thomas Monjalon , Bruce Richardson >> , Sergio Gonzalez Monroy >> , dev@dpdk.org, Jianfeng Tan >> , Santosh Shukla >> >> Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default >> User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 >> Thunderbird/52.4.0 >> >> On 10/31/2017 9:02 PM, Jerin Jacob wrote: >>> -----Original Message----- >>>> Date: Wed, 1 Nov 2017 01:07:26 +0000 >>>> From: Ferruh Yigit >>>> To: Thomas Monjalon , Bruce Richardson >>>> , Sergio Gonzalez Monroy >>>> >>>> CC: dev@dpdk.org, Ferruh Yigit , Jianfeng Tan >>>> , Santosh Shukla >>>> >>>> Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default >>>> X-Mailer: git-send-email 2.13.6 >>>> >>>> Fix kernel crash with KNI because KNI requires physical addresses. >>> The actual fix would be to make KNI IOMMU aware based on the DPDK mode. >>> >>> ie. On slow path, >>> >>> /* Get iommu domain for iova to physical addr conversion */ >>> if (rte_eal_iova_mode() == RTE_IOVA_VA) >>> kni->iommu_domain = iommu_get_domain_for_dev(dev); >>> else >>> kni->iommu_domain = NULL; >>> >>> On fast path, >>> >>> static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr) >>> { >>> /* Translation is installed only when IOMMU is present */ >>> if (kni->iommu_domain) >>> return iommu_iova_to_phys(kni->iommu_domain, dma_addr); >>> return dma_addr; >>> } >>> >>>> A config option introduced to disable IOVA mode detection and to set it >>>> to physical address by default. Disabling config option will enable IOVA >>>> mode detection. >>>> >>>> When there is no intension to use KNI, it is safe to enable detection. >>>> >>>> Config option disable IOVA mode detection by default to be sure only who >>>> is aware of result enable it. >>>> >>>> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy") >>>> >>>> Signed-off-by: Ferruh Yigit >>>> --- >>>> Cc: Jianfeng Tan >>>> Cc: Santosh Shukla >>>> Cc: Thomas Monjalon >>>> --- >>>> config/common_base | 5 +++++ >>>> lib/librte_eal/bsdapp/eal/eal.c | 4 ++++ >>>> lib/librte_eal/linuxapp/eal/eal.c | 4 ++++ >>>> 3 files changed, 13 insertions(+) >>>> >>>> diff --git a/config/common_base b/config/common_base >>>> index 82ee75456..903e7685b 100644 >>>> --- a/config/common_base >>>> +++ b/config/common_base >>>> @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n >>>> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n >>>> >>>> # >>>> +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution >>>> +# >>>> +CONFIG_RTE_EAL_USE_PHYS_IOVA=y >>> Defeat the purpose of all dynamic probing scheme. >>> Either we can fix the KNI or revert the following patch for this release. >>> >>> http://dpdk.org/commit/f37dfab2 >> This commit just enables IOVA VA mode for Intel drivers, that is how I can able >> to observe the issue, but it is not the source of the problem. Reverting that >> commit will not solve KNI crash with any other PMD that enables IOVA VA mode. > I don't understand why a PMD needs to enable IOVA_VA if it can support IOVA_PA. > IMO, IOVA_VA should be enabled only for those device it can WORK ONLY on > IOVA_VA mode. IOVA_VA is also necessary if we don't want/have root privilege to run DPDK app. > Forget about KNI, If we set CONFIG_RTE_EAL_USE_PHYS_IOVA > as y then the normal stuff wont work for if PMD can operate only in > IOVA_VA mode(like octeontx). Yes, agreed. > Regarding the KNI crash, it can be avoid by first checking the exiting > mode(rte_eal_iova_mode()). i.e since legacy driver like KNI need real > physical address to work "now", it can grace full exit on the init time if > mode == IOVA_VA; For the KNI crash, do we really need to correct the case: make those PMDs (requires IOVA_VA) also work well with KNI? - If no, we just need to report RTE_PCI_DRV_IOVA_AS_VA smartly. - If yes, we shall make KNI work with VA. Thanks, Jianfeng