From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 2EDE01B761 for ; Thu, 8 Feb 2018 12:20:19 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Feb 2018 03:20:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,478,1511856000"; d="scan'208";a="16935105" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.145]) ([10.237.220.145]) by orsmga006.jf.intel.com with ESMTP; 08 Feb 2018 03:20:17 -0800 To: Ravi Kerur Cc: dev@dpdk.org References: <8ddb30a3-1253-ff60-20bb-b735fef5a91c@intel.com> <10911b54-57ee-370b-a4f7-f34accf4811e@intel.com> <3464b900-8648-c128-e959-dc60a8883a2d@intel.com> <0120c68f-cf42-5d8b-5600-0514f76209b0@intel.com> <4e5ef551-b7b3-e12d-6254-b882bb952bbb@intel.com> From: "Burakov, Anatoly" Message-ID: Date: Thu, 8 Feb 2018 11:20:12 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] IXGBE, IOMMU DMAR DRHD handling fault issue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Feb 2018 11:20:20 -0000 On 06-Feb-18 5:55 PM, Ravi Kerur wrote: > > Hi Anatoly, > > I am actually confused with the state of vIOMMU + DPDK. Can you please > help me clarify? > > I tested following DPDK versions > > (1) DPDK 17.11, exhibits the issue (IOMMU width as reported by RedHat > and solution is to prevent using the patch) > (2) DPDK 17.05.02 (stable release) using 'testpmd' I was able to bind a > device in VM with VFIO driver and no DMAR error message on host > (3) DPDK 16.11.02 (stable release) using 'testpmd' I was able to bind a > device in VM with VFIO driver and no DMAR error message on host > > Clearly issue seen in 17.11 without the patch you mentioned is a > regression or the issue was masked in earlier DPDK version?  I did not > test traffic with any DPDK version because I wanted to first get DMAR > errors on host gone. > > Our application 'v1' is integrated with DPDK 16.11 and 'v2' is > integrated with DPDK 17.05.01. In both 'v1' and 'v2' cases I don't see > IOMMU width error messages on VM, however, DMAR error messages are seen > host. I am not able to relate what causes DMAR error messages on host? > > > Thanks. > Hi Ravi, vIOMMU support is out of our hands, really - we can only make use of hardware (or emulation of it) that is available. 39-bit wide address *can* work, you just have to be lucky and get PA addresses that would fit into 39 bits (so under 512G limit), because we set up IOVA addresses to be 1:1 to physical addresses. We could, in principle, set up IOVA addresses to go from zero instead of them being a 1:1 mapping to physical addresses, but that would introduce need to translate addresses between IOVA and physical in some cases (e.g. KNI). I'm not aware of any changes between 16.11 and 17.11 (and indeed 18.02) that would make or break support for 39-bit wide PA addresses for IOMMU. It is possible that VF/PF drivers do something differently which results in DMAR errors showing up sooner rather than later, but as far as VFIO support itself is concerned, there were no major changes in those releases. -- Thanks, Anatoly