From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 562F31B1B2; Mon, 8 Jan 2018 16:48:48 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7488978238; Mon, 8 Jan 2018 15:48:47 +0000 (UTC) Received: from [10.36.112.43] (ovpn-112-43.ams2.redhat.com [10.36.112.43]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 694B75D721; Mon, 8 Jan 2018 15:48:44 +0000 (UTC) To: Stephen Hemminger Cc: dev@dpdk.org, stable@dpdk.org, jianfeng.tan@intel.com, santosh.shukla@caviumnetworks.com, anatoly.burakov@intel.com, thomas@monjalon.net, peterx@redhat.com References: <20180108135127.25869-1-maxime.coquelin@redhat.com> <20180108073411.67f12201@xeon-e3> From: Maxime Coquelin Message-ID: Date: Mon, 8 Jan 2018 16:48:43 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <20180108073411.67f12201@xeon-e3> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Mon, 08 Jan 2018 15:48:47 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH] bus/pci: forbid VA as IOVA mode if IOMMU address width too small X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jan 2018 15:48:48 -0000 On 01/08/2018 04:34 PM, Stephen Hemminger wrote: > On Mon, 8 Jan 2018 14:51:27 +0100 > Maxime Coquelin wrote: > >> Intel VT-d supports different address widths for the IOVAs, from >> 39 bits to 56 bits. >> >> While recent processors support at least 48 bits, VT-d emulation >> currently only supports 39 bits. It makes DMA mapping to fail in this >> case when using VA as IOVA mode, as user-space virtual addresses uses >> up to 47 bits (see kernel's Documentation/x86/x86_64/mm.txt). >> >> This patch parses VT-d CAP register value available in sysfs, and >> forbid VA as IOVA mode if the GAW is 39 bits or unknown. >> >> Fixes: f37dfab21c98 ("drivers/net: enable IOVA mode for Intel PMDs") >> >> Cc: stable@dpdk.org >> Signed-off-by: Maxime Coquelin >> --- >> Hi, >> >> I'm not super happy with the patch as it does platform specific things in >> generic code, but there are no placeholder for IOMMU/VT-d at the moment. >> >> As this patch is to be backported to v17.11 LTS, it cannot be a big rework. >> >> If you have some suggestion to improve it, please let me know. >> >> The fix is quite urgent, as guest device assignment with vIOMMU is broken in >> mainline & v17.11 LTS. >> >> Advantage of this fix over forbidding VA as IOVA when running in emulation is >> that VT-d emulation will soon support 48 bits, so this is future proof. Also, >> VT-d spec supports 39 bits, so we could have physical CPUs supporting it, even >> if I don't know any. >> >> Thanks, >> Maxime > > You are assumming that if IOMMU is present that it is being used (ie VFIO). > What about the case of direct access to PF device via IGB_UIO? As soon as one device is bound to UIO or VFIO in noiomu mode, PA as IOVA mode will be selected. This is done in rte_pci_get_iommu_class(), by calling pci_one_device_bound_uio() and rte_vfio_noiommu_is_enabled(). >> +static inline bool >> +pci_one_device_iommu_support_va(struct rte_pci_device *dev) >> +{ > > This is not in fast path, there is no reason it should be inline > Ok, I will remove inlining in v2. I added it for consistency with the other functions declared above. Thanks, Maxime