From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 40569A00E6 for ; Mon, 5 Aug 2019 08:23:39 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C43B91BE46; Mon, 5 Aug 2019 08:23:38 +0200 (CEST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 607911BE3B for ; Mon, 5 Aug 2019 08:23:37 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8F00630FB8C1; Mon, 5 Aug 2019 06:23:36 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-235.brq.redhat.com [10.40.204.235]) by smtp.corp.redhat.com (Postfix) with ESMTP id 907A319C6A; Mon, 5 Aug 2019 06:23:34 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: anatoly.burakov@intel.com, tyos@jp.ibm.com, drc@linux.vnet.ibm.com, jerinj@marvell.com Date: Mon, 5 Aug 2019 08:23:26 +0200 Message-Id: <1564986207-20533-1-git-send-email-david.marchand@redhat.com> In-Reply-To: <1564740872-27225-1-git-send-email-david.marchand@redhat.com> References: <1564740872-27225-1-git-send-email-david.marchand@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Mon, 05 Aug 2019 06:23:36 +0000 (UTC) Subject: [dpdk-dev] [PATCH v2] bus/pci: always check IOMMU capabilities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" IOMMU capabilities won't change and must be checked even if no PCI device seem to be supported yet when EAL initialised. This is to accommodate with SPDK that registers its drivers after rte_eal_init(), especially on PPC platform where the IOMMU does not support VA. Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode") Signed-off-by: David Marchand Reviewed-by: David Christensen Acked-by: Jerin Jacob Tested-by: Jerin Jacob Tested-by: Takeshi Yoshimura --- Changelog since v1: - fixed log message - changed static global variable iommu_no_va as a local variable --- drivers/bus/pci/bsd/pci.c | 6 ++++++ drivers/bus/pci/linux/pci.c | 25 ++++++------------------- drivers/bus/pci/pci_common.c | 16 +++++++++++++++- drivers/bus/pci/private.h | 5 ++++- 4 files changed, 31 insertions(+), 21 deletions(-) diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c index a2de709..8f07ed9 100644 --- a/drivers/bus/pci/bsd/pci.c +++ b/drivers/bus/pci/bsd/pci.c @@ -376,6 +376,12 @@ error: return -1; } +bool +pci_device_iommu_support_va(__rte_unused const struct rte_pci_device *dev) +{ + return false; +} + enum rte_iova_mode pci_device_iova_mode(const struct rte_pci_driver *pdrv __rte_unused, const struct rte_pci_device *pdev) diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index f4fb742..43debaa 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -498,8 +498,8 @@ error: } #if defined(RTE_ARCH_X86) -static bool -pci_one_device_iommu_support_va(const struct rte_pci_device *dev) +bool +pci_device_iommu_support_va(const struct rte_pci_device *dev) { #define VTD_CAP_MGAW_SHIFT 16 #define VTD_CAP_MGAW_MASK (0x3fULL << VTD_CAP_MGAW_SHIFT) @@ -546,14 +546,14 @@ pci_one_device_iommu_support_va(const struct rte_pci_device *dev) return true; } #elif defined(RTE_ARCH_PPC_64) -static bool -pci_one_device_iommu_support_va(__rte_unused const struct rte_pci_device *dev) +bool +pci_device_iommu_support_va(__rte_unused const struct rte_pci_device *dev) { return false; } #else -static bool -pci_one_device_iommu_support_va(__rte_unused const struct rte_pci_device *dev) +bool +pci_device_iommu_support_va(__rte_unused const struct rte_pci_device *dev) { return true; } @@ -564,7 +564,6 @@ pci_device_iova_mode(const struct rte_pci_driver *pdrv, const struct rte_pci_device *pdev) { enum rte_iova_mode iova_mode = RTE_IOVA_DC; - static int iommu_no_va = -1; switch (pdev->kdrv) { case RTE_KDRV_VFIO: { @@ -595,18 +594,6 @@ pci_device_iova_mode(const struct rte_pci_driver *pdrv, iova_mode = RTE_IOVA_VA; break; } - - if (iova_mode != RTE_IOVA_PA) { - /* - * We can check this only once, because the IOMMU hardware is - * the same for all of them. - */ - if (iommu_no_va == -1) - iommu_no_va = pci_one_device_iommu_support_va(pdev) - ? 0 : 1; - if (iommu_no_va != 0) - iova_mode = RTE_IOVA_PA; - } return iova_mode; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 9794552..6b46b4f 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -616,8 +616,16 @@ rte_pci_get_iommu_class(void) const struct rte_pci_driver *drv; bool devices_want_va = false; bool devices_want_pa = false; + int iommu_no_va = -1; FOREACH_DEVICE_ON_PCIBUS(dev) { + /* + * We can check this only once, because the IOMMU hardware is + * the same for all of them. + */ + if (iommu_no_va == -1) + iommu_no_va = pci_device_iommu_support_va(dev) + ? 0 : 1; if (pci_ignore_device(dev)) continue; if (dev->kdrv == RTE_KDRV_UNKNOWN || @@ -643,7 +651,13 @@ rte_pci_get_iommu_class(void) devices_want_va = true; } } - if (devices_want_va && !devices_want_pa) { + if (iommu_no_va == 1) { + iova_mode = RTE_IOVA_PA; + if (devices_want_va) { + RTE_LOG(WARNING, EAL, "Some devices want 'VA' but IOMMU does not support 'VA'.\n"); + RTE_LOG(WARNING, EAL, "The devices that want 'VA' won't initialize.\n"); + } + } else if (devices_want_va && !devices_want_pa) { iova_mode = RTE_IOVA_VA; } else if (devices_want_pa && !devices_want_va) { iova_mode = RTE_IOVA_PA; diff --git a/drivers/bus/pci/private.h b/drivers/bus/pci/private.h index 8a55240..a205d4d 100644 --- a/drivers/bus/pci/private.h +++ b/drivers/bus/pci/private.h @@ -173,9 +173,12 @@ rte_pci_match(const struct rte_pci_driver *pci_drv, const struct rte_pci_device *pci_dev); /** - * OS specific callback for rte_pci_get_iommu_class + * OS specific callbacks for rte_pci_get_iommu_class * */ +bool +pci_device_iommu_support_va(const struct rte_pci_device *dev); + enum rte_iova_mode pci_device_iova_mode(const struct rte_pci_driver *pci_drv, const struct rte_pci_device *pci_dev); -- 1.8.3.1