From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0DB18A00E6 for ; Mon, 5 Aug 2019 08:16:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DD8E21BE45; Mon, 5 Aug 2019 08:16:52 +0200 (CEST) Received: from mail-vs1-f67.google.com (mail-vs1-f67.google.com [209.85.217.67]) by dpdk.org (Postfix) with ESMTP id 4028A1BE44 for ; Mon, 5 Aug 2019 08:16:52 +0200 (CEST) Received: by mail-vs1-f67.google.com with SMTP id m8so55294314vsj.0 for ; Sun, 04 Aug 2019 23:16:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9WJD1yGvHJiDQxFiboqIRlssoxq99s3kFY7X5O1wh/8=; b=gDsPIdyN1pgKgTzSMaPorJUPFQAhEk3R7vD0csBgijvAzrycB2surBc+k/fDdQ5WVj 1TSbLJEFgjf33ZUrg54jPfIeFkCPI4oLE6TiXd4VVkj9QaiOQzcEBUFs/IriKtbfCWIa n9jjXKEMCdLlGU+cguB/CP5BFOOCO3GW/k+nVIChY+VyTLxL/6PTNq7960IquL9ZnV2X obCK9ex91nxDpcfkKImzfC3caKlXTtg1sswdaUMCFPwfdfHL3hV4QsToQ7oMJ2ovcIKw EuD2okYzFPrEdjEKzkyZ0pirzUWJwZWA9wiJ6ULED9Q1xH7UhFL4mXRdEOyB5zw/PdDf k45A== X-Gm-Message-State: APjAAAVefSHNC6evo0EaS2gJ+zrfwbJ9QNSXYhNRkW2I2+xMvOMBZ2zU 0+1MVmWB5Ec5KztHw60xQc0PAqGYSXAl5D4FTcmaYw== X-Google-Smtp-Source: APXvYqwT4t+WPAzX5ljqO7yGsaLuov8/1uYdy2Xq8Msn2qheOK+rCFzv/Te/mbB1x4uacx35OFqX5rv5UADG7eDgojQ= X-Received: by 2002:a67:2ec8:: with SMTP id u191mr97012603vsu.39.1564985811555; Sun, 04 Aug 2019 23:16:51 -0700 (PDT) MIME-Version: 1.0 References: <1564740872-27225-1-git-send-email-david.marchand@redhat.com> In-Reply-To: From: David Marchand Date: Mon, 5 Aug 2019 08:16:40 +0200 Message-ID: To: Takeshi T Yoshimura Cc: dev , "Burakov, Anatoly" , David Christensen , Jerin Jacob Kollanukkaran Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH] bus/pci: always check IOMMU capabilities X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Aug 5, 2019 at 5:57 AM Takeshi T Yoshimura wrote: > > -----David Marchand wrote: ----- > > >To: dev@dpdk.org > >From: David Marchand > >Date: 08/02/2019 07:14PM > >Cc: anatoly.burakov@intel.com, tyos@jp.ibm.com, > >drc@linux.vnet.ibm.com > >Subject: [PATCH] bus/pci: always check IOMMU capabilities > > > >IOMMU capabilities won't change and must be checked even if no PCI > >device > >seem to be supported yet when EAL initialised. > > > >This is to accommodate with SPDK that registers its drivers after > >rte_eal_init(), especially on PPC platform where the IOMMU does not > >support VA. > > > >Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA > >mode") > > > >Signed-off-by: David Marchand > >--- > > drivers/bus/pci/bsd/pci.c | 6 ++++++ > > drivers/bus/pci/linux/pci.c | 25 ++++++------------------- > > drivers/bus/pci/pci_common.c | 16 +++++++++++++++- > > drivers/bus/pci/private.h | 5 ++++- > > 4 files changed, 31 insertions(+), 21 deletions(-) > > > >diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c > >index a2de709..8f07ed9 100644 > >--- a/drivers/bus/pci/bsd/pci.c > >+++ b/drivers/bus/pci/bsd/pci.c > >@@ -376,6 +376,12 @@ error: > > return -1; > > } > > > >+bool > >+pci_device_iommu_support_va(__rte_unused const struct rte_pci_device > >*dev) > >+{ > >+ return false; > >+} > >+ > > enum rte_iova_mode > > pci_device_iova_mode(const struct rte_pci_driver *pdrv __rte_unused, > > const struct rte_pci_device *pdev) > >diff --git a/drivers/bus/pci/linux/pci.c > >b/drivers/bus/pci/linux/pci.c > >index f4fb742..43debaa 100644 > >--- a/drivers/bus/pci/linux/pci.c > >+++ b/drivers/bus/pci/linux/pci.c > >@@ -498,8 +498,8 @@ error: > > } > > > > #if defined(RTE_ARCH_X86) > >-static bool > >-pci_one_device_iommu_support_va(const struct rte_pci_device *dev) > >+bool > >+pci_device_iommu_support_va(const struct rte_pci_device *dev) > > { > > #define VTD_CAP_MGAW_SHIFT 16 > > #define VTD_CAP_MGAW_MASK (0x3fULL << VTD_CAP_MGAW_SHIFT) > >@@ -546,14 +546,14 @@ pci_one_device_iommu_support_va(const struct > >rte_pci_device *dev) > > return true; > > } > > #elif defined(RTE_ARCH_PPC_64) > >-static bool > >-pci_one_device_iommu_support_va(__rte_unused const struct > >rte_pci_device *dev) > >+bool > >+pci_device_iommu_support_va(__rte_unused const struct rte_pci_device > >*dev) > > { > > return false; > > } > > #else > >-static bool > >-pci_one_device_iommu_support_va(__rte_unused const struct > >rte_pci_device *dev) > >+bool > >+pci_device_iommu_support_va(__rte_unused const struct rte_pci_device > >*dev) > > { > > return true; > > } > >@@ -564,7 +564,6 @@ pci_device_iova_mode(const struct rte_pci_driver > >*pdrv, > > const struct rte_pci_device *pdev) > > { > > enum rte_iova_mode iova_mode = RTE_IOVA_DC; > >- static int iommu_no_va = -1; > > > > switch (pdev->kdrv) { > > case RTE_KDRV_VFIO: { > >@@ -595,18 +594,6 @@ pci_device_iova_mode(const struct rte_pci_driver > >*pdrv, > > iova_mode = RTE_IOVA_VA; > > break; > > } > >- > >- if (iova_mode != RTE_IOVA_PA) { > >- /* > >- * We can check this only once, because the IOMMU hardware is > >- * the same for all of them. > >- */ > >- if (iommu_no_va == -1) > >- iommu_no_va = pci_one_device_iommu_support_va(pdev) > >- ? 0 : 1; > >- if (iommu_no_va != 0) > >- iova_mode = RTE_IOVA_PA; > >- } > > return iova_mode; > > } > > > >diff --git a/drivers/bus/pci/pci_common.c > >b/drivers/bus/pci/pci_common.c > >index 9794552..8d1d6ab 100644 > >--- a/drivers/bus/pci/pci_common.c > >+++ b/drivers/bus/pci/pci_common.c > >@@ -616,8 +616,16 @@ rte_pci_get_iommu_class(void) > > const struct rte_pci_driver *drv; > > bool devices_want_va = false; > > bool devices_want_pa = false; > >+ static int iommu_no_va = -1; > > > > FOREACH_DEVICE_ON_PCIBUS(dev) { > >+ /* > >+ * We can check this only once, because the IOMMU hardware is > >+ * the same for all of them. > >+ */ > >+ if (iommu_no_va == -1) > >+ iommu_no_va = pci_device_iommu_support_va(dev) > >+ ? 0 : 1; > > if (pci_ignore_device(dev)) > > continue; > > if (dev->kdrv == RTE_KDRV_UNKNOWN || > >@@ -643,7 +651,13 @@ rte_pci_get_iommu_class(void) > > devices_want_va = true; > > } > > } > >- if (devices_want_va && !devices_want_pa) { > >+ if (iommu_no_va == 1) { > >+ iova_mode = RTE_IOVA_PA; > >+ if (devices_want_va) { > >+ RTE_LOG(WARNING, EAL, "Some devices want 'VA' but because IOMMU > >does not support 'VA'.\n"); > >+ RTE_LOG(WARNING, EAL, "The devices that want 'VA' won't > >initialize.\n"); > >+ } > >+ } else if (devices_want_va && !devices_want_pa) { > > iova_mode = RTE_IOVA_VA; > > } else if (devices_want_pa && !devices_want_va) { > > iova_mode = RTE_IOVA_PA; > >diff --git a/drivers/bus/pci/private.h b/drivers/bus/pci/private.h > >index 8a55240..a205d4d 100644 > >--- a/drivers/bus/pci/private.h > >+++ b/drivers/bus/pci/private.h > >@@ -173,9 +173,12 @@ rte_pci_match(const struct rte_pci_driver > >*pci_drv, > > const struct rte_pci_device *pci_dev); > > > > /** > >- * OS specific callback for rte_pci_get_iommu_class > >+ * OS specific callbacks for rte_pci_get_iommu_class > > * > > */ > >+bool > >+pci_device_iommu_support_va(const struct rte_pci_device *dev); > >+ > > enum rte_iova_mode > > pci_device_iova_mode(const struct rte_pci_driver *pci_drv, > > const struct rte_pci_device *pci_dev); > >-- > >1.8.3.1 > > > > > > Tested-by: Takeshi Yoshimura Thank you all. I'll send a v2 with the comment from Jerin and the log message fix. -- David Marchand