From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 2B8252C2F for ; Tue, 5 Sep 2017 10:55:26 +0200 (CEST) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 05 Sep 2017 01:55:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.41,479,1498546800"; d="scan'208";a="147565234" Received: from irsmsx105.ger.corp.intel.com ([163.33.3.28]) by fmsmga005.fm.intel.com with ESMTP; 05 Sep 2017 01:55:14 -0700 Received: from irsmsx109.ger.corp.intel.com ([169.254.13.28]) by irsmsx105.ger.corp.intel.com ([169.254.7.75]) with mapi id 14.03.0319.002; Tue, 5 Sep 2017 09:55:13 +0100 From: "Burakov, Anatoly" To: santosh , "dev@dpdk.org" CC: "thomas@monjalon.net" , "jerin.jacob@caviumnetworks.com" , "hemant.agrawal@nxp.com" , "olivier.matz@6wind.com" , "maxime.coquelin@redhat.com" , "Gonzalez Monroy, Sergio" , "Richardson, Bruce" , "shreyansh.jain@nxp.com" , "gaetan.rivet@6wind.com" , "stephen@networkplumber.org" , "aconole@redhat.com" Thread-Topic: [PATCH v7 3/9] linuxapp/eal_pci: get iommu class Thread-Index: AQHTIgkLOTlx343FLkul6A75jwg/aKKk2NWAgAEZjYCAABKQ8A== Date: Tue, 5 Sep 2017 08:55:13 +0000 Message-ID: References: <20170814161059.6684-1-santosh.shukla@caviumnetworks.com> <20170831032618.7120-1-santosh.shukla@caviumnetworks.com> <20170831032618.7120-4-santosh.shukla@caviumnetworks.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_IC x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZGJiZTAzN2EtMzgzNC00M2E4LWFlNjUtYWE1NWI3ZjQ3MmFhIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6Inp3TXFnVHhMQzBGaEc0ZWxFMzVPZHBMd0t6Vjlpd2RqcGNLNEVuN0w1bFE9In0= dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v7 3/9] linuxapp/eal_pci: get iommu class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 08:55:26 -0000 > From: santosh [mailto:santosh.shukla@caviumnetworks.com] > Sent: Tuesday, September 5, 2017 9:48 AM > To: Burakov, Anatoly ; dev@dpdk.org > Cc: thomas@monjalon.net; jerin.jacob@caviumnetworks.com; > hemant.agrawal@nxp.com; olivier.matz@6wind.com; > maxime.coquelin@redhat.com; Gonzalez Monroy, Sergio > ; Richardson, Bruce > ; shreyansh.jain@nxp.com; > gaetan.rivet@6wind.com; stephen@networkplumber.org; > aconole@redhat.com > Subject: Re: [PATCH v7 3/9] linuxapp/eal_pci: get iommu class >=20 > Hi Anatoly, >=20 >=20 > On Monday 04 September 2017 08:38 PM, Burakov, Anatoly wrote: > >> From: Santosh Shukla [mailto:santosh.shukla@caviumnetworks.com] > >> Sent: Thursday, August 31, 2017 4:26 AM > >> To: dev@dpdk.org > >> Cc: thomas@monjalon.net; jerin.jacob@caviumnetworks.com; > >> hemant.agrawal@nxp.com; olivier.matz@6wind.com; > >> maxime.coquelin@redhat.com; Gonzalez Monroy, Sergio > >> ; Richardson, Bruce > >> ; shreyansh.jain@nxp.com; > >> gaetan.rivet@6wind.com; Burakov, Anatoly > ; > >> stephen@networkplumber.org; aconole@redhat.com; Santosh Shukla > >> > >> Subject: [PATCH v7 3/9] linuxapp/eal_pci: get iommu class > >> > >> Get iommu class of PCI device on the bus and returns preferred iova > >> mapping mode for that bus. > >> > >> Patch also introduces RTE_PCI_DRV_IOVA_AS_VA drv flag. > >> Flag used when driver needs to operate in iova=3Dva mode. > >> > >> Algorithm for iova scheme selection for PCI bus: > >> 0. If no device bound then return with RTE_IOVA_DC mapping mode, else > >> goto 1). > >> 1. Look for device attached to vfio kdrv and has .drv_flag set to > >> RTE_PCI_DRV_IOVA_AS_VA. > >> 2. Look for any device attached to UIO class of driver. > >> 3. Check for vfio-noiommu mode enabled. > >> > >> If 2) & 3) is false and 1) is true then select mapping scheme as > RTE_IOVA_VA. > >> Otherwise use default mapping scheme (RTE_IOVA_PA). > >> > >> Signed-off-by: Santosh Shukla > >> Signed-off-by: Jerin Jacob > >> Reviewed-by: Maxime Coquelin > >> Acked-by: Hemant Agrawal > >> --- > >> v6 --> v7: > >> - squashed v6 series patch no [01/12] & [05/12].. > >> i.e.. moved RTE_PCI_DRV_IOVA_AS_VA flag into this patch. (Aaron > >> comment). > >> > >> lib/librte_eal/common/include/rte_pci.h | 2 + > >> lib/librte_eal/linuxapp/eal/eal_pci.c | 95 > >> +++++++++++++++++++++++++ > >> lib/librte_eal/linuxapp/eal/eal_vfio.c | 19 +++++ > >> lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ > >> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 1 + > >> 5 files changed, 121 insertions(+) > >> > >> diff --git a/lib/librte_eal/common/include/rte_pci.h > >> b/lib/librte_eal/common/include/rte_pci.h > >> index 0e36de093..a67d77f22 100644 > >> --- a/lib/librte_eal/common/include/rte_pci.h > >> +++ b/lib/librte_eal/common/include/rte_pci.h > >> @@ -202,6 +202,8 @@ struct rte_pci_bus { #define > >> RTE_PCI_DRV_INTR_RMV 0x0010 > >> /** Device driver needs to keep mapped resources if unsupported dev > >> detected */ #define RTE_PCI_DRV_KEEP_MAPPED_RES 0x0020 > >> +/** Device driver supports iova as va */ #define > >> RTE_PCI_DRV_IOVA_AS_VA > >> +0X0040 > >> > >> /** > >> * A structure describing a PCI mapping. > >> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c > >> b/lib/librte_eal/linuxapp/eal/eal_pci.c > >> index 8951ce742..9725fd493 100644 > >> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > >> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > >> @@ -45,6 +45,7 @@ > >> #include "eal_filesystem.h" > >> #include "eal_private.h" > >> #include "eal_pci_init.h" > >> +#include "eal_vfio.h" > >> > >> /** > >> * @file > >> @@ -487,6 +488,100 @@ rte_pci_scan(void) > >> return -1; > >> } > >> > >> +/* > >> + * Is pci device bound to any kdrv > >> + */ > >> +static inline int > >> +pci_device_is_bound(void) > >> +{ > >> + struct rte_pci_device *dev =3D NULL; > >> + int ret =3D 0; > >> + > >> + FOREACH_DEVICE_ON_PCIBUS(dev) { > >> + if (dev->kdrv =3D=3D RTE_KDRV_UNKNOWN || > >> + dev->kdrv =3D=3D RTE_KDRV_NONE) { > >> + continue; > >> + } else { > >> + ret =3D 1; > >> + break; > >> + } > >> + } > >> + return ret; > >> +} > >> + > >> +/* > >> + * Any one of the device bound to uio */ static inline int > >> +pci_device_bound_uio(void) > >> +{ > >> + struct rte_pci_device *dev =3D NULL; > >> + > >> + FOREACH_DEVICE_ON_PCIBUS(dev) { > >> + if (dev->kdrv =3D=3D RTE_KDRV_IGB_UIO || > >> + dev->kdrv =3D=3D RTE_KDRV_UIO_GENERIC) { > >> + return 1; > >> + } > >> + } > >> + return 0; > >> +} > >> + > >> +/* > >> + * Any one of the device has iova as va */ static inline int > >> +pci_device_has_iova_va(void) > >> +{ > >> + struct rte_pci_device *dev =3D NULL; > >> + struct rte_pci_driver *drv =3D NULL; > >> + > >> + FOREACH_DRIVER_ON_PCIBUS(drv) { > >> + if (drv && drv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) { > >> + FOREACH_DEVICE_ON_PCIBUS(dev) { > >> + if (dev->kdrv =3D=3D RTE_KDRV_VFIO && > >> + rte_pci_match(drv, dev)) > >> + return 1; > >> + } > >> + } > >> + } > >> + return 0; > >> +} > >> + > >> +/* > >> + * Get iommu class of PCI devices on the bus. > >> + */ > >> +enum rte_iova_mode > >> +rte_pci_get_iommu_class(void) > >> +{ > >> + bool is_bound; > >> + bool is_vfio_noiommu_enabled =3D true; > >> + bool has_iova_va; > >> + bool is_bound_uio; > >> + > >> + is_bound =3D pci_device_is_bound(); > >> + if (!is_bound) > >> + return RTE_IOVA_DC; > >> + > >> + has_iova_va =3D pci_device_has_iova_va(); > >> + is_bound_uio =3D pci_device_bound_uio(); #ifdef VFIO_PRESENT > >> + is_vfio_noiommu_enabled =3D vfio_noiommu_is_enabled() =3D=3D 1 ? 1 : > >> 0; > > If you specify is_vfio_noiommu_enabled as bool, you should probably tre= at > it as such, and assign true/false. >=20 > queued for v8. >=20 > > Other than that, I'm curious why is it always set to "true" by default?= If we > don't have VFIO compiled, it seems like the error message would always > complain about vfio-noiommu mode being enabled, which is confusing. >=20 > Set to 'true' for case when VFIO_PRESENT unset.. meaning platform doesn't > support VFIO (linux versioned < 3.6) i.e.. using UIO - In that case, flag= makes > sure _pa policy selected. >=20 > On error message: It won't come in non-vfio case, as 'has_iova_va' will s= et to > 0. > Error message will show for those case where few device out of many bind > to uio, so message will pop-up and iova policy would be _pa in that case. >=20 > Thanks. Right. My apologies, I misunderstood the meaning of "has_iova_va" flag. Thanks, Anatoly >=20 > > Thanks, > > Anatoly