From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 29A122A58 for ; Wed, 5 Jul 2017 10:17:27 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1926530AF50; Wed, 5 Jul 2017 08:17:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 1926530AF50 Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx05.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=maxime.coquelin@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 1926530AF50 Received: from [10.36.112.37] (ovpn-112-37.ams2.redhat.com [10.36.112.37]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 32BA55C549; Wed, 5 Jul 2017 08:17:22 +0000 (UTC) To: Santosh Shukla , thomas@monjalon.net, bruce.richardson@intel.com, dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com References: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> <20170608110513.22548-3-santosh.shukla@caviumnetworks.com> From: Maxime Coquelin Message-ID: <6d3b43e4-3e49-5c20-5be5-e380f0a807d6@redhat.com> Date: Wed, 5 Jul 2017 10:17:20 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <20170608110513.22548-3-santosh.shukla@caviumnetworks.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 05 Jul 2017 08:17:26 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH 02/10] linuxapp/eal_pci: get iommu class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jul 2017 08:17:27 -0000 On 06/08/2017 01:05 PM, Santosh Shukla wrote: > Get iommu class of PCI device on the bus and returns preferred iova > mapping mode for that bus. > > IOVA mapping scheme for linuxapp case: > - uio/uio_generic/vfio_noiommu --> default i.e.. (RTE_IOVA_PA) > - vfio --> RTE_IOVA_VA. > - In case of no device attached to any driver, > return RTE_IOVA_DC. > > Signed-off-by: Santosh Shukla > Signed-off-by: Jerin Jacob > --- > lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_vfio.c | 23 +++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 +++ > lib/librte_eal/linuxapp/eal/rte_eal_version.map | 7 +++++ > 4 files changed, 72 insertions(+) > > diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c > index 595622b21..2772e883e 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_pci.c > +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c > @@ -45,6 +45,7 @@ > #include "eal_filesystem.h" > #include "eal_private.h" > #include "eal_pci_init.h" > +#include "eal_vfio.h" > > /** > * @file > @@ -488,6 +489,43 @@ rte_pci_scan(void) > return -1; > } > > +/* > + * Get iommu class of PCI devices on the bus. > + * Check that those devices are attached to iommu driver. > + * If attached then return iova_va or iova_pa mode, else > + * return with dont_care(_DC). > + */ > +enum rte_iova_mode > +rte_pci_get_iommu_class(void) > +{ > + struct rte_pci_device *dev = NULL; > + int ret = RTE_IOVA_DC; > + > + TAILQ_FOREACH(dev, &rte_pci_bus.device_list, next) { > + > + if (dev->kdrv == RTE_KDRV_UNKNOWN || > + dev->kdrv == RTE_KDRV_NONE) > + continue; > + > + if (dev->kdrv != RTE_KDRV_VFIO) { > + ret = RTE_IOVA_PA; > + return ret; > + } > + > + ret = RTE_IOVA_VA; > + } > + > + /* In case of iova_va, check for vfio_noiommu mode */ > + if (ret == RTE_IOVA_VA) { > +#ifdef VFIO_PRESENT > + if (vfio_noiommu_is_enabled() == 1) > +#endif > + ret = RTE_IOVA_PA; > + } > + > + return ret; > +} > + > /* Read PCI config space. */ > int rte_pci_read_config(const struct rte_pci_device *device, > void *buf, size_t len, off_t offset) > diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c > index 946df7e31..04914406f 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c > +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c > @@ -816,4 +816,27 @@ vfio_noiommu_dma_map(int __rte_unused vfio_container_fd) > return 0; > } > > +int > +vfio_noiommu_is_enabled(void) > +{ > +#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 2, 0) Please don't check against Kernel version, as distros may have backported the feature on older Kernels versions (RHEL for exmaple). Also, it doesn't look necessary since open should fail below on older kernels. > + return -1; > +#else > + int fd, ret, cnt __rte_unused; > + char c; > + > + ret = -1; > + fd = open(VFIO_NOIOMMU_MODE, O_RDONLY); > + if (fd < 0) > + return -1; > + > + cnt = read(fd, &c, 1); > + if (c == 'Y') > + ret = 1; > + > + close(fd); > + return ret; > +#endif > +} > + > #endif > diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.h b/lib/librte_eal/linuxapp/eal/eal_vfio.h > index 5ff63e5d7..26ea8e119 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_vfio.h > +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.h > @@ -150,6 +150,8 @@ struct vfio_config { > #define VFIO_NOIOMMU_GROUP_FMT "/dev/vfio/noiommu-%u" > #define VFIO_GET_REGION_ADDR(x) ((uint64_t) x << 40ULL) > #define VFIO_GET_REGION_IDX(x) (x >> 40) > +#define VFIO_NOIOMMU_MODE \ > + "/sys/module/vfio/parameters/enable_unsafe_noiommu_mode" > > /* DMA mapping function prototype. > * Takes VFIO container fd as a parameter. > @@ -210,6 +212,8 @@ int pci_vfio_is_enabled(void); > > int vfio_mp_sync_setup(void); > > +int vfio_noiommu_is_enabled(void); > + > #define SOCKET_REQ_CONTAINER 0x100 > #define SOCKET_REQ_GROUP 0x200 > #define SOCKET_CLR_GROUP 0x300 > diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map > index 670bab3a5..2cea7c272 100644 > --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map > +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map > @@ -198,3 +198,10 @@ DPDK_17.05 { > vfio_get_group_no; > > } DPDK_17.02; > + > +DPDK_17.08 { > + global: > + > + rte_pci_get_iommu_class; > + > +} DPDK_17.05; >