From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id EAB74A0487 for ; Wed, 3 Jul 2019 12:45:43 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B0D891E25; Wed, 3 Jul 2019 12:45:43 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 6025B2AB for ; Wed, 3 Jul 2019 12:45:41 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Jul 2019 03:45:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,446,1557212400"; d="scan'208";a="157930849" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.126]) ([10.237.220.126]) by orsmga008.jf.intel.com with ESMTP; 03 Jul 2019 03:45:37 -0700 To: David Marchand , dev@dpdk.org Cc: benjamin.walker@intel.com, jerinj@marvell.com, maxime.coquelin@redhat.com, thomas@monjalon.net References: <20190530174819.1160221-1-benjamin.walker@intel.com> <1560505157-9769-1-git-send-email-david.marchand@redhat.com> <1560505157-9769-4-git-send-email-david.marchand@redhat.com> From: "Burakov, Anatoly" Message-ID: <4fabfd5f-2ba9-ff45-59dd-cfd01b8d49d5@intel.com> Date: Wed, 3 Jul 2019 11:45:36 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <1560505157-9769-4-git-send-email-david.marchand@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2 3/3] bus/pci: only consider usable devices to select IOVA mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 14-Jun-19 10:39 AM, David Marchand wrote: > From: Ben Walker > > When selecting the preferred IOVA mode of the pci bus, the current > heuristic ("are devices bound?", "are devices bound to UIO?", "are pmd > drivers supporting IOVA as VA?" etc..) should honor the device > white/blacklist so that an unwanted device does not impact the decision. > > There is no reason to consider a device which has no driver available. > > This applies to all OS, so implements this in common code then call a > OS specific callback. > > On Linux side: > - the VFIO special considerations should be evaluated only if VFIO > support is built, > - there is no strong requirement on using VA rather than PA if a driver > supports VA, so defaulting to DC in such a case. > > Signed-off-by: Ben Walker > Signed-off-by: David Marchand > --- > + const struct rte_pci_device *pdev) > { > - struct rte_pci_device *dev = NULL; > - struct rte_pci_driver *drv = NULL; > + enum rte_iova_mode iova_mode = RTE_IOVA_DC; > + static int iommu_no_va = -1; > > - FOREACH_DRIVER_ON_PCIBUS(drv) { > - FOREACH_DEVICE_ON_PCIBUS(dev) { > - if (!rte_pci_match(drv, dev)) > - continue; > - /* > - * just one PCI device needs to be checked out because > - * the IOMMU hardware is the same for all of them. > - */ > - return pci_one_device_iommu_support_va(dev); > + switch (pdev->kdrv) { > + case RTE_KDRV_VFIO: { > +#ifdef VFIO_PRESENT > + static int is_vfio_noiommu_enabled = -1; > + > + if (is_vfio_noiommu_enabled == -1) { > + if (rte_vfio_noiommu_is_enabled() == 1) > + is_vfio_noiommu_enabled = 1; > + else > + is_vfio_noiommu_enabled = 0; > + } > + if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) { > + iova_mode = RTE_IOVA_PA; > + } else if (is_vfio_noiommu_enabled != 0) { > + RTE_LOG(DEBUG, EAL, "Forcing to 'PA', vfio-noiommu mode configured\n"); > + iova_mode = RTE_IOVA_PA; > } > +#endif > + break; I'm not too well-versed in bus code, so please excuse my ignorance of this codebase. It seems that we would be ignoring drv_flags in case VFIO wasn't compiled - if the driver has no RTE_PCI_DRV_IOVA_AS_VA flag, i'm pretty sure we can set IOVA mode to PA without caring about VFIO at all. I think it would be better to have something like this: if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) { iova_mode = RTE_IOVA_PA; break; // early exit } #ifdef VFIO_PRESENT static int is_vfio_noiommu_enabled = -1; if (is_vfio_noiommu_enabled == -1) { if (rte_vfio_noiommu_is_enabled() == 1) is_vfio_noiommu_enabled = 1; else is_vfio_noiommu_enabled = 0; } if (is_vfio_noiommu_enabled != 0) { iova_mode = RTE_IOVA_PA; } #endif break; In fact, could we not check if devices support both flags, and do an early exit in case they don't? Something like this, for example: if ((pdrv->drv_flags & RTE_PCI_DRV_IOVA_AS_VA) == 0) { return RTE_IOVA_PA; // early exit - device only wants PA } // device supports both PA and VA modes, so do more checks switch (pdev->kdrv) { ... } Unless i'm missing something, this would look much simpler and easier to understand. -- Thanks, Anatoly