From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by dpdk.org (Postfix) with ESMTP id 0E7BB4CB3 for ; Thu, 4 Oct 2018 15:35:12 +0200 (CEST) Received: by mail-ed1-f68.google.com with SMTP id v18-v6so7505746edq.12 for ; Thu, 04 Oct 2018 06:35:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=eT+WTl0eqy6ifr3l3Q1lYqonSFrdQsF0lR81fr8RDyA=; b=0rQyI23Xg9c7/bO480YCNQ19sGsyF7ruy19R5atZGsH77lOVKQTnd5GkBAVTJjqK7+ jZQE3yRTnsd3crWovNDXsyPYdGcm56tDlkncUV/EMVmZl3EQdewmaxjZbhAwYQLY/uJf 4Y71sDMT26X/hSTZu9dJHlOp0N0zLRoFCfnMw76vsTiPiBzEjMbUwLF7cZLuOaJvWIRF w0NR1w/Bx20GT2KvisEkVAhdnuJtNBTrvHDZEuyK3/uMmxEI5wms/26jk6y4UPYT1CvH YdOC9uAasM0YUDRtECb8TkOdNpxvFf3xVV3jgAKw54YtvLYszorB1vNwcdT6xsVPTkqV ANaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=eT+WTl0eqy6ifr3l3Q1lYqonSFrdQsF0lR81fr8RDyA=; b=EWc+GE9iJ5u2DaqAFXaHxFPedQl67sOQhGG2tVb7uwt7gBZ0ocmQ1i4+fTWYABIM+6 O3rbqmQZJmvv8OSXpLCjSNLBSfXkUkpYOlZOXsid/HrfqRJD8UZ+c86hzcEV7ktmB+NR wt9fOV24e3gJW+Sx24A3l1FYya1f4XZnQUx6bN5Da8fbpMO3eYzjZy5JCkov+7sQi0XE 0ml+Z1aVsN846xNxILVdmCPzuKvZvVE/kyrHaFDbAe9SvrebO0iOrr/Ra8/+dw2G/tqr fHM1ziBoHgSOZcc3YA2tFguU34ym+18i3Kv9MZC6TwtxWkJtwy/hXBY2YM4QkXR0tWuD vRSA== X-Gm-Message-State: ABuFfoigfUUCgshkDduRWe1tvJkUW24hvDrX3A+p3OmtH/T5Te8WB549 kACIurXOheovsHr6weOmJg+m45X4fvMR40E/Jii4mQ== X-Google-Smtp-Source: ACcGV61t1PtplC4OeGUBbLZTYOzk6q0X3Pu4gefVqG99SoQMcUVNxnmxmhUBV22skLfQXD/x2+4EqJtv9jWJBKZDKGg= X-Received: by 2002:a50:ef03:: with SMTP id m3-v6mr4001293eds.136.1538660111760; Thu, 04 Oct 2018 06:35:11 -0700 (PDT) MIME-Version: 1.0 References: <1535719857-19092-1-git-send-email-alejandro.lucero@netronome.com> <1535719857-19092-4-git-send-email-alejandro.lucero@netronome.com> <6b0b9159-d25b-56aa-0ca0-dd61b5206b04@intel.com> In-Reply-To: <6b0b9159-d25b-56aa-0ca0-dd61b5206b04@intel.com> From: Alejandro Lucero Date: Thu, 4 Oct 2018 14:35:00 +0100 Message-ID: To: "Burakov, Anatoly" , dev , Maxime Coquelin Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v2 3/5] bus/pci: use IOVAs check when setting IOVA mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2018 13:35:12 -0000 On Wed, Oct 3, 2018 at 1:56 PM Burakov, Anatoly wrote: > On 31-Aug-18 1:50 PM, Alejandro Lucero wrote: > > Although VT-d emulation currently only supports 39 bits, it could > > be iovas being within that supported range. This patch allows > > IOVA mode in such a case. > > > > Indeed, memory initialization code can be modified for using lower > > virtual addresses than those used by the kernel for 64 bits processes > > by default, and therefore memsegs iovas can use 39 bits or less for > > most system. And this is likely 100% true for VMs. > > > > Signed-off-by: Alejandro Lucero > > --- > > drivers/bus/pci/linux/pci.c | 15 ++++++++++----- > > 1 file changed, 10 insertions(+), 5 deletions(-) > > > > diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c > > index 04648ac..215dc10 100644 > > --- a/drivers/bus/pci/linux/pci.c > > +++ b/drivers/bus/pci/linux/pci.c > > @@ -588,10 +588,11 @@ > > fclose(fp); > > > > mgaw = ((vtd_cap_reg & VTD_CAP_MGAW_MASK) >> VTD_CAP_MGAW_SHIFT) + > 1; > > - if (mgaw < X86_VA_WIDTH) > > - return false; > > > > - return true; > > + if (!rte_eal_check_dma_mask(mgaw)) > > + return true; > > + else > > + return false; > > return rte_eal_check_dma_mask(mgaw) == 0; ? > I guess that works and is more elegant. Thanks. > > > } > > #elif defined(RTE_ARCH_PPC_64) > > static bool > > @@ -615,13 +616,17 @@ > > { > > struct rte_pci_device *dev = NULL; > > struct rte_pci_driver *drv = NULL; > > + int iommu_dma_mask_check_done = 0; > > > > FOREACH_DRIVER_ON_PCIBUS(drv) { > > FOREACH_DEVICE_ON_PCIBUS(dev) { > > if (!rte_pci_match(drv, dev)) > > continue; > > - if (!pci_one_device_iommu_support_va(dev)) > > - return false; > > + if (!iommu_dma_mask_check_done) { > > + if (!pci_one_device_iommu_support_va(dev)) > > + return false; > > + iommu_dma_mask_check_done = 1; > > + } > > } > > The commit message doesn't explain why are we only checking a single > device. Indeed, i am not 100% clear as to why, so some explanation in > the commit message and preferably a comment in code would be more than > welcome :) > > Because the pci_one_device_iommu_support_va function does always the same whatever the device is used in the call. The code uses the device for looking at /sys/bus/pci/devices/ but then it uses a link to iommu which will be the same for all the devices. Note that some can refer to dmar0 and others to dmar1, but the IOMMU capabilities are the same. The limitation here is not a PCI device but the IOMMU hardware itself. The first call to pci_one_device_iommu_support_va will check if all the hugepages addresses are within the supported DMA range by the IOMMU hw. If it fails, that is. Now that I'm explaining this, I notice it is the same for any case. If the check is good, no more checks are needed. This assumes there is just one IOMMU hardware or if more than one (I have NUMA systems with one IOMMU unit per socket) they are all the same hardware version. Adding Maxime in the thread for confirming this and asking him about my previous statement. > > } > > return true; > > > > > -- > Thanks, > Anatoly >