From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 796022C2B for ; Fri, 21 Jul 2017 10:07:59 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 22F1F8553F; Fri, 21 Jul 2017 08:07:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 22F1F8553F Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=maxime.coquelin@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 22F1F8553F Received: from [10.36.112.25] (ovpn-112-25.ams2.redhat.com [10.36.112.25]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5138A7E10D; Fri, 21 Jul 2017 08:07:53 +0000 (UTC) To: Santosh Shukla , thomas@monjalon.net, dev@dpdk.org Cc: bruce.richardson@intel.com, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com, sergio.gonzalez.monroy@intel.com, anatoly.burakov@intel.com, stephen@networkplumber.org, olivier.matz@6wind.com References: <20170711061631.5018-1-santosh.shukla@caviumnetworks.com> <20170718055950.10208-1-santosh.shukla@caviumnetworks.com> From: Maxime Coquelin Message-ID: <20ae9657-9fc0-7e64-20c4-138c7f49a836@redhat.com> Date: Fri, 21 Jul 2017 10:07:51 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <20170718055950.10208-1-santosh.shukla@caviumnetworks.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 21 Jul 2017 08:07:58 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH v4 00/12] Infrastructure to detect iova mapping on the bus X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Jul 2017 08:07:59 -0000 Hi Santosh, On 07/18/2017 07:59 AM, Santosh Shukla wrote: > v4: > Introducing RTE_PCI_DRV_IOVA_AS_VA flag for autodetection of iova va mapping. > If a PCI driver demand for IOVA as VA scheme then the driver can add it in the > PCI driver registration function. > > Algorithm to select IOVA as VA for PCI bus case: > 0. If no device bound then return with RTE_IOVA_DC mapping mode, > else goto 1). > 1. Look for device attached to vfio kdrv and has .drv_flag set > to RTE_PCI_DRV_IOVA_AS_VA. > 2. Look for any device attached to UIO class of driver. > 3. Check for vfio-noiommu mode enabled. > > If 2) & 3) is false and 1) is true then select > mapping scheme as RTE_IOVA_VA. Otherwise use default > mapping scheme (RTE_IOVA_PA). > > That way, Bus can truly autodetect the iova mapping mode for > a device Or a set of the device. > > > Patch series rebased on 'a599eb31f2e477674fc6176cdf989ee17432b552'. > > * Re-introduced RTE_IOVA_DC (Don't care mode) for no-device found case. > (Identified by Hemant [5]). > * Renamed flag from RTE_PCI_DRV_NEED_IOVA_VA to RTE_PCI_DRV_IOVA_AS_VA > (Suggested by Maxime[6]). > * Based on the discussion on the thread [3], [6] and [5]. > > v3 --> v4: > - Re-introduced RTE_IOVA_DEC mode (Suggested by Hemant [5]). > - Renamed flag to RTE_PCI_DRV_IOVA_AS_VA (Suggested by Maxime). > - Reworded WARNING message(suggested by Maxime[7]). > - Created a separate patch for rte_pci_get_iommu_class (suggested by Maxime[]). > - Added VFIO_PRESENT ifdef build fix. > > v2 --> v3: > - Removed rte_mempool_virt2phy (suggested by Olivier [4]) > > v1 --> v2: > - Removed override eal option i.e. (--iova-mode=<>) Because we have means to > truly autodetect the iova mode. > - Introduced RTE_PCI_DRV_NEED_IOVA_VA drv_flag (Suggested by Maxime [3]). > - Using NEED_IOVA_VA drv_flag in autodetection logic. > - Removed Linux version check macro in vfio code, As per Maxime feedback. > - Moved rte_pci_match API from local to global. > > Patch Summary: > 0) 1st: Introducing a new flag in rte_pci_drv > 1) 2nd: declare rte_pci_match api in pci header. Required for autodetection in > follow up patches. > 2) 3rd: declare rte_pci_get_iommu_class. > 3) 4nd - 5th: autodetection mapping infrastructure for Linux/bsdapp. > 4) 6th: Introduces global bus API named rte_bus_get_iommu_class. > 5) 7th: iova mode helper API. > 6) 8th - 9th: Calls rte_bus_get_iommu_class API for Linux/bsdapp and returns > their iova mode. > 7) 10th: Check iova mode and accordingly map vfio.dma_map to _pa or _va. > 8) 11th - 12th: Check for IOVA_VA mode in below APIs > - rte_mem_virt2phy > - rte_malloc_virt2phy > > Test History: > - Tested for x86/XL710 40G NIC card for both modes (iova_va/pa). > - Tested for arm64/thunderx vNIC Integrated NIC for both modes > - Tested for arm64/Octeontx integrated NICs for only > Iova_va mode(It supports only one mode.) > - Ran standalone tests like mempool_autotest, mbuf_autotest. > - Verified for Doxygen. > > Work History: > For v1, Refer [1]. > For v2, Refer [2]. > For v3, Refer [9]. > > > Checkpatch result: > * Debug message - WARNING: line over 80 characters > > Thanks., > > [1] https://www.mail-archive.com/dev@dpdk.org/msg67438.html > [2] https://www.mail-archive.com/dev@dpdk.org/msg70674.html > [3] https://www.mail-archive.com/dev@dpdk.org/msg70279.html > [4] https://www.mail-archive.com/dev@dpdk.org/msg70692.html > [5] http://dpdk.org/ml/archives/dev/2017-July/071282.html > [6] http://dpdk.org/ml/archives/dev/2017-July/070951.html > [7] http://dpdk.org/ml/archives/dev/2017-July/070941.html > [8] http://dpdk.org/ml/archives/dev/2017-July/070952.html > [9] http://dpdk.org/ml/archives/dev/2017-July/070918.html > > > Santosh Shukla (12): > eal/pci: introduce PCI driver iova as va flag > eal/pci: export match function > eal/pci: get iommu class > bsdapp/eal_pci: get iommu class > linuxapp/eal_pci: get iommu class > bus: get iommu class > eal: introduce iova mode helper api > linuxapp/eal: auto detect iova mode > bsdapp/eal: auto detect iova mapping mode > linuxapp/eal_vfio: honor iova mode before mapping > linuxapp/eal_memory: honor iova mode in virt2phy > eal/rte_malloc: honor iova mode in virt2phy > > lib/librte_eal/bsdapp/eal/eal.c | 21 ++++-- > lib/librte_eal/bsdapp/eal/eal_pci.c | 10 +++ > lib/librte_eal/bsdapp/eal/rte_eal_version.map | 4 ++ > lib/librte_eal/common/eal_common_bus.c | 23 ++++++ > lib/librte_eal/common/eal_common_pci.c | 11 +-- > lib/librte_eal/common/include/rte_bus.h | 32 +++++++++ > lib/librte_eal/common/include/rte_eal.h | 12 ++++ > lib/librte_eal/common/include/rte_pci.h | 28 ++++++++ > lib/librte_eal/common/rte_malloc.c | 9 ++- > lib/librte_eal/linuxapp/eal/eal.c | 21 ++++-- > lib/librte_eal/linuxapp/eal/eal_memory.c | 3 + > lib/librte_eal/linuxapp/eal/eal_pci.c | 95 +++++++++++++++++++++++++ > lib/librte_eal/linuxapp/eal/eal_vfio.c | 29 +++++++- > lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 ++ > lib/librte_eal/linuxapp/eal/rte_eal_version.map | 4 ++ > 15 files changed, 282 insertions(+), 24 deletions(-) > With Hermant's comments on patch 6 taken into account, feel free to add my: Reviewed-by: Maxime Coquelin Thanks, Maxime