From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) by dpdk.org (Postfix) with ESMTP id 236685F11 for ; Fri, 7 Sep 2018 22:13:07 +0200 (CEST) Received: from ALA-HCA.corp.ad.wrs.com ([147.11.189.40]) by mail.windriver.com (8.15.2/8.15.1) with ESMTPS id w87KD4FB005049 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 7 Sep 2018 13:13:04 -0700 (PDT) Received: from [128.224.56.213] (128.224.56.213) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server (TLS) id 14.3.408.0; Fri, 7 Sep 2018 13:13:04 -0700 To: "Burakov, Anatoly" , Jerin Jacob CC: santosh , , =?UTF-8?Q?Ga=c3=abtan_Rivet?= , , , , References: <1535558289-10336-1-git-send-email-eric.zhang@windriver.com> <20180830094323.37xkgud4fz3mflbg@bidouze.vm.6wind.com> <0f19de06-fccd-eff2-b33e-71d49a005dbb@nxp.com> <9ae6e3dd-5f33-d874-9a58-730435b58de3@windriver.com> <20180906073449.GA20576@jerin> <1781101b-ceed-ac80-5564-cc01232cb29f@intel.com> From: Eric Zhang Message-ID: <29ab4eb7-8af5-f66d-5359-dcfd7fbb2ba5@windriver.com> Date: Fri, 7 Sep 2018 16:13:02 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <1781101b-ceed-ac80-5564-cc01232cb29f@intel.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [128.224.56.213] Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2018 20:13:08 -0000 On 09/07/2018 05:26 AM, Burakov, Anatoly wrote: > On 06-Sep-18 8:34 AM, Jerin Jacob wrote: >> -----Original Message----- >>> Date: Tue, 4 Sep 2018 23:40:36 -0400 >>> From: Eric Zhang >>> To: santosh , >>> hemant.agrawal@nxp.com, >>>   Gaëtan Rivet , "Burakov, Anatoly" >>>   >>> CC: bruce.richardson@intel.com, dev@dpdk.org, >>> Allain.Legacy@windriver.com, >>>   Matt.Peters@windriver.com >>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical >>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 >>>   Thunderbird/52.9.1 >>> >>> On 08/30/2018 08:59 AM, santosh wrote: >>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote: >>>>> External Email >>>>> >>>>> Hi, >>>>> >>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote: >>>>>> Hi, >>>>>> >>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote: >>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote: >>>>>>>> This patch adds a configuration option to force the IOVA mode to >>>>>>>> physical address (PA). There exists virtual devices that are not >>>>>>>> directly attached to the PCI bus, and therefore the auto detection >>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU >>>>>>>> configuration >>>>>>>> may not report the required addressing mode. Having the >>>>>>>> configuration >>>>>>>> option permits the mode to be explicitly configured in this >>>>>>>> scenario. >>>>>>>> >>>>>>>> Signed-off-by: eric zhang >>>>>>>> --- >>>>>>> Defining this at compile-time seems like an overkill. Wouldn't >>>>>>> it be better >>>>>>> to just add an EAL command-line option to force IOVA mode to a >>>>>>> particular >>>>>>> value? >>>>> That is a good suggestion. >>>>>>> -- >>>>>>> Thanks, >>>>>>> Anatoly >>>>>> What is the bus of these devices and why not implement >>>>>> get_iommu_class >>>>>> in it? >>>>> There are cases, where you are using dpdk libraries with external >>>>> libraries and you need to change the default behavior DPDK lib to use >>>>> physical address instead of virtual address. >>>>> Providing an option to user will help. >>>>> >>>>> >>>> More appropriate solution could be: >>>> * Either fix it at bus layer .. i.e.. get_iommu_class().. >>>> * Or introduce something like [1] --iova-mode= param. >>>> >>>> Former is better solution than latter if autodetection is a key >>>> criteria. >>>> Thanks. >>>> >>>> [1] http://patchwork.dpdk.org/patch/25192/ >>>> >>> It's not generic which couldn't be fixed at bus layer. >>> So what's the preference of EAL option or compile time solution? >>> Adding --iova-mode as patch [1] will overrivde auto-detection >>> rte_bus_get_iommu_class() >>> make it no use; compile time solution will align with upstream and keep >>> new atuodetection >>> solution in #ifndef. >> >> If it is for vdev devices, why not introduce something like >> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality. >> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class() >> of vdev can decide the mode just like PCI bus. >> > > That seems like a better option to me, +1. As far as i know, at the > moment if there are no devices attached at all, or if there are only > vdev devices attached, DPDK will default to IOVA as PA mode for no > good reason; such a change would certainly fix this. Thanks for the suggestions however our virtual device doesn't run dpdk vdev code so we can't use the flag. Notice that in eal.c there is one workaround that force iova to be PA per virtual device is not directly attached to pci. That case is checking kni module. Ours is a similar case that virtual device not attach pci directly. So we have to turn to force iova to PA either 1. compilation option 2. eal option.  Which one should be the preference by taking into consideration that align with upstream? Thanks > >> >>> >>> Thanks >>> Eric >>> >> > >