From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail5.wrs.com (mail5.windriver.com [192.103.53.11]) by dpdk.org (Postfix) with ESMTP id 58FAE2BFA for ; Tue, 11 Sep 2018 19:23:10 +0200 (CEST) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail5.wrs.com (8.15.2/8.15.2) with ESMTPS id w8BHM0Pp030309 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Tue, 11 Sep 2018 10:22:11 -0700 Received: from [128.224.56.213] (128.224.56.213) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server (TLS) id 14.3.408.0; Tue, 11 Sep 2018 10:21:49 -0700 From: Eric Zhang To: "Burakov, Anatoly" , Jerin Jacob CC: santosh , , =?UTF-8?Q?Ga=c3=abtan_Rivet?= , , , , References: <1535558289-10336-1-git-send-email-eric.zhang@windriver.com> <20180830094323.37xkgud4fz3mflbg@bidouze.vm.6wind.com> <0f19de06-fccd-eff2-b33e-71d49a005dbb@nxp.com> <9ae6e3dd-5f33-d874-9a58-730435b58de3@windriver.com> <20180906073449.GA20576@jerin> <1781101b-ceed-ac80-5564-cc01232cb29f@intel.com> <29ab4eb7-8af5-f66d-5359-dcfd7fbb2ba5@windriver.com> Message-ID: <2dab0826-ee74-89fe-c012-ad9dd8e5f0cc@windriver.com> Date: Tue, 11 Sep 2018 13:21:47 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <29ab4eb7-8af5-f66d-5359-dcfd7fbb2ba5@windriver.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Originating-IP: [128.224.56.213] Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2018 17:23:10 -0000 On 09/07/2018 04:13 PM, Eric Zhang wrote: > > > On 09/07/2018 05:26 AM, Burakov, Anatoly wrote: >> On 06-Sep-18 8:34 AM, Jerin Jacob wrote: >>> -----Original Message----- >>>> Date: Tue, 4 Sep 2018 23:40:36 -0400 >>>> From: Eric Zhang >>>> To: santosh , >>>> hemant.agrawal@nxp.com, >>>>   Gaëtan Rivet , "Burakov, Anatoly" >>>>   >>>> CC: bruce.richardson@intel.com, dev@dpdk.org, >>>> Allain.Legacy@windriver.com, >>>>   Matt.Peters@windriver.com >>>> Subject: Re: [dpdk-dev] [PATCH] eal: force IOVA mode to physical >>>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 >>>>   Thunderbird/52.9.1 >>>> >>>> On 08/30/2018 08:59 AM, santosh wrote: >>>>> On Thursday 30 August 2018 05:43 PM, Hemant wrote: >>>>>> External Email >>>>>> >>>>>> Hi, >>>>>> >>>>>> On 8/30/2018 3:13 PM, Gaëtan Rivet wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On Thu, Aug 30, 2018 at 10:09:04AM +0100, Burakov, Anatoly wrote: >>>>>>>> On 29-Aug-18 4:58 PM, eric zhang wrote: >>>>>>>>> This patch adds a configuration option to force the IOVA mode to >>>>>>>>> physical address (PA). There exists virtual devices that are not >>>>>>>>> directly attached to the PCI bus, and therefore the auto >>>>>>>>> detection >>>>>>>>> of the IOVA mode based on probing the PCI bus and IOMMU >>>>>>>>> configuration >>>>>>>>> may not report the required addressing mode. Having the >>>>>>>>> configuration >>>>>>>>> option permits the mode to be explicitly configured in this >>>>>>>>> scenario. >>>>>>>>> >>>>>>>>> Signed-off-by: eric zhang >>>>>>>>> --- >>>>>>>> Defining this at compile-time seems like an overkill. Wouldn't >>>>>>>> it be better >>>>>>>> to just add an EAL command-line option to force IOVA mode to a >>>>>>>> particular >>>>>>>> value? >>>>>> That is a good suggestion. >>>>>>>> -- >>>>>>>> Thanks, >>>>>>>> Anatoly >>>>>>> What is the bus of these devices and why not implement >>>>>>> get_iommu_class >>>>>>> in it? >>>>>> There are cases, where you are using dpdk libraries with external >>>>>> libraries and you need to change the default behavior DPDK lib to >>>>>> use >>>>>> physical address instead of virtual address. >>>>>> Providing an option to user will help. >>>>>> >>>>>> >>>>> More appropriate solution could be: >>>>> * Either fix it at bus layer .. i.e.. get_iommu_class().. >>>>> * Or introduce something like [1] --iova-mode= param. >>>>> >>>>> Former is better solution than latter if autodetection is a key >>>>> criteria. >>>>> Thanks. >>>>> >>>>> [1] http://patchwork.dpdk.org/patch/25192/ >>>>> >>>> It's not generic which couldn't be fixed at bus layer. >>>> So what's the preference of EAL option or compile time solution? >>>> Adding --iova-mode as patch [1] will overrivde auto-detection >>>> rte_bus_get_iommu_class() >>>> make it no use; compile time solution will align with upstream and >>>> keep >>>> new atuodetection >>>> solution in #ifndef. >>> >>> If it is for vdev devices, why not introduce something like >>> RTE_PCI_DRV_IOVA_AS_VA and let vdev device describe its personality. >>> And based on the devices(flags) on vdev bus, rte_bus_get_iommu_class() >>> of vdev can decide the mode just like PCI bus. >>> >> >> That seems like a better option to me, +1. As far as i know, at the >> moment if there are no devices attached at all, or if there are only >> vdev devices attached, DPDK will default to IOVA as PA mode for no >> good reason; such a change would certainly fix this. > Thanks for the suggestions however our virtual device doesn't run dpdk > vdev code so we can't use the flag. > Notice that in eal.c there is one workaround that force iova to be PA > per virtual device is not directly attached to pci. That case is > checking kni module. Ours is a similar case that virtual device not > attach pci directly. > So we have to turn to force iova to PA either 1. compilation option 2. > eal option.  Which one should be the preference by taking into > consideration that align with upstream? > > Thanks > Any comments? >> >>> >>>> >>>> Thanks >>>> Eric >>>> >>> >> >> >