From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by dpdk.org (Postfix) with ESMTP id 2B31210A3 for ; Tue, 30 Oct 2018 13:58:47 +0100 (CET) Received: by mail-ed1-f68.google.com with SMTP id t10-v6so10365101eds.12 for ; Tue, 30 Oct 2018 05:58:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=B8vfLlO+dfGLj+WZ//13koav9OqscLsJ3YbqGrSANi0=; b=UwFTHcNrPUd9Jz/wRgyKYk8qmqFQBJ5J2+wjZKqivBEl4CdrYJji5D4nm9FF71SGC4 Wj54SfAd63EdBMTAdjiRPMZGDGGyd8KDQl1G8NBePFDJMt1++2857Ekpc6OzPx9f0xx1 De2w1ASvBABxVgKnCAvCKgHU8CQSCT+jMcqzb18XeUzqiqjUtUReUAqChyPSqMEGVUlp MqpjOV1LaGJ2cjIML0KbIA2oNvwBpsrXqC+RKuxR29bKJl0+94c+7RQR98Cri9uhaK17 AVDPHAPNIn8eSZc2dNOeu2IltTi6SoMWG76zuQofXB1e00+JYUVIEz6fXcIdqZJQLnhS RE5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=B8vfLlO+dfGLj+WZ//13koav9OqscLsJ3YbqGrSANi0=; b=iepRnqE68xyoOF2oHCYgPfkmhh06OYCT4X3Ignco83jmqAYjaCYvJxm+CyBeCzuKIv orsaQbr0fMel4D6qiiuygIYl8Z2arjjgePijF9uM2RdUYqjEegsIX7casUgYmQiWEz/4 j1GMz+TUsNeYfi3RDt0P+mtorm39dzbckEiuD3PF204mwgOeqtBkLWZ/l8yZM/HToj4K hp64CSm+rIQEHAjzRG3EC5pGADrG7q9BGxYufufLoUuubcew/zcdK/3lelDhBpfZ9AHq pRPR6Wo/i24VQo1VHdHsYmq0+HEYNrHVtkLJdU52nTt/6I0fPlOZ7ewve0b8x2/rm06j swhg== X-Gm-Message-State: AGRZ1gLcmQ3+YlhKQeY9VlnouuxHJ8QONj6lR5uyr1Eh4oJXR2l6GMlg FXv/RV275lGKkiFngV9zY0dFNtCvsjy9417Pfd4XBQ== X-Google-Smtp-Source: AJdET5dGZGUADYvIRilxITZtxK0n26udBV/etfS1WflyPUP5RU+6lCDXd1uYrkDt4Nx2F4gci0qntmBrRlhAJZbv7Xo= X-Received: by 2002:a50:b704:: with SMTP id g4-v6mr16915113ede.139.1540904326702; Tue, 30 Oct 2018 05:58:46 -0700 (PDT) MIME-Version: 1.0 References: <20180907154703.83316-1-dariusz.stojaczyk@intel.com> <20180907155843.96465-1-dariusz.stojaczyk@intel.com> In-Reply-To: From: Alejandro Lucero Date: Tue, 30 Oct 2018 12:58:35 +0000 Message-ID: To: dariusz.stojaczyk@intel.com Cc: "Burakov, Anatoly" , dev , Santosh Shukla , Hemant Agrawal , Jerin Jacob , Maxime Coquelin , chas3@att.com Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH v2] eal/bus: use RTE_IOVA_PA only if phys addresses are available X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2018 12:58:47 -0000 On Mon, Sep 17, 2018 at 2:06 PM Stojaczyk, Dariusz < dariusz.stojaczyk@intel.com> wrote: > > > > -----Original Message----- > > From: Burakov, Anatoly > > Sent: Monday, September 17, 2018 12:34 PM > > To: Stojaczyk, Dariusz ; dev@dpdk.org; > > Santosh Shukla ; Hemant Agrawal > > ; Jerin Jacob > > > > Cc: Maxime Coquelin ; Chas Williams > > > > Subject: Re: [PATCH v2] eal/bus: use RTE_IOVA_PA only if phys addresses > > are available > > > > On 07-Sep-18 4:58 PM, Darek Stojaczyk wrote: > > > When neither RTE_IOVA_VA nor RTE_IOVA_PA was explicitly requested, > > > DPDK would currently fallback to the default RTE_IOVA_PA mode and > > > possibly encounter a failure later on if running as a non-priviledged > > > user. Attempting to use RTE_IOVA_VA if no phys addresses are available > > > may help in this case. > > > > > > Signed-off-by: Darek Stojaczyk > > > --- > > > Changes since v1: > > > * added a missing rte_memory.h include > > > > > > lib/librte_eal/common/eal_common_bus.c | 19 +++++++++++++++---- > > > 1 file changed, 15 insertions(+), 4 deletions(-) > > > > > > diff --git a/lib/librte_eal/common/eal_common_bus.c > > > b/lib/librte_eal/common/eal_common_bus.c > > > index 0943851cc..68c581b8a 100644 > > > --- a/lib/librte_eal/common/eal_common_bus.c > > > +++ b/lib/librte_eal/common/eal_common_bus.c > > > @@ -37,6 +37,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > > > > #include "eal_private.h" > > > > > > @@ -236,9 +237,19 @@ rte_bus_get_iommu_class(void) > > > mode |= bus->get_iommu_class(); > > > } > > > > > > - if (mode != RTE_IOVA_VA) { > > > - /* Use default IOVA mode */ > > > - mode = RTE_IOVA_PA; > > > + if (mode == RTE_IOVA_VA) > > > + return RTE_IOVA_VA; > > > + > > > + if (mode & RTE_IOVA_PA) { > > > + /* Not all buses support RTE_IOVA_VA, fallback to > > RTE_IOVA_PA */ > > > + return RTE_IOVA_PA; > > > + } > > > + > > > + if (rte_eal_using_phys_addrs()) { > > > + /* Default to RTE_IOVA_PA only if it's supported */ > > > + return RTE_IOVA_PA; > > > } > > > - return mode; > > > + > > > + /* Since RTE_IOVA_PA is unsupported, fallback to RTE_IOVA_VA */ > > > + return RTE_IOVA_VA; > > > } > > > > > > > This is a good change, however I think that this is too pessimistic. If > i don't > > have any devices that explictly require IOVA_PA, i should be running in > > IOVA_VA mode. > > Another problem may occur when trying to hotplug devices that support only > 39bit DMA. You may not be able to map any memory with vfio when in > RTE_IOVA_VA mode, as virtual addresses likely occupy more than 39 bits. > > There is now a hint for trying to map memory as low as possible instead of using default Linux mmap base address. This makes devices with addressing limitations being usable as long as the physical memory to map is not more than what those devices allow. > The rte_pci bus enforces RTE_IOVA_PA whenever it finds such devices on > init. > > I have no doubt the logic can be improved here, but for now RTE_IOVA_PA is > the only safe default. > > D. > > > > > This of course doesn't take hotplug into account, so a command-line > switch > > to force one or the other should also be available. > > > > For example, at startup, i might have devices bound to VFIO, so IOVA_VA > > mode is picked. However, even though at a time of startup none of the > > devices require physical addresses, i also know that i might later > hotplug a > > device that requires IOVA_PA (leaving the question of hotplug brokenness > > aside for now...) - currently, this scenario will not work, as i will be > forced to > > use IOVA_VA mode unless i happen to have a IOVA_PA device available at > > startup. > > > > Similarly, if i'm running DPDK as root but am only using virtual devices > like > > pcap, i should be able to force DPDK into using VA addresses [*], yet > > currently i will be forced to use IOVA_PA if i don't *also* have a few > devices > > bound exclusively to VFIO. > > > > [*] Do we have vdev devices that require IOVA_PA? I can't think of any... > > > > -- > > Thanks, > > Anatoly >