From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id CBBBAA0487
for <public@inbox.dpdk.org>; Mon, 29 Jul 2019 13:18:06 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id B622F1BF21;
Mon, 29 Jul 2019 13:18:05 +0200 (CEST)
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
by dpdk.org (Postfix) with ESMTP id CFD7E1BF10
for <dev@dpdk.org>; Mon, 29 Jul 2019 13:18:03 +0200 (CEST)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga007.jf.intel.com ([10.7.209.58])
by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
29 Jul 2019 04:18:02 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.64,322,1559545200"; d="scan'208";a="162149875"
Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.92])
([10.237.220.92])
by orsmga007.jf.intel.com with ESMTP; 29 Jul 2019 04:18:00 -0700
To: David Marchand <david.marchand@redhat.com>
Cc: dev <dev@dpdk.org>, John McNamara <john.mcnamara@intel.com>,
Marko Kovacevic <marko.kovacevic@intel.com>, dariusz.stojaczyk@intel.com,
Thomas Monjalon <thomas@monjalon.net>,
Jerin Jacob Kollanukkaran <jerinj@marvell.com>
References: <8b7148c100678ee2f4cd9b94168b006e851fe6ef.1564052753.git.anatoly.burakov@intel.com>
<cda08c84d00201139718e15eeeca0dc5e67ead31.1564155444.git.anatoly.burakov@intel.com>
<CAJFAV8weSCL=--j=J04hdxiFJiL7P5vF4i2ud8rW0Ep0ecU4YA@mail.gmail.com>
From: "Burakov, Anatoly" <anatoly.burakov@intel.com>
Message-ID: <4b3b8843-1354-6e38-26af-893b3a436e51@intel.com>
Date: Mon, 29 Jul 2019 12:18:00 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
Thunderbird/60.8.0
MIME-Version: 1.0
In-Reply-To: <CAJFAV8weSCL=--j=J04hdxiFJiL7P5vF4i2ud8rW0Ep0ecU4YA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Subject: Re: [dpdk-dev] [PATCH v4] eal: pick IOVA as PA if IOMMU is not
available
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On 29-Jul-19 10:31 AM, David Marchand wrote:
> On Fri, Jul 26, 2019 at 5:37 PM Anatoly Burakov
> <anatoly.burakov@intel.com> wrote:
>>
>> When IOMMU is not available, /sys/kernel/iommu_groups will not be
>> populated. This is happening since at least 3.6 when VFIO support
>> was added. If the directory is empty, EAL should not pick IOVA as
>> VA as the default IOVA mode.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> Tested-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
>> Tested-by: Jerin Jacob <jerinj@marvell.com>
>> Reviewed-by: Jerin Jacob <jerinj@marvell.com>
>> ---
>>
>> Notes:
>> v4:
>> - Fix indentation in release notes' known issues
>>
>> v3:
>> - Add documentation changes
>> - Fix a typo pointed out by checkpatch
>>
>> v2:
>> - Decouple IOMMU from VFIO
>> - Add a check for physical addresses availability
>>
>> .../prog_guide/env_abstraction_layer.rst | 27 +++++++++++------
>> doc/guides/rel_notes/known_issues.rst | 26 ++++++++++++++++
>> doc/guides/rel_notes/release_19_08.rst | 16 ++++++++++
>> lib/librte_eal/linux/eal/eal.c | 21 +++++++++++--
>> lib/librte_eal/linux/eal/eal_vfio.c | 30 +++++++++++++++++++
>> lib/librte_eal/linux/eal/eal_vfio.h | 2 ++
>> 6 files changed, 111 insertions(+), 11 deletions(-)
>>
>> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
>> index 1487ea550..e6e70e5a8 100644
>> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
>> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
>> @@ -425,6 +425,9 @@ IOVA Mode Detection
>> IOVA Mode is selected by considering what the current usable Devices on the
>> system require and/or support.
>>
>> +On FreeBSD, RTE_IOVA_VA mode is not supported, so RTE_IOVA_PA is always used.
>
> We still allow setting it via --iova-mode=
> Is it really unsupported ? vdev like rings could work.
Oh, right, we don't *really* support IOVA as VA mode, but you can still
run in --no-huge mode (or using only virtual devices) and pretend like
it works ;)
Still, i think FreeBSD should default to PA unless it's not running as root.
>
>
>> +On Linux, the IOVA mode is detected based on a heuristic.
>> +
>> Below is the 2-step heuristic for this choice.
>
> We can combine those two sentences as a single one.
Sure.
>
>
>>
>> For the first step, EAL asks each bus its requirement in terms of IOVA mode
>> @@ -438,20 +441,26 @@ and decides on a preferred IOVA mode.
>> RTE_IOVA_VA), then the preferred IOVA mode is RTE_IOVA_DC (see below with the
>> check on Physical Addresses availability),
>>
>> +If the buses have expressed no preference on which IOVA mode to pick, then a
>> +default is selected using the following logic:
>> +
<snip>
>> @@ -1061,8 +1061,25 @@ rte_eal_init(int argc, char **argv)
>> enum rte_iova_mode iova_mode = rte_bus_get_iommu_class();
>>
>> if (iova_mode == RTE_IOVA_DC) {
>> - iova_mode = RTE_IOVA_VA;
>> - RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode, select IOVA as VA mode.\n");
>> + RTE_LOG(DEBUG, EAL, "Buses did not request a specific IOVA mode.\n");
>> +
>> + if (!phys_addrs) {
>> + /* if we have no access to physical addresses,
>> + * pick IOVA as VA mode.
>> + */
>> + iova_mode = RTE_IOVA_VA;
>> + RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n");
>> + } else if (vfio_iommu_enabled()) {
>
> How about:
> s/vfio_iommu_enabled/is_iommu_available/
>
> And the code would move from vfio specific files to eal.c.
Can do.
--
Thanks,
Anatoly