[PATCH 0/1] rebase iova fixes

[PATCH 0/1] rebase iova fixes

[christian.ehrhardt]

From: Christian Ehrhardt <christian.ehrhardt@canonical.com>

Testing 23.11 has shown that [1] is still needed, all tests
on ppc64 with no-huge still fail due to iova=pa breaking without
pa being available.

This is the rebase of [1] to 23.11 and I've added the matching
change to the documentation as well.

David Wilder (1):
  eal/linux: force iova-mode va without pa available

 doc/guides/prog_guide/env_abstraction_layer.rst |  9 ++++++---
 lib/eal/linux/eal.c                             | 14 ++++++++------
 2 files changed, 14 insertions(+), 9 deletions(-)

-- 
2.34.1

[PATCH 1/1] eal/linux: force iova-mode va without pa available

[christian.ehrhardt]

From: David Wilder <dwilder@us.ibm.com>

When using --no-huge option physical address are not guaranteed
to be persistent.

This change effectively makes "--no-huge" the same as
"--no-huge --iova-mode=va".

When --no-huge is used (or any other condition making physical
addresses unavailable) setting --iova-mode=pa will have no effect.

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
---
 doc/guides/prog_guide/env_abstraction_layer.rst |  9 ++++++---
 lib/eal/linux/eal.c                             | 14 ++++++++------
 2 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
index 6debf54efb..20c7355e0f 100644
--- a/doc/guides/prog_guide/env_abstraction_layer.rst
+++ b/doc/guides/prog_guide/env_abstraction_layer.rst
@@ -559,9 +559,12 @@ IOVA Mode is selected by considering what the current usable Devices on the
 system require and/or support.
 
 On FreeBSD, RTE_IOVA_PA is always the default. On Linux, the IOVA mode is
-detected based on a 2-step heuristic detailed below.
+detected based on a heuristic detailed below.
 
-For the first step, EAL asks each bus its requirement in terms of IOVA mode
+For the first step, if no Physical Addresses are available RTE_IOVA_VA is
+selected.
+
+Then EAL asks each bus its requirement in terms of IOVA mode
 and decides on a preferred IOVA mode.
 
 - if all buses report RTE_IOVA_PA, then the preferred IOVA mode is RTE_IOVA_PA,
@@ -575,7 +578,7 @@ and decides on a preferred IOVA mode.
 If the buses have expressed no preference on which IOVA mode to pick, then a
 default is selected using the following logic:
 
-- if physical addresses are not available, RTE_IOVA_VA mode is used
+- if enable_iova_as_pa was not set at build RTE_IOVA_VA mode is used
 - if /sys/kernel/iommu_groups is not empty, RTE_IOVA_VA mode is used
 - otherwise, RTE_IOVA_PA mode is used
 
diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c
index 57da058cec..7d0eedef57 100644
--- a/lib/eal/linux/eal.c
+++ b/lib/eal/linux/eal.c
@@ -1067,6 +1067,14 @@ rte_eal_init(int argc, char **argv)
 
 	phys_addrs = rte_eal_using_phys_addrs() != 0;
 
+	if (!phys_addrs) {
+		/* if we have no access to physical addresses,
+		 * pick IOVA as VA mode.
+		 */
+		iova_mode = RTE_IOVA_VA;
+		RTE_LOG(INFO, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n");
+	}
+
 	/* if no EAL option "--iova-mode=<pa|va>", use bus IOVA scheme */
 	if (internal_conf->iova_mode == RTE_IOVA_DC) {
 		/* autodetect the IOVA mapping mode */
@@ -1078,12 +1086,6 @@ rte_eal_init(int argc, char **argv)
 			if (!RTE_IOVA_IN_MBUF) {
 				iova_mode = RTE_IOVA_VA;
 				RTE_LOG(DEBUG, EAL, "IOVA as VA mode is forced by build option.\n");
-			} else if (!phys_addrs) {
-				/* if we have no access to physical addresses,
-				 * pick IOVA as VA mode.
-				 */
-				iova_mode = RTE_IOVA_VA;
-				RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n");
 			} else if (is_iommu_enabled()) {
 				/* we have an IOMMU, pick IOVA as VA mode */
 				iova_mode = RTE_IOVA_VA;
-- 
2.34.1

Re: [PATCH 1/1] eal/linux: force iova-mode va without pa available

[Christian Ehrhardt]

[-- Attachment #1: Type: text/plain, Size: 4163 bytes --]

On Thu, Nov 23, 2023 at 4:52 PM <christian.ehrhardt@canonical.com> wrote:

> From: David Wilder <dwilder@us.ibm.com>
>
> When using --no-huge option physical address are not guaranteed
> to be persistent.
>
> This change effectively makes "--no-huge" the same as
> "--no-huge --iova-mode=va".
>
> When --no-huge is used (or any other condition making physical
> addresses unavailable) setting --iova-mode=pa will have no effect.
>
> Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
> ---
>  doc/guides/prog_guide/env_abstraction_layer.rst |  9 ++++++---
>  lib/eal/linux/eal.c                             | 14 ++++++++------
>  2 files changed, 14 insertions(+), 9 deletions(-)
>
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst
> b/doc/guides/prog_guide/env_abstraction_layer.rst
> index 6debf54efb..20c7355e0f 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -559,9 +559,12 @@ IOVA Mode is selected by considering what the current
> usable Devices on the
>  system require and/or support.
>
>  On FreeBSD, RTE_IOVA_PA is always the default. On Linux, the IOVA mode is
> -detected based on a 2-step heuristic detailed below.
> +detected based on a heuristic detailed below.
>
> -For the first step, EAL asks each bus its requirement in terms of IOVA
> mode
> +For the first step, if no Physical Addresses are available RTE_IOVA_VA is
> +selected.
> +
> +Then EAL asks each bus its requirement in terms of IOVA mode
>  and decides on a preferred IOVA mode.
>
>  - if all buses report RTE_IOVA_PA, then the preferred IOVA mode is
> RTE_IOVA_PA,
> @@ -575,7 +578,7 @@ and decides on a preferred IOVA mode.
>  If the buses have expressed no preference on which IOVA mode to pick,
> then a
>  default is selected using the following logic:
>
> -- if physical addresses are not available, RTE_IOVA_VA mode is used
> +- if enable_iova_as_pa was not set at build RTE_IOVA_VA mode is used
>  - if /sys/kernel/iommu_groups is not empty, RTE_IOVA_VA mode is used
>  - otherwise, RTE_IOVA_PA mode is used
>
> diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c
> index 57da058cec..7d0eedef57 100644
> --- a/lib/eal/linux/eal.c
> +++ b/lib/eal/linux/eal.c
> @@ -1067,6 +1067,14 @@ rte_eal_init(int argc, char **argv)
>
>         phys_addrs = rte_eal_using_phys_addrs() != 0;
>
> +       if (!phys_addrs) {
> +               /* if we have no access to physical addresses,
> +                * pick IOVA as VA mode.
> +                */
> +               iova_mode = RTE_IOVA_VA;
>

^^ this won't work, thanks for trying to rush it :-/
I'll fix and test it before v2
But since it wasn't my patch initially I'm happy if anyone else wants to
take over ...


> +               RTE_LOG(INFO, EAL, "Physical addresses are unavailable,
> selecting IOVA as VA mode.\n");
> +       }
> +
>         /* if no EAL option "--iova-mode=<pa|va>", use bus IOVA scheme */
>         if (internal_conf->iova_mode == RTE_IOVA_DC) {
>                 /* autodetect the IOVA mapping mode */
> @@ -1078,12 +1086,6 @@ rte_eal_init(int argc, char **argv)
>                         if (!RTE_IOVA_IN_MBUF) {
>                                 iova_mode = RTE_IOVA_VA;
>                                 RTE_LOG(DEBUG, EAL, "IOVA as VA mode is
> forced by build option.\n");
> -                       } else if (!phys_addrs) {
> -                               /* if we have no access to physical
> addresses,
> -                                * pick IOVA as VA mode.
> -                                */
> -                               iova_mode = RTE_IOVA_VA;
> -                               RTE_LOG(DEBUG, EAL, "Physical addresses
> are unavailable, selecting IOVA as VA mode.\n");
>                         } else if (is_iommu_enabled()) {
>                                 /* we have an IOMMU, pick IOVA as VA mode
> */
>                                 iova_mode = RTE_IOVA_VA;
> --
> 2.34.1
>
>

-- 
Christian Ehrhardt
Director of Engineering, Ubuntu Server
Canonical Ltd

[-- Attachment #2: Type: text/html, Size: 5482 bytes --]