On 11/24/23 2:09 AM, christian.ehrhardt@canonical.com wrote:
> From: David Wilder <dwilder@us.ibm.com>
>
> When using --no-huge option physical address are not guaranteed
> to be persistent.
>
> This change effectively makes "--no-huge" the same as
> "--no-huge --iova-mode=va".
>
> When --no-huge is used (or any other condition making physical
> addresses unavailable) setting --iova-mode=pa will have no effect.
>
> Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
> ---
>   doc/guides/prog_guide/env_abstraction_layer.rst |  9 ++++++---
>   lib/eal/linux/eal.c                             | 16 ++++++++++------
>   2 files changed, 16 insertions(+), 9 deletions(-)
>
> diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
> index 6debf54efb..20c7355e0f 100644
> --- a/doc/guides/prog_guide/env_abstraction_layer.rst
> +++ b/doc/guides/prog_guide/env_abstraction_layer.rst
> @@ -559,9 +559,12 @@ IOVA Mode is selected by considering what the current usable Devices on the
>   system require and/or support.
>
>   On FreeBSD, RTE_IOVA_PA is always the default. On Linux, the IOVA mode is
> -detected based on a 2-step heuristic detailed below.
> +detected based on a heuristic detailed below.
>
> -For the first step, EAL asks each bus its requirement in terms of IOVA mode
> +For the first step, if no Physical Addresses are available RTE_IOVA_VA is
> +selected.
> +
> +Then EAL asks each bus its requirement in terms of IOVA mode
>   and decides on a preferred IOVA mode.
>
>   - if all buses report RTE_IOVA_PA, then the preferred IOVA mode is RTE_IOVA_PA,
> @@ -575,7 +578,7 @@ and decides on a preferred IOVA mode.
>   If the buses have expressed no preference on which IOVA mode to pick, then a
>   default is selected using the following logic:
>
> -- if physical addresses are not available, RTE_IOVA_VA mode is used
> +- if enable_iova_as_pa was not set at build RTE_IOVA_VA mode is used
>   - if /sys/kernel/iommu_groups is not empty, RTE_IOVA_VA mode is used
>   - otherwise, RTE_IOVA_PA mode is used
>
> diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c
> index 57da058cec..2f1fce3c54 100644
> --- a/lib/eal/linux/eal.c
> +++ b/lib/eal/linux/eal.c
> @@ -1067,6 +1067,16 @@ rte_eal_init(int argc, char **argv)
>
>       phys_addrs = rte_eal_using_phys_addrs() != 0;
>
> +     if (!phys_addrs) {
> +             /* if we have no access to physical addresses, pick IOVA as VA mode. */
> +             if (internal_conf->iova_mode == RTE_IOVA_PA)
> +                     RTE_LOG(WARNING, EAL, "WARNING: --iova-mode=pa, but Physical addresses are unavailable, selecting IOVA as VA mode.\n");
> +             else
> +                     RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n");
> +             internal_conf->iova_mode = RTE_IOVA_VA;
> +             rte_eal_get_configuration()->iova_mode = internal_conf->iova_mode;
> +     }
> +
>       /* if no EAL option "--iova-mode=<pa|va>", use bus IOVA scheme */
>       if (internal_conf->iova_mode == RTE_IOVA_DC) {
>               /* autodetect the IOVA mapping mode */
> @@ -1078,12 +1088,6 @@ rte_eal_init(int argc, char **argv)
>                       if (!RTE_IOVA_IN_MBUF) {
>                               iova_mode = RTE_IOVA_VA;
>                               RTE_LOG(DEBUG, EAL, "IOVA as VA mode is forced by build option.\n");
> -                     } else if (!phys_addrs) {
> -                             /* if we have no access to physical addresses,
> -                              * pick IOVA as VA mode.
> -                              */
> -                             iova_mode = RTE_IOVA_VA;
> -                             RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n");
>                       } else if (is_iommu_enabled()) {
>                               /* we have an IOMMU, pick IOVA as VA mode */
>                               iova_mode = RTE_IOVA_VA;

What tests are you running that generate an error without explicitly
selecting the iova-mode?  When I run the fast-tests I see the system
correctly selecting VA without any additional command-line parameters
when running 23.11:

21:47:05 DPDK_TEST=acl_autotest MALLOC_PERTURB_=201
/home/drc/src/dpdk/build/app/dpdk-test --no-huge -m 2048
----------------------------------- output
-----------------------------------
stdout:
RTE>>acl_autotest
acl context <acl_ctx>@0x179cbd300
   socket_id=-1
   alg=5
   first_load_sz=0
   max_rules=196608
   rule_size=128
   num_rules=0
   num_categories=0
   num_tries=0
acl context <acl_ctx>@0x179cbd300
   socket_id=-1
   alg=5
   first_load_sz=0
   max_rules=196608
   rule_size=128
   num_rules=0
   num_categories=0
   num_tries=0
running test_convert_rules(acl_ipv4vlan_tuple)
running test_convert_rules(acl_ipv4vlan_tuple,
RTE_ACL_FIELD_TYPE_BITMASK type for IPv4)
running test_convert_rules(acl_ipv4vlan_tuple, RTE_ACL_FIELD_TYPE_RANGE
type for IPv4)
running test_convert_rules(acl_ipv4vlan_tuple: swap VLAN and PORTs order)
running test_convert_rules(acl_ipv4vlan_tuple: swap SRC and DST IPv4 order)
test_u32_range#1704 starting range test from 0 to 264192
Test OK
RTE>>
stderr:
EAL: Detected CPU lcores: 128
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'

Dave