From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C1CF743407; Wed, 29 Nov 2023 22:51:02 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 53AC7402E6; Wed, 29 Nov 2023 22:51:02 +0100 (CET) Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by mails.dpdk.org (Postfix) with ESMTP id 8E2D7402E1 for ; Wed, 29 Nov 2023 22:51:00 +0100 (CET) Received: from pps.filterd (m0353724.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3ATLkGmu004279 for ; Wed, 29 Nov 2023 21:50:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : mime-version : subject : to : references : from : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=LLCnZ1/gVm3Pbn9BgBVMLP1iam/YfSHjU8G35nrE3QU=; b=V8RNwZt96gXDyKPf95yFB4O3r0+xnrBJcnobnornx+hpK4pm1D0ZU1292vqqSonBPDwC tQSadrZI85PaPKLcTXH8V+o6Dd+C3ozUgSsjowbU97ivXvUhC4vczKtoNHB7o6iva6FS V3pwDhyROlrYXiczwuBgyV9sxcGWBJ0RdeX4QrXOeNIJ8sCwuIC/KjFjYMCVw8SZWZic U2W9gEg/GhnioBq4ORM4I4fAQ/GxyMUJ35cbdid8da+ji5VylZhhv+GR7o/OevnGASFo DFbc7Ov+1Bmj50c6OO1XMD0q5jPxODgZMHJ8XdiVgefo5leFhQs9Q3ca1AQF0iWa9Bir vg== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3updgh87mb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 29 Nov 2023 21:50:59 +0000 Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3ATKXwC7010730 for ; Wed, 29 Nov 2023 21:50:59 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 3ukwfk9u0b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 29 Nov 2023 21:50:59 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 3ATLownK066164 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 29 Nov 2023 21:50:58 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E034E58058 for ; Wed, 29 Nov 2023 21:50:57 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA12858057 for ; Wed, 29 Nov 2023 21:50:57 +0000 (GMT) Received: from [9.61.74.19] (unknown [9.61.74.19]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTPS for ; Wed, 29 Nov 2023 21:50:57 +0000 (GMT) Message-ID: Date: Wed, 29 Nov 2023 13:50:57 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] eal/linux: force iova-mode va without pa available To: dev@dpdk.org References: <20231124100904.388453-1-christian.ehrhardt@canonical.com> Content-Language: en-US From: David Christensen In-Reply-To: <20231124100904.388453-1-christian.ehrhardt@canonical.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: pfBAekLTPcGygdumueczagNkGpHAuKjf X-Proofpoint-GUID: pfBAekLTPcGygdumueczagNkGpHAuKjf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-11-29_20,2023-11-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 suspectscore=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 adultscore=0 mlxlogscore=999 spamscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000 definitions=main-2311290169 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 11/24/23 2:09 AM, christian.ehrhardt@canonical.com wrote: > From: David Wilder > > When using --no-huge option physical address are not guaranteed > to be persistent. > > This change effectively makes "--no-huge" the same as > "--no-huge --iova-mode=va". > > When --no-huge is used (or any other condition making physical > addresses unavailable) setting --iova-mode=pa will have no effect. > > Signed-off-by: Christian Ehrhardt > --- > doc/guides/prog_guide/env_abstraction_layer.rst | 9 ++++++--- > lib/eal/linux/eal.c | 16 ++++++++++------ > 2 files changed, 16 insertions(+), 9 deletions(-) > > diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst > index 6debf54efb..20c7355e0f 100644 > --- a/doc/guides/prog_guide/env_abstraction_layer.rst > +++ b/doc/guides/prog_guide/env_abstraction_layer.rst > @@ -559,9 +559,12 @@ IOVA Mode is selected by considering what the current usable Devices on the > system require and/or support. > > On FreeBSD, RTE_IOVA_PA is always the default. On Linux, the IOVA mode is > -detected based on a 2-step heuristic detailed below. > +detected based on a heuristic detailed below. > > -For the first step, EAL asks each bus its requirement in terms of IOVA mode > +For the first step, if no Physical Addresses are available RTE_IOVA_VA is > +selected. > + > +Then EAL asks each bus its requirement in terms of IOVA mode > and decides on a preferred IOVA mode. > > - if all buses report RTE_IOVA_PA, then the preferred IOVA mode is RTE_IOVA_PA, > @@ -575,7 +578,7 @@ and decides on a preferred IOVA mode. > If the buses have expressed no preference on which IOVA mode to pick, then a > default is selected using the following logic: > > -- if physical addresses are not available, RTE_IOVA_VA mode is used > +- if enable_iova_as_pa was not set at build RTE_IOVA_VA mode is used > - if /sys/kernel/iommu_groups is not empty, RTE_IOVA_VA mode is used > - otherwise, RTE_IOVA_PA mode is used > > diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c > index 57da058cec..2f1fce3c54 100644 > --- a/lib/eal/linux/eal.c > +++ b/lib/eal/linux/eal.c > @@ -1067,6 +1067,16 @@ rte_eal_init(int argc, char **argv) > > phys_addrs = rte_eal_using_phys_addrs() != 0; > > + if (!phys_addrs) { > + /* if we have no access to physical addresses, pick IOVA as VA mode. */ > + if (internal_conf->iova_mode == RTE_IOVA_PA) > + RTE_LOG(WARNING, EAL, "WARNING: --iova-mode=pa, but Physical addresses are unavailable, selecting IOVA as VA mode.\n"); > + else > + RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n"); > + internal_conf->iova_mode = RTE_IOVA_VA; > + rte_eal_get_configuration()->iova_mode = internal_conf->iova_mode; > + } > + > /* if no EAL option "--iova-mode=", use bus IOVA scheme */ > if (internal_conf->iova_mode == RTE_IOVA_DC) { > /* autodetect the IOVA mapping mode */ > @@ -1078,12 +1088,6 @@ rte_eal_init(int argc, char **argv) > if (!RTE_IOVA_IN_MBUF) { > iova_mode = RTE_IOVA_VA; > RTE_LOG(DEBUG, EAL, "IOVA as VA mode is forced by build option.\n"); > - } else if (!phys_addrs) { > - /* if we have no access to physical addresses, > - * pick IOVA as VA mode. > - */ > - iova_mode = RTE_IOVA_VA; > - RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n"); > } else if (is_iommu_enabled()) { > /* we have an IOMMU, pick IOVA as VA mode */ > iova_mode = RTE_IOVA_VA; What tests are you running that generate an error without explicitly selecting the iova-mode? When I run the fast-tests I see the system correctly selecting VA without any additional command-line parameters when running 23.11: 21:47:05 DPDK_TEST=acl_autotest MALLOC_PERTURB_=201 /home/drc/src/dpdk/build/app/dpdk-test --no-huge -m 2048 ----------------------------------- output ----------------------------------- stdout: RTE>>acl_autotest acl context @0x179cbd300 socket_id=-1 alg=5 first_load_sz=0 max_rules=196608 rule_size=128 num_rules=0 num_categories=0 num_tries=0 acl context @0x179cbd300 socket_id=-1 alg=5 first_load_sz=0 max_rules=196608 rule_size=128 num_rules=0 num_categories=0 num_tries=0 running test_convert_rules(acl_ipv4vlan_tuple) running test_convert_rules(acl_ipv4vlan_tuple, RTE_ACL_FIELD_TYPE_BITMASK type for IPv4) running test_convert_rules(acl_ipv4vlan_tuple, RTE_ACL_FIELD_TYPE_RANGE type for IPv4) running test_convert_rules(acl_ipv4vlan_tuple: swap VLAN and PORTs order) running test_convert_rules(acl_ipv4vlan_tuple: swap SRC and DST IPv4 order) test_u32_range#1704 starting range test from 0 to 264192 Test OK RTE>> stderr: EAL: Detected CPU lcores: 128 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' Dave