From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 67A81433F4; Tue, 28 Nov 2023 15:40:05 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 002A542DF2; Tue, 28 Nov 2023 15:40:04 +0100 (CET) Received: from smtp-relay-internal-0.canonical.com (smtp-relay-internal-0.canonical.com [185.125.188.122]) by mails.dpdk.org (Postfix) with ESMTP id 4AAF042DDB for ; Tue, 28 Nov 2023 15:40:03 +0100 (CET) Received: from mail-pj1-f70.google.com (mail-pj1-f70.google.com [209.85.216.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 038E43F1C7 for ; Tue, 28 Nov 2023 14:40:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1701182401; bh=8+IVgmX67uDM0bDMw9yQqA01W8bdaOt8tLmTD1didVA=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=Lv68sub6xA3xVOhYxZQZG3evnWdnVAu/2wgB6JbBuv/HY4ADKQvbArlEDZBJ3XBa+ TT2Wj7FN22qLJ4Nx+LfHgDlYruavacNFnHw9/PaP0FgqW4zDD5HY8gfdg/gLR6Eti6 6/k0N+ZrvUdmscD1TqTt2J1BPwAV7GbIJ2TSlNZfcKKESRgWjgjPP2DSxR74kdNirL /BLKWfv0IY0cifbg5CKz9izB/+JItw57lIRHHK98gyRTqSQwoAORAhh3mtUzegR2K9 ZHzW3Yr1XEfCAMSK8rwFcoz3BKio/w15CwY+RqCEEXK3qhEbkXpDI2honRNcriE98a +aLLgz3PIK8CQ== Received: by mail-pj1-f70.google.com with SMTP id 98e67ed59e1d1-2855e4715e4so8107880a91.0 for ; Tue, 28 Nov 2023 06:40:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701182395; x=1701787195; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8+IVgmX67uDM0bDMw9yQqA01W8bdaOt8tLmTD1didVA=; b=r84mvGjDzX8PiKaaAraZHXqDpDLBbkPDI3H5qN5JGfC4Dlc8J6j2OanlMo9rZMcHXe Wq14sRzjCfhksd8fTjPyoShDEQ/kiWna7wV9ZBKm/7sZVQXaZO5OOZzUkleyVJuAsbqd 3A2lzSbeLuiUkqwrvPq5slMptoFOlroZNgYdoq2YELv+UHYXkEYAE4607dB0t3+4kaao 7yNc3Mq/umiMy776AeQJvmF7gEM4k6jVo/I5wqj5pYH8GhdjwHZfYgr4G4i5nLIUh1gL UDKGrq2JHSnhW20WHVCIFGh1ouQvFDngu8pvZzvImq3QOC6ehmZ3p7ia0E4SVoZQhkxU AVuQ== X-Gm-Message-State: AOJu0YxmQZCnmBbFTjipS/UG41fAWli/DzWxf65GfgQU/rGyqp3ZCwFG WJBnHMSoM2nRHxjjy94Pdi9AGA/g1aog4ZG6nzKvPu6Fn2/1ZLDaZlA8WD9KVF2HnxIqE6m8F4s lyHvISckivb/v4CpSFD1Gzf9t4tkioyO8ImKrrHV0ftyu X-Received: by 2002:a17:90a:1997:b0:280:5e66:a1e2 with SMTP id 23-20020a17090a199700b002805e66a1e2mr15481496pji.22.1701182394920; Tue, 28 Nov 2023 06:39:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IExydoB94jNvquYKK+omUNwWPPNqJ1r/88r5S4XWeAngEJBN+AI358T9T9olLDJftpBsMZDXdvdMlVdPotko/k= X-Received: by 2002:a17:90a:1997:b0:280:5e66:a1e2 with SMTP id 23-20020a17090a199700b002805e66a1e2mr15481475pji.22.1701182394623; Tue, 28 Nov 2023 06:39:54 -0800 (PST) MIME-Version: 1.0 References: <20231124100904.388453-1-christian.ehrhardt@canonical.com> <20231124132941.4fe0f846@sovereign> In-Reply-To: <20231124132941.4fe0f846@sovereign> From: Christian Ehrhardt Date: Tue, 28 Nov 2023 15:39:28 +0100 Message-ID: Subject: Re: [PATCH v2] eal/linux: force iova-mode va without pa available To: Dmitry Kozlyuk Cc: dev , Luca Boccassi , David Wilder Content-Type: multipart/alternative; boundary="000000000000548b7e060b37644d" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000548b7e060b37644d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 24, 2023 at 11:29=E2=80=AFAM Dmitry Kozlyuk wrote: > 2023-11-24 11:09 (UTC+0100), christian.ehrhardt@canonical.com: > [...] > > diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c > > index 57da058cec..2f1fce3c54 100644 > > --- a/lib/eal/linux/eal.c > > +++ b/lib/eal/linux/eal.c > > @@ -1067,6 +1067,16 @@ rte_eal_init(int argc, char **argv) > > > > phys_addrs =3D rte_eal_using_phys_addrs() !=3D 0; > > > > + if (!phys_addrs) { > > + /* if we have no access to physical addresses, pick IOVA > as VA mode. */ > > + if (internal_conf->iova_mode =3D=3D RTE_IOVA_PA) > > + RTE_LOG(WARNING, EAL, "WARNING: --iova-mode=3Dpa, > but Physical addresses are unavailable, selecting IOVA as VA mode.\n"); > > If an impossible combination of options is requested, > initialization should fail instead. > You are absolutely right Dmitry. In fact I was only trying to rebase an old patch that we used to carry. But the more I look at and think about it the less I like the approach. A production setup has an admin that should do this consciously. What we actually should change is not the behavior of EAL, but just the test automation to work on no-huge ppc64. That is simpler and has much less impact and therefore probably is the right solution. Consider this patch here withdrawn. I'll submit the fix to the tests in a few seconds. > > + else > > + RTE_LOG(DEBUG, EAL, "Physical addresses are > unavailable, selecting IOVA as VA mode.\n"); > > + internal_conf->iova_mode =3D RTE_IOVA_VA; > > + rte_eal_get_configuration()->iova_mode =3D > internal_conf->iova_mode; > > + } > > + > > /* if no EAL option "--iova-mode=3D", use bus IOVA scheme = */ > > if (internal_conf->iova_mode =3D=3D RTE_IOVA_DC) { > > /* autodetect the IOVA mapping mode */ > > What do you think about keeping the existing code structure: > > if (--iova-mode not specified) { > iova_mode =3D VA if !phys_addrs or !RTE_IOVA_IN_MBUF (with logs) > if (iova_mode =3D=3D DC) { > // autodetect from bus requirements and IOMMU (with logs) > } > rte_eal_get_configuration()->iova_mode =3D iova_mode; > } else { > rte_eal_get_configuration()->iova_mode =3D > internal_conf->iova_mode; > } > // verify rte_eal_get_configuration()->iova_mode > > Note: the logic should be consistent across OS when possible. > > --=20 Christian Ehrhardt Director of Engineering, Ubuntu Server Canonical Ltd --000000000000548b7e060b37644d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Nov 24, 2023 at 11:29=E2=80= =AFAM Dmitry Kozlyuk <dmitry= .kozliuk@gmail.com> wrote:
2023-11-24 11:09 (UTC+0100), christian.ehrhardt@canonical.com<= /a>:
[...]
> diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c
> index 57da058cec..2f1fce3c54 100644
> --- a/lib/eal/linux/eal.c
> +++ b/lib/eal/linux/eal.c
> @@ -1067,6 +1067,16 @@ rte_eal_init(int argc, char **argv)
>=C2=A0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0phys_addrs =3D rte_eal_using_phys_addrs() != =3D 0;
>=C2=A0
> +=C2=A0 =C2=A0 =C2=A0if (!phys_addrs) {
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* if we have no acce= ss to physical addresses, pick IOVA as VA mode. */
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (internal_conf->= ;iova_mode =3D=3D RTE_IOVA_PA)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0RTE_LOG(WARNING, EAL, "WARNING: --iova-mode=3Dpa, but Physical = addresses are unavailable, selecting IOVA as VA mode.\n");

If an impossible combination of options is requested,
initialization should fail instead.



<= /div>
Consider this patch here withdrawn.
I'll submit the= fix to the tests in a few seconds.
=C2=A0
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0else
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, select= ing IOVA as VA mode.\n");
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0internal_conf->iov= a_mode =3D RTE_IOVA_VA;
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rte_eal_get_configura= tion()->iova_mode =3D internal_conf->iova_mode;
> +=C2=A0 =C2=A0 =C2=A0}
> +
>=C2=A0 =C2=A0 =C2=A0 =C2=A0/* if no EAL option "--iova-mode=3D<= pa|va>", use bus IOVA scheme */
>=C2=A0 =C2=A0 =C2=A0 =C2=A0if (internal_conf->iova_mode =3D=3D RTE_I= OVA_DC) {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* autodetect th= e IOVA mapping mode */

What do you think about keeping the existing code structure:

if (--iova-mode not specified) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 iova_mode =3D VA if !phys_addrs or !RTE_IOVA_IN= _MBUF (with logs)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (iova_mode =3D=3D DC) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 // autodetect from = bus requirements and IOMMU (with logs)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 rte_eal_get_configuration()->iova_mode =3D i= ova_mode;
} else {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 rte_eal_get_configuration()->iova_mode =3D =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 internal_conf->i= ova_mode;
}
// verify rte_eal_get_configuration()->iova_mode

Note: the logic should be consistent across OS when possible.



--
Christian Ehrhardt
Di= rector of Engineering, Ubuntu Server
Canonical Ltd
--000000000000548b7e060b37644d--