From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 8133B43FE3;
Wed, 8 May 2024 16:14:41 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 1864B43769;
Wed, 8 May 2024 16:14:35 +0200 (CEST)
Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com
[209.85.218.49]) by mails.dpdk.org (Postfix) with ESMTP id 7809A43579
for <dev@dpdk.org>; Wed, 8 May 2024 13:48:44 +0200 (CEST)
Received: by mail-ej1-f49.google.com with SMTP id
a640c23a62f3a-a5a0013d551so82778266b.2
for <dev@dpdk.org>; Wed, 08 May 2024 04:48:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1715168924; x=1715773724; darn=dpdk.org;
h=content-transfer-encoding:cc:to:subject:message-id:date:from
:in-reply-to:references:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=FUjojWhkdGKyElklQCU4/g42GG9d04pgk6zPF/iFz+I=;
b=N5kIgW0CW8CbwXu8itcjIy+U1Ibxxpk5oD7jmroUs7xAuEO5svlRMwVKAtVYjTwkxz
Oi+uJ5Hj40uPdI71kXmSsdjBkzfpLX+IR83RB9LmvQYWxOlwzk0cxusoT84oSeOZCYjf
74XzbNsYm7fUyTsLT28SQqBcGG+hLY/rLM07qTq+8n6CQIjHu+fr6Ut0KM1F3ZI0QbIE
JBij9ZJF9e3iBEwzVP5XnWH1spMXKBYWVyUllYqrMTpOuZ/8UOzPF1Wr0GOMIhV6OOiJ
+cXMpDlCLWo+lssjheEJAAu9ZVQpQY70KXEwq/+wLtjFfKS+FrmORJ4VZVvxYvVbFDA+
Exmw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1715168924; x=1715773724;
h=content-transfer-encoding:cc:to:subject:message-id:date:from
:in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=FUjojWhkdGKyElklQCU4/g42GG9d04pgk6zPF/iFz+I=;
b=FcNTWFoeZNBxQgbHDKO+f6VRjxCZcyb6GOc3GImJ41UNk4ctGtrHBjZ1Wq59A27bx0
rn/wKR/P8EnNmeMnf5uPbaVOT3UjvswRu3Wv0hwgHfthzwjlzh0BHGRdRyEnqWFjknbr
GMarvvN7vcmTD6rN0nesftCN/l4hcU1fXOqksqefUi65OdvF7kGABX1/2r95lYsrcgi2
2oWsd0b0LB2pUMZCFXtE701yTrTTruD857mEzaX4o681GNBxOBTD66Z3rYmBbjVQgbih
QrImpEAIG74nVd+dutzmj3Bp7oEqjg+4ULSxQ3y/B1Mv5FxBjtC+FU7z5iaUsV4oJUuh
8VEw==
X-Forwarded-Encrypted: i=1;
AJvYcCWWzGQbkx5okv+Pjjf9SyMegzYESGn2fgInnAExVu1rsoav+4MkX11dORoXjzXu4g7GxGS8ZxH6G16VC7Q=
X-Gm-Message-State: AOJu0Yx3GQ+gH6ff3YpM4rOaB0w15tqF+BGkrloqCcyZ/BzBfeArEXZ7
VHSDTI5d7nqYwOt8XHWYsr7/pOH43VhzKGwHtlBaPo7DdYmV5tLAMpHrVEErAik5yX8H9mKFVA8
NDbdSpq+caIZFOCJwCkiv61bdZ4w=
X-Google-Smtp-Source: AGHT+IEVnH0wsdi1oxfHQBzyySl57XK7TzGbt3ncyNqui/Uyixtf8UCTZDn+c8D+DfEYlAMVhI8iqHSovgOjSUHancY=
X-Received: by 2002:a50:8ac2:0:b0:572:5630:d8d6 with SMTP id
4fb4d7f45d1cf-5731d9d1b7amr2121850a12.22.1715168923757; Wed, 08 May 2024
04:48:43 -0700 (PDT)
MIME-Version: 1.0
References: <20240502144149.66446-1-daniel.gregory@bytedance.com>
In-Reply-To: <20240502144149.66446-1-daniel.gregory@bytedance.com>
From: =?UTF-8?Q?Stanis=C5=82aw_Kardach?= <stanislaw.kardach@gmail.com>
Date: Wed, 8 May 2024 13:48:32 +0200
Message-ID: <CAJcPQBrMq5tmJA-E25ccjG8dUXy3Jj_D63=6aRTBCeprm_Neng@mail.gmail.com>
Subject: Re: [RFC PATCH] eal/riscv: add support for zawrs extension
To: Daniel Gregory <daniel.gregory@bytedance.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>, dev@dpdk.org,
Liang Ma <liangma@bytedance.com>, Punit Agrawal <punit.agrawal@bytedance.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Mailman-Approved-At: Wed, 08 May 2024 16:14:33 +0200
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
I'm terribly sorry for the delay in review. I'll try to get to it in
the next 2 days.
On Thu, May 2, 2024 at 4:44=E2=80=AFPM Daniel Gregory
<daniel.gregory@bytedance.com> wrote:
>
> The zawrs extension adds a pair of instructions that stall a core until
> a memory location is written to. This patch uses one of them to
> implement RISCV-specific versions of the rte_wait_until_equal_*
> functions. This is potentially more energy efficient than the default
> implementation that uses rte_pause/Zihintpause.
>
> The technique works as follows:
>
> * Create a reservation set containing the address we want to wait on
> using an atomic load (lr.dw)
> * Call wrs.nto - this blocks until the reservation set is invalidated by
> someone else writing to that address
> * Execution can also resume arbitrarily, so we still need to check
> whether a change occurred and loop if not
>
> Due to RISC-V atomics only supporting naturally aligned word (32 bit)
> and double word (64 bit) loads, I've used pointer rounding and bit
> shifting to implement waiting on 16-bit values.
>
> This new functionality is controlled by a Meson flag that is disabled by
> default.
>
> Signed-off-by: Daniel Gregory <daniel.gregory@bytedance.com>
> Suggested-by: Punit Agrawal <punit.agrawal@bytedance.com>
> ---
>
> Posting as an RFC to get early feedback and enable testing by others
> with Zawrs-enabled hardware. Whilst I have been able to test it compiles
> & passes tests using QEMU, I am waiting on some Zawrs-enabled hardware
> to become available before I carry out performance tests.
>
> Nonetheless, I would be glad to hear any feedback on the general
> approach. Thanks, Daniel
>
> config/riscv/meson.build | 5 ++
> lib/eal/riscv/include/rte_pause.h | 139 ++++++++++++++++++++++++++++++
> 2 files changed, 144 insertions(+)
>
> diff --git a/config/riscv/meson.build b/config/riscv/meson.build
> index 07d7d9da23..4cfdc42ecb 100644
> --- a/config/riscv/meson.build
> +++ b/config/riscv/meson.build
> @@ -26,6 +26,11 @@ flags_common =3D [
> # read from /proc/device-tree/cpus/timebase-frequency. This property=
is
> # guaranteed on Linux, as riscv time_init() requires it.
> ['RTE_RISCV_TIME_FREQ', 0],
> +
> + # Enable use of RISC-V Wait-on-Reservation-Set extension (Zawrs)
> + # Mitigates looping when polling on memory locations
> + # Make sure to add '_zawrs' to your target's -march below
> + ['RTE_RISCV_ZAWRS', false]
> ]
>
> ## SoC-specific options.
> diff --git a/lib/eal/riscv/include/rte_pause.h b/lib/eal/riscv/include/rt=
e_pause.h
> index cb8e9ca52d..e7b43dffa3 100644
> --- a/lib/eal/riscv/include/rte_pause.h
> +++ b/lib/eal/riscv/include/rte_pause.h
> @@ -11,6 +11,12 @@
> extern "C" {
> #endif
>
> +#ifdef RTE_RISCV_ZAWRS
> +#define RTE_WAIT_UNTIL_EQUAL_ARCH_DEFINED
> +#endif
> +
> +#include <rte_debug.h>
> +
> #include "rte_atomic.h"
>
> #include "generic/rte_pause.h"
> @@ -24,6 +30,139 @@ static inline void rte_pause(void)
> asm volatile(".int 0x0100000F" : : : "memory");
> }
>
> +#ifdef RTE_RISCV_ZAWRS
> +
> +/*
> + * Atomic load from an address, it returns either a sign-extended word o=
r
> + * doubleword and creates a 'reservation set' containing the read memory
> + * location. When someone else writes to the reservation set, it is inva=
lidated,
> + * causing any stalled WRS instructions to resume.
> + *
> + * Address needs to be naturally aligned.
> + */
> +#define __RTE_RISCV_LR_32(src, dst, memorder) do { \
> + if ((memorder) =3D=3D rte_memory_order_relaxed) { \
> + asm volatile("lr.w %0, (%1)" \
> + : "=3Dr" (dst) \
> + : "r" (src) \
> + : "memory"); \
> + } else { \
> + asm volatile("lr.w.aq %0, (%1)" \
> + : "=3Dr" (dst) \
> + : "r" (src) \
> + : "memory"); \
> + } } while (0)
> +#define __RTE_RISCV_LR_64(src, dst, memorder) do { \
> + if ((memorder) =3D=3D rte_memory_order_relaxed) { \
> + asm volatile("lr.d %0, (%1)" \
> + : "=3Dr" (dst) \
> + : "r" (src) \
> + : "memory"); \
> + } else { \
> + asm volatile("lr.d.aq %0, (%1)" \
> + : "=3Dr" (dst) \
> + : "r" (src) \
> + : "memory"); \
> + } } while (0)
> +
> +/*
> + * There's not a RISC-V atomic load primitive for halfwords, so cast up =
to a
> + * _naturally aligned_ word and extract the halfword we want
> + */
> +#define __RTE_RISCV_LR_16(src, dst, memorder) do { =
\
> + uint32_t word; \
> + __RTE_RISCV_LR_32(((uintptr_t)(src) & (~3)), word, (memorder)); \
> + if ((size_t)(src) & 3) \
> + (dst) =3D (typeof(dst))(word >> 16); =
\
> + else \
> + (dst) =3D (typeof(dst))(word & 0xFFFF); =
\
> +} while (0)
> +
> +#define __RTE_RISCV_LR(src, dst, memorder, size) { \
> + RTE_BUILD_BUG_ON(size !=3D 16 && size !=3D 32 && size !=3D 64); \
> + if (size =3D=3D 16) \
> + __RTE_RISCV_LR_16(src, dst, memorder); \
> + else if (size =3D=3D 32) \
> + __RTE_RISCV_LR_32(src, dst, memorder); \
> + else if (size =3D=3D 64) \
> + __RTE_RISCV_LR_64(src, dst, memorder); \
> +}
> +
> +/*
> + * Wait-on-Reservation-Set extension instruction, it stalls execution un=
til the
> + * reservation set is invalidated or an interrupt is observed.
> + * A loop is likely still needed as it may stop stalling arbitrarily.
> + */
> +#define __RTE_RISCV_WRS_NTO() { asm volatile("wrs.nto" : : : "memory"); =
}
> +
> +static __rte_always_inline void
> +rte_wait_until_equal_16(volatile uint16_t *addr, uint16_t expected,
> + int memorder)
> +{
> + uint16_t value;
> +
> + RTE_ASSERT(memorder =3D=3D rte_memory_order_acquire ||
> + memorder =3D=3D rte_memory_order_relaxed);
> + RTE_ASSERT(((size_t)addr & 1) =3D=3D 0);
> +
> + __RTE_RISCV_LR_16(addr, value, memorder);
> + while (value !=3D expected) {
> + __RTE_RISCV_WRS_NTO();
> + __RTE_RISCV_LR_16(addr, value, memorder);
> + }
> +}
> +
> +static __rte_always_inline void
> +rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected,
> + int memorder)
> +{
> + uint32_t value;
> +
> + RTE_ASSERT(memorder =3D=3D rte_memory_order_acquire ||
> + memorder =3D=3D rte_memory_order_relaxed);
> + RTE_ASSERT(((size_t)addr & 3) =3D=3D 0);
> +
> + __RTE_RISCV_LR_32(addr, value, memorder);
> + while (value !=3D expected) {
> + __RTE_RISCV_WRS_NTO();
> + __RTE_RISCV_LR_32(addr, value, memorder);
> + }
> +}
> +
> +static __rte_always_inline void
> +rte_wait_until_equal_64(volatile uint64_t *addr, uint64_t expected,
> + int memorder)
> +{
> + uint64_t value;
> +
> + RTE_ASSERT(memorder =3D=3D rte_memory_order_acquire ||
> + memorder =3D=3D rte_memory_order_relaxed);
> + RTE_ASSERT(((size_t)addr & 7) =3D=3D 0);
> +
> + __RTE_RISCV_LR_64(addr, value, memorder);
> + while (value !=3D expected) {
> + __RTE_RISCV_WRS_NTO();
> + __RTE_RISCV_LR_64(addr, value, memorder);
> + }
> +}
> +
> +#define RTE_WAIT_UNTIL_MASKED(addr, mask, cond, expected, memorder) do {=
\
> + RTE_BUILD_BUG_ON(!__builtin_constant_p(memorder)); =
\
> + RTE_BUILD_BUG_ON(memorder !=3D rte_memory_order_acquire && =
\
> + memorder !=3D rte_memory_order_relaxed); =
\
> + RTE_ASSERT(((size_t)(addr) & (sizeof(*(addr)) - 1)) !=3D 0); =
\
> + const uint32_t size =3D sizeof(*(addr)) << 3; =
\
> + typeof(*(addr)) expected_value =3D (expected); =
\
> + typeof(*(addr)) value; =
\
> + __RTE_RISCV_LR((addr), value, memorder, size); =
\
> + while (!((value & (mask)) cond expected_value)) { =
\
> + __RTE_RISCV_WRS_NTO(); =
\
> + __RTE_RISCV_LR((addr), value, memorder, size); =
\
> + } =
\
> +} while (0)
> +
> +#endif /* RTE_RISCV_ZAWRS */
> +
> #ifdef __cplusplus
> }
> #endif
> --
> 2.39.2
>