From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2725744114; Thu, 30 May 2024 19:21:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3F9A7410F2; Thu, 30 May 2024 19:20:54 +0200 (CEST) Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by mails.dpdk.org (Postfix) with ESMTP id 1724F40689 for ; Thu, 30 May 2024 19:20:50 +0200 (CEST) Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-a630ff4ac84so15109966b.1 for ; Thu, 30 May 2024 10:20:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1717089650; x=1717694450; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3hrZuESU+aTV9fNqMzCENu43ecVacJ7EPRhOiWynP9M=; b=K+XnrAvmgUl+oFvjWGKslovqxyMGMahzycoWW7l3j8NQkcPWw9WQBxs1Djpy00eBvs CXrGFlDNDBD+LrHEk65dlvXzlVwGwJKF2NuIv7j0IQmz7+dB20bMYSNKbL8cQ6tOPd1G s59C122fgkXpgJXH8BAI0X9u1RX9Z8Im8hCNCHpSS3WR5ATa/GeTpb0aHWg9utrFLUEP sP/po3V4YEjRcjkNLgtkPw7hbHmiVndSzv6SdcI+Sox+Rs7lPDvIjDp9X3Wo8egDYpG6 8E761D/HoU00ufU+0Wrn59oIzkzhG5cX8NPQeH8TJesao7236Vj5yitTae+XXs6bHdoL +MOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717089650; x=1717694450; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3hrZuESU+aTV9fNqMzCENu43ecVacJ7EPRhOiWynP9M=; b=UOT1cBR5/2IihcCNzBQjaTse47iYHp7c6zYhKgN0kTfuhh6/kgEOl4NRcu3zjj+cfu 8piyzxOqiHHdyRigBDn3uH3J8Hxov2OrCzuoHrZ8LdamrV3seNyqxUH9TvDbevRaFgVy I6MdaEUOu18GegLXs4RlaXOhFdZ9/OR/liI2y6nILgTJosWcYziEgS3gWinps46xSxxw b7M2FRNru3XJESK9ptPRDV+oyBhTyKUI2h03YfV4N08jsfSjyTjY9TWJGjohF3A/bwTc QnTIOLVoAZlzr/ZK7lqf6w1ErPXbI6Fkvi/RVtm2xkTc36Mni3grJtJ0zCAosPTGdrZH KSpQ== X-Forwarded-Encrypted: i=1; AJvYcCUPkCxWC6ORQSqCe/7uCat/CYukAkFDJD8p0Q44dh+afdQRfItNa5ZvO1marevaED6QRHX1mARtLaL40HQ= X-Gm-Message-State: AOJu0YyMtqEExLLDN6Qz1B7/hx0bhFk2vKl5JaPNtHNqGJhH3KsO9Wc1 A8QVggMp/V/5wYZeuBBVaejfQeSKjAmtQJJBv9BiS/u+pnAfuC76/s54P4J2j1Y= X-Google-Smtp-Source: AGHT+IEgTVrXsJJBaJ8WOXNXQW6zd2L/D9g7YFw4dmzeb3I8bahh3Unm2bfcy2HEix6tN3tn/NANjg== X-Received: by 2002:a17:907:104a:b0:a62:c5f:e34a with SMTP id a640c23a62f3a-a65e8e6d5femr179374966b.39.1717089650251; Thu, 30 May 2024 10:20:50 -0700 (PDT) Received: from C02FF2N1MD6T.bytedance.net ([93.115.195.2]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a644f347fd4sm197141366b.212.2024.05.30.10.20.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 May 2024 10:20:49 -0700 (PDT) From: Daniel Gregory To: Stanislaw Kardach Cc: Bruce Richardson , Tyler Retzlaff , dev@dpdk.org, Liang Ma , Punit Agrawal , Daniel Gregory Subject: [PATCH 2/2] eal/riscv: add support for zicbop extension Date: Thu, 30 May 2024 18:19:48 +0100 Message-Id: <20240530171948.19763-3-daniel.gregory@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240530171948.19763-1-daniel.gregory@bytedance.com> References: <20240530171948.19763-1-daniel.gregory@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The zicbop extension adds instructions for prefetching data into cache. Use them to implement RISCV-specific versions of the rte_prefetch* and rte_prefetch*_write functions. - prefetch.r indicates to hardware that the cache block will be accessed by a data read soon - prefetch.w indicates to hardware that the cache block will be accessed by a data write soon These instructions are emitted by __builtin_prefetch on modern versions of Clang (17.0.1+) and GCC (13.1.0+). For earlier versions, we may not have support for assembling Zicbop instructions, so emit the word that encodes a 'prefetch.[rw] 0(a0)' instruction. This new functionality is controlled by a Meson flag that is disabled by default. Whilst it's a hint, like rte_pause(), and so has no effect if the target doesn't support the extension, it requires the address prefetched to be loaded into a0, which may be costly. Signed-off-by: Daniel Gregory Suggested-by: Punit Agrawal --- config/riscv/meson.build | 6 +++ lib/eal/riscv/include/rte_prefetch.h | 57 ++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 4 deletions(-) diff --git a/config/riscv/meson.build b/config/riscv/meson.build index 07d7d9da23..ecf9da1c39 100644 --- a/config/riscv/meson.build +++ b/config/riscv/meson.build @@ -26,6 +26,12 @@ flags_common = [ # read from /proc/device-tree/cpus/timebase-frequency. This property is # guaranteed on Linux, as riscv time_init() requires it. ['RTE_RISCV_TIME_FREQ', 0], + + # When true override the default implementation of the prefetching functions + # (rte_prefetch*) with a version that explicitly uses the Zicbop extension. + # Do not enable when using modern versions of GCC (13.1.0+) or Clang + # (17.0.1+). They will emit these instructions in the default implementation + ['RTE_RISCV_ZICBOP', false], ] ## SoC-specific options. diff --git a/lib/eal/riscv/include/rte_prefetch.h b/lib/eal/riscv/include/rte_prefetch.h index 748cf1b626..82cad526b3 100644 --- a/lib/eal/riscv/include/rte_prefetch.h +++ b/lib/eal/riscv/include/rte_prefetch.h @@ -14,21 +14,42 @@ extern "C" { #include #include + +#ifdef RTE_RISCV_ZICBOP +#define RTE_PREFETCH_WRITE_ARCH_DEFINED +#endif + #include "generic/rte_prefetch.h" +/* + * Modern versions of GCC & Clang will emit prefetch instructions for + * __builtin_prefetch when the Zicbop extension is present. + * The RTE_RISCV_ZICBOP option controls whether we emit them manually for older + * compilers that may not have the support to assemble them. + */ static inline void rte_prefetch0(const volatile void *p) { - RTE_SET_USED(p); +#ifndef RTE_RISCV_ZICBOP + /* by default __builtin_prefetch prepares for a read */ + __builtin_prefetch((const void *)p); +#else + /* prefetch.r 0(a0) */ + register const volatile void *a0 asm("a0") = p; + asm volatile (".int 0x00156013" : : "r" (a0)); +#endif } +/* + * The RISC-V Zicbop extension doesn't have instructions to prefetch to only a + * subset of cache levels, so fallback to rte_prefetch0 + */ static inline void rte_prefetch1(const volatile void *p) { - RTE_SET_USED(p); + rte_prefetch0(p); } - static inline void rte_prefetch2(const volatile void *p) { - RTE_SET_USED(p); + rte_prefetch0(p); } static inline void rte_prefetch_non_temporal(const volatile void *p) @@ -44,6 +65,34 @@ rte_cldemote(const volatile void *p) RTE_SET_USED(p); } +#ifdef RTE_RISCV_ZICBOP +__rte_experimental +static inline void +rte_prefetch0_write(const void *p) +{ + /* prefetch.w 0(a0) */ + register const void *a0 asm("a0") = p; + asm volatile (".int 0x00356013" : : "r" (a0)); +} + +/* + * The RISC-V Zicbop extension doesn't have instructions to prefetch to only a + * subset of cache levels, so fallback to rte_prefetch0_write + */ +__rte_experimental +static inline void +rte_prefetch1_write(const void *p) +{ + rte_prefetch0_write(p); +} +__rte_experimental +static inline void +rte_prefetch2_write(const void *p) +{ + rte_prefetch0_write(p); +} +#endif /* RTE_RISCV_ZICBOP */ + #ifdef __cplusplus } #endif -- 2.39.2