From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CC3CEA0549; Thu, 16 Jul 2020 14:33:06 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 54B541C23E; Thu, 16 Jul 2020 14:32:48 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 11D4B1C23C for ; Thu, 16 Jul 2020 14:32:45 +0200 (CEST) IronPort-SDR: JcZVI95S3SlBYcSiR1mzYt4+uJ4t6PYq22J0+bfqCLaibRJABhNmfiQYWUrtxST7e/IOGk1lwd Cd7KNHaZ5yOg== X-IronPort-AV: E=McAfee;i="6000,8403,9683"; a="234224350" X-IronPort-AV: E=Sophos;i="5.75,359,1589266800"; d="scan'208";a="234224350" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jul 2020 05:32:45 -0700 IronPort-SDR: 58hksonqUG3SzB9x8eMsVRcWFxQLXWRk03iq6PSjD0ycwTKAEerzRkwL2WwJa9bOBPxKNgVNFZ GZZ+1YTN5YKQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,359,1589266800"; d="scan'208";a="286466675" Received: from silpixa00383879.ir.intel.com ([10.237.222.142]) by orsmga006.jf.intel.com with ESMTP; 16 Jul 2020 05:32:42 -0700 From: Radu Nicolau To: dev@dpdk.org Cc: beilei.xing@intel.com, jia.guo@intel.com, bruce.richardson@intel.com, konstantin.ananyev@intel.com, jerinjacobk@gmail.com, david.marchand@redhat.com, fiona.trahe@intel.com, wei.zhao1@intel.com, ruifeng.wang@arm.com, Radu Nicolau Date: Thu, 16 Jul 2020 13:29:13 +0100 Message-Id: <1594902556-19511-2-git-send-email-radu.nicolau@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1594902556-19511-1-git-send-email-radu.nicolau@intel.com> References: <1591870283-7776-1-git-send-email-radu.nicolau@intel.com> <1594902556-19511-1-git-send-email-radu.nicolau@intel.com> Subject: [dpdk-dev] [PATCH v7 1/4] eal: add WC store functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add rte_write32_wc and rte_write32_wc_relaxed functions that implement 32bit stores using write combining memory protocol. Provided generic stubs and x86 implementation. Signed-off-by: Radu Nicolau Acked-by: Bruce Richardson --- v7: fix ARM64 build lib/librte_eal/arm/include/rte_io_64.h | 12 +++++++ lib/librte_eal/include/generic/rte_io.h | 48 ++++++++++++++++++++++++++++ lib/librte_eal/x86/include/rte_io.h | 56 +++++++++++++++++++++++++++++++++ 3 files changed, 116 insertions(+) diff --git a/lib/librte_eal/arm/include/rte_io_64.h b/lib/librte_eal/arm/include/rte_io_64.h index e534624..d07d9cb 100644 --- a/lib/librte_eal/arm/include/rte_io_64.h +++ b/lib/librte_eal/arm/include/rte_io_64.h @@ -164,6 +164,18 @@ rte_write64(uint64_t value, volatile void *addr) rte_write64_relaxed(value, addr); } +static __rte_always_inline void +rte_write32_wc(uint32_t value, volatile void *addr) +{ + rte_write32(value, addr); +} + +static __rte_always_inline void +rte_write32_wc_relaxed(uint32_t value, volatile void *addr) +{ + rte_write32_relaxed(value, addr); +} + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/include/generic/rte_io.h b/lib/librte_eal/include/generic/rte_io.h index da457f7..0669baa 100644 --- a/lib/librte_eal/include/generic/rte_io.h +++ b/lib/librte_eal/include/generic/rte_io.h @@ -229,6 +229,40 @@ rte_write32(uint32_t value, volatile void *addr); static inline void rte_write64(uint64_t value, volatile void *addr); +/** + * Write a 32-bit value to I/O device memory address addr using write + * combining memory write protocol. Depending on the platform write combining + * may not be available and/or may be treated as a hint and the behavior may + * fallback to a regular store. + * + * @param value + * Value to write + * @param addr + * I/O memory address to write the value to + */ +__rte_experimental +static inline void +rte_write32_wc(uint32_t value, volatile void *addr); + +/** + * Write a 32-bit value to I/O device memory address addr using write + * combining memory write protocol. Depending on the platform write combining + * may not be available and/or may be treated as a hint and the behavior may + * fallback to a regular store. + * + * The relaxed version does not have additional I/O memory barrier, useful in + * accessing the device registers of integrated controllers which implicitly + * strongly ordered with respect to memory access. + * + * @param value + * Value to write + * @param addr + * I/O memory address to write the value to + */ +__rte_experimental +static inline void +rte_write32_wc_relaxed(uint32_t value, volatile void *addr); + #endif /* __DOXYGEN__ */ #ifndef RTE_OVERRIDE_IO_H @@ -345,6 +379,20 @@ rte_write64(uint64_t value, volatile void *addr) rte_write64_relaxed(value, addr); } +#ifndef RTE_NATIVE_WRITE32_WC +static __rte_always_inline void +rte_write32_wc(uint32_t value, volatile void *addr) +{ + rte_write32(value, addr); +} + +static __rte_always_inline void +rte_write32_wc_relaxed(uint32_t value, volatile void *addr) +{ + rte_write32_relaxed(value, addr); +} +#endif /* RTE_NATIVE_WRITE32_WC */ + #endif /* RTE_OVERRIDE_IO_H */ #endif /* _RTE_IO_H_ */ diff --git a/lib/librte_eal/x86/include/rte_io.h b/lib/librte_eal/x86/include/rte_io.h index 2db71b1..c95ed67 100644 --- a/lib/librte_eal/x86/include/rte_io.h +++ b/lib/librte_eal/x86/include/rte_io.h @@ -9,8 +9,64 @@ extern "C" { #endif +#include "rte_cpuflags.h" + +#define RTE_NATIVE_WRITE32_WC #include "generic/rte_io.h" +/** + * @internal + * MOVDIRI wrapper. + */ +static __rte_always_inline void +_rte_x86_movdiri(uint32_t value, volatile void *addr) +{ + asm volatile( + /* MOVDIRI */ + ".byte 0x40, 0x0f, 0x38, 0xf9, 0x02" + : + : "a" (value), "d" (addr)); +} + +static __rte_always_inline void +rte_write32_wc(uint32_t value, volatile void *addr) +{ + static int _x86_movdiri_flag = -1; + if (_x86_movdiri_flag == 1) { + rte_wmb(); + _rte_x86_movdiri(value, addr); + } else if (_x86_movdiri_flag == 0) { + rte_write32(value, addr); + } else { + _x86_movdiri_flag = + (rte_cpu_get_flag_enabled(RTE_CPUFLAG_MOVDIRI) > 0); + if (_x86_movdiri_flag == 1) { + rte_wmb(); + _rte_x86_movdiri(value, addr); + } else { + rte_write32(value, addr); + } + } +} + +static __rte_always_inline void +rte_write32_wc_relaxed(uint32_t value, volatile void *addr) +{ + static int _x86_movdiri_flag = -1; + if (_x86_movdiri_flag == 1) { + _rte_x86_movdiri(value, addr); + } else if (_x86_movdiri_flag == 0) { + rte_write32_relaxed(value, addr); + } else { + _x86_movdiri_flag = + (rte_cpu_get_flag_enabled(RTE_CPUFLAG_MOVDIRI) > 0); + if (_x86_movdiri_flag == 1) + _rte_x86_movdiri(value, addr); + else + rte_write32_relaxed(value, addr); + } +} + #ifdef __cplusplus } #endif -- 2.7.4