From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f68.google.com (mail-pg0-f68.google.com [74.125.83.68]) by dpdk.org (Postfix) with ESMTP id 70F901B03C for ; Fri, 26 Jan 2018 03:01:56 +0100 (CET) Received: by mail-pg0-f68.google.com with SMTP id q67so6284157pga.9 for ; Thu, 25 Jan 2018 18:01:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QJQuSMJyDIYvYjXoEy7AmN0KzKJ5ZrbEmQrUaa46k8A=; b=dc5dTPQTz1Y/jN/xT0o7WqsTCE3IDdzQDbqd3dnB9PPqoLZNHmgQGXm9VskOoF7g/j RJilNoSLkMG1qyKiZyOVh8rXHw/GjOpmUUXQSu1E8teVCm5s2ao9VrQbf2f5aTQT8xzJ dt+5edLxCF9q1zK9W1K+VQXA0PcmSXdVpUUtq2yWjELhqLcwZIfNyE5hVt2ZrrEv77mY 10/lp8FpiBTx+3O41NqfjoFr93d1pBwzQvBsaxZ785dBuBj1oiNAo2/erU4ayfczahGc mXi6WYbPuIT0IrnwXR5sz3h58VekGdRzJxl58HcHBQ+cl8CbEwKhQ1MYz96BorDy0+w1 Okqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QJQuSMJyDIYvYjXoEy7AmN0KzKJ5ZrbEmQrUaa46k8A=; b=e9oMJkMwJ38+oi8OW8LKMirhVwNaW0OdIm4igEi+R3Fjpi07FFjwIYy3rgIpmsS9lN qUzHh4SrHSi+jJD0z37y+4icbWAUPNzgma85MzPayuBfylLGrY8Wfc3FzErhrQZQP2+a F/sLaYN/t3lpbe0Wq+iDiJ8cpDRWKy2s49hyyZtwC2ABzf7ZkOxS4khFXs46ZlDI5kZb 6XdNcYoUDqG1vxoyjagqpKz2Gq61cBnbD7fAigAuW1NTuGmploithb0zTYV+U/Kc5Ob9 D7lWq1FPTJNqlCZfRgGVO1HoOfJlORE2MfhS8jdfj9csYub064e49hMo721Yg7QNWagt IDEw== X-Gm-Message-State: AKwxytdK3FrVasKFvIAQoQVQRRcewHiw6Psn72ZZkr8ayHi0J7/R6i41 qOeMxFb1v5+CPvzDyw06LdboVzMpWJs= X-Google-Smtp-Source: AH8x225iIcL2Ed0VAnP2zo5is0VJqkCkPSNrTJn79EUWDDx0C+cdeiqGlxhQDOq5HinSvIur8PuMmw== X-Received: by 2002:a17:902:6a81:: with SMTP id n1-v6mr13193811plk.11.1516932115097; Thu, 25 Jan 2018 18:01:55 -0800 (PST) Received: from xeon-e3.lan (204-195-18-133.wavecable.com. [204.195.18.133]) by smtp.gmail.com with ESMTPSA id d5sm22141492pfk.2.2018.01.25.18.01.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Jan 2018 18:01:54 -0800 (PST) From: Stephen Hemminger To: dev@dpdk.org Cc: Stephen Hemminger Date: Thu, 25 Jan 2018 18:01:37 -0800 Message-Id: <20180126020151.979-2-stephen@networkplumber.org> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180126020151.979-1-stephen@networkplumber.org> References: <20180126020151.979-1-stephen@networkplumber.org> Subject: [dpdk-dev] [PATCH v8 01/15] eal: introduce atomic exchange operation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jan 2018 02:01:56 -0000 To handle atomic update of link status (64 bit), every driver was doing its own version using cmpset. Atomic exchange is a useful primitive in its own right; therefore make it a EAL routine. Signed-off-by: Stephen Hemminger --- .../common/include/arch/ppc_64/rte_atomic.h | 21 +++++- .../common/include/arch/x86/rte_atomic.h | 24 +++++++ .../common/include/arch/x86/rte_atomic_32.h | 12 ++++ .../common/include/arch/x86/rte_atomic_64.h | 12 ++++ lib/librte_eal/common/include/generic/rte_atomic.h | 78 ++++++++++++++++++++++ 5 files changed, 146 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h b/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h index 150810cdb365..11fa1117a255 100644 --- a/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h +++ b/lib/librte_eal/common/include/arch/ppc_64/rte_atomic.h @@ -132,6 +132,12 @@ static inline int rte_atomic16_dec_and_test(rte_atomic16_t *v) return __atomic_sub_fetch(&v->cnt, 1, __ATOMIC_ACQUIRE) == 0; } +static inline uint16_t +rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val) +{ + return __atomic_exchange_2(dst, val, __ATOMIC_SEQ_CST); +} + /*------------------------- 32 bit atomic operations -------------------------*/ static inline int @@ -233,6 +239,13 @@ static inline int rte_atomic32_dec_and_test(rte_atomic32_t *v) return ret == 0; } + +static inline uint32_t +rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val) +{ + return __atomic_exchange_4(dst, val, __ATOMIC_SEQ_CST); +} + /*------------------------- 64 bit atomic operations -------------------------*/ static inline int @@ -427,7 +440,6 @@ static inline int rte_atomic64_test_and_set(rte_atomic64_t *v) { return rte_atomic64_cmpset((volatile uint64_t *)&v->cnt, 0, 1); } - /** * Atomically set a 64-bit counter to 0. * @@ -438,6 +450,13 @@ static inline void rte_atomic64_clear(rte_atomic64_t *v) { v->cnt = 0; } + +static inline uint64_t +rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val) +{ + return __atomic_exchange_4(dst, val, __ATOMIC_SEQ_CST); +} + #endif #ifdef __cplusplus diff --git a/lib/librte_eal/common/include/arch/x86/rte_atomic.h b/lib/librte_eal/common/include/arch/x86/rte_atomic.h index 36cfabc38f09..55bfc3903cdc 100644 --- a/lib/librte_eal/common/include/arch/x86/rte_atomic.h +++ b/lib/librte_eal/common/include/arch/x86/rte_atomic.h @@ -60,6 +60,18 @@ rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src) return res; } +static inline uint16_t +rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val) +{ + asm volatile( + MPLOCKED + "xchgw %0, %1;" + : "=r" (val), "=m" (*dst) + : "0" (val), "m" (*dst) + : "memory"); /* no-clobber list */ + return val; +} + static inline int rte_atomic16_test_and_set(rte_atomic16_t *v) { return rte_atomic16_cmpset((volatile uint16_t *)&v->cnt, 0, 1); @@ -134,6 +146,18 @@ rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src) return res; } +static inline uint32_t +rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val) +{ + asm volatile( + MPLOCKED + "xchgl %0, %1;" + : "=r" (val), "=m" (*dst) + : "0" (val), "m" (*dst) + : "memory"); /* no-clobber list */ + return val; +} + static inline int rte_atomic32_test_and_set(rte_atomic32_t *v) { return rte_atomic32_cmpset((volatile uint32_t *)&v->cnt, 0, 1); diff --git a/lib/librte_eal/common/include/arch/x86/rte_atomic_32.h b/lib/librte_eal/common/include/arch/x86/rte_atomic_32.h index fb3abf187998..8d711b6f685d 100644 --- a/lib/librte_eal/common/include/arch/x86/rte_atomic_32.h +++ b/lib/librte_eal/common/include/arch/x86/rte_atomic_32.h @@ -98,6 +98,18 @@ rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src) return res; } +static inline uint64_t +rte_atomic64_exchange(volatile uint64_t *dest, uint64_t val) +{ + uint64_t old; + + do { + old = *dest; + } while (rte_atomic64_cmpset(dest, old, val)); + + return old; +} + static inline void rte_atomic64_init(rte_atomic64_t *v) { diff --git a/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h b/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h index 1a53a766bd72..fd2ec9c53796 100644 --- a/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h +++ b/lib/librte_eal/common/include/arch/x86/rte_atomic_64.h @@ -71,6 +71,18 @@ rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src) return res; } +static inline uint64_t +rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val) +{ + asm volatile( + MPLOCKED + "xchgq %0, %1;" + : "=r" (val), "=m" (*dst) + : "0" (val), "m" (*dst) + : "memory"); /* no-clobber list */ + return val; +} + static inline void rte_atomic64_init(rte_atomic64_t *v) { diff --git a/lib/librte_eal/common/include/generic/rte_atomic.h b/lib/librte_eal/common/include/generic/rte_atomic.h index 3ba7245a3e17..bf12162d02f4 100644 --- a/lib/librte_eal/common/include/generic/rte_atomic.h +++ b/lib/librte_eal/common/include/generic/rte_atomic.h @@ -139,6 +139,32 @@ rte_atomic16_cmpset(volatile uint16_t *dst, uint16_t exp, uint16_t src) } #endif +/** + * Atomic exchange. + * + * (atomic) equivalent to: + * ret = *dst + * *dst = val; + * return ret; + * + * @param dst + * The destination location into which the value will be written. + * @param val + * The new value. + * @return + * The original value at that location + */ +static inline uint16_t +rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val); + +#ifdef RTE_FORCE_INTRINSICS +static inline uint16_t +rte_atomic16_exchange(volatile uint16_t *dst, uint16_t val) +{ + return __atomic_exchange_2(dst, val, __ATOMIC_SEQ_CST); +} +#endif + /** * The atomic counter structure. */ @@ -392,6 +418,32 @@ rte_atomic32_cmpset(volatile uint32_t *dst, uint32_t exp, uint32_t src) } #endif +/** + * Atomic exchange. + * + * (atomic) equivalent to: + * ret = *dst + * *dst = val; + * return ret; + * + * @param dst + * The destination location into which the value will be written. + * @param val + * The new value. + * @return + * The original value at that location + */ +static inline uint32_t +rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val); + +#ifdef RTE_FORCE_INTRINSICS +static inline uint32_t +rte_atomic32_exchange(volatile uint32_t *dst, uint32_t val) +{ + return __atomic_exchange_4(dst, val, __ATOMIC_SEQ_CST); +} +#endif + /** * The atomic counter structure. */ @@ -644,6 +696,32 @@ rte_atomic64_cmpset(volatile uint64_t *dst, uint64_t exp, uint64_t src) } #endif +/** + * Atomic exchange. + * + * (atomic) equivalent to: + * ret = *dst + * *dst = val; + * return ret; + * + * @param dst + * The destination location into which the value will be written. + * @param val + * The new value. + * @return + * The original value at that location + */ +static inline uint64_t +rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val); + +#ifdef RTE_FORCE_INTRINSICS +static inline uint64_t +rte_atomic64_exchange(volatile uint64_t *dst, uint64_t val) +{ + return __atomic_exchange_8(dst, val, __ATOMIC_SEQ_CST); +} +#endif + /** * The atomic counter structure. */ -- 2.15.1