From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 546E7A046B for ; Wed, 24 Jul 2019 13:53:00 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5A3841C1C4; Wed, 24 Jul 2019 13:52:59 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 74ADE1C1A1 for ; Wed, 24 Jul 2019 13:52:58 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6OBoClX031078; Wed, 24 Jul 2019 04:52:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=i62JqXhmZdm3aj61IJp+I3pRsFFanTdYlUiKrYiWbtU=; b=f5z8GtcQNMjp4KAtnXudzyw/jmrYtL1RBrWBiYGPhUYyiWFfWVyKHIaf4MIM8GBfy2Ig zXBLLYAsmCb5RQ0cTSc+ZPelue6tq/aGP+g4JY5I0c0boQx0mtSl+oXnwTXj3CE+YXRD GA8ww2ACmi3ej3CHKEGBOXSZiTXiCvMNMZnghM0iB9ss9J3zQS14n2QBn16auBiLZo4D YK3j014rLw6+PycFfFlkKmNQFMmBjz6ucro7T5Oyzo63M0A/tiuUU6U0/6YNO+80aXO5 swuxg1mZASXYoDW/3UWd9m5YK4kAor+JTMtVKcfaDg732kbL/QoQGVNSN5vFZXgRP0Va YA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2tx61rbv61-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 24 Jul 2019 04:52:54 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 24 Jul 2019 04:52:53 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.59) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Wed, 24 Jul 2019 04:52:53 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Dr17nQHMNds8WRtJEhHKWQDM75QoyAVgBC9AJTJElVynAsBHj0x2c4HUuCJS0VCu37MttAxwQcRDJdzBateZoyBRRcjvYc8r9KH/SsXU0FFFtjYA66e9BL7WsVdnY0P31/vfAn5WMB8G3pVwKVz1A+YxGH1uIb9CkxIlxoH6hAj0NUEqI0mupbgGdrFTQAGTA374TTVk5uFIBNiUzN7Md56GsMA2tFvoL64yS4l0nvb2EvoG0QAikm6KflFa+jZuuJNkA/JmnYKSUxXwqMaBRLjNxj7ukC544Rik4pc3WT+FXZ+y6tP41ybc2FXd2WmTWmHMpn07KgTO8kdqnEQ4wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i62JqXhmZdm3aj61IJp+I3pRsFFanTdYlUiKrYiWbtU=; b=lsOBs52jiM1GAb1DtFpUcG8s6rDapobfe87Ode5aTe73HfhWwg9pKrl66SeJGUDOBHB7a83EsdJL3Cnw6o/AhZZX9STYjnUEnvmbWx3W0YMW7Hd4gWr4diOZK9JdwmMBKXK9R9UzXMgSuXZHiTrMBRgo7XT6dD5JGQQN0FCPzW3f0bmv3e+G6Mm84XALQDbXQK+MlS6NQnjZMI10KTLJSm/xnSyQFs6iFaKaVN+b2wkv2ng1KgWkKkFxJUf9LHwe6MhCrB+z2Af6ufc2KKZu8c8knZe7bDF+s3lDreoh0ahgcDpYi1jNxVQiaL9MJNQaKRzkyU3ZbAzUfCByzNYG6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=marvell.com;dmarc=pass action=none header.from=marvell.com;dkim=pass header.d=marvell.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i62JqXhmZdm3aj61IJp+I3pRsFFanTdYlUiKrYiWbtU=; b=sECgyMyMGhFlXkfV5HuHkv2PDrztjBLy3wCVO3lOEfa8x1Haf2JDkJdqPHeAlj8eC6Ej7r994E8ywfjVbVNGPoOQZ32jOeudc5LSQO+e20qvVNTS4/a2PKWhE9xpfrQLCnJD95/eKGPQl3J2+leHXhSv35xBM71fJam+hDA3SAk= Received: from BYAPR18MB2424.namprd18.prod.outlook.com (20.179.91.149) by BYAPR18MB2918.namprd18.prod.outlook.com (20.179.59.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2094.17; Wed, 24 Jul 2019 11:52:51 +0000 Received: from BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::2d42:12b6:aa2e:2862]) by BYAPR18MB2424.namprd18.prod.outlook.com ([fe80::2d42:12b6:aa2e:2862%4]) with mapi id 15.20.2094.013; Wed, 24 Jul 2019 11:52:51 +0000 From: Jerin Jacob Kollanukkaran To: Gavin Hu , "dev@dpdk.org" CC: "nd@arm.com" , "thomas@monjalon.net" , "stephen@networkplumber.org" , "Pavan Nikhilesh Bhagavatula" , "Honnappa.Nagarahalli@arm.com" Thread-Topic: [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal Thread-Index: AQHVQW111en1NLqNQEi6PszJ0pbCO6bZppVg Date: Wed, 24 Jul 2019 11:52:50 +0000 Message-ID: References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1563896626-44862-2-git-send-email-gavin.hu@arm.com> In-Reply-To: <1563896626-44862-2-git-send-email-gavin.hu@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [106.200.248.176] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 14146df9-d205-4664-3007-08d7102d747c x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BYAPR18MB2918; x-ms-traffictypediagnostic: BYAPR18MB2918: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0108A997B2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(136003)(39860400002)(376002)(366004)(396003)(346002)(199004)(189003)(13464003)(54906003)(14454004)(6116002)(110136005)(2501003)(316002)(71190400001)(305945005)(68736007)(71200400001)(3846002)(33656002)(7696005)(99286004)(74316002)(7736002)(66066001)(2906002)(446003)(11346002)(476003)(8676002)(6506007)(55016002)(5660300002)(81156014)(486006)(26005)(6436002)(76116006)(6306002)(6246003)(52536014)(966005)(66556008)(66476007)(64756008)(66446008)(25786009)(229853002)(66946007)(8936002)(14444005)(81166006)(4326008)(186003)(256004)(9686003)(53546011)(478600001)(86362001)(102836004)(76176011)(53936002); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR18MB2918; H:BYAPR18MB2424.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: L30wShP0sD4n1+xq9rBL1+65nRbNEG1epz6HpPQMaHUZE/9Ep/8unGSJEHr1uGe0gcEwC/FTwBskO9gWIXn3ZjFlrQrP5SBnNFFkARC29FD3CTAwcLS2KOID/SiFf5CMq5k70vNoHocgqWUUrTKTAhSKCMXQvITXFquVl5imMs58Lk7UaDhBENhtr5cII5Dijz4yJEkuwh6fXHCkUvZD+m9yPDZ+Hec1XgFRv5oCZ//qQPiG2ZY5uHjtwVI84iodbzFUBH0+JodufEHSOGVx3JsgvaFxinEUen8x9o/1VWrKXXxm4vrwdR5yzdcqM6Hc9XuT+17kTD/T1NQg6RmDogPhyd79usHGVVrfpkmy+CbJdY1tLe0I8QFFAP9Zctc63ofvDY15Qymhqcdd9Jj1V0vlpqwAjG5o42aYa8eegUQ= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 14146df9-d205-4664-3007-08d7102d747c X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jul 2019 11:52:50.8543 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: jerinj@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR18MB2918 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-24_05:2019-07-24,2019-07-24 signatures=0 Subject: Re: [dpdk-dev] [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Gavin Hu > Sent: Tuesday, July 23, 2019 9:14 PM > To: dev@dpdk.org > Cc: nd@arm.com; thomas@monjalon.net; stephen@networkplumber.org; > Jerin Jacob Kollanukkaran ; Pavan Nikhilesh > Bhagavatula ; > Honnappa.Nagarahalli@arm.com; gavin.hu@arm.com > Subject: [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal >=20 > The rte_wait_until_equalxx APIs abstract the functionality of 'polling fo= r a > memory location to become equal to a given value'. >=20 > Signed-off-by: Gavin Hu > Reviewed-by: Ruifeng Wang > Reviewed-by: Steve Capper > Reviewed-by: Ola Liljedahl > Reviewed-by: Honnappa Nagarahalli > Acked-by: Pavan Nikhilesh > --- > .../common/include/arch/arm/rte_atomic_64.h | 4 + > .../common/include/arch/arm/rte_pause_64.h | 106 > +++++++++++++++++++++ > lib/librte_eal/common/include/generic/rte_pause.h | 39 +++++++- > 3 files changed, 148 insertions(+), 1 deletion(-) >=20 > diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > index 97060e4..8d742c6 100644 > --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > @@ -15,8 +15,12 @@ extern "C" { >=20 > #include "generic/rte_atomic.h" >=20 > +#ifndef dsb > #define dsb(opt) asm volatile("dsb " #opt : : : "memory") > +#endif > +#ifndef dmb > #define dmb(opt) asm volatile("dmb " #opt : : : "memory") > +#endif Is this change required? Please fix the root cause. I do see some build error too. In file included from /home/jerin/dpdk.org/build/include/rte_pause_64.h:13, from /home/jerin/dpdk.org/build/include/rte_pause.h:13, from /home/jerin/dpdk.org/build/include/generic/rte_spinlo= ck.h:25, from /home/jerin/dpdk.org/build/include/rte_spinlock.h:17, from /home/jerin/dpdk.org/drivers/bus/fslmc/mc/mc_sys.c:10= : /home/jerin/dpdk.org/build/include/generic/rte_pause.h: In function 'rte_wa= it_until_equal16': /home/jerin/dpdk.org/build/include/generic/rte_pause.h:44:49: error: macro = "dmb" passed 1 arguments, but takes just 0 44 | __rte_wait_until_equal(addr, expected, memorder); Command to reproduce(gcc 9.1) rm -rf build && unset RTE_KERNELDIR && make -j T=3Darm64-thunderx-linux-gc= c CROSS=3Daarch64-linux-gnu- && sed -ri 's,(CONFIG_RTE_KNI_KMOD=3D)y,\1= n,' build/.config && sed -ri 's,(CONFIG_RTE_LIBRTE_VHOST_NUMA=3D)y,\1n,' b= uild/.config && sed -ri 's,(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=3D)y,\1n,= ' build/.config && sed -ri 's,(CONFIG_RTE_EAL_IGB_UIO=3D)y,\1n,' build/.co= nfig && CC=3D"ccache gcc" make -j test-build CROSS=3Daarch64-linux-gnu- >=20 > #define rte_mb() dsb(sy) >=20 > diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > index 93895d3..1f7be0a 100644 > --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > @@ -17,6 +17,112 @@ static inline void rte_pause(void) > asm volatile("yield" ::: "memory"); > } >=20 > +#ifdef RTE_USE_WFE Do we need it to be under RTE_USE_WFE? If there is no harm, no need to add Conditional compilation to detect build errors, especially config is disabl= ed by default. > +/* Wait for *addr to be updated with expected value */ static > +__rte_always_inline void rte_wait_until_equal16(volatile uint16_t > +*addr, uint16_t expected, int memorder) { > + uint16_t tmp; > + if (memorder =3D=3D __ATOMIC_RELAXED) > + asm volatile( > + "ldxrh %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "b.eq 2f\n" > + "sevl\n" > + "1: wfe\n" > + "ldxrh %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "bne 1b\n" > + "2:\n" > + : [tmp] "=3D&r" (tmp) > + : [addr] "Q"(*addr), [expected] "r"(expected) > + : "cc", "memory"); > + else > + asm volatile( > + "ldaxrh %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "b.eq 2f\n" > + "sevl\n" > + "1: wfe\n" > + "ldaxrh %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "bne 1b\n" > + "2:\n" > + : [tmp] "=3D&r" (tmp) > + : [addr] "Q"(*addr), [expected] "r"(expected) > + : "cc", "memory"); > +} > + > +static __rte_always_inline void > +rte_wait_until_equal32(volatile uint32_t *addr, uint32_t expected, int > +memorder) { > + uint32_t tmp; > + if (memorder =3D=3D __ATOMIC_RELAXED) > + asm volatile( > + "ldxr %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "b.eq 2f\n" > + "sevl\n" > + "1: wfe\n" > + "ldxr %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "bne 1b\n" > + "2:\n" > + : [tmp] "=3D&r" (tmp) > + : [addr] "Q"(*addr), [expected] "r"(expected) > + : "cc", "memory"); > + else > + asm volatile( > + "ldaxr %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "b.eq 2f\n" > + "sevl\n" > + "1: wfe\n" > + "ldaxr %w[tmp], %w[addr]\n" > + "cmp %w[tmp], %w[expected]\n" > + "bne 1b\n" > + "2:\n" > + : [tmp] "=3D&r" (tmp) > + : [addr] "Q"(*addr), [expected] "r"(expected) > + : "cc", "memory"); > +} > + > +static __rte_always_inline void > +rte_wait_until_equal64(volatile uint64_t *addr, uint64_t expected, int > +memorder) { > + uint64_t tmp; > + if (memorder =3D=3D __ATOMIC_RELAXED) > + asm volatile( > + "ldxr %x[tmp], %x[addr]\n" > + "cmp %x[tmp], %x[expected]\n" > + "b.eq 2f\n" > + "sevl\n" > + "1: wfe\n" > + "ldxr %x[tmp], %x[addr]\n" > + "cmp %x[tmp], %x[expected]\n" > + "bne 1b\n" > + "2:\n" > + : [tmp] "=3D&r" (tmp) > + : [addr] "Q"(*addr), [expected] "r"(expected) > + : "cc", "memory"); > + else > + asm volatile( > + "ldaxr %x[tmp], %x[addr]\n" > + "cmp %x[tmp], %x[expected]\n" > + "b.eq 2f\n" > + "sevl\n" > + "1: wfe\n" > + "ldaxr %x[tmp], %x[addr]\n" > + "cmp %x[tmp], %x[expected]\n" > + "bne 1b\n" > + "2:\n" > + : [tmp] "=3D&r" (tmp) > + : [addr] "Q"(*addr), [expected] "r"(expected) > + : "cc", "memory"); Duplication of code. Please introduce a macro for assembly Skelton. Something like http://patches.dpdk.org/patch/56949/ > +} > + > +#endif > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_eal/common/include/generic/rte_pause.h > b/lib/librte_eal/common/include/generic/rte_pause.h > index 52bd4db..8f5f025 100644 > --- a/lib/librte_eal/common/include/generic/rte_pause.h > +++ b/lib/librte_eal/common/include/generic/rte_pause.h > @@ -4,7 +4,6 @@ >=20 > #ifndef _RTE_PAUSE_H_ > #define _RTE_PAUSE_H_ > - > /** > * @file > * > @@ -12,6 +11,10 @@ > * > */ >=20 > +#include > +#include > +#include > + > /** > * Pause CPU execution for a short while > * > @@ -20,4 +23,38 @@ > */ > static inline void rte_pause(void); >=20 > +#if !defined(RTE_USE_WFE) > +#ifdef RTE_USE_C11_MEM_MODEL > +#define __rte_wait_until_equal(addr, expected, memorder) do {\ > + while (__atomic_load_n(addr, memorder) !=3D expected) \ > + rte_pause();\ > +} while (0) > +#else > +#define __rte_wait_until_equal(addr, expected, memorder) do {\ > + while (*addr !=3D expected)\ > + rte_pause();\ > + if (memorder !=3D __ATOMIC_RELAXED)\ > + rte_smp_rmb();\ Is this correct wrt all memorder? If there is no specific gain on no C11 mem model, let have only C11 memmode= l Aka remove RTE_USE_C11_MEM_MODEL > +} while (0) > +#endif > + Spotted public API. Lets have prototype with very good documentation on the API details. > +static __rte_always_inline void > +rte_wait_until_equal16(volatile uint16_t *addr, uint16_t expected, int > +memorder) { > + __rte_wait_until_equal(addr, expected, memorder); } > + > +static __rte_always_inline void > +rte_wait_until_equal32(volatile uint32_t *addr, uint32_t expected, int > +memorder) { > + __rte_wait_until_equal(addr, expected, memorder); } > + > +static __rte_always_inline void > +rte_wait_until_equal64(volatile uint64_t *addr, uint64_t expected, int > +memorder) { > + __rte_wait_until_equal(addr, expected, memorder); } #endif /* > +RTE_USE_WFE */ > + > #endif /* _RTE_PAUSE_H_ */ > -- > 2.7.4