From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0BD3A046B for ; Wed, 24 Jul 2019 20:10:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0F53C1C244; Wed, 24 Jul 2019 20:10:20 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40051.outbound.protection.outlook.com [40.107.4.51]) by dpdk.org (Postfix) with ESMTP id ED2B01C241 for ; Wed, 24 Jul 2019 20:10:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Jy+CaBemMKkMg8xSP1icseedvo6a9Xpp++2ID2fdmvXrAHOvCiLZ/wQO3H7dsqs/PWEG9qSh3j+Ya8ZQr3oMTPeuAfNw6FoK8QPsI4fUCyAZ0TKBAu0zscKTljs2ADWlsYdxWLuw9Yl6qfsidrliG/Gc4jb006ohoqTt8bm7tngTRIMVeh2aiSAm7KEGNsDgX+v99DDPw3lYJ2ETaw49nXbLF0n0In91gLuTi3UwJr2rAJ0H51j2zPdxH1ZmgFrc3xyMlOu4pDfnomT8i2teNH61X3THoQHcopLWaV/AzyAlHdO5FfUwKftN2+hgkR7o9Dm3ix7tyxuc4fh33vvxng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NiUdzffXTutyBc8VuwCF87Hgbfx/y7M5NWsI1Bf/TKY=; b=bpDl7swT43zEV6ZHUAdFk0HN1huA7C4jPtq9e/g/mVIKN0pZ/jx4XaS21q8jdY5iRzb7BmRpafUu720Yd3MAVB/wwf0m52+sIwHyHr9wBf78xXZneHPA1oR4VKtU6rs4MSYOSC9Qi8YY0gW8OHUB9rkdm7MPw/+rczDfjlnq9pPVSoYnXarPsPPhjG0BIBnKntfCEaNzjqcuUPTjIW7fuyVARPx6VMbUcJjfTXqGPKZo7FiCCPxoeaf+0X8qovI5hBtOSo7a/pIKXiTOTXLgWfCRIcjRY+XYfZR2or2wIrmxXAqsQ17yxmDSIhAVsDdjRieaqcVM+KA7pGstgoHQgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=arm.com;dmarc=pass action=none header.from=arm.com;dkim=pass header.d=arm.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NiUdzffXTutyBc8VuwCF87Hgbfx/y7M5NWsI1Bf/TKY=; b=gWdJSjvQwOgbEdRVTgkYwkuTS2rGZnPC551bYpjJ5fGVj1IpLrBy9p1Ccfo9KiXTIMiJR+pnVwji240zFCLDjlsirStpUQRahybPwLcQ08V+nkkxZ2ksYpTWRdYO0EI1BH0Kjy36vr1HmhmdDgS/7IUqgiHKQIGgvF2LP/goG7A= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (52.133.244.200) by VI1PR08MB3008.eurprd08.prod.outlook.com (52.133.14.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2115.10; Wed, 24 Jul 2019 18:10:17 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::8130:867e:2371:9c0]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::8130:867e:2371:9c0%2]) with mapi id 15.20.2094.017; Wed, 24 Jul 2019 18:10:17 +0000 From: "Gavin Hu (Arm Technology China)" To: "jerinj@marvell.com" , "dev@dpdk.org" CC: nd , "thomas@monjalon.net" , "stephen@networkplumber.org" , Pavan Nikhilesh Bhagavatula , Honnappa Nagarahalli Thread-Topic: [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal Thread-Index: AQHVQW1zyMb0c33Rs0ePsnUqyB32JabZqccAgABpCGA= Date: Wed, 24 Jul 2019 18:10:16 +0000 Message-ID: References: <1561911676-37718-1-git-send-email-gavin.hu@arm.com> <1563896626-44862-2-git-send-email-gavin.hu@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: de3edb3f-20aa-41a8-b2a7-ca120a9862f9.0 x-checkrecipientchecked: true authentication-results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e86399c3-38b0-4381-d4e5-08d710622e98 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:VI1PR08MB3008; x-ms-traffictypediagnostic: VI1PR08MB3008: x-ms-exchange-purlcount: 1 x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-microsoft-antispam-prvs: nodisclaimer: True x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0108A997B2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(396003)(376002)(39860400002)(136003)(346002)(189003)(199004)(13464003)(7696005)(33656002)(66066001)(25786009)(2501003)(6246003)(2906002)(99286004)(7736002)(478600001)(305945005)(14444005)(966005)(4326008)(256004)(71200400001)(71190400001)(74316002)(14454004)(76176011)(6306002)(446003)(68736007)(6436002)(11346002)(476003)(486006)(186003)(26005)(102836004)(76116006)(8676002)(5660300002)(229853002)(55236004)(55016002)(53546011)(6506007)(6116002)(3846002)(53936002)(110136005)(54906003)(9686003)(52536014)(66476007)(66556008)(64756008)(66446008)(66946007)(81156014)(81166006)(316002)(8936002)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB3008; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 5PZPBt4woQvg0NDamMd64Ncmpk62MrxKrPT/FcBjhRAkY5VqiFRS8K+QpPVfhqXsDUKQX8caf0BxBlqXUm24XS7JPOpU18f1fumh858EIf6j/KQ+vo3C04rZ736wEFQ+BxzpYG2hZZ2MCzCchwwOAXTs2ePsuo2qlxYNps31lrYMfJoSJF12ynr6b008Kop+RduyssXU8Y8iS62AA6uT9wlZGbHVIKRWp6CJW7JB6eSXxzd4LAOZQxkHXuNomO9V2m1oEOdLkR4UlWkqG3WneOja2J/sdbUXyDXBJilNL1SAu2cwtKiog2qmYlb+pA8YEAuQBd7eY545M/gJV/0wMyNaB11LUrvnFpUmKo0ArHO3fgUhFxFadr6U7gzkzopkLt1LsDy6gjVTtUYyJ9CvHXf6ALzSphq669Ie7hPsUk0= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: e86399c3-38b0-4381-d4e5-08d710622e98 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jul 2019 18:10:16.9560 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Gavin.Hu@arm.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3008 Subject: Re: [dpdk-dev] [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Jerin, > -----Original Message----- > From: Jerin Jacob Kollanukkaran > Sent: Wednesday, July 24, 2019 7:53 PM > To: Gavin Hu (Arm Technology China) ; dev@dpdk.org > Cc: nd ; thomas@monjalon.net; > stephen@networkplumber.org; Pavan Nikhilesh Bhagavatula > ; Honnappa Nagarahalli > > Subject: RE: [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal >=20 > > -----Original Message----- > > From: Gavin Hu > > Sent: Tuesday, July 23, 2019 9:14 PM > > To: dev@dpdk.org > > Cc: nd@arm.com; thomas@monjalon.net; stephen@networkplumber.org; > > Jerin Jacob Kollanukkaran ; Pavan Nikhilesh > > Bhagavatula ; > > Honnappa.Nagarahalli@arm.com; gavin.hu@arm.com > > Subject: [EXT] [PATCH v3 1/5] eal: add the APIs to wait until equal > > > > The rte_wait_until_equalxx APIs abstract the functionality of 'polling = for a > > memory location to become equal to a given value'. > > > > Signed-off-by: Gavin Hu > > Reviewed-by: Ruifeng Wang > > Reviewed-by: Steve Capper > > Reviewed-by: Ola Liljedahl > > Reviewed-by: Honnappa Nagarahalli > > Acked-by: Pavan Nikhilesh > > --- > > .../common/include/arch/arm/rte_atomic_64.h | 4 + > > .../common/include/arch/arm/rte_pause_64.h | 106 > > +++++++++++++++++++++ > > lib/librte_eal/common/include/generic/rte_pause.h | 39 +++++++- > > 3 files changed, 148 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > > b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > > index 97060e4..8d742c6 100644 > > --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > > +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h > > @@ -15,8 +15,12 @@ extern "C" { > > > > #include "generic/rte_atomic.h" > > > > +#ifndef dsb > > #define dsb(opt) asm volatile("dsb " #opt : : : "memory") > > +#endif > > +#ifndef dmb > > #define dmb(opt) asm volatile("dmb " #opt : : : "memory") > > +#endif >=20 > Is this change required? Please fix the root cause. > I do see some build error too. >=20 > In file included from /home/jerin/dpdk.org/build/include/rte_pause_64.h:1= 3, > from /home/jerin/dpdk.org/build/include/rte_pause.h:13, > from > /home/jerin/dpdk.org/build/include/generic/rte_spinlock.h:25, > from /home/jerin/dpdk.org/build/include/rte_spinlock.h:1= 7, > from /home/jerin/dpdk.org/drivers/bus/fslmc/mc/mc_sys.c:= 10: > /home/jerin/dpdk.org/build/include/generic/rte_pause.h: In function > 'rte_wait_until_equal16': > /home/jerin/dpdk.org/build/include/generic/rte_pause.h:44:49: error: > macro "dmb" passed 1 arguments, but takes just 0 > 44 | __rte_wait_until_equal(addr, expected, memorder); >=20 > Command to reproduce(gcc 9.1) >=20 > rm -rf build && unset RTE_KERNELDIR && make -j T=3Darm64-thunderx-linux- > gcc CROSS=3Daarch64-linux-gnu- && sed -ri > 's,(CONFIG_RTE_KNI_KMOD=3D)y,\1n,' build/.config && sed -ri > 's,(CONFIG_RTE_LIBRTE_VHOST_NUMA=3D)y,\1n,' build/.config && sed -ri > 's,(CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=3D)y,\1n,' build/.config && > sed -ri 's,(CONFIG_RTE_EAL_IGB_UIO=3D)y,\1n,' build/.config && CC=3D"cca= che > gcc" make -j test-build CROSS=3Daarch64-linux-gnu- >=20 >=20 > > > > #define rte_mb() dsb(sy) > > > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > index 93895d3..1f7be0a 100644 > > --- a/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > +++ b/lib/librte_eal/common/include/arch/arm/rte_pause_64.h > > @@ -17,6 +17,112 @@ static inline void rte_pause(void) > > asm volatile("yield" ::: "memory"); > > } > > > > +#ifdef RTE_USE_WFE >=20 > Do we need it to be under RTE_USE_WFE? If there is no harm, no need to > add > Conditional compilation to detect build errors, especially config is disa= bled > by default. >=20 > > +/* Wait for *addr to be updated with expected value */ static > > +__rte_always_inline void rte_wait_until_equal16(volatile uint16_t > > +*addr, uint16_t expected, int memorder) { > > + uint16_t tmp; > > + if (memorder =3D=3D __ATOMIC_RELAXED) > > + asm volatile( > > + "ldxrh %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "b.eq 2f\n" > > + "sevl\n" > > + "1: wfe\n" > > + "ldxrh %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "bne 1b\n" > > + "2:\n" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "Q"(*addr), [expected] "r"(expected) > > + : "cc", "memory"); > > + else > > + asm volatile( > > + "ldaxrh %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "b.eq 2f\n" > > + "sevl\n" > > + "1: wfe\n" > > + "ldaxrh %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "bne 1b\n" > > + "2:\n" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "Q"(*addr), [expected] "r"(expected) > > + : "cc", "memory"); > > +} > > + > > +static __rte_always_inline void > > +rte_wait_until_equal32(volatile uint32_t *addr, uint32_t expected, int > > +memorder) { > > + uint32_t tmp; > > + if (memorder =3D=3D __ATOMIC_RELAXED) > > + asm volatile( > > + "ldxr %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "b.eq 2f\n" > > + "sevl\n" > > + "1: wfe\n" > > + "ldxr %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "bne 1b\n" > > + "2:\n" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "Q"(*addr), [expected] "r"(expected) > > + : "cc", "memory"); > > + else > > + asm volatile( > > + "ldaxr %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "b.eq 2f\n" > > + "sevl\n" > > + "1: wfe\n" > > + "ldaxr %w[tmp], %w[addr]\n" > > + "cmp %w[tmp], %w[expected]\n" > > + "bne 1b\n" > > + "2:\n" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "Q"(*addr), [expected] "r"(expected) > > + : "cc", "memory"); > > +} > > + > > +static __rte_always_inline void > > +rte_wait_until_equal64(volatile uint64_t *addr, uint64_t expected, int > > +memorder) { > > + uint64_t tmp; > > + if (memorder =3D=3D __ATOMIC_RELAXED) > > + asm volatile( > > + "ldxr %x[tmp], %x[addr]\n" > > + "cmp %x[tmp], %x[expected]\n" > > + "b.eq 2f\n" > > + "sevl\n" > > + "1: wfe\n" > > + "ldxr %x[tmp], %x[addr]\n" > > + "cmp %x[tmp], %x[expected]\n" > > + "bne 1b\n" > > + "2:\n" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "Q"(*addr), [expected] "r"(expected) > > + : "cc", "memory"); > > + else > > + asm volatile( > > + "ldaxr %x[tmp], %x[addr]\n" > > + "cmp %x[tmp], %x[expected]\n" > > + "b.eq 2f\n" > > + "sevl\n" > > + "1: wfe\n" > > + "ldaxr %x[tmp], %x[addr]\n" > > + "cmp %x[tmp], %x[expected]\n" > > + "bne 1b\n" > > + "2:\n" > > + : [tmp] "=3D&r" (tmp) > > + : [addr] "Q"(*addr), [expected] "r"(expected) > > + : "cc", "memory"); >=20 > Duplication of code. Please introduce a macro for assembly Skelton. > Something like >=20 > http://patches.dpdk.org/patch/56949/ >=20 > > +} > > + > > +#endif > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/librte_eal/common/include/generic/rte_pause.h > > b/lib/librte_eal/common/include/generic/rte_pause.h > > index 52bd4db..8f5f025 100644 > > --- a/lib/librte_eal/common/include/generic/rte_pause.h > > +++ b/lib/librte_eal/common/include/generic/rte_pause.h > > @@ -4,7 +4,6 @@ > > > > #ifndef _RTE_PAUSE_H_ > > #define _RTE_PAUSE_H_ > > - > > /** > > * @file > > * > > @@ -12,6 +11,10 @@ > > * > > */ > > > > +#include > > +#include > > +#include > > + > > /** > > * Pause CPU execution for a short while > > * > > @@ -20,4 +23,38 @@ > > */ > > static inline void rte_pause(void); > > > > +#if !defined(RTE_USE_WFE) > > +#ifdef RTE_USE_C11_MEM_MODEL > > +#define __rte_wait_until_equal(addr, expected, memorder) do {\ > > + while (__atomic_load_n(addr, memorder) !=3D expected) \ > > + rte_pause();\ > > +} while (0) > > +#else > > +#define __rte_wait_until_equal(addr, expected, memorder) do {\ > > + while (*addr !=3D expected)\ > > + rte_pause();\ > > + if (memorder !=3D __ATOMIC_RELAXED)\ > > + rte_smp_rmb();\ >=20 > Is this correct wrt all memorder? > If there is no specific gain on no C11 mem model, let have only C11 > memmodel > Aka remove RTE_USE_C11_MEM_MODEL I am looking into all your comments(thanks!) and will submit a new version.= =20 > > +} while (0) > > +#endif > > + >=20 > Spotted public API. Lets have prototype with very good documentation on > the > API details. >=20 >=20 > > +static __rte_always_inline void > > +rte_wait_until_equal16(volatile uint16_t *addr, uint16_t expected, int > > +memorder) { > > + __rte_wait_until_equal(addr, expected, memorder); } > > + > > +static __rte_always_inline void > > +rte_wait_until_equal32(volatile uint32_t *addr, uint32_t expected, int > > +memorder) { > > + __rte_wait_until_equal(addr, expected, memorder); } > > + > > +static __rte_always_inline void > > +rte_wait_until_equal64(volatile uint64_t *addr, uint64_t expected, int > > +memorder) { > > + __rte_wait_until_equal(addr, expected, memorder); } #endif /* > > +RTE_USE_WFE */ > > + > > #endif /* _RTE_PAUSE_H_ */ > > -- > > 2.7.4