From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 85EAEA0597; Wed, 8 Apr 2020 23:56:32 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 65A041C1F0; Wed, 8 Apr 2020 23:56:31 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70059.outbound.protection.outlook.com [40.107.7.59]) by dpdk.org (Postfix) with ESMTP id 4B6DD1C1D3 for ; Wed, 8 Apr 2020 23:56:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IRAga+iEx4YoTB8R8nkFREk5VrkcD95bVRX/+XJTAJE=; b=dG1puBVFV5PWfKTjHAyMf/imTQx8rcTTqyBW0JaHEgqfjGVlZjKKsI/7qeFM/UyfIynb4JijbNJGcjR7JfsYbWZPxj2EQpPUbGzR5PR0UF3Fdur48H19TinduCUXhqfwdSbj5KqYHFWDubW6akZqgcLzL2gY4hcH0Jt5vLm6xxc= Received: from AM6P193CA0106.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:88::47) by DB7PR08MB3068.eurprd08.prod.outlook.com (2603:10a6:5:18::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.15; Wed, 8 Apr 2020 21:56:27 +0000 Received: from VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:88:cafe::7e) by AM6P193CA0106.outlook.office365.com (2603:10a6:209:88::47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2878.17 via Frontend Transport; Wed, 8 Apr 2020 21:56:27 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT016.mail.protection.outlook.com (10.152.18.115) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.17 via Frontend Transport; Wed, 8 Apr 2020 21:56:26 +0000 Received: ("Tessian outbound e2c88df8bbbe:v50"); Wed, 08 Apr 2020 21:56:26 +0000 X-CR-MTA-TID: 64aa7808 Received: from 0f27258647d9.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 5DC48C68-8AC9-47B1-B7B1-FA41809AF3C1.1; Wed, 08 Apr 2020 21:56:21 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0f27258647d9.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 08 Apr 2020 21:56:21 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=StoZxLLzyCPFKnggHhqgnn3KrRmIUGBHdL4Ieol6qdcwRuR7BytxJR76NsxCzXtcC+XJw6Zya976KRm3JFAW2CN9ZaYcOBVpTgOChFC+o7e0GEZfhsWx11gZOvHTYugXfdkMF+tx6hpKnidETSN9dUoUaPcsRl1RMAuBf1h4Sy3r9ZCf/eVCcrDVf0kDg8iYSO37v1j1LCOTJhUXDQgHGmFcmppQ4xyKpYL3iREsG7WA3sO0TqDPk3rrkdnJbixEIxwnH9lsrYaEq+LMpmK6m0PZSRd2yUJYLN01wHns/OjWuhVL3Fugb/0gpW0emXqAarVQOlwYB4ABNTtgR7AlhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IRAga+iEx4YoTB8R8nkFREk5VrkcD95bVRX/+XJTAJE=; b=DU6zf2pVqNk4zW7gqI6hZ2gKt8sbhqbnwLcgvm9z7+cWMfRe8GtxbQWNzElsssISWfZzsjEgUUGg6XoHgSdMQA4Vlz1gutCaYPE7aiY6Qiu6haF8mzInu0u+A9Ws4Y/UIcQ26U2ZobQQ/QP9KunqlHTjqJWoM/IzY5QYpJBCdscz+KlsCwwHWApwEmy4NgbC5CHKhucp8Yf61xTu478jDPKjxYng/0U5yHeoSQ98BJmlj65vl2Fts+aVDgntfCpDOZ8+C1QG59IpFcRS8G8ACmHPzaDerLUY4x+Pdb7sQ7sZsvhGz2w/1jztwFla2fqmQ0W/ltWGLz9gebVTKTXDkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IRAga+iEx4YoTB8R8nkFREk5VrkcD95bVRX/+XJTAJE=; b=dG1puBVFV5PWfKTjHAyMf/imTQx8rcTTqyBW0JaHEgqfjGVlZjKKsI/7qeFM/UyfIynb4JijbNJGcjR7JfsYbWZPxj2EQpPUbGzR5PR0UF3Fdur48H19TinduCUXhqfwdSbj5KqYHFWDubW6akZqgcLzL2gY4hcH0Jt5vLm6xxc= Received: from DBBPR08MB4646.eurprd08.prod.outlook.com (10.255.79.144) by DBBPR08MB4282.eurprd08.prod.outlook.com (20.179.43.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15; Wed, 8 Apr 2020 21:56:18 +0000 Received: from DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d]) by DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d%5]) with mapi id 15.20.2878.021; Wed, 8 Apr 2020 21:56:18 +0000 From: Honnappa Nagarahalli To: "Carrillo, Erik G" , Phil Yang , "rsanford@akamai.com" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "Burakov, Anatoly" , "thomas@monjalon.net" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH 2/2] lib/timer: relax barrier for status update Thread-Index: AQHWDeodFNsTow5NnU2kr/yq0gxiTahvuM1ggAAD/ACAAAGw8A== Date: Wed, 8 Apr 2020 21:56:18 +0000 Message-ID: References: <1582526539-14360-1-git-send-email-phil.yang@arm.com> <1582526539-14360-2-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: fb3e9212-be78-416c-91dc-e75bb4130c23.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 6e7758fe-912d-4691-6ee7-08d7dc07afde x-ms-traffictypediagnostic: DBBPR08MB4282:|DBBPR08MB4282:|DB7PR08MB3068: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882; x-forefront-prvs: 0367A50BB1 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBBPR08MB4646.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(366004)(396003)(136003)(39850400004)(376002)(346002)(478600001)(30864003)(110136005)(186003)(81156014)(7696005)(316002)(86362001)(52536014)(33656002)(2906002)(54906003)(64756008)(66446008)(66476007)(9686003)(66556008)(8936002)(66946007)(81166007)(15650500001)(26005)(8676002)(4326008)(55016002)(6506007)(5660300002)(71200400001)(76116006); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: fBUmAkEBbnYc6jiXaKFSzekr1NvB+Jk1SCv0zmV3g0F6H/U0e0U1QVmB9U0lcDQfDdrIVf5lRKXKsUruD+zMc2Fv5frxlDnaVcOncXu+RWceEXatu+S8c6h2h1fORq6Q+g2BGuom7xVSL9cNted6ZUBWkmxA4AvaQY6pDMj+jpBzCFIzj2LevuCcow/GkqbuBvHKh9Qu6Y7AdpuQYXXAJNZRSLKjhGd9djdbsS9jbzRJqfyUEVSTwoIjS4JqOaWFgCRxpUy71CNljHD/Vvg1tg5/Sme5x6ZVf6QQilu/4D94y90NTAPbmiGeGjyWg32s0nCHx6RxK5+xJ1LcFc0PcTHQ1KX7SBkioiPGINkjCP0yxd2OUjplviy6QhAYQNhaSNVg9Ov8uYxcPYY5Lib+m9jw9//gajIqulA8cLaiPWTUKnUuq0C0GCrJejWgfjXf x-ms-exchange-antispam-messagedata: j2Ft/iAKvY+ScA0aW+Dv+O+gHLPt5Lw+hX9QhpQdvo3d7Kc5/zdalPR9RbBpaQBxWMtxssKlev2qFiDezkPxyBlW8khzSrMns26JpvcH5KYFYJlC+38bYpmTY6eVVnEOmBLS7UYq3N/dUxQO5ZfSFw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4282 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT016.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(346002)(396003)(39850400004)(136003)(46966005)(33656002)(36906005)(70206006)(110136005)(9686003)(316002)(26826003)(356004)(2906002)(6506007)(478600001)(54906003)(55016002)(4326008)(82740400003)(5660300002)(8676002)(30864003)(15650500001)(52536014)(70586007)(81156014)(81166007)(86362001)(47076004)(336012)(26005)(7696005)(8936002)(186003); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: b853beb8-ac74-4172-ba1c-08d7dc07ab0c X-Forefront-PRVS: 0367A50BB1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 41HcsY9k28ZafALSWMP/d2jmXUAY8ChdS4fWbBPXpP/a2ru39UiSgTLoY8zyXTzcsWQrSwCsWFfxvaAQ3GIoc4ib+bmLgWpUetGo4+NcQMhtNgvJptN+u2fSOgqGILwjA+RfMiY/GF+zRWu+k48Mb7TpYqMfH3gRkmPnCqnPpKMwmGhQmPxLWrQ63yMKPXJvlfeFPF9IGY/buKxZ9TJyzR3i1LELY8h2AlSPFOcMZDhVZA0PTZZVvF6jm1UGif6zIWo5cT97gceBkIND9f0Qkdq9Rxv+1XAX+8N0qP8ImgEFqMZs+t+aXNJxCAb1OxlB8zcvz9UIEq9Eq7xX6Xdz/Ls3CNbpd3xqN+rJH5duTIWlo0Cql1uH7HQkv/oQxzlv19EelYplU2zVHu+HfGzxzsLSsEvg1aKFBghTJpUvZ0X9eo9AXgnNSRMMDHRv5p0tEpAqWjcxAVvCrZPPK6kYRFlhPRuin4sXUXXpGQQ3lWKDQ9owSqJ/31+w+FHj1SeqoyRo3diINT0r95OYi2pkYQ== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2020 21:56:26.8605 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6e7758fe-912d-4691-6ee7-08d7dc07afde X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3068 Subject: Re: [dpdk-dev] [PATCH 2/2] lib/timer: relax barrier for status update X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > Subject: [PATCH 2/2] lib/timer: relax barrier for status update > > > > > > > > Volatile has no ordering semantics. The rte_timer structure > > > > defines timer status as a volatile variable and uses the rte_r/wmb > > > > barrier to guarantee inter-thread visibility. > > > > > > > > This patch optimized the volatile operation with c11 atomic > > > > operations and one-way barrier to save the performance penalty. > > > > According to the timer_perf_autotest benchmarking results, this > > > > patch can uplift 10%~16% timer appending performance, 3%~20% timer > > > > resetting performance and 45% timer callbacks scheduling > > > > performance on aarch64 and no loss in performance for x86. > > > > > > > > Suggested-by: Honnappa Nagarahalli > > > > Signed-off-by: Phil Yang > > > > Reviewed-by: Gavin Hu > > > > > > Hi Phil, > > > > > > It seems like the consensus is to generally avoid replacing rte_atomi= c_* > > > interfaces with the GCC builtins directly. In other areas of DPDK t= hat are > > > being patched, are the C11 APIs going to be investigat= ed? > > It > > > seems like that decision will apply here as well. > > Agree. The new APIs are going to be 1 to 1 mapped with the built-in > > intrinsics (the memory orderings used themselves will not change). We > > should go ahead with the review and conclude any issues. Once the > > decision is made on what APIs to use, we can submit the next version us= ing > the APIs decided. > > > Thanks, Honnappa. >=20 > I have reviewed the memory orderings and I see no issues with them. I d= o > have a question regarding a comment - I'll pose it inline: Fantastic, thank you. I have an unrelated (to this patch) question for you below. >=20 > > > > > > Thanks, > > > Erik > > > > > > > --- > > > > lib/librte_timer/rte_timer.c | 90 > > > > +++++++++++++++++++++++++++++++---- > > > > --------- > > > > lib/librte_timer/rte_timer.h | 2 +- > > > > 2 files changed, 65 insertions(+), 27 deletions(-) > > > > > > > > diff --git a/lib/librte_timer/rte_timer.c > > > > b/lib/librte_timer/rte_timer.c index 269e921..be0262d 100644 > > > > --- a/lib/librte_timer/rte_timer.c > > > > +++ b/lib/librte_timer/rte_timer.c > > > > @@ -10,7 +10,6 @@ > > > > #include > > > > #include > > > > > > > > -#include > > > > #include > > > > #include > > > > #include > > > > @@ -218,7 +217,7 @@ rte_timer_init(struct rte_timer *tim) > > > > > > > > status.state =3D RTE_TIMER_STOP; > > > > status.owner =3D RTE_TIMER_NO_OWNER; > > > > - tim->status.u32 =3D status.u32; > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > __ATOMIC_RELAXED); > > > > } > > > > > > > > /* > > > > @@ -239,9 +238,9 @@ timer_set_config_state(struct rte_timer *tim, > > > > > > > > /* wait that the timer is in correct status before update, > > > > * and mark it as being configured */ > > > > - while (success =3D=3D 0) { > > > > - prev_status.u32 =3D tim->status.u32; > > > > + prev_status.u32 =3D __atomic_load_n(&tim->status.u32, > > > > __ATOMIC_RELAXED); > > > > > > > > + while (success =3D=3D 0) { > > > > /* timer is running on another core > > > > * or ready to run on local core, exit > > > > */ > > > > @@ -258,9 +257,20 @@ timer_set_config_state(struct rte_timer *tim, > > > > * mark it atomically as being configured */ > > > > status.state =3D RTE_TIMER_CONFIG; > > > > status.owner =3D (int16_t)lcore_id; > > > > - success =3D rte_atomic32_cmpset(&tim->status.u32, > > > > - prev_status.u32, > > > > - status.u32); > > > > + /* If status is observed as RTE_TIMER_CONFIG earlier, > > > > + * that's not going to cause any issues because the > > > > + * pattern is read for status then read the other members. >=20 > I don't follow the above comment. What is meant by "earlier"? >=20 > Thanks, > Erik I would rather change this comment to something similar to what is mentione= d while changing to 'RUNNING' state. 'CONFIG' is also a locking state. I think it is much easier to understand. >=20 > > > > + * In one of the callers to timer_set_config_state > > > > + * (the __rte_timer_reset) we set other members to the > > > > + * structure (period, expire, f, arg) we want these > > > > + * changes to be observed after our change to status. > > > > + * So we need __ATOMIC_ACQUIRE here. > > > > + */ > > > > + success =3D __atomic_compare_exchange_n(&tim- > > > > >status.u32, > > > > + &prev_status.u32, > > > > + status.u32, 0, > > > > + __ATOMIC_ACQUIRE, > > > > + __ATOMIC_RELAXED); > > > > } > > > > > > > > ret_prev_status->u32 =3D prev_status.u32; @@ -279,20 +289,27 @@ > > > > timer_set_running_state(struct rte_timer *tim) > > > > > > > > /* wait that the timer is in correct status before update, > > > > * and mark it as running */ > > > > - while (success =3D=3D 0) { > > > > - prev_status.u32 =3D tim->status.u32; > > > > + prev_status.u32 =3D __atomic_load_n(&tim->status.u32, > > > > __ATOMIC_RELAXED); > > > > > > > > + while (success =3D=3D 0) { > > > > /* timer is not pending anymore */ > > > > if (prev_status.state !=3D RTE_TIMER_PENDING) > > > > return -1; > > > > > > > > /* here, we know that timer is stopped or pending, > > > > - * mark it atomically as being configured */ > > > > + * mark it atomically as being running > > > > + */ > > > > status.state =3D RTE_TIMER_RUNNING; > > > > status.owner =3D (int16_t)lcore_id; > > > > - success =3D rte_atomic32_cmpset(&tim->status.u32, > > > > - prev_status.u32, > > > > - status.u32); > > > > + /* RUNNING states are acting as locked states. If the > > > > + * timer is in RUNNING state, the state cannot be changed > > > > + * by other threads. So, we should use ACQUIRE here. > > > > + */ > > > > + success =3D __atomic_compare_exchange_n(&tim- > > > > >status.u32, > > > > + &prev_status.u32, > > > > + status.u32, 0, > > > > + __ATOMIC_ACQUIRE, > > > > + __ATOMIC_RELAXED); > > > > } > > > > > > > > return 0; > > > > @@ -520,10 +537,12 @@ __rte_timer_reset(struct rte_timer *tim, > > > > uint64_t expire, > > > > > > > > /* update state: as we are in CONFIG state, only us can modify > > > > * the state so we don't need to use cmpset() here */ > > > > - rte_wmb(); > > > > status.state =3D RTE_TIMER_PENDING; > > > > status.owner =3D (int16_t)tim_lcore; > > > > - tim->status.u32 =3D status.u32; > > > > + /* The "RELEASE" ordering guarantees the memory operations above > > > > + * the status update are observed before the update by all thread= s > > > > + */ > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > __ATOMIC_RELEASE); > > > > > > > > if (tim_lcore !=3D lcore_id || !local_is_locked) > > > > rte_spinlock_unlock(&priv_timer[tim_lcore].list_lock); > > > > @@ -600,10 +619,12 @@ __rte_timer_stop(struct rte_timer *tim, int > > > > local_is_locked, > > > > } > > > > > > > > /* mark timer as stopped */ > > > > - rte_wmb(); > > > > status.state =3D RTE_TIMER_STOP; > > > > status.owner =3D RTE_TIMER_NO_OWNER; > > > > - tim->status.u32 =3D status.u32; > > > > + /* The "RELEASE" ordering guarantees the memory operations above > > > > + * the status update are observed before the update by all thread= s > > > > + */ > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > __ATOMIC_RELEASE); > > > > > > > > return 0; > > > > } > > > > @@ -637,7 +658,8 @@ rte_timer_stop_sync(struct rte_timer *tim) > > > > int rte_timer_pending(struct rte_timer *tim) { > > > > - return tim->status.state =3D=3D RTE_TIMER_PENDING; > > > > + return __atomic_load_n(&tim->status.state, > > > > + __ATOMIC_RELAXED) =3D=3D > > > > RTE_TIMER_PENDING; > > > > } > > > > > > > > /* must be called periodically, run all timer that expired */ @@ > > > > -739,8 > > > > +761,12 @@ __rte_timer_manage(struct rte_timer_data *timer_data) > > > > /* remove from done list and mark timer as stopped > > > */ > > > > status.state =3D RTE_TIMER_STOP; > > > > status.owner =3D RTE_TIMER_NO_OWNER; > > > > - rte_wmb(); > > > > - tim->status.u32 =3D status.u32; > > > > + /* The "RELEASE" ordering guarantees the memory > > > > + * operations above the status update are observed > > > > + * before the update by all threads > > > > + */ > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > + __ATOMIC_RELEASE); > > > > } > > > > else { > > > > /* keep it in list and mark timer as pending */ @@ - > > > > 748,8 +774,12 @@ __rte_timer_manage(struct rte_timer_data > > *timer_data) > > > > status.state =3D RTE_TIMER_PENDING; Is it better to set this to STOPPED since it is out of the run list? I thin= k it is better for the understanding as well. > > > > __TIMER_STAT_ADD(priv_timer, pending, 1); > > > > status.owner =3D (int16_t)lcore_id; > > > > - rte_wmb(); > > > > - tim->status.u32 =3D status.u32; > > > > + /* The "RELEASE" ordering guarantees the memory > > > > + * operations above the status update are observed > > > > + * before the update by all threads > > > > + */ > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > + __ATOMIC_RELEASE); > > > > __rte_timer_reset(tim, tim->expire + tim->period, > > > > tim->period, lcore_id, tim->f, tim->arg, 1, > > > > timer_data); > > > > @@ -919,8 +949,12 @@ rte_timer_alt_manage(uint32_t timer_data_id, > > > > /* remove from done list and mark timer as stopped > > > */ > > > > status.state =3D RTE_TIMER_STOP; > > > > status.owner =3D RTE_TIMER_NO_OWNER; > > > > - rte_wmb(); > > > > - tim->status.u32 =3D status.u32; > > > > + /* The "RELEASE" ordering guarantees the memory > > > > + * operations above the status update are observed > > > > + * before the update by all threads > > > > + */ > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > + __ATOMIC_RELEASE); > > > > } else { > > > > /* keep it in list and mark timer as pending */ > > > > rte_spinlock_lock( > > > > @@ -928,8 +962,12 @@ rte_timer_alt_manage(uint32_t timer_data_id, > > > > status.state =3D RTE_TIMER_PENDING; > > > > __TIMER_STAT_ADD(data->priv_timer, pending, 1); > > > > status.owner =3D (int16_t)this_lcore; > > > > - rte_wmb(); > > > > - tim->status.u32 =3D status.u32; > > > > + /* The "RELEASE" ordering guarantees the memory > > > > + * operations above the status update are observed > > > > + * before the update by all threads > > > > + */ > > > > + __atomic_store_n(&tim->status.u32, status.u32, > > > > + __ATOMIC_RELEASE); > > > > __rte_timer_reset(tim, tim->expire + tim->period, > > > > tim->period, this_lcore, tim->f, tim->arg, 1, > > > > data); > > > > diff --git a/lib/librte_timer/rte_timer.h > > > > b/lib/librte_timer/rte_timer.h index c6b3d45..df533fa 100644 > > > > --- a/lib/librte_timer/rte_timer.h > > > > +++ b/lib/librte_timer/rte_timer.h > > > > @@ -101,7 +101,7 @@ struct rte_timer { > > > > uint64_t expire; /**< Time when timer expire. */ > > > > struct rte_timer *sl_next[MAX_SKIPLIST_DEPTH]; > > > > - volatile union rte_timer_status status; /**< Status of timer. */ > > > > + union rte_timer_status status; /**< Status of timer. */ > > > > uint64_t period; /**< Period of timer (0 if not periodic). = */ > > > > rte_timer_cb_t f; /**< Callback function. */ > > > > void *arg; /**< Argument to callback function. */ > > > > -- > > > > 2.7.4