From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 239D9A034F; Tue, 9 Nov 2021 21:59:49 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D5A2040E03; Tue, 9 Nov 2021 21:59:48 +0100 (CET) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80081.outbound.protection.outlook.com [40.107.8.81]) by mails.dpdk.org (Postfix) with ESMTP id DA78B4068B for ; Tue, 9 Nov 2021 21:59:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b6945kc4BJUAmHNlTbvNFMp07XteB6gKO6O6+YTdrZ4=; b=8Mz7nVYO4Ti7krQ4H/6bETn4U6vAmFHbCenKBuuWoPEUdORckAid4UOAU84pwIDnMAcbmHgNv7vJzBJskS98JI9rONxJE8RGeBSPhKKtj7gpn8oCQKSh3H1jfBKhmHtV8m38nw4dAThsFtbdR24JowOtA7+5FD4ZhWkgwGOwPdQ= Received: from AM6PR0502CA0046.eurprd05.prod.outlook.com (2603:10a6:20b:56::23) by DB7PR08MB3354.eurprd08.prod.outlook.com (2603:10a6:5:1d::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.13; Tue, 9 Nov 2021 20:59:43 +0000 Received: from AM5EUR03FT027.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:56:cafe::3) by AM6PR0502CA0046.outlook.office365.com (2603:10a6:20b:56::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.11 via Frontend Transport; Tue, 9 Nov 2021 20:59:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT027.mail.protection.outlook.com (10.152.16.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.14 via Frontend Transport; Tue, 9 Nov 2021 20:59:42 +0000 Received: ("Tessian outbound 6ebd41198c5d:v108"); Tue, 09 Nov 2021 20:59:42 +0000 X-CR-MTA-TID: 64aa7808 Received: from c663f8ccd039.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id BA077F26-8A17-412C-8617-DF279D5E185D.1; Tue, 09 Nov 2021 20:59:33 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c663f8ccd039.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 09 Nov 2021 20:59:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hs806jjiTiq4P7UTT0GhYr7zp+aAzWKpt3Nqh7rxFnlISbqHcHQADSG4j/DbeogVFnIPItJ2dr1d5YUALysd/OXrrDC7mUq/Q2x/SZ5N1bNDYfB3LE1a2rXZ6VEAQH45Lt9QIquUz/wlAQ625pgs6iz72lsf62cFvVH6WnzABG/6IteTSySvR8d7P2ap7vZMieUkfvzA3J2zomuXyaUgTHUfAvsqyCly+VNqZmMi3ZirmgSsR/QW9rdPVwpOFFziNvN1U8ZjW9iliY/CTVJQhQOvOiK2A9Zw2va+b/zLi5ZRIlGjS8dKocx+8OgIaYC2mR3SQnxdfaZ52lI5LYIpzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b6945kc4BJUAmHNlTbvNFMp07XteB6gKO6O6+YTdrZ4=; b=OpoV/vdnkoq8ME11bD6YAIbWEQfl1SlEx87Y+5MUu2caElmkcKBkKYzL7CyL2Qh/xUIWzd4hGIdtP556EsBQXNaOdMcLAglzWsENvw/Dh+PlFQEupoeSAH9yibvipqB/JcEEBlbuQv6QyqWhkFrtkeAAmFRfFlW2g9vCpdHjOYwUkdqyat04CDfToizzvTewlNxnAGtpfy5Peb6NR+HEvaK9rP02/18yEXKZWZRwDGaVTf9Pa0QN9Fe8MgXe1jDbYDHE/JNV/0Wtda9cUJX/VOq+JFwH7kJ9t6g48CMTGMvfKBCrSIgDoYV1yBxvWUG4dsnov27ydok+0doObNj6xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b6945kc4BJUAmHNlTbvNFMp07XteB6gKO6O6+YTdrZ4=; b=8Mz7nVYO4Ti7krQ4H/6bETn4U6vAmFHbCenKBuuWoPEUdORckAid4UOAU84pwIDnMAcbmHgNv7vJzBJskS98JI9rONxJE8RGeBSPhKKtj7gpn8oCQKSh3H1jfBKhmHtV8m38nw4dAThsFtbdR24JowOtA7+5FD4ZhWkgwGOwPdQ= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DB9PR08MB6410.eurprd08.prod.outlook.com (2603:10a6:10:262::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.15; Tue, 9 Nov 2021 20:59:28 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::8501:86ca:a393:17b7]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::8501:86ca:a393:17b7%9]) with mapi id 15.20.4690.015; Tue, 9 Nov 2021 20:59:28 +0000 From: Honnappa Nagarahalli To: Joyce Kong , "thomas@monjalon.net" , "david.marchand@redhat.com" , Ruifeng Wang , "konstantin.ananyev@intel.com" , "rsanford@akamai.com" , "erik.g.carrillo@intel.com" , "olivier.matz@6wind.com" , "yipeng1.wang@intel.com" , "sameh.gobriel@intel.com" , "bruce.richardson@intel.com" , "vladimir.medvedkin@intel.com" , "anatoly.burakov@intel.com" , "andrew.rybchenko@oktetlabs.ru" , "jerinj@marvell.com" , "declan.doherty@intel.com" , "ciara.power@intel.com" , "xiaoyun.li@intel.com" , "nicolas.chautru@intel.com" , "maryam.tahhan@intel.com" , "reshma.pattan@intel.com" , "cristian.dumitrescu@intel.com" CC: "dev@dpdk.org" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v1 03/12] test/timer: use compiler atomic builtins for sync Thread-Index: AQHXh4gA4wsG34LBl0eWlZgtGMjWY6v8PUfQ Date: Tue, 9 Nov 2021 20:59:28 +0000 Message-ID: References: <20210802101847.3462-1-joyce.kong@arm.com> <20210802101847.3462-4-joyce.kong@arm.com> In-Reply-To: <20210802101847.3462-4-joyce.kong@arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 52B6ED8D33D0D645A71FCB9A99583B58.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 3b8afec5-8d90-4a90-02dd-08d9a3c3da65 x-ms-traffictypediagnostic: DB9PR08MB6410:|DB7PR08MB3354: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: i9RKLfADE52BseclG3sUaSphdLXuaxhJy/RmvytqtXPbu31EAdIEdBtqH5jgQFjsBEet+yuPz4Y/SXfZOccLySUYy0aP6Tl/2giZTTXrk6WuRtsDbSBsu3j5XRiDQCN+Bd8KMN4bxPQJJL/DaBTrcS9ZvGDKc0PlPNqvRDM5yV5IJRS+V/RoEHm0QpJ3thEAtctLyHhcWrVk3TV3dD7hBR/iecWr2cEVGkmEPUjzrYeEtzjdFfttOETwuunkpRcINaJrDvU0HrdAOXaHNJIdVCy9VHov/K8LcRZDjkBAgTwvEVqrQu2GIJajwLI23EGgpk9jjS8/SuR0115PqPAKTX4PAcgf+MrjsvwUjahb2yfOXS4r2KdLgAp0HrFP8gumFp1+PR0oEQK3ETnbwOQdwaoEfzE+oxPM3K5CwhShf7BF14IX6UphH7ro2hLV46dCU1lrG2Ow08VkgqgiXhW2OlOA+yYTNNLtkqGADtaaSYOwNyG7ug0IUZ4t2fqpBVRZp3a0GjYEHGhRyyqEWI4isFc7RCMI8u1Up2MhOI5560B+Jo4ETlcbGq3VzIAdZyMHM3selZoPkEdOLevqkgH/QJyRpCVZRjyr02CVvMubfW03UCEU6b09GYIEWyMg+QA9l1Fsg1ACoRUWxVm2UaDbk2E/02nL3yEfHwoZ60Q4BOlJarHdZhFHzWjPMELtU0xga/D42GpE9lsdt2wG+VJi4mr2jz2v4+8KPNjyo855KKKWz/32lpRAeSvrCMnAIHJX X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(122000001)(5660300002)(55016002)(921005)(38100700002)(86362001)(38070700005)(54906003)(26005)(110136005)(2906002)(66556008)(6506007)(66446008)(71200400001)(66476007)(76116006)(64756008)(186003)(7416002)(66946007)(316002)(33656002)(83380400001)(508600001)(7696005)(8936002)(9686003)(8676002)(52536014)(4326008)(87944003); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6410 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT027.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: a714cc86-abe3-4e66-c009-08d9a3c3d218 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sVL7KImJi68P4fL+juMeh6S+HcajIF2OzFG2P1LDkDL+qVKnIeRlhtV6bCFb6mYxmYDjOKxJp4R3aRclL+5HCDC3YhL18TDzhR1WJrjMUofIVb2rdZ9cvnI0Dl1m7Q2ZjKEION05hR1Jea/BSr/Usyc/tEZJ4NKBHKhVENe35SW5DaBNI9WLRzP/O7+ESI3oDVlVK/7ibznRq8MRbVZR90QrL/7Q9gCk9+FAhkQsNq7vrIrerFbQamtBiy+1m468nIKNvMMpXPs9xNKX3Ue5GDqNEKufkp8uf1bpfXqIC34g4KPv2KZ4z5ctPiRDWYvURqbFYog5Yi4q26LQ84EwRmDxICUOTR+jjbH79Ls3pqlYP29O803uWGjNjC6MOhkqNq8kBBcKSSbPQ1XN78Z450ubgkGVX0GWpVHR0pKqZC5fCkOw3cGOnqO+zUpPcXDH+QnDmcFRQIiKkC0zx0K5UZfrMX2MLYqtmuu9Z403mSrAur7/UlikzKYPAx3bxS7GpoykDmssMp7SrbTG0R3LFcf/zodymtKP4EB3cDJtI90F1V2LZXcnV0wtvDOcHfu2o0mxux1DTuD77T7agCF4Ojk23zXo+xcyeotPYqTX9oVxrO27eIUfn0WpHCxJQNu5wdoC+gvWFRw3CQYrwYA1HsKN0AJMW6PbUz/HnN8V0tktvLSAxTs6I3a8zyM6sJA/bML5gnfdUUqkx0LIi4kgzz92CVnsTxEOlJ0cygXtrw1kiA7vZLbOmyGoooNP2Pz2 X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(7696005)(86362001)(33656002)(47076005)(921005)(82310400003)(83380400001)(336012)(52536014)(55016002)(70586007)(186003)(54906003)(6506007)(70206006)(9686003)(356005)(26005)(110136005)(8676002)(81166007)(316002)(508600001)(5660300002)(4326008)(8936002)(2906002)(36860700001)(87944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2021 20:59:42.6835 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3b8afec5-8d90-4a90-02dd-08d9a3c3da65 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT027.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3354 Subject: Re: [dpdk-dev] [PATCH v1 03/12] test/timer: use compiler atomic builtins for sync X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > Subject: [PATCH v1 03/12] test/timer: use compiler atomic builtins for sy= nc >=20 > Convert rte_atomic usages to compiler atomic built-ins for lcore_state an= d > collisions sync. >=20 > Signed-off-by: Joyce Kong > Reviewed-by: Ruifeng Wang > --- > app/test/test_timer.c | 28 ++++++++++++---------------- > app/test/test_timer_secondary.c | 1 - > 2 files changed, 12 insertions(+), 17 deletions(-) >=20 > diff --git a/app/test/test_timer.c b/app/test/test_timer.c index > a10b2fe9da..9a3e1b53e4 100644 > --- a/app/test/test_timer.c > +++ b/app/test/test_timer.c > @@ -102,7 +102,6 @@ > #include > #include > #include > -#include > #include > #include > #include > @@ -203,7 +202,7 @@ timer_stress_main_loop(__rte_unused void *arg) >=20 > /* Need to synchronize worker lcores through multiple steps. */ enum { > WORKER_WAITING =3D 1, WORKER_RUN_SIGNAL, WORKER_RUNNING, > WORKER_FINISHED }; -static rte_atomic16_t lcore_state[RTE_MAX_LCORE]; > +static uint16_t lcore_state[RTE_MAX_LCORE]; >=20 > static void > main_init_workers(void) > @@ -211,7 +210,7 @@ main_init_workers(void) > unsigned i; >=20 > RTE_LCORE_FOREACH_WORKER(i) { > - rte_atomic16_set(&lcore_state[i], WORKER_WAITING); > + __atomic_store_n(&lcore_state[i], WORKER_WAITING, > __ATOMIC_RELAXED); > } > } >=20 > @@ -221,11 +220,10 @@ main_start_workers(void) > unsigned i; >=20 > RTE_LCORE_FOREACH_WORKER(i) { > - rte_atomic16_set(&lcore_state[i], WORKER_RUN_SIGNAL); > + __atomic_store_n(&lcore_state[i], WORKER_RUN_SIGNAL, > +__ATOMIC_RELAXED); We need to use RELEASE order here. When this function is called in timer_st= ress2_main_loop, the main_lcore is releasing memory operations to worker co= res. > } > RTE_LCORE_FOREACH_WORKER(i) { > - while (rte_atomic16_read(&lcore_state[i]) !=3D > WORKER_RUNNING) > - rte_pause(); > + rte_wait_until_equal_16(&lcore_state[i], WORKER_RUNNING, > +__ATOMIC_RELAXED); This should be ACQUIRE order. > } > } >=20 > @@ -235,8 +233,7 @@ main_wait_for_workers(void) > unsigned i; >=20 > RTE_LCORE_FOREACH_WORKER(i) { > - while (rte_atomic16_read(&lcore_state[i]) !=3D > WORKER_FINISHED) > - rte_pause(); > + rte_wait_until_equal_16(&lcore_state[i], WORKER_FINISHED, > +__ATOMIC_RELAXED); We should use ACQUIRE here as the workers are releasing memory updates to t= he main lcore. > } > } >=20 > @@ -245,9 +242,8 @@ worker_wait_to_start(void) { > unsigned lcore_id =3D rte_lcore_id(); >=20 > - while (rte_atomic16_read(&lcore_state[lcore_id]) !=3D > WORKER_RUN_SIGNAL) > - rte_pause(); > - rte_atomic16_set(&lcore_state[lcore_id], WORKER_RUNNING); > + rte_wait_until_equal_16(&lcore_state[lcore_id], > WORKER_RUN_SIGNAL, __ATOMIC_RELAXED); Since the worker threads will use the memory operations that happened in ma= in_lcore, we need ACQUIRE memory ordering here. > + __atomic_store_n(&lcore_state[lcore_id], WORKER_RUNNING, > +__ATOMIC_RELAXED); This should be RELEASE memory order. > } >=20 > static void > @@ -255,7 +251,7 @@ worker_finish(void) > { > unsigned lcore_id =3D rte_lcore_id(); >=20 > - rte_atomic16_set(&lcore_state[lcore_id], WORKER_FINISHED); > + __atomic_store_n(&lcore_state[lcore_id], WORKER_FINISHED, > +__ATOMIC_RELAXED); Should be RELEASE as workers are releasing updates to the main lcore. > } >=20 >=20 > @@ -281,12 +277,12 @@ timer_stress2_main_loop(__rte_unused void *arg) > unsigned int lcore_id =3D rte_lcore_id(); > unsigned int main_lcore =3D rte_get_main_lcore(); > int32_t my_collisions =3D 0; > - static rte_atomic32_t collisions; > + static uint32_t collisions; >=20 > if (lcore_id =3D=3D main_lcore) { > cb_count =3D 0; > test_failed =3D 0; > - rte_atomic32_set(&collisions, 0); > + __atomic_store_n(&collisions, 0, __ATOMIC_RELAXED); > main_init_workers(); I think the existing code is probably incorrect in calling 'main_init_worke= rs' in this function. This should be called outside of ' timer_stress2_main= _loop' so that lcore_state is guaranteed to be initialized correctly before= the threads are launched. Please call 'main_init_workers()' in ' test_timer' function. > timers =3D rte_malloc(NULL, sizeof(*timers) * > NB_STRESS2_TIMERS, 0); > if (timers =3D=3D NULL) { > @@ -315,7 +311,7 @@ timer_stress2_main_loop(__rte_unused void *arg) > my_collisions++; > } > if (my_collisions !=3D 0) > - rte_atomic32_add(&collisions, my_collisions); > + __atomic_fetch_add(&collisions, my_collisions, > __ATOMIC_RELAXED); >=20 > /* wait long enough for timers to expire */ > rte_delay_ms(100); > @@ -329,7 +325,7 @@ timer_stress2_main_loop(__rte_unused void *arg) >=20 > /* now check that we get the right number of callbacks */ > if (lcore_id =3D=3D main_lcore) { > - my_collisions =3D rte_atomic32_read(&collisions); > + my_collisions =3D __atomic_load_n(&collisions, > __ATOMIC_RELAXED); > if (my_collisions !=3D 0) > printf("- %d timer reset collisions (OK)\n", > my_collisions); > rte_timer_manage(); > diff --git a/app/test/test_timer_secondary.c > b/app/test/test_timer_secondary.c index 16a9f1878b..5795c97f07 100644 > --- a/app/test/test_timer_secondary.c > +++ b/app/test/test_timer_secondary.c > @@ -9,7 +9,6 @@ > #include > #include > #include > -#include > #include > #include > #include > -- > 2.17.1