From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 03EC8A00C5; Fri, 8 Jul 2022 21:22:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D7130406B4; Fri, 8 Jul 2022 21:22:30 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140043.outbound.protection.outlook.com [40.107.14.43]) by mails.dpdk.org (Postfix) with ESMTP id D840D4021E for ; Fri, 8 Jul 2022 21:22:28 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Ild821xqp8WhgCZ6i1BznWirFgchybGsueelACaHG8qnRYRQzWFULfojjYV7kskqpGDL5emUxUjX3KhTal2n9w5gm8yjoZbYtHIdTQhYbtbWCLWT0hF9aQhmiuIyrMl+SET4eYZXl7RrjZr93SiaEnLaJGfzBJjE5l5Ve18VGoF4oZW5neg6gSuNOqtA/fnN/wHgSQBJPuaS8qJrQmiLZKoJ0ItRNCYOGOdSCYa0zEN86zoDD4oxNjyRIZBydiXujXt2WL5E+y7cDLMi740+jKahR8szZZ99DvL73nnS5zZun3LiOyiBEBWyW2bhl7/Nf045bFgqAGtpZrBZ+hkQ7g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BB3a4jcgC5jtr+2rO+cdlVMp/ByUYdqahykIM0DxDBQ=; b=Pjunc5CAB3fVwqkKsfnx27sCXCSywSsomuMvdjsYSrC9zJOayLl7wyiJhwtY3HwfbLk95ibFekSLPVm5ArAVpm4JjMc+fLcAncS4u4DgBx5DoiDjwKnoe0zKEiFP0Z8xz1EPv+vQsfdQo4WjpCBrAyXJnD7+5EYVxfTMVOmFPKcn/83F0ZlqzMZNRpLN8H1oArUqKsXPNP0j1mkA7Hni7s3nC6Qv1+P3hnyDTY0knjxYX0IZJYBqEswR727fUj7BbgHENaTz+WY3VIaX2tbMszT0ZKpaht5zVq/sUiT/ww+/Ep5SpzZkEchwjrjaGoMkLIjvCF+ERW5H+KtFm2pa2Q== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BB3a4jcgC5jtr+2rO+cdlVMp/ByUYdqahykIM0DxDBQ=; b=VNlHLkj4b27gXvyMz5oU3H/4bbY2C7yI3KfsWA7t1E9x+GPNRk3JGNtfoX+w8EIPN4Ex9wgxoCetq4BwtoB/LBS2IXrGHa62aR7qQltZbAoENz1qOBm9ElGGSyhUCvbPJEnmgAdeM1V7U1TNQGjfrp7lpzL3vwXJfJqv12pN54U= Received: from DB8PR06CA0065.eurprd06.prod.outlook.com (2603:10a6:10:120::39) by AM0PR08MB4017.eurprd08.prod.outlook.com (2603:10a6:208:12a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.17; Fri, 8 Jul 2022 19:22:26 +0000 Received: from DBAEUR03FT027.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:120:cafe::5f) by DB8PR06CA0065.outlook.office365.com (2603:10a6:10:120::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 19:22:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT027.mail.protection.outlook.com (100.127.142.237) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 19:22:26 +0000 Received: ("Tessian outbound f9f15f9daab2:v122"); Fri, 08 Jul 2022 19:22:26 +0000 X-CR-MTA-TID: 64aa7808 Received: from 8f081d60a52c.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id D68DA395-C7FE-4953-A83D-6AAD6657011A.1; Fri, 08 Jul 2022 19:22:19 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8f081d60a52c.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 19:22:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V+1a/t67MOaBajm4MXcTEhkSZss92Js2xTB4zT3qJGPRTCma3EhdGeceYY9Vuz8MtOJ4a6yAw7d5nt4Tl5l/ffEBOm8iJmHVICBdPzl8yA98ho4fr2sSA0zzGhJrKJs9zRUNam4PnhlbbADm0UR7kFSE8kF2HilH+0VI2pw9e5uwr82R7FcDswhCcQPGlp/6DKTqezcav+W58Ku2YB+9cdn6THsAZZmwve4F8v3HvLoozz86R1WwC7Wsga91edO0WJgjbrWtCDCLebbv5mRmJkZ3zsX+zXKlBhl2NeSddE3UpzE6ADM0oR7XoFA736JFGl8Lt7slIrpMhou+cKONDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BB3a4jcgC5jtr+2rO+cdlVMp/ByUYdqahykIM0DxDBQ=; b=MRmjavQ6NWiVmN1qbY4h5Wriu+cpNXoOQ/cziv3D0Ya40f1aLtgzj7D9+P3/1fYdCjwVLjtWEtg9Mf7dX8KUGg6tP6qym6NrAyf+diPoXqjshT08kDesZO64e3TEFP6liwmKe4tqZFvCtJimiAaqIBKmA7KUB06f9IbKQTTG4+upgAspI6GH8fugrsIlN7Ivf3A5m451kGwNDE6iWhLvnUNK1+w7YEhVMlC2Vzs5+xGuZco3Z8CL1oJytQ7LgLYrCCn7XOOcssvSdSohicWGjjdHWEaUNwT7SntH16hYCyztdIs1z3BOtecEv+iLsx5PjcA6BMxh+ytlH6Ahup+A/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BB3a4jcgC5jtr+2rO+cdlVMp/ByUYdqahykIM0DxDBQ=; b=VNlHLkj4b27gXvyMz5oU3H/4bbY2C7yI3KfsWA7t1E9x+GPNRk3JGNtfoX+w8EIPN4Ex9wgxoCetq4BwtoB/LBS2IXrGHa62aR7qQltZbAoENz1qOBm9ElGGSyhUCvbPJEnmgAdeM1V7U1TNQGjfrp7lpzL3vwXJfJqv12pN54U= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DB9PR08MB6732.eurprd08.prod.outlook.com (2603:10a6:10:2af::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20; Fri, 8 Jul 2022 19:22:18 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1c7f:6a8d:b518:f972]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1c7f:6a8d:b518:f972%3]) with mapi id 15.20.5417.016; Fri, 8 Jul 2022 19:22:18 +0000 From: Honnappa Nagarahalli To: Stephen Hemminger , "dev@dpdk.org" CC: nd , nd Subject: RE: [RFC] rwlock: prevent readers from starving writers Thread-Topic: [RFC] rwlock: prevent readers from starving writers Thread-Index: AQHYkj3qoG6JNj5SAkukgWeYSzkbi6102xTA Date: Fri, 8 Jul 2022 19:22:18 +0000 Message-ID: References: <20220707201226.618611-1-stephen@networkplumber.org> In-Reply-To: <20220707201226.618611-1-stephen@networkplumber.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 6C92169F8A12F8469C6993CA32BB9AF7.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: 555ac6b1-b540-4404-11d8-08da6117311c x-ms-traffictypediagnostic: DB9PR08MB6732:EE_|DBAEUR03FT027:EE_|AM0PR08MB4017:EE_ x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 3VymI6hwrv8BxWqJ2vXmq/geRIjqk2NqaEMxCg69C/1Pw41mGbxMo4MtdG+C18dC/4m3+gp2sF8C7zyqmDelJOPeLkwyKHrDExREdMLLYH+pLBoxDAACKHk5nenRMOwHt34iDSTHWvsn9gdaVwFpwgS67eNcT4zcEyyUXnHQK5/B1PGpkM1pJx8hZ2QOJ/VaDlnO8AJ9wv0o+sHaZV1UhKGWMNvySx0hHR2DdX34LGYx+YvGUuT6JyagjdKq+tHpyV4aaPxdfrUu6RYJp70CrcPQ+onlODzCMWtZ9b3IwKZb9cuKQLsbtd20UzXpZFc9Vpi9Pp18fFG1AkwKbmM6zuF9oPPZBjKmfmii+IM517bNB2URdpP3FsMc8e33TXWJE/5nxXkexlSZ45LH4X6cFgpnk02cl1Qa3IGZPvG9OC2e6VSJpQ9ss52MagHqJXVGh+81J56khZw2JcWj3oiVk2QaN7/7eTikE35TdPkYiYYqapPxX04zK9cER9v8Xhjbjz2MU51FHjmf0UU/VZpj44qcSXA7xHQGjicQz4jutiZ2h8UOaWzVVM6vO2wBa35qd371p4yniEDC+F1Y0x2+O4qGshbFUtJx+r94Gz4ROHmhVfhdqpGHIDU52KPkXAdTr/fKkiTHkKSWL8l8i3gcubjU46JAQSrit36A+BSaIpi0oS4zJdnbXcBGQToNF4tuTCVDQJXCRcpGLWU6aVxAO60SLwiHS/JQNq5OCytCdILMLUWTx78Vm7ZomtkQz7CpscexmVTakxLtjBqPc2O6EtbUF+ze06w2XWCy9HQqkktGUF5l2K2ZvR9dJd1X9bh6 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(136003)(366004)(396003)(346002)(376002)(39860400002)(9686003)(38070700005)(55016003)(83380400001)(66556008)(66946007)(66476007)(76116006)(4326008)(316002)(66446008)(64756008)(54906003)(8676002)(122000001)(110136005)(186003)(41300700001)(6506007)(7696005)(52536014)(26005)(71200400001)(478600001)(2906002)(86362001)(33656002)(5660300002)(8936002)(38100700002); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6732 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT027.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 5a96ec89-8520-48dd-6ae5-08da61172c98 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lcLKsClq6RRlWEJVMsdjMx+KaLSoNuHP4whcids0nbFDIxUfjeEovHqp6I+0QYaPEBa2gOmIsJvf54AAPYdZrH+FwLrMiia+2CtQaYb+3YpmbDYk2yZUv9hENTLJE+8KrkisUSW5Ax0vYz/KsF1mNEG1f29oISu6NfsRuoWASG9fSGFaNdh3FCBiW+92CeIyzcfN14FuB4wBXqVrHKdk9+3AunJw5j3YotDh7J7NXw4U1AZhI9x9K6uhy9Af87oxD2bKEbr+4VUru3nnJWqFSrIFwMiBxfyYtqcnA+TwiERTrDG+Jlfn4xH6BPAXg5YlydVpMCp5zXh5NhL340DSApfg3db+lMkbW/y4+b0LvZdToxBkiS0pajHwAiLZIDdMBf0we+Wr3ku+0ttyWfYAnpCU5sdYxpDHeI1a1T6b2XvF5jj60BXfaGkLT+tm2CW8DJmFPiDmSaG5EL61LUfWk6Nf4tvlox9TyhCaB0WQKMsdt6CnUgkB60IP+XyPHOI/4La35f/utTVHUd5h6N6lbE9AnMEk2RXeXtHZezs4kfkertEjcZCqvK27N9L3ZgF11Ylm6zhNX5mMK7MgBWrxp2lv0l3t4aM8yk3Mxy+ktYTCyzWMaxAUCNivIwpc8gEP7ejv7hi6kJlDdQUUBOwdKY2LHjtwfMicgg2P59I8oT3K9MwxOsMOsYdwD89fy+KNgUArMoFw9H5dtsZqF4X6uaOGnGjmMs/UWyZsE7FAMNufzXaKuEN1W/OXwyo13zQD2Jou4L5fbLMIZommnAzGpxWaRrAWWBA9fwnuXNO8Y1kl1duvJs1uLhGPICqBKE7h X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230016)(4636009)(39860400002)(396003)(136003)(346002)(376002)(40470700004)(36840700001)(46966006)(70586007)(4326008)(8676002)(54906003)(40460700003)(86362001)(82740400003)(356005)(70206006)(52536014)(110136005)(478600001)(5660300002)(316002)(8936002)(81166007)(36860700001)(33656002)(82310400005)(2906002)(40480700001)(41300700001)(6506007)(7696005)(55016003)(9686003)(83380400001)(186003)(47076005)(336012)(26005); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 19:22:26.2255 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 555ac6b1-b540-4404-11d8-08da6117311c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT027.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4017 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >=20 > The original reader/writer lock in DPDK can cause a stream of readers to > starve writers. >=20 > The new version uses an additional bit to indicate that a writer is waiti= ng and > which keeps readers from starving the writer. This addition makes sense. I am wondering if we should create a new lock. Is it possible that some app= lications are dependent on the current behavior? >=20 > Signed-off-by: Stephen Hemminger > --- > Would like this to be in 22.11, but needs some more review >=20 > lib/eal/include/generic/rte_rwlock.h | 93 ++++++++++++++++++---------- > 1 file changed, 61 insertions(+), 32 deletions(-) >=20 > diff --git a/lib/eal/include/generic/rte_rwlock.h > b/lib/eal/include/generic/rte_rwlock.h > index da9bc3e9c0e2..725cd19ffb27 100644 > --- a/lib/eal/include/generic/rte_rwlock.h > +++ b/lib/eal/include/generic/rte_rwlock.h > @@ -13,7 +13,7 @@ > * This file defines an API for read-write locks. The lock is used to > * protect data that allows multiple readers in parallel, but only > * one writer. All readers are blocked until the writer is finished > - * writing. > + * writing. This version will not starve writers. > * > */ >=20 > @@ -28,10 +28,17 @@ extern "C" { > /** > * The rte_rwlock_t type. > * > - * cnt is -1 when write lock is held, and > 0 when read locks are held. > + * Readers increment the counter by RW_READ (4) > + * Writers set the RWLOCK_WRITE bit when lock is held > + * and set the RWLOCK_WAIT bit while waiting. > */ > + > +#define RTE_RWLOCK_WAIT 0x1 /* Writer is waiting */ > +#define RTE_RWLOCK_WRITE 0x2 /* Writer has the lock */ > +#define RTE_RWLOCK_READ 0x4 /* Reader increment */ > + > typedef struct { > - volatile int32_t cnt; /**< -1 when W lock held, > 0 when R locks held. > */ > + volatile int32_t cnt; > } rte_rwlock_t; >=20 > /** > @@ -61,17 +68,24 @@ static inline void > rte_rwlock_read_lock(rte_rwlock_t *rwl) { > int32_t x; > - int success =3D 0; >=20 > - while (success =3D=3D 0) { > + while (1) { > x =3D __atomic_load_n(&rwl->cnt, __ATOMIC_RELAXED); > /* write lock is held */ > - if (x < 0) { > + if (x & (RTE_RWLOCK_WAIT | RTE_RWLOCK_WRITE)) { > rte_pause(); > continue; > } > - success =3D __atomic_compare_exchange_n(&rwl->cnt, &x, x > + 1, 1, > - __ATOMIC_ACQUIRE, > __ATOMIC_RELAXED); > + > + /* Try to get read lock */ > + x =3D __atomic_add_fetch(&rwl->cnt, RTE_RWLOCK_READ, > + __ATOMIC_ACQUIRE); > + if (!(x & (RTE_RWLOCK_WAIT | RTE_RWLOCK_WRITE))) > + return; > + > + /* Undo */ > + __atomic_fetch_sub(&rwl->cnt, RTE_RWLOCK_READ, > + __ATOMIC_RELEASE); > } > } >=20 > @@ -93,17 +107,23 @@ static inline int > rte_rwlock_read_trylock(rte_rwlock_t *rwl) { > int32_t x; > - int success =3D 0; >=20 > - while (success =3D=3D 0) { > - x =3D __atomic_load_n(&rwl->cnt, __ATOMIC_RELAXED); > - /* write lock is held */ > - if (x < 0) > - return -EBUSY; > - success =3D __atomic_compare_exchange_n(&rwl->cnt, &x, x > + 1, 1, > - __ATOMIC_ACQUIRE, > __ATOMIC_RELAXED); > - } > + x =3D __atomic_load_n(&rwl->cnt, __ATOMIC_RELAXED); > + > + /* write lock is held */ > + if (x & (RTE_RWLOCK_WAIT | RTE_RWLOCK_WRITE)) > + return -EBUSY; > + > + /* Try to get read lock */ > + x =3D __atomic_add_fetch(&rwl->cnt, RTE_RWLOCK_READ, > + __ATOMIC_ACQUIRE); > + > + if (x & (RTE_RWLOCK_WAIT | RTE_RWLOCK_WRITE)) { > + __atomic_fetch_sub(&rwl->cnt, RTE_RWLOCK_READ, > + __ATOMIC_RELEASE); >=20 > + return -EBUSY; > + } > return 0; > } >=20 > @@ -116,7 +136,7 @@ rte_rwlock_read_trylock(rte_rwlock_t *rwl) static > inline void rte_rwlock_read_unlock(rte_rwlock_t *rwl) { > - __atomic_fetch_sub(&rwl->cnt, 1, __ATOMIC_RELEASE); > + __atomic_fetch_sub(&rwl->cnt, RTE_RWLOCK_READ, > __ATOMIC_RELEASE); > } >=20 > /** > @@ -139,11 +159,12 @@ rte_rwlock_write_trylock(rte_rwlock_t *rwl) > int32_t x; >=20 > x =3D __atomic_load_n(&rwl->cnt, __ATOMIC_RELAXED); > - if (x !=3D 0 || __atomic_compare_exchange_n(&rwl->cnt, &x, -1, 1, > - __ATOMIC_ACQUIRE, __ATOMIC_RELAXED) =3D=3D 0) > + if (x < RTE_RWLOCK_WRITE && > + __atomic_compare_exchange_n(&rwl->cnt, &x, x + > RTE_RWLOCK_WRITE, > + 1, __ATOMIC_ACQUIRE, > __ATOMIC_RELAXED)) > + return 0; > + else > return -EBUSY; > - > - return 0; > } >=20 > /** > @@ -156,18 +177,26 @@ static inline void > rte_rwlock_write_lock(rte_rwlock_t *rwl) { > int32_t x; > - int success =3D 0; >=20 > - while (success =3D=3D 0) { > + while (1) { > x =3D __atomic_load_n(&rwl->cnt, __ATOMIC_RELAXED); > - /* a lock is held */ > - if (x !=3D 0) { > - rte_pause(); > - continue; > + > + /* No readers or writers */ > + if (x < RTE_RWLOCK_WRITE) { > + /* Turn off RTE_RWLOCK_WAIT, turn on > RTE_RWLOCK_WRITE */ > + if (__atomic_compare_exchange_n(&rwl->cnt, &x, > RTE_RWLOCK_WRITE, 1, > + > __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) > + return; > } > - success =3D __atomic_compare_exchange_n(&rwl->cnt, &x, - > 1, 1, > - __ATOMIC_ACQUIRE, > __ATOMIC_RELAXED); > - } > + > + /* Turn on writer wait bit */ > + if (!(x & RTE_RWLOCK_WAIT)) > + __atomic_fetch_or(&rwl->cnt, RTE_RWLOCK_WAIT, > __ATOMIC_RELAXED); > + > + /* Wait until can try to take the lock */ > + while (__atomic_load_n(&rwl->cnt, __ATOMIC_RELAXED) > > RTE_RWLOCK_WAIT) > + rte_pause(); > + } > } >=20 > /** > @@ -179,7 +208,7 @@ rte_rwlock_write_lock(rte_rwlock_t *rwl) static > inline void rte_rwlock_write_unlock(rte_rwlock_t *rwl) { > - __atomic_store_n(&rwl->cnt, 0, __ATOMIC_RELEASE); > + __atomic_fetch_sub(&rwl->cnt, RTE_RWLOCK_WRITE, > __ATOMIC_RELEASE); > } >=20 > /** > -- > 2.35.1