From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D305EA04B2; Wed, 26 Aug 2020 12:17:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BC13F58C4; Wed, 26 Aug 2020 12:17:26 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70074.outbound.protection.outlook.com [40.107.7.74]) by dpdk.org (Postfix) with ESMTP id C0DEC1F1C for ; Wed, 26 Aug 2020 12:17:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ytoz29RG9uaOZ31QSatlzMkA/uKWAiJ3h5U7nlr/Gpo=; b=gw5CRsq++A0MwD2gTXPyTpLadHSymkBs5aoj5mCc7utl8drKFFwuOnuYg+rOMUy9ZOJ3VHu/FS40ks83k179Q9zd2yZLeN2IskCwCUuTxtVSgMxBMcKOK22lw5Zot4QmhqVOqDtF8ui2Dfwrkn2zYzBzrBDwxOxmkzii2QUA4sg= Received: from DB6PR0601CA0045.eurprd06.prod.outlook.com (2603:10a6:4:17::31) by AM0PR08MB5266.eurprd08.prod.outlook.com (2603:10a6:208:162::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3305.25; Wed, 26 Aug 2020 10:17:22 +0000 Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:17:cafe::69) by DB6PR0601CA0045.outlook.office365.com (2603:10a6:4:17::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3305.25 via Frontend Transport; Wed, 26 Aug 2020 10:17:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3326.19 via Frontend Transport; Wed, 26 Aug 2020 10:17:22 +0000 Received: ("Tessian outbound 7a6fb63c1e64:v64"); Wed, 26 Aug 2020 10:17:22 +0000 X-CR-MTA-TID: 64aa7808 Received: from 8b5754b74892.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 710CAB4E-47E1-4538-94B6-2F0A07199978.1; Wed, 26 Aug 2020 10:17:17 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 8b5754b74892.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 26 Aug 2020 10:17:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kb8Oz8WowhPbZrthUkNl5JgMg0l362wEtnCZAYnbCqlRy6tIHtDII2FSMSlh8ALT+yT1NaJwMWdvh026l1hVAzlaXS7b49e545eqXl2TTZz4RtYhqLT1Jbxwz0GplsPwFclwPmU8jPb8oNlUFm6jtc/iiL6G3FaiHl76EzftC3sJCKzPSXMmLlpVjwrmxbXu3fQ1L8NAEpMC4iu45nBIENouuLjhXbMCL/rqnwheVekcOKLjYCTexAzmQjpyrfiO4GnPAmC/xNp9rmid+A/xvULv+RzyGw2ybeV2mFkecSMjgBYJPOAxX65UR2kiZdUTY3BEDq0G9Xt46OmWbuJaNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ytoz29RG9uaOZ31QSatlzMkA/uKWAiJ3h5U7nlr/Gpo=; b=QqPL32OlfYTJpxWMq/RIL5BP8lIcwmD7hsnbrO8tcTMT/ezWXeXNFvF4vcXZ0jCcqGIbz9IAJzqhvqRflnvqWmLsQNwKF/nIQCyxkk1NlKOsr/ZxscjOSoNeYiWW1tW9bP9su6uX5YOUyXglDr3XtnPO8CZUQBgI3D75nUErz2DUZQqhGZqngQGenQRRzxvqysHqmiG8MYlWJKt9oIitrSqBAFyk94NGmGZ8T3QCtviWyIUYEflUQmi6HVj7v311TpETU7spIYPEc2M5r+m6fhZdbQaytNpw52oeiZq0qRvSv729bMDK/8vOfRMuNC8yIQBQOqoElmMQ3JVb6tXytg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ytoz29RG9uaOZ31QSatlzMkA/uKWAiJ3h5U7nlr/Gpo=; b=gw5CRsq++A0MwD2gTXPyTpLadHSymkBs5aoj5mCc7utl8drKFFwuOnuYg+rOMUy9ZOJ3VHu/FS40ks83k179Q9zd2yZLeN2IskCwCUuTxtVSgMxBMcKOK22lw5Zot4QmhqVOqDtF8ui2Dfwrkn2zYzBzrBDwxOxmkzii2QUA4sg= Received: from DB7PR08MB3865.eurprd08.prod.outlook.com (2603:10a6:10:74::25) by DB6PR08MB2632.eurprd08.prod.outlook.com (2603:10a6:6:17::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3326.19; Wed, 26 Aug 2020 10:17:15 +0000 Received: from DB7PR08MB3865.eurprd08.prod.outlook.com ([fe80::519c:72bd:e189:625b]) by DB7PR08MB3865.eurprd08.prod.outlook.com ([fe80::519c:72bd:e189:625b%7]) with mapi id 15.20.3305.026; Wed, 26 Aug 2020 10:17:15 +0000 From: Phil Yang To: Diogo Behrens CC: "dev@dpdk.org" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH] librte_eal: fix mcslock hang on weak memory Thread-Index: AQHWe4pZeMmgY2mDiEqkB/VtKH/xk6lKKsYA Date: Wed, 26 Aug 2020 10:17:15 +0000 Message-ID: References: <20200826092002.19395-1-diogo.behrens@huawei.com> In-Reply-To: <20200826092002.19395-1-diogo.behrens@huawei.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: C748942AC7983049849250EEDA3BED05.0 x-checkrecipientchecked: true Authentication-Results-Original: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [203.126.0.111] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: df905609-5eb4-4a76-79c5-08d849a938da x-ms-traffictypediagnostic: DB6PR08MB2632:|AM0PR08MB5266: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: tdKWlTnN5hVvFVubVHwkYQKE92Ax/st+bp9R4yLZ17TRQftz75eXjc0oPEaZU0OztJ6kO1h9JfclR/ku+MJq9lmHqei2AheCkhac/nENkBIf38Xwa5aYHDuWN8ypz8HXXLNaCY4NZRK4xabYcnQm5rRu5k6LHjpJEPo7ID38EAUzUUYLkhzChZDLkKAjmdLvHnhtvQrkdtLehabNUm7oI6OnJW2GZyW6Qpd3O01uwPl0Mb1/JtFexudxEerc+FDt5mWGbLTsAbi1/vQCzsfsvG8uZrT9AHNnnuY3uLnqWM38V1Yx+sodk7aAHV3q5/WJ0O781f1veStYHX32Y/aZWhJ9DMi6CTdwPTlda8EWztX5QjCK0Mbu/AkV5KfwZpoSNhMzxGWJXQESnM0VEoMlfQ8LApgcVgtyBHTqv5Z7aqko7Zr7Iu2we2lqyKl/Gph5EHiDQWvB6EFgFLjeEU0FkQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB7PR08MB3865.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(366004)(39860400002)(136003)(396003)(346002)(5660300002)(7696005)(71200400001)(316002)(186003)(6916009)(8676002)(66476007)(66946007)(76116006)(52536014)(83380400001)(66446008)(64756008)(66556008)(4326008)(8936002)(478600001)(966005)(86362001)(6506007)(33656002)(55016002)(26005)(9686003)(2906002)(54906003)(87944003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: MHs7OWEwqHMjTxkclnwVFZ+xFBaMIU7gvdzu+rMkLnM6AefAFiVMIZKP7LURyjQfvcffqivMjHvt1K9CvVJKEBbyk8k7BcPUX9JwfdBvrlFfQd1NRrAaQKex+usQ9GItjkjKPVZhfhwpM3MUlUD44iGxoWcv8iBEAlTueOhfXNaOYA5QoJZF0YjeDpZH9Ce/X90plU4Jll/mO48r5T82s2Z9R76ICyJe3dPRv1EJfAUCZgJRayWHy0pnmYPoUhJhiQ6qpDT8HGJHAS6CoxCM0J7rBzdtCDnKShcErNIVdFztzP3ZmbQ2Vp/eS/3X650o9sfahAe3aARSffcYQneBQYI70/1CCkxv9gLYd6PvtQNkQrvXC01kBcqyKBI9hy1Q5MqFw82bz2g0bepq78eR3Ya0Zo0FbAetsKc0p9kVS4XQ1Tim5WqGIaSkVPyvKPwDFYqpoGGlBRX2jxE0vTS94bEiuftvXy6RbVnJ6Ar9p2JnV4+ECH149upBp2VUvo4JZay9jhr+M19ArfXdeuBQ0+3cVp/7VxzBjXDJXXuQ8OcmZ+XaJBcHnl6tX8GLflgUemT770z6NviZ8ymMxS/zK1DftrGM2YUCnwJE64BKHeEbbNi7Rz+Gsa+2JS/jzKCSMUyO9zMCUpLIlz1f8dOIjQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR08MB2632 Original-Authentication-Results: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: d30aa1c4-00ed-436e-b0f8-08d849a934c5 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cAemgreK1jr4PvtxrqbVbUTBIQqyRvLvXQ6KTgSK6RwvwzS9JeS9nWTKffAOAa4lncxiHe/DOV9ZMPswdroNc9geGhCcs7U74+H4XAku1yUIxoX050hD6ih/HYTur/GJb4PIz1K0NPz4Gi0wb1tDzsx4ZHfQQXpEL12E5GIbDsvjR84ty6b5Hv/MoUEHmeao4xnlXU7NySSrGdfjMtwuayDm8mMAsxacG7GBVp0Vs3xeAMmaI3eR7QJ8m3wIhhwBj06E1g9J1CkHIKBeXLRmPydF9H+fVQamTD34T7oFEu64BbaD3EnDJG1SJ7hMAVm6Yk0XMykE6PTLdMOEQAIIrhmW2LkQdIxZ0zk9XCuksRJpoqp+vY+tUZ9UMmv2e5BfYZJT1EPLzp085Vz/jBhcyRCw+NR0wZti01ZgnYzm40Y69LZrK18/DSvoBvwbrfk7YNKxJ9Bqs66/n5HnzoxKwnfc5XvqsD17zw1iNLjuFp9fFzY+7o/ejP4MkhEyYJLd X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(396003)(39860400002)(376002)(136003)(346002)(46966005)(82310400002)(9686003)(55016002)(8936002)(316002)(6506007)(54906003)(2906002)(70586007)(70206006)(8676002)(83380400001)(336012)(356005)(86362001)(478600001)(4326008)(966005)(47076004)(82740400003)(26005)(186003)(33656002)(5660300002)(52536014)(7696005)(6862004)(81166007)(87944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2020 10:17:22.4923 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: df905609-5eb4-4a76-79c5-08d849a938da X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5266 Subject: Re: [dpdk-dev] [PATCH] librte_eal: fix mcslock hang on weak memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Diogo Behrens writes: > Subject: [PATCH] librte_eal: fix mcslock hang on weak memory >=20 > The initialization me->locked=3D1 in lock() must happen before > next->locked=3D0 in unlock(), otherwise a thread may hang forever, > waiting me->locked become 0. On weak memory systems (such as ARMv8), > the current implementation allows me->locked=3D1 to be reordered with > announcing the node (pred->next=3Dme) and, consequently, to be > reordered with next->locked=3D0 in unlock(). >=20 > This fix adds a release barrier to pred->next=3Dme, forcing > me->locked=3D1 to happen before this operation. >=20 > Signed-off-by: Diogo Behrens > --- > lib/librte_eal/include/generic/rte_mcslock.h | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) >=20 > diff --git a/lib/librte_eal/include/generic/rte_mcslock.h > b/lib/librte_eal/include/generic/rte_mcslock.h > index 2bef28351..ce553f547 100644 > --- a/lib/librte_eal/include/generic/rte_mcslock.h > +++ b/lib/librte_eal/include/generic/rte_mcslock.h > @@ -68,7 +68,14 @@ rte_mcslock_lock(rte_mcslock_t **msl, rte_mcslock_t > *me) > */ > return; > } > - __atomic_store_n(&prev->next, me, __ATOMIC_RELAXED); > + /* The store to me->next above should also complete before the > node is > + * visible to predecessor thread releasing the lock. Hence, the store > + * prev->next also requires release semantics. Note that, for > example, > + * on ARM, the release semantics in the exchange operation is not > + * strong as a release fence and is not sufficient to enforce the > + * desired order here. Hi Diogo, I didn't quite understand why the exchange operation with ACQ_REL memory or= dering is not sufficient. It emits 'stlxr' on armv8.0-a and 'swpal' on armv8.1-a (with LSE support).= =20 Both of these two instructions contain a release semantics.=20 Please check the URL for the detail. https://gcc.godbolt.org/z/Mc4373 BTW, if you captured a deadlock issue on your testbed, could you please sha= re your test case here? Thanks, Phil > + */ > + __atomic_store_n(&prev->next, me, __ATOMIC_RELEASE); >=20 > /* The while-load of me->locked should not move above the > previous > * store to prev->next. Otherwise it will cause a deadlock. Need a > -- > 2.28.0 >=20