From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2C18EA04B1; Sun, 22 Nov 2020 19:07:30 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1A9F172DF; Sun, 22 Nov 2020 19:07:29 +0100 (CET) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by dpdk.org (Postfix) with ESMTP id 1D82A72DD for ; Sun, 22 Nov 2020 19:07:27 +0100 (CET) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id C051A5C00AD; Sun, 22 Nov 2020 13:07:23 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Sun, 22 Nov 2020 13:07:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= +bsRnhORYConluBI7JyW91DZ9mrwLY1tW3ZLy9vlDHk=; b=tkUJcTMQySUZPnRQ AC1KkNzPJ5R+Fut60puiq4ScqszQESzja3azz1pLO/LH6iCHUlexiTlHH5Hbpx7y wMuZUjxe6MxY7O4c3BNVmNfhNyxSOi57eMpxPAbzDAeIsjJf645rU5soXf4Mn/bt zITd2MIJvTYAErDEiVU2VqYrpJWOFcERZH1EXgHGUkVcv2lBkNbjA7X63BsR7aIQ JZs0QZ5h9/AtPhaw4ndCkgEjBqP5ugnd3tT0bQ+0ClAv0wK7tvuZfJQveuL1IJpE +lSlxx2Ru5SFXWY9TSB66P33qqAwe5BkzXLiq+r7ail7PeYLBT2isi3EKBGwpMhL cvgRVg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=+bsRnhORYConluBI7JyW91DZ9mrwLY1tW3ZLy9vlD Hk=; b=AZtimlmXqL6XUsNuE5pCaItlfpVMJJ7hN5vYNTSdQbgiDV3AgcxZW+Ba7 3mj5O5A4z5a50kJJA3UpsftlLnWLQtFG2aut7wrQ0zkl/Wo7+LAkZMw5Ydmn03t2 sZLreR8anwTl1ffnXttgkpV/9XyyXtNuzbNU8Bfd8ypbMervAWZSgKckUQHbEqT1 XzwFHq5XRJrbsqKwn7dR+GBtkxeodwIM159BLUMljgyT3qQwhjRL3aBZLaQ3YpdW 3VUEDEZxP2oliayI1Ey7vJduUUjL2lwZORz3oLXtafrOlIteJr7J1avU2ORL3o/+ GiHf3AFv8t0QmL3Agt1gQQd1qNFxw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrudeggedgudduvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpeefgfevteffhfehveetueejjeduledtveeitedvvefgudeuhfel veejtdduuedvtdenucffohhmrghinheprghrmhdrtghomhdpihhnrhhirgdrfhhrnecukf hppeejjedrudefgedrvddtfedrudekgeenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 425D53064AA6; Sun, 22 Nov 2020 13:07:22 -0500 (EST) From: Thomas Monjalon To: Honnappa Nagarahalli Cc: dev@dpdk.org, Phil Yang , "dev@dpdk.org" , nd , Diogo Behrens , "david.marchand@redhat.com" Date: Sun, 22 Nov 2020 19:07:21 +0100 Message-ID: <3356496.NAzAc6GACp@thomas> In-Reply-To: References: <20200826092002.19395-1-diogo.behrens@huawei.com> <7423385.l6g0CaCsxP@thomas> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH] librte_eal: fix mcslock hang on weak memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 20/10/2020 23:49, Honnappa Nagarahalli: > > > > > > Honnappa? > > > > 07/10/2020 11:55, Diogo Behrens: > > > Hi Thomas, > > > > > > we are still waiting for the comments from Honnappa. In our > > > understanding, the missing barrier is a bug according to the model. We > > > reproduced the scenario in herd7, which represents the authoritative > > > memory model: > > > https://developer.arm.com/architectures/cpu-architecture/a-profile/mem > > > ory-model-tool > > > > > > Here is a litmus code that shows that the XCHG (when compiled to LDAXR > > and STLR) is not atomic wrt memory updates to other locations: > > > ----- > > > AArch64 XCHG-nonatomic > > > { > > > 0:X1=locked; 0:X3=next; > > > 1:X1=locked; 1:X3=next; 1:X5=tail; > > > } > > > P0 | P1; > > > LDR W0, [X3] | MOV W0, #1; > > > CBZ W0, end | STR W0, [X1]; (* init locked *) > > > MOV W2, #2 | MOV W2, #0; > > > STR W2, [X1] | xchg:; > > > end: | LDAXR W6, [X5]; > > > NOP | STLXR W4, W0, [X5]; > > > NOP | CBNZ W4, xchg; > > > NOP | STR W0, [X3]; (* set next *) > > > exists > > > (0:X2=2 /\ locked=1) > > > ----- > > > (web version of herd7: http://diy.inria.fr/www/?record=aarch64) > > > > > > P1 is trying to acquire the lock: > > > - initializes locked > > > - does the xchg on the tail of the mcslock > > > - sets the next > > > > > > P0 is releasing the lock: > > > - if next is not set, just terminates > > > - if next is set, stores 2 in locked > > > > > > The initialization of locked should never overwrite the store 2 to locked, but > > it does. > > > To avoid that reordering to happen, one should make the last store of P1 to > > have a "release" barrier, ie, STLR. > > > > > > This is equivalent to the reordering occurring in the mcslock of librte_eal. > > > > > > Best regards, > > > -Diogo > > > > > > -----Original Message----- > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > Sent: Tuesday, October 6, 2020 11:50 PM > > > To: Phil Yang ; Diogo Behrens > > > ; Honnappa Nagarahalli > > > > > > Cc: dev@dpdk.org; nd > > > Subject: Re: [dpdk-dev] [PATCH] librte_eal: fix mcslock hang on weak > > > memory > > > > > > 31/08/2020 20:45, Honnappa Nagarahalli: > > > > > > > > Hi Diogo, > > > > > > > > Thanks for your explanation. > > > > > > > > As documented in > > https://developer.arm.com/documentation/ddi0487/fc B2.9.5 Load- > > Exclusive and Store-Exclusive instruction usage restrictions: > > > > " Between the Load-Exclusive and the Store-Exclusive, there are no > > > > explicit memory accesses, preloads, direct or indirect System > > > > register writes, address translation instructions, cache or TLB > > maintenance instructions, exception generating instructions, exception > > returns, or indirect branches." > > > > [Honnappa] This is a requirement on the software, not on the micro- > > architecture. > > > > We are having few discussions internally, will get back soon. > > > > > > > > So it is not allowed to insert (1) & (4) between (2, 3). The cmpxchg > > operation is atomic. > > > > > > > > > Please what is the conclusion? > Apologies for not updating on this sooner. > > Unfortunately, memory ordering questions are hard topics. I have been discussing this internally with few experts and it is still ongoing, hope to conclude soon. > > My focus has been to replace __atomic_exchange_n(msl, me, __ATOMIC_ACQ_REL) with __atomic_exchange_n(msl, me, __ATOMIC_SEQ_CST). However, the generated code is the same in the second case as well (for load-store exclusives), which I am not sure if it is correct. > > I think we have 2 choices here: > 1) Accept the patch - when my internal discussion concludes, I can make the change and backport according to the conclusion. > 2) Wait till the discussion is over - it might take another couple of weeks One month passed since this last update. We are keeping this issue in DPDK 20.11.0 I guess.