From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7531AA0506; Tue, 29 Mar 2022 10:21:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1735041151; Tue, 29 Mar 2022 10:21:06 +0200 (CEST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150052.outbound.protection.outlook.com [40.107.15.52]) by mails.dpdk.org (Postfix) with ESMTP id 6CA6D40041 for ; Mon, 28 Mar 2022 16:06:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uG0v4aya+Dl/M+ctqpuCJpxpTAxRU0NUH4tX1ZCTsmU=; b=QYGBtH1NU0XQsxOc9HCRbiedIHq503E+P+lCngMLoiF2Mo8OdcN+bxybbgroRqMIgHJlCKfSo1m8TZHoul1pLGwNk5LwbLKlkIE1QhYMdj7oYoJU0ZWuqtIobFChiQg0OU33R3DNHvVGhAkiAhlUN2+7bEqiudahMssx5/nw5qM= Received: from AS8PR05CA0012.eurprd05.prod.outlook.com (2603:10a6:20b:311::17) by AM6PR08MB4801.eurprd08.prod.outlook.com (2603:10a6:20b:c1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.19; Mon, 28 Mar 2022 14:06:43 +0000 Received: from VE1EUR03FT056.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:311:cafe::46) by AS8PR05CA0012.outlook.office365.com (2603:10a6:20b:311::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.18 via Frontend Transport; Mon, 28 Mar 2022 14:06:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT056.mail.protection.outlook.com (10.152.19.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.18 via Frontend Transport; Mon, 28 Mar 2022 14:06:43 +0000 Received: ("Tessian outbound 826a6d8e58c3:v113"); Mon, 28 Mar 2022 14:06:42 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 036e0237ef70bc8f X-CR-MTA-TID: 64aa7808 Received: from 08f2a32287b0.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id AF9EDF5C-132A-4EAC-9E42-7B38D36F5D43.1; Mon, 28 Mar 2022 14:06:33 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 08f2a32287b0.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 28 Mar 2022 14:06:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V6KSQ7G0D2NitCgBvCdoXVTnvlDF9+2eQ6LjhfTBWeAvNTeDnLrMf8FHGu5pn8UThdAfarMU7b3uO9fPGr1kYCCeAOPDzJ/3q72/gl/f1fab0d9PHUx2nyER2Mp1osyauto2Czv0IhbJjENhUlS/gkEDtj4J8kaWztR+RIfQNsVT9D2Mi/pkviNwP7BWjYQGqj7N3cgXQzhLjTYFRRMeSFHImme2NNSlJtxgRbOXFkYwCyyKAs1K0dK70gdtgeY78aaVuqqsqenmsFMrsnPKSh4nb3CuWEpJnrsuuMGVhLMmFAM3sxbEA3+l3tAIZWN2Yi3sPXaSB/MCF3g4n3bOTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uG0v4aya+Dl/M+ctqpuCJpxpTAxRU0NUH4tX1ZCTsmU=; b=L+nsmk1OUGIQUMuxrUh8ZonCws9jUj7+VJHjiU7aufDoWfDSi6AThzr+VNZcYUOuiQGAo/vCgvv+YWeeTzf3/rgBHnnZ4N+w7m1z7xYgA/GRhINnYBut5heNpHOLYwEMMCAGmAAlV/S9f+g39xw74K/H8bIlql4d8Rr6/7SGLM4poxMPGQSyBp4Ze27qliZts9AT7tZTHwo4Lu9NaCRPQ9Dtvce3GFY+PztP47mlKGPSIq0Ejd6w0VR0nHbY/DtQtVzoOsPAD5wdfMSbhjpFPc8f7ExHU/laXFjdW3lKSV1UESudxQfFQA+oNYBuHxDOx6eRYBsbosBF9iwEw+dR4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uG0v4aya+Dl/M+ctqpuCJpxpTAxRU0NUH4tX1ZCTsmU=; b=QYGBtH1NU0XQsxOc9HCRbiedIHq503E+P+lCngMLoiF2Mo8OdcN+bxybbgroRqMIgHJlCKfSo1m8TZHoul1pLGwNk5LwbLKlkIE1QhYMdj7oYoJU0ZWuqtIobFChiQg0OU33R3DNHvVGhAkiAhlUN2+7bEqiudahMssx5/nw5qM= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from AM8PR08MB6498.eurprd08.prod.outlook.com (2603:10a6:20b:364::11) by VI1PR0801MB1662.eurprd08.prod.outlook.com (2603:10a6:800:52::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.21; Mon, 28 Mar 2022 14:06:29 +0000 Received: from AM8PR08MB6498.eurprd08.prod.outlook.com ([fe80::ec4f:1ad:e332:9f39]) by AM8PR08MB6498.eurprd08.prod.outlook.com ([fe80::ec4f:1ad:e332:9f39%7]) with mapi id 15.20.5102.023; Mon, 28 Mar 2022 14:06:29 +0000 Message-ID: Date: Mon, 28 Mar 2022 16:06:25 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [RFC] eal: add seqlock Content-Language: en-US To: "Ananyev, Konstantin" , "mattias.ronnblom" , "dev@dpdk.org" Cc: Thomas Monjalon , David Marchand , "Olsen, Onar" , "Honnappa.Nagarahalli@arm.com" , "nd@arm.com" , "mb@smartsharesystems.com" , "stephen@networkplumber.org" References: <20220325202428.94628-1-mattias.ronnblom@ericsson.com> <77168168-eadb-9a0b-b51e-9ccdf8ad7230@ericsson.com> From: Ola Liljedahl In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO2P265CA0458.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a2::14) To AM8PR08MB6498.eurprd08.prod.outlook.com (2603:10a6:20b:364::11) MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 1d12054f-b666-4ec4-2edf-08da10c4301f X-MS-TrafficTypeDiagnostic: VI1PR0801MB1662:EE_|VE1EUR03FT056:EE_|AM6PR08MB4801:EE_ X-LD-Processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: nJpZzVh7Zfxcf8jcxxKStzeMRs9O5dCKFDdS174bfJTZMeFgWl6SsRMlxYPcEPCjjeVPZBU02O1jCHCiLbpF/yGSbOJTU6SGgy2sZ1KNgCrpUWLZCznHgvp7Vjegde982FvTJyWSVlGYjGGUiTeTE8WwRnsKh1PRjO4UQsL4bPRez5Ie82OX+2TEnPbRvpdWW8u/UH/nox6hnUkbTI+FNt/YWhsS6KiyCKceFpmNzuNo1JLoNvZUxM8mQr3AsQUFVJSlRElqgVkyn01WMFObR+3HSLiVRmfIs85imEWGnTOeJsmANDocaFu0HxtT9FTTVL87asI07UoMiV5ZNAF8opQo9+rx2notGhcjA4rErlDqhba53cE+6rL9fFLMtlFbOgqgaVeEhlcV1wEedB0VrtSoMDAoSNnIP+fflgR7M63sW+mOPlZWErqGmu4/32NwKLSieeKN7jsXvNY7a3UKybsGYRq6jXiJu0NvMexxbGdbDKgdKN+sn3XX5xWKe9Xz/TiuCjAYKGgJbWUm/pga4/464mNFT9LM5cb6djsbqPeqFuLcE3X96fiSqYYeK4ZLmIE2iV/4uuMyJZTxQLbk1PPgBWmim+hpOV3DzdD/WwDcj+uCoWFBEu96mD/zkWSAlcrXVYKjBZ+CinfCsxHY7FzxjsWNwmZ2RJqJZEXqNCwutVLZ0M0fgBbr2euMUgFoEjiH5Dc6+7IeG7Rsew2fedmVE2pOrro6VhTQvWcG2GjP78MwMgzsMRghhhv1mRSYHB2q0mD9JlYcfezFsgQYVBYJmsBFV9yRuT+plGCOafQuZZgzeKSAK8H+r3MHuTqQ X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR08MB6498.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(53546011)(2906002)(6512007)(110136005)(44832011)(316002)(31686004)(66946007)(36756003)(38100700002)(6666004)(83380400001)(54906003)(8936002)(4326008)(66556008)(966005)(8676002)(186003)(26005)(5660300002)(508600001)(31696002)(66476007)(86362001)(6486002)(2616005)(6506007)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1662 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT056.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 65304bdd-6947-4321-501b-08da10c4276f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bd8T/A+ooWcVEtrp+v9i4he+rJCeACEXPcaMDck3DpVSu4CeIQAZwj+m2spuO23rcRonx1fC4duBj5wPpFYpq4nLZvcB/f4MxkcvsHoG5bOojg2obpen+4778Sb1fQlDKGEtVrW2wpyURJagl1fPZQsNygilT5O11OR1uk8JdyfwrYY9jDSBHXiF0qCctbPj0Rzc5AM5d3hu787gUd8aCJZHHo1M3gwlYR9CcfB1xAq4WvdIsQS5D+IJuGWLISQGgnzuShS7+yf7tqek+go3nDOUbrwi7W/wEye2lrmF+EVsXW8hjRayNiVbLOOnvudEMDXPMF7K2/rLJGnCG7UxYqGxf/FZ64dNXLHkhmfv3vPNaxzaD5jkBEwB3UWwMoGWn4ZEc3o/143ckyQF0Ko6FXZzTTFt/d5asQYE5cdQy/ugKYBu8vQzDERy9WiNVIMJEZ53PCsGWu0w5bRual8jpuFRNrHszYzTfaf2lrSrw3Me+4Fr2rolLpmMrd0ONR9WB97mhXqRAXhRo8F+xy3Y7EPc6D1XqhOuhWH+c57CGtAT7MkZbxS2bJYx3Ocbm9lyOGhC1lvB5TmWrXqmhB/AUqNwEwR2JReWr/cK/tlHlHP4t5qyF//SCNgepfsHwm1R/fbHGnqM7jJJkOYeG8py4LxNcgjFhqQBK/Xk8EXQsBPOnXYaWGYNcz9Fl8gwA2dBYCFbQSxf6Oxu6CRyv4v9jqsOim2ud/fKld5+9Zsw53wLoXfSXE4NDpOkhyLghZKDwgh44dCpVAjTouS7M9rLKzEi1cNLKayM9jM+mkN4TiE= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(8676002)(86362001)(44832011)(2616005)(356005)(4326008)(2906002)(336012)(8936002)(36860700001)(82310400004)(81166007)(40460700003)(31696002)(83380400001)(53546011)(966005)(5660300002)(47076005)(508600001)(6486002)(110136005)(6506007)(316002)(6666004)(36756003)(26005)(54906003)(70206006)(70586007)(186003)(107886003)(31686004)(6512007)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Mar 2022 14:06:43.1924 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1d12054f-b666-4ec4-2edf-08da10c4301f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT056.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4801 X-Mailman-Approved-At: Tue, 29 Mar 2022 10:21:04 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 3/28/22 12:53, Ananyev, Konstantin wrote: > >>>> diff --git a/lib/eal/include/meson.build b/lib/eal/include/meson.build >>>> index 9700494816..48df5f1a21 100644 >>>> --- a/lib/eal/include/meson.build >>>> +++ b/lib/eal/include/meson.build >>>> @@ -36,6 +36,7 @@ headers += files( >>>> 'rte_per_lcore.h', >>>> 'rte_random.h', >>>> 'rte_reciprocal.h', >>>> + 'rte_seqlock.h', >>>> 'rte_service.h', >>>> 'rte_service_component.h', >>>> 'rte_string_fns.h', >>>> diff --git a/lib/eal/include/rte_seqlock.h b/lib/eal/include/rte_seqlock.h >>>> new file mode 100644 >>>> index 0000000000..b975ca848a >>>> --- /dev/null >>>> +++ b/lib/eal/include/rte_seqlock.h >>>> @@ -0,0 +1,84 @@ >>>> +/* SPDX-License-Identifier: BSD-3-Clause >>>> + * Copyright(c) 2022 Ericsson AB >>>> + */ >>>> + >>>> +#ifndef _RTE_SEQLOCK_H_ >>>> +#define _RTE_SEQLOCK_H_ >>>> + >>>> +#include >>>> +#include >>>> + >>>> +#include >>>> +#include >>>> +#include >>>> + >>>> +struct rte_seqlock { >>>> + uint64_t sn; >>>> + rte_spinlock_t lock; >>>> +}; >>>> + >>>> +typedef struct rte_seqlock rte_seqlock_t; >>>> + >>>> +__rte_experimental >>>> +void >>>> +rte_seqlock_init(rte_seqlock_t *seqlock); >>> Probably worth to have static initializer too. >>> >> >> I will add that in the next version, thanks. >> >>>> + >>>> +__rte_experimental >>>> +static inline uint64_t >>>> +rte_seqlock_read_begin(const rte_seqlock_t *seqlock) >>>> +{ >>>> + /* __ATOMIC_ACQUIRE to prevent loads after (in program order) >>>> + * from happening before the sn load. Syncronizes-with the >>>> + * store release in rte_seqlock_end(). >>>> + */ >>>> + return __atomic_load_n(&seqlock->sn, __ATOMIC_ACQUIRE); >>>> +} >>>> + >>>> +__rte_experimental >>>> +static inline bool >>>> +rte_seqlock_read_retry(const rte_seqlock_t *seqlock, uint64_t begin_sn) >>>> +{ >>>> + uint64_t end_sn; >>>> + >>>> + /* make sure the data loads happens before the sn load */ >>>> + rte_atomic_thread_fence(__ATOMIC_ACQUIRE); >>> That's sort of 'read_end' correct? >>> If so, shouldn't it be '__ATOMIC_RELEASE' instead here, >>> and >>> end_sn = __atomic_load_n(..., (__ATOMIC_ACQUIRE) >>> on the line below? >> >> A release fence prevents reordering of stores. The reader doesn't do any >> stores, so I don't understand why you would use a release fence here. >> Could you elaborate? > > From my understanding: > rte_atomic_thread_fence(__ATOMIC_ACQUIRE); > serves as a hoist barrier here, so it would only prevent later instructions > to be executed before that point. > But it wouldn't prevent earlier instructions to be executed after that point. > While we do need to guarantee that cpu will finish all previous reads before > progressing further. > > Suppose we have something like that: > > struct { > uint64_t shared; > rte_seqlock_t lock; > } data; > > ... > sn = ... > uint64_t x = data.shared; > /* inside rte_seqlock_read_retry(): */ > ... > rte_atomic_thread_fence(__ATOMIC_ACQUIRE); > end_sn = __atomic_load_n(&data.lock.sn, __ATOMIC_RELAXED); > > Here we need to make sure that read of data.shared will always happen > before reading of data.lock.sn. > It is not a problem on IA (as reads are not reordered), but on machines with > relaxed memory ordering (ARM, etc.) it can happen. > So to prevent it we do need a sink barrier here first (ATOMIC_RELEASE) We can't use store-release since there is no write on the reader-side. And fence-release orders against later stores, not later loads. > > Honnappa and other ARM & atomics experts, please correct me if I am wrong here. The C standard (chapter 7.17.4 in the C11 (draft)) isn't so easy to digest. If we trust Preshing, he has a more accessible description here: https://preshing.com/20130922/acquire-and-release-fences/ "An acquire fence prevents the memory reordering of any read which precedes it in program order with any read or write which follows it in program order." and here: https://preshing.com/20131125/acquire-and-release-fences-dont-work-the-way-youd-expect/ (for C++ but the definition seems to be identical to that of C11). Essentially a LoadLoad+LoadStore barrier which is what we want to achieve. GCC 10.3 for AArch64/A64 ISA generates a "DMB ISHLD" instruction. This waits for all loads preceding (in program order) the memory barrier to be observed before any memory accesses after (in program order) the memory barrier. I think the key to understanding atomic thread fences is that they are not associated with a specific memory access (unlike load-acquire and store-release) so they can't order earlier or later memory accesses against some specific memory access. Instead the fence orders any/all earlier loads and/or stores against any/all later loads or stores (depending on acquire or release). > >>>> + >>>> + end_sn = __atomic_load_n(&seqlock->sn, __ATOMIC_RELAXED); >>>> + >>>> + return unlikely(begin_sn & 1 || begin_sn != end_sn); >>>> +} >>>> + >>>> +__rte_experimental >>>> +static inline void >>>> +rte_seqlock_write_begin(rte_seqlock_t *seqlock) >>>> +{ >>>> + uint64_t sn; >>>> + >>>> + /* to synchronize with other writers */ >>>> + rte_spinlock_lock(&seqlock->lock); >>>> + >>>> + sn = seqlock->sn + 1; >>>> + >>>> + __atomic_store_n(&seqlock->sn, sn, __ATOMIC_RELAXED); >>>> + >>>> + /* __ATOMIC_RELEASE to prevent stores after (in program order) >>>> + * from happening before the sn store. >>>> + */ >>>> + rte_atomic_thread_fence(__ATOMIC_RELEASE); >>> I think it needs to be '__ATOMIC_ACQUIRE' here instead of '__ATOMIC_RELEASE'. >> >> Please elaborate on why. > > As you said in the comments above, we need to prevent later stores > to be executed before that point. So we do need a hoist barrier here. > AFAIK to guarantee a hoist barrier '__ATOMIC_ACQUIRE' is required. An acquire fence wouldn't order an earlier store (the write to seqlock->sn) from being reordered with some later store (e.g. writes to the protected data), thus it would allow readers to see updated data (possibly torn) with a pre-update sequence number. We need a StoreStore barrier for ordering the SN store and data stores => fence(release). Acquire and releases fences can (also) be used to create synchronize-with relationships (this is how the C standard defines them). Preshing has a good example on this. Basically Thread 1: data = 242; atomic_thread_fence(atomic_release); atomic_store_n(&guard, 1, atomic_relaxed); Thread 2: while (atomic_load_n(&guard, atomic_relaxed) != 1) ; atomic_thread_fence(atomic_acquire); do_something(data); These are obvious analogues to store-release and load-acquire, thus the acquire & release names of the fences. - Ola > >> >>>> +} >>>> + >>>> +__rte_experimental >>>> +static inline void >>>> +rte_seqlock_write_end(rte_seqlock_t *seqlock) >>>> +{ >>>> + uint64_t sn; >>>> + >>>> + sn = seqlock->sn + 1; >>>> + >>>> + /* synchronizes-with the load acquire in rte_seqlock_begin() */ >>>> + __atomic_store_n(&seqlock->sn, sn, __ATOMIC_RELEASE); >>>> + >>>> + rte_spinlock_unlock(&seqlock->lock); >>>> +} >>>> + >