From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7E42EA0511; Sun, 8 May 2022 21:41:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A5AA4068F; Sun, 8 May 2022 21:41:03 +0200 (CEST) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id E6DE540395 for ; Sun, 8 May 2022 21:41:01 +0200 (CEST) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id AD0511F3DF for ; Sun, 8 May 2022 21:41:01 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id AB7081F6A4; Sun, 8 May 2022 21:41:01 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED, AWL, NICE_REPLY_A, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.6 X-Spam-Score: -2.9 Received: from [192.168.1.59] (unknown [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 0606A1F4F5; Sun, 8 May 2022 21:40:59 +0200 (CEST) Message-ID: <55ed0b2b-ebe6-fd48-48b8-d173a69d541f@lysator.liu.se> Date: Sun, 8 May 2022 21:40:58 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: [PATCH v6] eal: add seqlock Content-Language: en-US To: Stephen Hemminger , =?UTF-8?Q?Mattias_R=c3=b6nnblom?= Cc: Thomas Monjalon , David Marchand , dev@dpdk.org, onar.olsen@ericsson.com, Honnappa.Nagarahalli@arm.com, nd@arm.com, konstantin.ananyev@intel.com, mb@smartsharesystems.com, Chengwen Feng , Ola Liljedahl References: <20220508121242.290008-1-mattias.ronnblom@ericsson.com> <20220508091034.53b23b3e@hermes.local> From: =?UTF-8?Q?Mattias_R=c3=b6nnblom?= In-Reply-To: <20220508091034.53b23b3e@hermes.local> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2022-05-08 18:10, Stephen Hemminger wrote: > On Sun, 8 May 2022 14:12:42 +0200 > Mattias Rönnblom wrote: >> A sequence lock (seqlock) is a synchronization primitive which allows >> for data-race free, low-overhead, high-frequency reads, suitable for >> data structures shared across many cores and which are updated >> relatively infrequently. >> >> A seqlock permits multiple parallel readers. The variant of seqlock >> implemented in this patch supports multiple writers as well. A >> spinlock is used for writer-writer serialization. >> >> To avoid resource reclamation and other issues, the data protected by >> a seqlock is best off being self-contained (i.e., no pointers [except >> to constant data]). >> >> One way to think about seqlocks is that they provide means to perform >> atomic operations on data objects larger than what the native atomic >> machine instructions allow for. >> >> DPDK seqlocks are not preemption safe on the writer side. A thread >> preemption affects performance, not correctness. >> >> A seqlock contains a sequence number, which can be thought of as the >> generation of the data it protects. >> >> A reader will >> 1. Load the sequence number (sn). >> 2. Load, in arbitrary order, the seqlock-protected data. >> 3. Load the sn again. >> 4. Check if the first and second sn are equal, and even numbered. >> If they are not, discard the loaded data, and restart from 1. >> >> The first three steps need to be ordered using suitable memory fences. >> >> A writer will >> 1. Take the spinlock, to serialize writer access. >> 2. Load the sn. >> 3. Store the original sn + 1 as the new sn. >> 4. Perform load and stores to the seqlock-protected data. >> 5. Store the original sn + 2 as the new sn. >> 6. Release the spinlock. >> >> Proper memory fencing is required to make sure the first sn store, the >> data stores, and the second sn store appear to the reader in the >> mentioned order. >> >> The sn loads and stores must be atomic, but the data loads and stores >> need not be. >> >> The original seqlock design and implementation was done by Stephen >> Hemminger. This is an independent implementation, using C11 atomics. >> >> For more information on seqlocks, see >> https://en.wikipedia.org/wiki/Seqlock > > I think would be good to have the sequence count (read side only) like > the kernel and sequence lock (sequence count + spinlock) as separate things. > > That way the application could use sequence count + ticket lock if it > needed to scale to more writers. > Sounds reasonable. Would that be something like: typedef struct { uint32_t sn; } rte_seqlock_t; rte_seqlock_read_begin() rte_seqlock_read_retry() rte_seqlock_write_begin() rte_seqlock_write_end() typedef struct { rte_seqlock_t seqlock; rte_spinlock_t wlock; } rte__t; rte__read_begin() rte__read_retry() rte__write_lock() rte__write_unlock() or are you suggesting removing the spinlock altogether, and leave writer-side synchronization to the application (at least in this DPDK release)? >> diff --git a/lib/eal/common/rte_seqlock.c b/lib/eal/common/rte_seqlock.c >> new file mode 100644 >> index 0000000000..d4fe648799 >> --- /dev/null >> +++ b/lib/eal/common/rte_seqlock.c >> @@ -0,0 +1,12 @@ >> +/* SPDX-License-Identifier: BSD-3-Clause >> + * Copyright(c) 2022 Ericsson AB >> + */ >> + >> +#include >> + >> +void >> +rte_seqlock_init(rte_seqlock_t *seqlock) >> +{ >> + seqlock->sn = 0; >> + rte_spinlock_init(&seqlock->lock); >> +} > > So small, worth just making inline? I don't think so, but it is small. Especially if rte_spinlock_init() now goes away. :)