From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 633E6A0503; Thu, 31 Mar 2022 11:39:27 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0C0F9410FA; Thu, 31 Mar 2022 11:39:27 +0200 (CEST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150087.outbound.protection.outlook.com [40.107.15.87]) by mails.dpdk.org (Postfix) with ESMTP id 8889640DF6 for ; Thu, 31 Mar 2022 11:39:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=14ACnzuB9x9wYCT/C8iVQjXo59sknc3PAeseAr6Q4oI=; b=MOwC5M8NsWMh0GLYbVGQD+eHwKGqAbkiX5F3MEAgAHdpyHYRHaPsGyw1U44IJFLDgNFzczA9bcRKqiUCMMmytjCJTUTEbT22+GjC9zFHwXzoaAhi85xoqgFPKW7upUDVwH4YcgOfKqWn4DktT077tCJ6SrOPL7XL5FV6FJ45NBE= Received: from AM5P194CA0005.EURP194.PROD.OUTLOOK.COM (2603:10a6:203:8f::15) by HE1PR0802MB2234.eurprd08.prod.outlook.com (2603:10a6:3:c6::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.22; Thu, 31 Mar 2022 09:39:20 +0000 Received: from AM5EUR03FT045.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:8f:cafe::9b) by AM5P194CA0005.outlook.office365.com (2603:10a6:203:8f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.22 via Frontend Transport; Thu, 31 Mar 2022 09:39:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT045.mail.protection.outlook.com (10.152.17.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.19 via Frontend Transport; Thu, 31 Mar 2022 09:39:18 +0000 Received: ("Tessian outbound 2d401af10eb3:v118"); Thu, 31 Mar 2022 09:39:18 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: c97f95af8288c189 X-CR-MTA-TID: 64aa7808 Received: from 87925864bbf6.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 3EB07C9C-E72B-48B6-B14A-97EBE4A8B0D1.1; Thu, 31 Mar 2022 09:39:07 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 87925864bbf6.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 31 Mar 2022 09:39:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Zv0TKrzYWDLYPWf+MHsJKKjSn24r3SAHV9fWssKcY7+e2MOYPIv5YQJjkwoqC6IAWKqR2ulnq2FHghP/1voLY6VnMaPJgd9pvVh+FsAqmQWB/yImYjOpsDSF0ITRsMhqHw+zr/HQnNQfW09XIkUaAzid4QrLQ6RWtV+PE6/cShkBjXjaN6vuZtSlCFohgMw+BP4zGQAaAiRcbI5mTQkQhy0HLfvw2cNcR8KxLpbvDr+0Hg9sGyV2lxR1QTX4a32Ck6uI+fejY8rUFlmocWD+tLvZb0J7DOnBn6LiUCl7u15PoQiLp5WW20kFDCFYbT0v+IEK/tBFf4arSFgjOGW/9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=14ACnzuB9x9wYCT/C8iVQjXo59sknc3PAeseAr6Q4oI=; b=HBa9QLQDKCvfZiWU1lxzVZTJCHbt9RHHAUJnfT+Vbr3IKJgxXXsIf49ZBOia58bLE77wIFs2wRACr0p2OjXM2ZPzCXt5e7HVd3+zZTbJt/NN8RHNoS8jB06pXi9j1xMXELA4YptIKszIcV8i9Q6WkFbzFv8mHp08GOfHdxv593wHeR62tdTtXRRjv5SkVoEXxtytzCgE0yZaXI8RZYyhDcsiaGYJIUZVXFjB6Am4+JM9wcWlx4sgO/rYMCFtqST20dFfzmB9vACz7YWkkQ/jzfocrp/5vXYjuxpyIoRn2e+sVOksf5t9+zvafLpbDCVd3PlQQU48Bb8rS505gHAYAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=14ACnzuB9x9wYCT/C8iVQjXo59sknc3PAeseAr6Q4oI=; b=MOwC5M8NsWMh0GLYbVGQD+eHwKGqAbkiX5F3MEAgAHdpyHYRHaPsGyw1U44IJFLDgNFzczA9bcRKqiUCMMmytjCJTUTEbT22+GjC9zFHwXzoaAhi85xoqgFPKW7upUDVwH4YcgOfKqWn4DktT077tCJ6SrOPL7XL5FV6FJ45NBE= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from AM8PR08MB6498.eurprd08.prod.outlook.com (2603:10a6:20b:364::11) by HE1PR0801MB2090.eurprd08.prod.outlook.com (2603:10a6:3:4c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5123.21; Thu, 31 Mar 2022 09:38:57 +0000 Received: from AM8PR08MB6498.eurprd08.prod.outlook.com ([fe80::ec4f:1ad:e332:9f39]) by AM8PR08MB6498.eurprd08.prod.outlook.com ([fe80::ec4f:1ad:e332:9f39%5]) with mapi id 15.20.5123.019; Thu, 31 Mar 2022 09:38:57 +0000 Message-ID: Date: Thu, 31 Mar 2022 11:38:54 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH v2] eal: add seqlock Content-Language: en-US To: =?UTF-8?Q?Morten_Br=c3=b8rup?= , =?UTF-8?Q?Mattias_R=c3=b6nnblom?= , dev@dpdk.org Cc: Thomas Monjalon , David Marchand , Onar Olsen , Honnappa.Nagarahalli@arm.com, nd@arm.com, konstantin.ananyev@intel.com, stephen@networkplumber.org References: <98CBD80474FA8B44BF855DF32C47DC35D86F84@smartserver.smartshare.dk> <20220330142602.108061-1-mattias.ronnblom@ericsson.com> <3888e595-de18-3cf0-707b-309b153c2b02@ericsson.com> <37d60d0f-9911-7692-cdb6-62cd5da540ac@arm.com> <98CBD80474FA8B44BF855DF32C47DC35D86F8D@smartserver.smartshare.dk> From: Ola Liljedahl In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86F8D@smartserver.smartshare.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO2P265CA0403.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:f::31) To AM8PR08MB6498.eurprd08.prod.outlook.com (2603:10a6:20b:364::11) MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: af2631bc-461a-4a81-6aff-08da12fa5430 X-MS-TrafficTypeDiagnostic: HE1PR0801MB2090:EE_|AM5EUR03FT045:EE_|HE1PR0802MB2234:EE_ X-LD-Processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: qHrl5R7ESo4yP4FUrdA2ufoPNmLuNm+CCRx8YyrA6HDyr0DxmJw0s+lb/54kk4+ajmoHMbVH1m21/HHUhJTOs5r4sS6FwdBltEtyLYb4JLup07wnGPCF/oBHRFo3lN690MxF2ipkOwxCRkvsjtgdeZoHS3nhJKQHKne+Au5n4KAdo1TCDV7HNll/0JuyB2wd2oUk2+rQh1IDrKya0w4YkwhWJ2Tkokd6IffIFtyK6jy1CodRVllMwHdkLjcsMBgnPD7Cto16+/w6i1hegjs3/E33H5PWGGxIYemas51lfsDV5Wk92TGK1JufFWZZT+E85VfuXk9tguGjq5D6k00BL2Y3X1xXr5MIirQ7b3T+FUCF3sQAfyF9sMz13WsPnwC5QDuzqWeWvBe5jTP+Lm9HgriH1CGhIgAW5cHBlRTTUE1CSbexy8uOYIEB7cx2WwWiX8IH73kefhw9VsElvA+teeQokSkkEPVwy+ULaaKF40fIyjg/z0Qgi5Xar2mKuHmRnhAgmOZM5b9QD7wqL8K3Cm+ym9Xccxy8yvRUf9X9YuY0IeL3zUUbhTgvKavaA0CLFmwsw6yJJueTWeJypBriAbMErJeBNxcj2pBgprMR9aSGWPhYthuugTrBhbRFKkyBEh2EGqVIEGdXnF1Tk/aBiQ5DCeoAf7PAM4s8l60bQL/VIlfOXHAsZK3rsmp6sfD1dLw7+BOT9W7I4jjqLp5TW1SNQlPZp8smkIMprShz9R4= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM8PR08MB6498.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(38100700002)(54906003)(86362001)(31696002)(316002)(110136005)(5660300002)(8676002)(4326008)(8936002)(44832011)(66476007)(66946007)(66556008)(6486002)(26005)(83380400001)(186003)(66574015)(2616005)(508600001)(53546011)(6666004)(6512007)(6506007)(36756003)(2906002)(31686004)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB2090 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 5204c7c6-4938-4708-335d-08da12fa4734 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BJQE4CUOIg7mIUMubnBrpbki2mKl2STTJHyivs7jOzw7A1B1voGLEv5U6y0UxAiXyTYXC16P4hNTmu/bPtG25/ZAtL22+9ekYYJXkDrRq1m3Xe0zFN36eGVR5RijKrGRlQH+3DVWO7Nv0vZt/x9qREeBYzyYYtNSCDUEa9gnEkPNx8Ro8Ohhp33P620g9wccTF8Ugz8KxMjTLR56fswBA8Krl9qEX9hcyW3zCvltzYc1UpxGmV0ccO4471VW3Hs+De5bFsPV8gi/wg2YIWOhKqIJUVU7UCMRqABjNdQVQdDTBZ+Ar/hL6wufl6/hGzlAN9cYgERTfRXE+Kouw5bdJHyk2PW2MqktjqsStR5M/6I/dYL+Y1kiZ1svrA+ZdMC/CjmOOyIAFMM2dQvQTKiOhE9Wv/rvbVPMBVLqiiwPK7Unh26IdO4F/ox9/iCsQDf9IwpztSu+s9lEaWIBG+M7ckxnBs5sTkkMkjxZInqsACwzPu38PL/ywps6I/fEADb+jJNLbY1rp74zHtHF7ceG/yp9HgbwCUJOhWl+cETa1jpMW/O+X0HOZoSPy3GjUfAIBS1IhPR6n7MG6/+ZVJ4zxopPUTGKl1TR4hZ7S+9RPiw8M1kvOhcoQRQdHx3n++ilNpKdITtS8pRq1/xeM4yoxhm6DxvoTIfgSMf+iWmpSHwbR7Y1vyC8kCXoVJFHqgaAhbRvNP2MMtYEG8ylH6JjJQ== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(46966006)(36840700001)(40470700004)(110136005)(36860700001)(66574015)(31686004)(53546011)(47076005)(336012)(36756003)(6512007)(8676002)(107886003)(2906002)(4326008)(70586007)(70206006)(6506007)(2616005)(316002)(40460700003)(508600001)(86362001)(81166007)(6486002)(186003)(82310400004)(5660300002)(8936002)(44832011)(54906003)(356005)(26005)(31696002)(83380400001)(6666004)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Mar 2022 09:39:18.9461 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: af2631bc-461a-4a81-6aff-08da12fa5430 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0802MB2234 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 3/31/22 11:25, Morten Brørup wrote: >> From: Ola Liljedahl [mailto:ola.liljedahl@arm.com] >> Sent: Thursday, 31 March 2022 11.05 >> >> On 3/31/22 09:46, Mattias Rönnblom wrote: >>> On 2022-03-30 16:26, Mattias Rönnblom wrote: >>>> A sequence lock (seqlock) is synchronization primitive which allows >>>> for data-race free, low-overhead, high-frequency reads, especially >> for >>>> data structures shared across many cores and which are updated with >>>> relatively infrequently. >>>> >>>> >>> >>> >>> >>> Some questions I have: >>> >>> Is a variant of the seqlock without the spinlock required? The reason >> I >>> left such out was that I thought that in most cases where only a >> single >>> writer is used (or serialization is external to the seqlock), the >>> spinlock overhead is negligible, since updates are relatively >> infrequent. > > Mattias, when you suggested adding the seqlock, I considered this too, and came to the same conclusion as you. > >> You can combine the spinlock and the sequence number. Odd sequence >> number means the seqlock is busy. That would replace a non-atomic RMW >> of >> the sequence number with an atomic RMW CAS and avoid the spin lock >> atomic RMW operation. Not sure how much it helps. >> >>> >>> Should the rte_seqlock_read_retry() be called rte_seqlock_read_end(), >> or >>> some third alternative? I wanted to make clear it's not just a >> "release >>> the lock" function. You could use >>> the|||__attribute__((warn_unused_result)) annotation to make clear >> the >>> return value cannot be ignored, although I'm not sure DPDK ever use >> that >>> attribute. > > I strongly support adding __attribute__((warn_unused_result)) to the function. There's a first time for everything, and this attribute is very relevant here! > >> We have to decide how to use the seqlock API from the application >> perspective. >> Your current proposal: >> do { >> sn = rte_seqlock_read_begin(&seqlock) >> //read protected data >> } while (rte_seqlock_read_retry(&seqlock, sn)); >> >> or perhaps >> sn = rte_seqlock_read_lock(&seqlock); >> do { >> //read protected data >> } while (!rte_seqlock_read_tryunlock(&seqlock, &sn)); >> >> Tryunlock should signal to the user that the unlock operation might not >> succeed and something needs to be repeated. > > Perhaps rename rte_seqlock_read_retry() to rte_seqlock_read_tryend()? As Ola mentions, this also inverses the boolean result value. If you consider this, please check that the resulting assembly output remains efficient. > > I think lock()/unlock() should be avoided in the read operation names, because no lock is taken during read. I like the critical region begin()/end() names. I was following the naming convention of rte_rwlock. Isn't the seqlock just a more scalable implementation of a reader/writer lock? > > Regarding naming, you should also consider renaming rte_seqlock_write_begin/end() to rte_seqlock_write_lock/unlock(), following the naming convention of the other locks. This could prepare for future extensions, such as rte_seqlock_write_trylock(). Just a thought; I don't feel strongly about this. > > Ola, the rte_seqlock_read_lock(&seqlock) must remain inside the loop, because retries can be triggered by a write operation happening between the read_begin() and read_tryend(), and then the new sn must be used by the read operation. That's why my rte_seqlock_read_tryunlock() function takes the sequence number as a parameter passed by reference. Then the sequence number can be updated if necessary. I didn't want to force a new call to rte_seqlock_read_lock() because there should be a one-to-one match between rte_seqlock_read_lock() and a successful call to rte_seqlock_read_tryunlock(). - Ola >