From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AC59CA0524; Fri, 8 Jan 2021 20:14:04 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2EB75140E95; Fri, 8 Jan 2021 20:14:04 +0100 (CET) Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com [209.85.215.169]) by mails.dpdk.org (Postfix) with ESMTP id 057D9140E92 for ; Fri, 8 Jan 2021 20:14:02 +0100 (CET) Received: by mail-pg1-f169.google.com with SMTP id i7so8218036pgc.8 for ; Fri, 08 Jan 2021 11:14:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:subject:message-id:mime-version :content-transfer-encoding; bh=0oKWoI41iT3x7z4ZN+0o9Vdt6nD673EBGV6LuOWxrew=; b=RwBpmmZcZ4etS1pRBpYzPvf2RdNcboifmcHMesd/cwGk7rtlu/fO1ys+YcbdQzKN4q I0HGsKXplY2DGzh4V4rU9qnXBmxoOYY5b2FnpvIus3S1OzVzJtPrqTtp1pQqG/L7wmSs 55JCkmIxzHjkqd2OsPihE/A7GDn8A71exBpEJhiv+zxnV/EivR+IYYooCkFlmNrQi9ts 6E9kEksqyZ/T9bSC42gx/KlARjpYfw77hvwy2+O3fPcMA22WvFLvPeS0Za9gmD3KbmrM 6CVHf5IlzIVc9tz2J5wjowffkv9NS8skQNoNNTJOEWC6hSLPCB3QaoA6Vbe+8pQh/itz 5c0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-transfer-encoding; bh=0oKWoI41iT3x7z4ZN+0o9Vdt6nD673EBGV6LuOWxrew=; b=ttLTQDg9OJukkNabtRJ+vVt+fv80w1sq9WBLiZJhFodEqlJ6u+akqN2HvU4MGWKWO2 JVznxp8I4v4RI2TcjsAP2OKENzGE8cyY05jC7oiPgCq2RbQhsSjF1xWpAZuUE4zeUl96 ZxYSKLYQ/duBGhkZUDcj07oK23BlnBKCzXRKuL+J6eWn+7tINP67NctsIWCWFo8CxEpJ uaFZ1pugJcOBTAw9RSCJlvUUAYs5QZ9G9OFsrkwyFtInjyjIhqI5whzWS3oGAlMB47EB ogNHr06MFHu1M4Jri+LPwBuZunEy8pZd/PlApOHlLwT1y/Pv5bdozoBIASOmY+F69WSG oXFg== X-Gm-Message-State: AOAM533TiiLv+I/Do26+TrpI1ha2CIOneJBp+3aGUvONXav4myozk1JM ZkqVS35uMU3YLZuyhBFhsZIglOtHwAZQXA== X-Google-Smtp-Source: ABdhPJxjuqYzNzZg6wnnHWs0UAQwDMbiC8nOpLQ78+oM/lqfvmUhJkxrkCUsH2/h6JX2Bzxbn3DYWQ== X-Received: by 2002:a65:5b47:: with SMTP id y7mr8327162pgr.221.1610133241454; Fri, 08 Jan 2021 11:14:01 -0800 (PST) Received: from hermes.local (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id 193sm9776830pfz.36.2021.01.08.11.14.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jan 2021 11:14:01 -0800 (PST) Date: Fri, 8 Jan 2021 11:13:58 -0800 From: Stephen Hemminger To: dev@dpdk.org Message-ID: <20210108111358.3b88d51c@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] Reader-Writer lock starvation issues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The current version of rte_rwlock doesn't do what it says in the documentation. " The lock is used to protect data that allows multiple readers in parallel, but only one writer. All readers are blocked until the writer is finished writing." The problem is that the current implementation does not stop a a new reader from acquiring the lock while a writer is waiting. Writer: repeat until x = __atomic_load(&counter) == 0; __atomic_compare_exchange(&counter, &x, -1); Reader: x = __atomic_load(&counter); __atomic_compare_exchange(&counter, &x, x + 1); Fixing it likely would require an ABI change to add additional state. For more background on reader-writer locks see: https://www.cs.rochester.edu/research/synchronization/pseudocode/rw.html The code in DPDK is actually effectively the same as the first example "Simple, non-scalable reader-preference lock" It looks like doing the right thing will require increasing the size of the rte_rwlock structure and cause an ABI breakage. I am running with an alternative which uses ticket locks to do: "Simple, non-scalable writer-preference lock" My recommendation would be: 1. Fix documentation in rte_rwlock.h (and add release note) and put this in 20.02 and LTS. 2. Add new rte_ticket_rwlock.h which provides the correct semantics. Comments?