From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id AC59CA0524;
Fri, 8 Jan 2021 20:14:04 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 2EB75140E95;
Fri, 8 Jan 2021 20:14:04 +0100 (CET)
Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com
[209.85.215.169])
by mails.dpdk.org (Postfix) with ESMTP id 057D9140E92
for <dev@dpdk.org>; Fri, 8 Jan 2021 20:14:02 +0100 (CET)
Received: by mail-pg1-f169.google.com with SMTP id i7so8218036pgc.8
for <dev@dpdk.org>; Fri, 08 Jan 2021 11:14:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=networkplumber-org.20150623.gappssmtp.com; s=20150623;
h=date:from:to:subject:message-id:mime-version
:content-transfer-encoding;
bh=0oKWoI41iT3x7z4ZN+0o9Vdt6nD673EBGV6LuOWxrew=;
b=RwBpmmZcZ4etS1pRBpYzPvf2RdNcboifmcHMesd/cwGk7rtlu/fO1ys+YcbdQzKN4q
I0HGsKXplY2DGzh4V4rU9qnXBmxoOYY5b2FnpvIus3S1OzVzJtPrqTtp1pQqG/L7wmSs
55JCkmIxzHjkqd2OsPihE/A7GDn8A71exBpEJhiv+zxnV/EivR+IYYooCkFlmNrQi9ts
6E9kEksqyZ/T9bSC42gx/KlARjpYfw77hvwy2+O3fPcMA22WvFLvPeS0Za9gmD3KbmrM
6CVHf5IlzIVc9tz2J5wjowffkv9NS8skQNoNNTJOEWC6hSLPCB3QaoA6Vbe+8pQh/itz
5c0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:subject:message-id:mime-version
:content-transfer-encoding;
bh=0oKWoI41iT3x7z4ZN+0o9Vdt6nD673EBGV6LuOWxrew=;
b=ttLTQDg9OJukkNabtRJ+vVt+fv80w1sq9WBLiZJhFodEqlJ6u+akqN2HvU4MGWKWO2
JVznxp8I4v4RI2TcjsAP2OKENzGE8cyY05jC7oiPgCq2RbQhsSjF1xWpAZuUE4zeUl96
ZxYSKLYQ/duBGhkZUDcj07oK23BlnBKCzXRKuL+J6eWn+7tINP67NctsIWCWFo8CxEpJ
uaFZ1pugJcOBTAw9RSCJlvUUAYs5QZ9G9OFsrkwyFtInjyjIhqI5whzWS3oGAlMB47EB
ogNHr06MFHu1M4Jri+LPwBuZunEy8pZd/PlApOHlLwT1y/Pv5bdozoBIASOmY+F69WSG
oXFg==
X-Gm-Message-State: AOAM533TiiLv+I/Do26+TrpI1ha2CIOneJBp+3aGUvONXav4myozk1JM
ZkqVS35uMU3YLZuyhBFhsZIglOtHwAZQXA==
X-Google-Smtp-Source: ABdhPJxjuqYzNzZg6wnnHWs0UAQwDMbiC8nOpLQ78+oM/lqfvmUhJkxrkCUsH2/h6JX2Bzxbn3DYWQ==
X-Received: by 2002:a65:5b47:: with SMTP id y7mr8327162pgr.221.1610133241454;
Fri, 08 Jan 2021 11:14:01 -0800 (PST)
Received: from hermes.local (204-195-22-127.wavecable.com. [204.195.22.127])
by smtp.gmail.com with ESMTPSA id 193sm9776830pfz.36.2021.01.08.11.14.00
for <dev@dpdk.org>
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 08 Jan 2021 11:14:01 -0800 (PST)
Date: Fri, 8 Jan 2021 11:13:58 -0800
From: Stephen Hemminger <stephen@networkplumber.org>
To: dev@dpdk.org
Message-ID: <20210108111358.3b88d51c@hermes.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Subject: [dpdk-dev] Reader-Writer lock starvation issues
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
The current version of rte_rwlock doesn't do what it says in the documentation.
" The lock is used to protect data that allows multiple readers in parallel,
but only one writer. All readers are blocked until the writer is finished
writing."
The problem is that the current implementation does not stop a a new reader
from acquiring the lock while a writer is waiting.
Writer:
repeat until x = __atomic_load(&counter) == 0;
__atomic_compare_exchange(&counter, &x, -1);
Reader:
x = __atomic_load(&counter);
__atomic_compare_exchange(&counter, &x, x + 1);
Fixing it likely would require an ABI change to add additional state.
For more background on reader-writer locks see:
https://www.cs.rochester.edu/research/synchronization/pseudocode/rw.html
The code in DPDK is actually effectively the same as the first example
"Simple, non-scalable reader-preference lock"
It looks like doing the right thing will require increasing the size of
the rte_rwlock structure and cause an ABI breakage.
I am running with an alternative which uses ticket locks to do:
"Simple, non-scalable writer-preference lock"
My recommendation would be:
1. Fix documentation in rte_rwlock.h (and add release note) and put this in 20.02 and LTS.
2. Add new rte_ticket_rwlock.h which provides the correct semantics.
Comments?