From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (xvm-189-124.dc0.ghst.net [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id AC59CA0524;
	Fri,  8 Jan 2021 20:14:04 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 2EB75140E95;
	Fri,  8 Jan 2021 20:14:04 +0100 (CET)
Received: from mail-pg1-f169.google.com (mail-pg1-f169.google.com
 [209.85.215.169])
 by mails.dpdk.org (Postfix) with ESMTP id 057D9140E92
 for <dev@dpdk.org>; Fri,  8 Jan 2021 20:14:02 +0100 (CET)
Received: by mail-pg1-f169.google.com with SMTP id i7so8218036pgc.8
 for <dev@dpdk.org>; Fri, 08 Jan 2021 11:14:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=networkplumber-org.20150623.gappssmtp.com; s=20150623;
 h=date:from:to:subject:message-id:mime-version
 :content-transfer-encoding;
 bh=0oKWoI41iT3x7z4ZN+0o9Vdt6nD673EBGV6LuOWxrew=;
 b=RwBpmmZcZ4etS1pRBpYzPvf2RdNcboifmcHMesd/cwGk7rtlu/fO1ys+YcbdQzKN4q
 I0HGsKXplY2DGzh4V4rU9qnXBmxoOYY5b2FnpvIus3S1OzVzJtPrqTtp1pQqG/L7wmSs
 55JCkmIxzHjkqd2OsPihE/A7GDn8A71exBpEJhiv+zxnV/EivR+IYYooCkFlmNrQi9ts
 6E9kEksqyZ/T9bSC42gx/KlARjpYfw77hvwy2+O3fPcMA22WvFLvPeS0Za9gmD3KbmrM
 6CVHf5IlzIVc9tz2J5wjowffkv9NS8skQNoNNTJOEWC6hSLPCB3QaoA6Vbe+8pQh/itz
 5c0A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:date:from:to:subject:message-id:mime-version
 :content-transfer-encoding;
 bh=0oKWoI41iT3x7z4ZN+0o9Vdt6nD673EBGV6LuOWxrew=;
 b=ttLTQDg9OJukkNabtRJ+vVt+fv80w1sq9WBLiZJhFodEqlJ6u+akqN2HvU4MGWKWO2
 JVznxp8I4v4RI2TcjsAP2OKENzGE8cyY05jC7oiPgCq2RbQhsSjF1xWpAZuUE4zeUl96
 ZxYSKLYQ/duBGhkZUDcj07oK23BlnBKCzXRKuL+J6eWn+7tINP67NctsIWCWFo8CxEpJ
 uaFZ1pugJcOBTAw9RSCJlvUUAYs5QZ9G9OFsrkwyFtInjyjIhqI5whzWS3oGAlMB47EB
 ogNHr06MFHu1M4Jri+LPwBuZunEy8pZd/PlApOHlLwT1y/Pv5bdozoBIASOmY+F69WSG
 oXFg==
X-Gm-Message-State: AOAM533TiiLv+I/Do26+TrpI1ha2CIOneJBp+3aGUvONXav4myozk1JM
 ZkqVS35uMU3YLZuyhBFhsZIglOtHwAZQXA==
X-Google-Smtp-Source: ABdhPJxjuqYzNzZg6wnnHWs0UAQwDMbiC8nOpLQ78+oM/lqfvmUhJkxrkCUsH2/h6JX2Bzxbn3DYWQ==
X-Received: by 2002:a65:5b47:: with SMTP id y7mr8327162pgr.221.1610133241454; 
 Fri, 08 Jan 2021 11:14:01 -0800 (PST)
Received: from hermes.local (204-195-22-127.wavecable.com. [204.195.22.127])
 by smtp.gmail.com with ESMTPSA id 193sm9776830pfz.36.2021.01.08.11.14.00
 for <dev@dpdk.org>
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 08 Jan 2021 11:14:01 -0800 (PST)
Date: Fri, 8 Jan 2021 11:13:58 -0800
From: Stephen Hemminger <stephen@networkplumber.org>
To: dev@dpdk.org
Message-ID: <20210108111358.3b88d51c@hermes.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Subject: [dpdk-dev] Reader-Writer lock starvation issues
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

The current version of rte_rwlock doesn't do what it says in the documentation.
" The lock is used to protect data that allows multiple readers in parallel,
 but only one writer. All readers are blocked until the writer is finished
 writing."

The problem is that the current implementation does not stop a a new reader
from acquiring the lock while a writer is waiting.

Writer:
      repeat until x = __atomic_load(&counter) == 0;
      __atomic_compare_exchange(&counter, &x, -1);
                                         
Reader:
      x = __atomic_load(&counter);
      __atomic_compare_exchange(&counter, &x, x + 1);


Fixing it likely would require an ABI change to add additional state.

For more background on reader-writer locks see:
  https://www.cs.rochester.edu/research/synchronization/pseudocode/rw.html

The code in DPDK is actually effectively the same as the first example
 "Simple, non-scalable reader-preference lock"

It looks like doing the right thing will require increasing the size of
the rte_rwlock structure and cause an ABI breakage.

I am running with an alternative which uses ticket locks to do:
  "Simple, non-scalable writer-preference lock"

My recommendation would be:

 1. Fix documentation in rte_rwlock.h (and add release note) and put this in 20.02 and LTS.
 2. Add new rte_ticket_rwlock.h which provides the correct semantics.

Comments?