From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9B6B5A04B5; Mon, 11 Jan 2021 12:52:18 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1B19F140CB5; Mon, 11 Jan 2021 12:52:18 +0100 (CET) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id D5BD3140CAF for ; Mon, 11 Jan 2021 12:52:16 +0100 (CET) IronPort-SDR: Jnh+buFSDmvF054hjANbENJHHIQJCmHJQ2T1+gl1ypSW+rTCtTo+g9D61ZArvxfAEirfJlj1wt dBsh1aqbopDg== X-IronPort-AV: E=McAfee;i="6000,8403,9860"; a="177941186" X-IronPort-AV: E=Sophos;i="5.79,338,1602572400"; d="scan'208";a="177941186" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2021 03:52:10 -0800 IronPort-SDR: rcZfWWGtj7frHnEP62js5xsFdtWPAaD7Ka+ThLfIaN0iUygZ4OC2TbXD68qLvHRlqLoSqhI81T TdPDPA/7oVGQ== X-IronPort-AV: E=Sophos;i="5.79,338,1602572400"; d="scan'208";a="352574323" Received: from fyigit-mobl1.ger.corp.intel.com (HELO [10.213.254.111]) ([10.213.254.111]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jan 2021 03:52:09 -0800 To: Honnappa Nagarahalli , Stephen Hemminger , "dev@dpdk.org" Cc: nd References: <20210108111358.3b88d51c@hermes.local> From: Ferruh Yigit Message-ID: Date: Mon, 11 Jan 2021 11:52:04 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] Reader-Writer lock starvation issues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 1/8/2021 9:27 PM, Honnappa Nagarahalli wrote: > > >> >> The current version of rte_rwlock doesn't do what it says in the >> documentation. >> " The lock is used to protect data that allows multiple readers in parallel, but >> only one writer. All readers are blocked until the writer is finished writing." >> >> The problem is that the current implementation does not stop a a new reader >> from acquiring the lock while a writer is waiting. > Agree, essentially the arbitration is left to the hardware. > >> >> Writer: >> repeat until x = __atomic_load(&counter) == 0; >> __atomic_compare_exchange(&counter, &x, -1); >> >> Reader: >> x = __atomic_load(&counter); >> __atomic_compare_exchange(&counter, &x, x + 1); >> >> >> Fixing it likely would require an ABI change to add additional state. >> >> For more background on reader-writer locks see: >> >> https://www.cs.rochester.edu/research/synchronization/pseudocode/rw.htm >> l >> >> The code in DPDK is actually effectively the same as the first example >> "Simple, non-scalable reader-preference lock" > I do not think the DPDK implementation has reader-preference. There is no code to control the arbitration between writers and readers. It is possible that if there are multiple writers the readers might be starved depending on how the hardware does the arbitration. > As far as I can see, in current implementation: When writer has the lock, both writers and readers needs to wait, and when writer releases reader or writer has chance to acquire the lock. When reader has the lock, other readers can acquire the lock and writers has to wait, and if readers keep coming it can cause writer starvation. Overall this doesn't look fair reader-writer lock ... >> >> It looks like doing the right thing will require increasing the size of the >> rte_rwlock structure and cause an ABI breakage. >> >> I am running with an alternative which uses ticket locks to do: >> "Simple, non-scalable writer-preference lock" > Does it provide good scalability? > >> >> My recommendation would be: >> >> 1. Fix documentation in rte_rwlock.h (and add release note) and put this in >> 20.02 and LTS. > Agree, the document is not clear on the arbitration. > >> 2. Add new rte_ticket_rwlock.h which provides the correct semantics. > Agree. > >> >> Comments?