[dpdk-dev] Reader-Writer lock starvation issues

[Stephen Hemminger <stephen@networkplumber.org>]

The current version of rte_rwlock doesn't do what it says in the documentation.
" The lock is used to protect data that allows multiple readers in parallel,
 but only one writer. All readers are blocked until the writer is finished
 writing."

The problem is that the current implementation does not stop a a new reader
from acquiring the lock while a writer is waiting.

Writer:
      repeat until x = __atomic_load(&counter) == 0;
      __atomic_compare_exchange(&counter, &x, -1);
                                         
Reader:
      x = __atomic_load(&counter);
      __atomic_compare_exchange(&counter, &x, x + 1);


Fixing it likely would require an ABI change to add additional state.

For more background on reader-writer locks see:
  https://www.cs.rochester.edu/research/synchronization/pseudocode/rw.html

The code in DPDK is actually effectively the same as the first example
 "Simple, non-scalable reader-preference lock"

It looks like doing the right thing will require increasing the size of
the rte_rwlock structure and cause an ABI breakage.

I am running with an alternative which uses ticket locks to do:
  "Simple, non-scalable writer-preference lock"

My recommendation would be:

 1. Fix documentation in rte_rwlock.h (and add release note) and put this in 20.02 and LTS.
 2. Add new rte_ticket_rwlock.h which provides the correct semantics.

Comments?