Re: [dpdk-dev] [RFC PATCH 3/3] lib/lpm: integrate RCU QSBR

[Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>]

<snip>
Thank you Stephen for your comments, appreciate your inputs.

> > On Thu, 22 Aug 2019 14:34:57 +0800
> > Ruifeng Wang <ruifeng.wang@arm.com> wrote:
> >
> > > Currently, the tbl8 group is freed even though the readers might be
> > > using the tbl8 group entries. The freed tbl8 group can be
> > > reallocated quickly. This results in incorrect lookup results.
> > >
> > > RCU QSBR process is integrated for safe tbl8 group reclaim.
> > > Refer to RCU documentation to understand various aspects of
> > > integrating RCU library into other libraries.
> > >
> > > Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> >
> > Having RCU in LPM is a good idea but difficult to find out how to do it in
> DPDK.
> > Not everyone wants to use RCU, so making a required part of how LPM is
> > used will impact users.
> 
> LPM users will not be imposed to use RCU. New API is provided to enable the
> RCU functionality in LPM library. For users not using RCU, code path is intact,
> and there will be no performance drop.
> 
> >
> > Also, it looks like DPDK RCU lacks a good generic way to handle deferred
> free.
Both rcu_defer and call_rcu from 'userspace RCU library' are wrappers on top of the underlying basic mechanisms. Such wrappers can be added. However, I would prefer to integrate RCU into couple of libraries to clearly show the need for wrappers. Integrating RCU in the libraries also removes some burden from the application.

> > Having to introduce a ring to handle is adding more complexity when a
> > generic solution would be better (see userspace RCU library for example).
A ring is required in rcu_defer as well as call_rcu since the pointer needs to be stored while waiting for quiescent state updates. The ring is used in the proposed solution for the same purpose.
I briefly looked through rcu_defer. The solution proposed here seems to be similar to rcu_defer. However, there are several differences.
1) rcu_defer uses a single queue for each updater thread, the proposed solution uses per data structure ring. IMO, this provides a better control over the resources to reclaim. Note that currently, the ring usage itself is not optimized (intentionally) to keep the patches focused on the understanding the design.
2) rcu_defer also launches another thread which wakes up periodically and reclaims the resources in the ring (along with the updater thread calling synchronize_rcu, which blocks, when the queue is full). This requires additional synchronization between the updater thread and the reclaimer thread. The solution proposed here does not need another thread as the DPDK RCU library provides non-blocking reclaiming mechanism, reclaiming is done in the context of the update thread.

> > Other parts of DPDK would benefit if deferred free was done better.
Which other parts are you talking about? The design proposed in 1/3 is a common solution that should apply to other libraries as well.

> 
> This requires support from RCU library.
> Needs Honnappa's comment.