Re: [dpdk-dev] [PATCH v5 1/3] rcu: add RCU library supporting QSBR mechanism

[Stephen Hemminger <stephen@networkplumber.org>]

On Mon, 15 Apr 2019 17:39:07 +0000
"Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:

> > -----Original Message-----
> > From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> > Sent: Monday, April 15, 2019 4:39 PM
> > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > Cc: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; paulmck@linux.ibm.com; Kovacevic, Marko
> > <marko.kovacevic@intel.com>; dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Dharmik Thakkar
> > <Dharmik.Thakkar@arm.com>; Malvika Gupta <Malvika.Gupta@arm.com>; nd <nd@arm.com>
> > Subject: Re: [PATCH v5 1/3] rcu: add RCU library supporting QSBR mechanism
> > 
> > On Mon, 15 Apr 2019 12:24:47 +0000
> > "Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:
> >   
> > > > -----Original Message-----
> > > > From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> > > > Sent: Saturday, April 13, 2019 12:06 AM
> > > > To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > > > Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; paulmck@linux.ibm.com; Kovacevic, Marko <marko.kovacevic@intel.com>;
> > > > dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Dharmik Thakkar <Dharmik.Thakkar@arm.com>; Malvika  
> > Gupta  
> > > > <Malvika.Gupta@arm.com>; nd <nd@arm.com>
> > > > Subject: Re: [PATCH v5 1/3] rcu: add RCU library supporting QSBR mechanism
> > > >
> > > > On Fri, 12 Apr 2019 22:24:45 +0000
> > > > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com> wrote:
> > > >  
> > > > > >
> > > > > > On Fri, 12 Apr 2019 15:20:37 -0500
> > > > > > Honnappa Nagarahalli <honnappa.nagarahalli@arm.com> wrote:
> > > > > >  
> > > > > > > Add RCU library supporting quiescent state based memory reclamation  
> > > > > > method.  
> > > > > > > This library helps identify the quiescent state of the reader threads
> > > > > > > so that the writers can free the memory associated with the lock less
> > > > > > > data structures.
> > > > > > >
> > > > > > > Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> > > > > > > Reviewed-by: Steve Capper <steve.capper@arm.com>
> > > > > > > Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > > > > > > Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
> > > > > > > Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>  
> > > > > >
> > > > > > After evaluating long term API/ABI issues, I think you need to get rid of almost
> > > > > > all use of inline and visible structures. Yes it might be marginally slower, but
> > > > > > you thank me the first time you have to fix something.
> > > > > >  
> > > > > Agree, I was planning on another version to address this (I am yet to take a look at your patch addressing the ABI).
> > > > > The structure visibility definitely needs to be addressed.
> > > > > For the inline functions, is the plan to convert all the inline functions in DPDK? If yes, I think we need to consider the performance  
> > > > difference. May be consider L3-fwd application, change all the inline functions in its path and run a test?
> > > >
> > > > Every function that is not in the direct datapath should not be inline.
> > > > Exceptions or things like rx/tx burst, ring enqueue/dequeue, and packet alloc/free  
> > >
> > > Plus synchronization routines: spin/rwlock/barrier, etc.
> > > I think rcu should be one of such exceptions - it is just another synchronization mechanism after all
> > > (just a bit more sophisticated).
> > > Konstantin  
> > 
> > If you look at the other userspace RCU, you wil see that the only inlines
> > are the rcu_read_lock,rcu_read_unlock and rcu_reference/rcu_assign_pointer.
> > 
> > The synchronization logic is all real functions.  
> 
> In fact, I think urcu provides both flavors:
> https://github.com/urcu/userspace-rcu/blob/master/include/urcu/static/urcu-qsbr.h
> I still don't understand why we have to treat it differently then let say spin-lock/ticket-lock or rwlock.
> If we gone all the way to create our own version of rcu, we probably want it to be as fast as possible
> (I know that main speedup should come from the fact that readers don't have to wait for writer to finish, but still...)
> 
> Konstantin
> 

Having locking functions inline is already a problem in current releases.
The implementation can not be improved without breaking ABI (or doing special
workarounds like lock v2)