Re: [dpdk-dev] ABI and inline functions

[Stephen Hemminger <stephen@networkplumber.org>]

On Wed, 24 Apr 2019 13:54:51 +0100
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:

> On 24-Apr-19 1:22 PM, Ray Kinsella wrote:
> > 
> > On 24/04/2019 12:08, Burakov, Anatoly wrote:  
> >> On 23-Apr-19 3:12 PM, Ray Kinsella wrote:  
> >>>
> >>>
> >>> On 18/04/2019 11:28, Bruce Richardson wrote:  
> >>>> On Thu, Apr 18, 2019 at 04:34:53AM +0000, Honnappa Nagarahalli wrote:  
> >>>>>>
> >>>>>> On Wed, Apr 17, 2019 at 05:12:43AM +0000, Honnappa Nagarahalli wrote:  
> >>>>>>> Hello,
> >>>>>>>      There was a conversation [1] in the context of RCU library. I
> >>>>>>> thought
> >>>>>>> it needs participation from broader audience. Summary for the context
> >>>>>>> (please look at [1] for full details)
> >>>>>>>  
> >>>>>>
> >>>>>> Thanks for kicking off this discussion
> >>>>>>  
> >>>>>>> 1) How do we provide ABI compatibility when the code base contains  
> >>>>>> inline functions? Unless we freeze development in these inline
> >>>>>> functions and
> >>>>>> corresponding structures, it might not be possible to claim ABI
> >>>>>> compatibility.
> >>>>>> Application has to be recompiled.
> >>>>>>
> >>>>>> I agree that in some cases the application "might" need to be
> >>>>>> recompiled,
> >>>>>> but on the other hand I also think that there are many cases where ABI
> >>>>>> function versioning can still be used to mitigate things. For
> >>>>>> example, if we
> >>>>>> think of a couple of various scenarios:
> >>>>>>
> >>>>>> 1. If everything is inline and variables are allocated by app, e.g.
> >>>>>> spinlock on stack, then there is no issue as everything is application
> >>>>>> contained.  
> >>>>> If there is a bug fix which requires the structure to change, the
> >>>>> application would need to recompile. I guess you are talking about a
> >>>>> scenario when nothing changed in the inline function/variables.
> >>>>>  
> >>>>
> >>>> If the application wants the bugfix, then yes. However, if the app is
> >>>> unaffected by the bug, then it should have the option of updating DPDK
> >>>> without a recompile.  
> >>>
> >>> I would also imagine that should be an extremely rare case, that a
> >>> bugfix would require a structure change ... perhaps for an alignment
> >>> issues?  
> >>
> >> Multiprocess threading issues is one case i've had to do that more than
> >> once.  
> > 
> > Another reason to dislike multi process I guess.
> >   
> >>
> >> <snip>
> >>
> >>  
> >>>
> >>> The reality is that most other system libraries provide strong
> >>> guarantees ... to date we have provided very little.
> >>>  
> >>
> >> To our credit, the libraries you're likely referring to aren't trying to
> >> reimplement the Linux kernel :) I don't think we do these API/ABI breaks
> >> just because we like doing them - DPDK is complex, and getting
> >> everything right the first time *and* allowing for future evolution is
> >> not a trivial undertaking.
> >>
> >>
> >> To me, part of the problem is that DPDK is an "everything and the
> >> kitchen sink" kind of library where there is a bunch of drivers, a whole
> >> quasi-OS layer of dealing with hardware in a cross-platform manner, a
> >> separate memory management system, a bunch of libraries such as hash/lpm
> >> tables, plus there's QOS, IP Pipeline, flow stuff, etc. - normally, "a
> >> library" would concentrate on doing one thing well. DPDK, on the other
> >> hand, tries to do *everything* well. The sheer breadth of DPDK's scope
> >> is, i think, contributing to the breakages. If you keep 99% of your
> >> libraries stable between version, but there's a small ABI tweak in an
> >> LPM library, the entire DPDK stability gets invalidated.  
> > 
> > Well I guess DPDK is no more complex than Java or .NET Framework in that
> > respect, as these also feature OS-layers, memory management systems,
> > application frameworks, primitives etc but do manage to give their
> > consumers strong guarantee's on API stability. Clearly ABI stability has
> > a no meaning when you always being JIT compiled.  
> 
> I was basing my response on your earlier comparisons of DPDK to 
> GStreamer :) Comparing it to .NET Framework is perhaps a better fit, but 
> then these frameworks generally go through much more rigorous 
> development/design cycle than DPDK does - DPDK's API design process is 
> pretty ad-hoc, while both Java and .NET have various kinds of procedures 
> by which things get into the standard library. If we're prepared to do 
> that - i'm all for it. What we can't have is stability *and* keep the 
> same approach to design/development that we have now.
> 

Lets step back a bit. I think some of the basic business rules which
go back to the "Innovators dilemma" are that it is NOT a good idea
to pay too much attention to existing customers. Instead, it is important
to pay attention to the need of the potential customers that could
use your product.

In this case, my argument is that it is important for the DPDK project
to pay more attention to the reasons that new users
do not want to use the project; and existing users do not want to
upgrade.  This problem has been repeatedly expressed as:
"DPDK is not stable". The project needs to address that concern
even if it means a 1% drop in performance. Surely that 1% can be
made up by improving other places.  Even using link-time-optimization
can give 2 to 5 % back.