Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon <thomas.monjalon@6wind.com>]

2015-06-19 11:27, Neil Horman:
> On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote:
> > 2015-06-19 09:02, Neil Horman:
> > > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > > > 2015-06-19 06:26, Neil Horman:
> > > > > On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > > > > > For the 2.1 release, I think we should agree to make patches that change
> > > > > > the ABI controllable via a compile-time option. I like Olivier's proposal
> > > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > > > > changes instead of a separate option per patch set (see
> > > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > > > > should rework the affected patch sets to use that approach for 2.1.
> > > > > 
> > > > > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > > > > maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> > > > > work (that is to say you make it impossible to really tell what ABI version you
> > > > > are building).
> > > > 
> > > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> > > > So one ABI version number refers always to the same ABI.
> > > > 
> > > > > If you have two compile time options that modify the ABI, you
> > > > > have to burn through 4 possible LIBABIVER version values to accomodate all
> > > > > possible combinations, and then you need to remember that when you make them
> > > > > statically applicable.
> > > > 
> > > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > > > 
> > > > Your intent when introducing ABI policy was to allow smooth porting of
> > > > applications from a DPDK version to another. Right?
> > > > The adopted solution was to provide backward compatibility during 1 release.
> > > > But there are cases where it's not possible. So the policy was to notice
> > > > the future change and wait one release cycle to break the ABI (failing
> > > > compatibility goals).
> > > > The compile-time option may provide an alternative DPDK packaging when the
> > > > ABI backward compatibility cannot be provided (case of mbuf changes).
> > > > In such case, it's still possible to upgrade DPDK by providing 2 versions of
> > > > DPDK libs. So the existing apps continue to link with the previous ABI and
> > > > have the possibility of migrating to the new one.
> > > > Another advantage of this approach is that we don't have to wait 1 release
> > > > to integrate the changes.
> > > > The last advantage is to benefit early of these changes with static libraries.
> > > 
> > > Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> > > Implementing an ABI preview option like this implies the notion that, after a
> > > release, you have to remove all the ifdefs that you inserted to create the new
> > > ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
> > > large, and is predicated on the centralization of work effort (that is to say
> > > you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
> > > from the build' patch every release.
> > 
> > It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
> > have easy backward compatibility with the compat macros you introduced.
> > I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> > At the same time, the deprecated API, using the compat macros, will be removed.
> > 
> I think that is something you can't really predict, as its not an issue of how
> stringent we are with its use, but rather a function of how much change
> developers want in a given release.  That is to say, if you only reserve it for
> the most important/urgently needed changes, thats fine, but if you have a
> release in which 50 developers want to make urgent and important changes that
> breaks ABI, you still have quite a job on your hands to back out the config
> changes.
> 
> Not to mention the fact that backing those changes out is a manual process.
> 
> > > What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
> > > the sort that Thomas Herbert proposed a few weeks ago).
> > 
> > This tree was created after Thomas' request:
> > 	http://dpdk.org/browse/next/dpdk-next/
> > 
> Awesome, Though I'm not sure thats entirely the right place either.  IIRC that
> location was intended to be an early integration site that took unreviewed
> patches.  I think this really calls for a branch from the mainline tree that
> exclusively accepts reviewed ABI changing patches, that can then be merged after
> the next release
> 
> > > Patches that aren't ABI stable can be put on the next-branch/tree in thier
> > > final format.  You can delcare the branch unstable (thereby reserving your
> > > right to rebase it).  People can use that to preview the next ABI version
> > > (complete with the update LIBABIVER bump), and when you release dpdk-X,
> > > the new ABI for dpdk-X+1 is achieved by simply merging.
> > 
> > Having this tree living would be a nice improvement but it won't provide any
> > stable (and enough validated) releases to rely on.
> > 
> I'm not sure I follow you entirely here.  If the goal is to find a place to
> accept patches that are ABI altering ahead of the main release, why do you need
> to provide stable/validated releases?  Just base it off the HEAD of the git tree
> during the DPDK release X merge window, any testing done in the base branch
> should roughly apply, save for functional changes made by the ABI patches you
> add in on the branch.

OK, I didn't get you. So you are saying that the changes for the next release
may be prepared in a branch. Yes, it's possible.
But before stating on practical method to execute the policy, we need to agree
on the policy.

This is my proposal:
- The ABI policy must be better explained: what happens to LIBABIVER and .map
when adding a field, adding or removing a function.
- The file doc/guides/rel_notes/abi.rst must contain the ABI notices but the
policy must be moved to doc/guides/guidelines/compat.rst
- The case "backward compatibility broken" must be replaced by the usage of
CONFIG_RTE_NEXT_ABI while describing cases where it can apply.
- The .map files must be generated in order to make it simpler and allows
the use of CONFIG_RTE_NEXT_ABI.