[dpdk-announce] important design choices - statistics - ABI

[dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

Hi all,

Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
	- the announce@ ML is moderated to keep a low traffic,
	- every announce email is forwarded to dev@ ML.
In case you want to reply to this email, please use dev@dpdk.org address.

There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
We need to take a decision shortly and discuss (or agree) this proposal:
	http://dpdk.org/ml/archives/dev/2015-June/019461.html

During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
There were some interesting discussions but not a lot of participants:
	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461

During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding gap:
	http://dpdk.org/ml/archives/dev/2015-June/019386.html
To support the flow director for VF, there is no proposal yet:
	http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release 2.2:
	http://dpdk.org/ml/archives/dev/2015-June/019225.html
To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
	http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
	http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
	http://dpdk.org/ml/archives/dev/2015-June/019172.html
We must also better document how to remove a deprecated ABI:
	http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
what it means and how to proceed. Even the macros are not yet well documented:
	http://dpdk.org/ml/archives/dev/2015-June/019357.html

Thanks for your attention and your participation in these important choices.

Re: [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> Hi all,
> 
> Sometimes there are some important discussions about architecture or design
> which require opinions from several developers. Unfortunately, we cannot
> read every threads. Maybe that using the announce mailing list will help
> to bring more audience to these discussions.
> Please note that
> 	- the announce@ ML is moderated to keep a low traffic,
> 	- every announce email is forwarded to dev@ ML.
> In case you want to reply to this email, please use dev@dpdk.org address.
> 
> There were some debates about software statistics disabling.
> Should they be always on or possibly disabled when compiled?
> We need to take a decision shortly and discuss (or agree) this proposal:
> 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> During the development of the release 2.0, there was an agreement to keep
> ABI compatibility or to bring new ABI while keeping old one during one release.
> In case it's not possible to have this transition, the (exceptional) break
> should be acknowledged by several developers.
> 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> There were some interesting discussions but not a lot of participants:
> 	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
> 
> During the current development cycle for the release 2.1, the ABI question
> arises many times in different threads.
> To add the hash key size field, it is proposed to use a struct padding gap:
> 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> To support the flow director for VF, there is no proposal yet:
> 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> To add the speed capability, it is proposed to break ABI in the release 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> To add the interrupt mode, it is proposed to add a build-time option
> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
> 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> To add the packet type, there is a proposal to add a build-time option
> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> We must also better document how to remove a deprecated ABI:
> 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> The ABI compatibility is a new constraint and we need to better understand
> what it means and how to proceed. Even the macros are not yet well documented:
> 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> 
> Thanks for your attention and your participation in these important choices.
> 

Thomas-
	Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions

1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions.  Distributions, by and large, favor
building libraries as DSO's, favoring security and updatability in favor of all
out performance.

2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK exposes
many data structures and instances that cannot be changed without breaking ABI

3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI

4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was introduced
to prevent the loading of an incompatible library with an older application

5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.


It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3).  I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a
result patches are getting delayed, which was never my intent.  Not sure whats
to be done about that, but we should probably address it.  Is use of the
versionnig interface just too hard or convoluted?

Neil