[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

Hi all,

Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
	- the announce@ ML is moderated to keep a low traffic,
	- every announce email is forwarded to dev@ ML.
In case you want to reply to this email, please use dev@dpdk.org address.

There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
We need to take a decision shortly and discuss (or agree) this proposal:
	http://dpdk.org/ml/archives/dev/2015-June/019461.html

During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
There were some interesting discussions but not a lot of participants:
	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461

During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding gap:
	http://dpdk.org/ml/archives/dev/2015-June/019386.html
To support the flow director for VF, there is no proposal yet:
	http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release 2.2:
	http://dpdk.org/ml/archives/dev/2015-June/019225.html
To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
	http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
	http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
	http://dpdk.org/ml/archives/dev/2015-June/019172.html
We must also better document how to remove a deprecated ABI:
	http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
what it means and how to proceed. Even the macros are not yet well documented:
	http://dpdk.org/ml/archives/dev/2015-June/019357.html

Thanks for your attention and your participation in these important choices.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Matthew Hall]

On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> There were some debates about software statistics disabling.
> Should they be always on or possibly disabled when compiled?
> We need to take a decision shortly and discuss (or agree) this proposal:
> 	http://dpdk.org/ml/archives/dev/2015-June/019461.html

This goes against the idea I have seen before that we should be moving toward 
a distro-friendly approach where one copy of DPDK can be used by multiple apps 
without having to rebuild it. It seems like it is also a bit ABI hostile 
according to the below goals / discussions.

Jemalloc is also very high-performance code and still manages to allow 
enabling and disabling statistics at runtime. Are we sure it's impossible for 
DPDK or just theorizing?

> During the development of the release 2.0, there was an agreement to keep
> ABI compatibility or to bring new ABI while keeping old one during one release.
> In case it's not possible to have this transition, the (exceptional) break
> should be acknowledged by several developers.

Personally to me it seems more important to preserve the ABI on patch 
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?

> During the current development cycle for the release 2.1, the ABI question
> arises many times in different threads.

Most but not all of these examples point to a different issue which sometimes 
happens in libraries... often seen as "old-style" versus "new-style" C library 
interface. For example, in old-style like libpcap there are a lot of structs, 
both opaque and non-opaque, which the caller must allocate in order to run libpcap. 

However new-style libraries such as libcurl usually just have init functions 
which initialize all the secret structs based on some defaults and some user 
parameters and hide the actual structs from the user. If you want to adjust 
some you call an adjuster function that modifies the actual secret struct 
contents, with some enum saying what field to adjust, and the new value you 
want it to have.

If you want to keep a stable ABI for a non-stable library like DPDK, there's a 
good chance you must begin hiding all these weird device specific structs all 
over the DPDK from the user needing to directly allocate and modify them. 
Otherwise the ABI breaks everytime you have to add adjustments, extensions, 
modifications to all these obscure special features.

Matthew.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Stephen Hemminger]

On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall <mhall@mhcomputing.net> wrote:

> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> >       http://dpdk.org/ml/archives/dev/2015-June/019461.html
>
> This goes against the idea I have seen before that we should be moving
> toward
> a distro-friendly approach where one copy of DPDK can be used by multiple
> apps
> without having to rebuild it. It seems like it is also a bit ABI hostile
> according to the below goals / discussions.
>
> Jemalloc is also very high-performance code and still manages to allow
> enabling and disabling statistics at runtime. Are we sure it's impossible
> for
> DPDK or just theorizing?
>
> > During the development of the release 2.0, there was an agreement to keep
> > ABI compatibility or to bring new ABI while keeping old one during one
> release.
> > In case it's not possible to have this transition, the (exceptional)
> break
> > should be acknowledged by several developers.
>
> Personally to me it seems more important to preserve the ABI on patch
> releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
>
> > During the current development cycle for the release 2.1, the ABI
> question
> > arises many times in different threads.
>
> Most but not all of these examples point to a different issue which
> sometimes
> happens in libraries... often seen as "old-style" versus "new-style" C
> library
> interface. For example, in old-style like libpcap there are a lot of
> structs,
> both opaque and non-opaque, which the caller must allocate in order to run
> libpcap.
>
> However new-style libraries such as libcurl usually just have init
> functions
> which initialize all the secret structs based on some defaults and some
> user
> parameters and hide the actual structs from the user. If you want to adjust
> some you call an adjuster function that modifies the actual secret struct
> contents, with some enum saying what field to adjust, and the new value you
> want it to have.
>
> If you want to keep a stable ABI for a non-stable library like DPDK,
> there's a
> good chance you must begin hiding all these weird device specific structs
> all
> over the DPDK from the user needing to directly allocate and modify them.
> Otherwise the ABI breaks everytime you have to add adjustments, extensions,
> modifications to all these obscure special features.
>
> Matthew.
>

The DPDK makes extensive use of inline functions which prevents data hiding
necessary for ABI stablility. This a fundamental tradeoff, and since the
whole
reason for DPDK is performance; the ABI is going to be a moving target.

It would make more sense to provide a higher level API which was abstracted,
slower, but stable for applications. But in doing so it would mean giving
up things
like inline lockless rings. Just don't go as far as the Open (not)
dataplane API;
which is just an excuse for closed source.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-16 22:28, Stephen Hemminger:
> On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall <mhall@mhcomputing.net> wrote:
> > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > > There were some debates about software statistics disabling.
> > > Should they be always on or possibly disabled when compiled?
> > > We need to take a decision shortly and discuss (or agree) this proposal:
> > >       http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > This goes against the idea I have seen before that we should be moving
> > toward a distro-friendly approach where one copy of DPDK can be used by
> > multiple apps without having to rebuild it. It seems like it is also a
> > bit ABI hostile according to the below goals / discussions.
> >
> > Jemalloc is also very high-performance code and still manages to allow
> > enabling and disabling statistics at runtime. Are we sure it's impossible
> > for DPDK or just theorizing?

Please Matthew, it is better to comment in the thread dedicated to statistics.

[...]
> > Personally to me it seems more important to preserve the ABI on patch
> > releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?

The goal of the ABI deprecation process was to provide a smooth integration
of the release 2.X+1.0. There are 4 months between releases 2.X.0 and 2.X+1.0.

[...]
> > However new-style libraries such as libcurl usually just have init
> > functions which initialize all the secret structs based on some defaults
> > and some user parameters and hide the actual structs from the user.
> > If you want to adjust some you call an adjuster function that modifies
> > the actual secret struct contents, with some enum saying what field to
> > adjust, and the new value you want it to have.
> >
> > If you want to keep a stable ABI for a non-stable library like DPDK,
> > there's a good chance you must begin hiding all these weird device
> > specific structs all over the DPDK from the user needing to directly
> > allocate and modify them.
> > Otherwise the ABI breaks everytime you have to add adjustments,
> > extensions, modifications to all these obscure special features.
> 
> The DPDK makes extensive use of inline functions which prevents data hiding
> necessary for ABI stablility. This a fundamental tradeoff, and since the
> whole reason for DPDK is performance; the ABI is going to be a moving target.
> 
> It would make more sense to provide a higher level API which was abstracted,
> slower, but stable for applications. But in doing so it would mean giving
> up things like inline lockless rings. Just don't go as far as the Open (not)
> dataplane API;
> which is just an excuse for closed source.

I don't understand what you mean.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Marc Sune]

On 17/06/15 07:28, Stephen Hemminger wrote:
> On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall <mhall@mhcomputing.net> wrote:
>
>> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
>>> There were some debates about software statistics disabling.
>>> Should they be always on or possibly disabled when compiled?
>>> We need to take a decision shortly and discuss (or agree) this proposal:
>>>        http://dpdk.org/ml/archives/dev/2015-June/019461.html
>> This goes against the idea I have seen before that we should be moving
>> toward
>> a distro-friendly approach where one copy of DPDK can be used by multiple
>> apps
>> without having to rebuild it. It seems like it is also a bit ABI hostile
>> according to the below goals / discussions.
>>
>> Jemalloc is also very high-performance code and still manages to allow
>> enabling and disabling statistics at runtime. Are we sure it's impossible
>> for
>> DPDK or just theorizing?
>>
>>> During the development of the release 2.0, there was an agreement to keep
>>> ABI compatibility or to bring new ABI while keeping old one during one
>> release.
>>> In case it's not possible to have this transition, the (exceptional)
>> break
>>> should be acknowledged by several developers.
>> Personally to me it seems more important to preserve the ABI on patch
>> releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
>>
>>> During the current development cycle for the release 2.1, the ABI
>> question
>>> arises many times in different threads.
>> Most but not all of these examples point to a different issue which
>> sometimes
>> happens in libraries... often seen as "old-style" versus "new-style" C
>> library
>> interface. For example, in old-style like libpcap there are a lot of
>> structs,
>> both opaque and non-opaque, which the caller must allocate in order to run
>> libpcap.
>>
>> However new-style libraries such as libcurl usually just have init
>> functions
>> which initialize all the secret structs based on some defaults and some
>> user
>> parameters and hide the actual structs from the user. If you want to adjust
>> some you call an adjuster function that modifies the actual secret struct
>> contents, with some enum saying what field to adjust, and the new value you
>> want it to have.
>>
>> If you want to keep a stable ABI for a non-stable library like DPDK,
>> there's a
>> good chance you must begin hiding all these weird device specific structs
>> all
>> over the DPDK from the user needing to directly allocate and modify them.
>> Otherwise the ABI breaks everytime you have to add adjustments, extensions,
>> modifications to all these obscure special features.
>>
>> Matthew.
>>
> The DPDK makes extensive use of inline functions which prevents data hiding
> necessary for ABI stablility. This a fundamental tradeoff, and since the
> whole
> reason for DPDK is performance; the ABI is going to be a moving target.
>
> It would make more sense to provide a higher level API which was abstracted,
> slower, but stable for applications. But in doing so it would mean giving
> up things
> like inline lockless rings. Just don't go as far as the Open (not)
> dataplane API;
> which is just an excuse for closed source.
+1 to what Stephen says. I don't think though a higher level API is 
really worth.

I unfortunately could not participate in the ABI discussion (there are 
times in which you cannot be following each and every single thread in 
the mailing list, and then it got lost in my inbox, my bad).

To me, ABI compatibility is a must if we'd have long term support 
releases (1.8.0, 1.8.1...), which we don't (I proposed to have them a 
couple of times).

On the other side, if the argument for ABI compatibility is shared 
libraries and to make it easy to package for distros, I don't see 
anything better than using the MAJOR.MIN version to identify the ABI 
(1.8, 1.9). Having LTS support would also help distros be able to stay 
in an ABI compatible version of DPDK and get recent bug fixes, and to 
better identify ABi changes (MAJ.MIN).

Moreover, I doubt that many people use shared libraries in DPDK. DPDK is 
built for performance and has a lot of code inlining which makes anyway 
your binary to be recompiled, as Stephen says.

DPDK is still growing, and the strict ABI policy is forcing patches to 
be delayed, to add hacky work-arounds to not break the ABI policy ( Re: 
[dpdk-dev] [PATCH v6 01/18] mbuf: redefine packet_type in rte_mbuf as an 
example)  that are announced that have to be removed afterwards. I see 
the entire ABI policy as a blocker and inducing to have botches all over 
the code, than anything else.

Marc

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Morten Brørup]

Dear Thomas,

I don't have time to follow the DPDK Developers mailing list, but since you call for feedback, I would like to share my thoughts regarding these design choices.


Regarding the statistics discussion:

1. The suggested solution assumes that, when statistics is disabled, the cost of allocating and maintaining zero-value statistics is negligible. If statistics counters are only available through accessor functions, this is probably true.

However, if statistics counters are directly accessible, e.g. as elements in the fast path data structures of a library, maintaining zero-value statistics may a have memory and/or performance impact.

Since the compile time flag CONFIG_RTE_<LIBRARY_NAME>_STATS_COLLECT already tells the application if the statistics are present or not, the application should simply use this flag to determine if statistics are accessible or not.

2. The suggested solution with only one single flag per library prevents implementing statistics with varying granularity for different purposes. E.g. a library could have one set of statistics counters for ordinary SNMP purposes, and another set of statistics counters for debugging/optimization purposes.

Multiple flags per library should be possible. A hierarchy of flags per library is probably not required.


Regarding the PHY speed ABI:

1. The Ethernet PHY ABI for speed, duplex, etc. should be common throughout the entire DPDK. It might be confusing if some structures/functions use a bitmask to indicate PHY speed/duplex/personality/etc. and other structures/functions use a combination of an unsigned integer, duplex flag, personality enumeration etc. (By personality enumeration, I am referring to PHYs with multiple electrical interfaces. E.g. a dual personality PHY might have both an RJ45 copper interface and an SFP module interface, whereof only one can be active at any time.)

2. The auto-negotiation standard allows the PHY to announce (to its link partner) any subset of its capabilities to its link partner. E.g. a standard 10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both half and full duplex and 1 Gbit/s full duplex) can be configured to announce 10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link partner. (Of course, more useful combinations are normally announced, but the purpose of the example is to show that any combination is possible.)

The ABI for auto-negotiation should include options to select the list of capabilities to announce to the link partner. The Linux PHY ABI only allows forcing a selected speed and duplex (thereby disabling auto-negotiation) or enabling auto-negotiation (thereby announcing all possible speeds and duplex combinations the PHY is capable of). Don't make the same mistake in DPDK.

PS: While working for Vitesse Semiconductors (an Ethernet chip company) a long time ago, I actually wrote the API for their line of Ethernet PHYs. So I have hands on experience in this area.


Regarding the discussion about backwards/forwards compatibility in the ABI:

1. Sometimes, ABI breakage is required. That is the cost the users pay for getting the benefits from upgrading to the latest and greatest version of any library. The current solution of requiring acknowledgement from several qualified developers is fine - these developers will consider the cost/benefit on behalf of all the DPDK users and make a qualified decision.

2. It is my general experience that documentation is not always updated to reflect the fine details of the source code, and this also applies to release notes. For open source software, the primary point of documentation is usually the source code itself.

2a. It should be clearly visible directly in the DPDK source code (including makefiles etc.) which ABI (i.e. functions, macros, type definitions etc.) is the current, the deprecated, and the future.

2b. When a developer migrates a project using DPDK from a previous version of the DPDK, it should be easy for the developer to identify all DPDK ABI modifications and variants, e.g. by using a common indicator in the DPDK source code, such as LIBAPIVER, that developer can simply search for.

3. Adding special feature flags, e.g. CONFIG_RTE_EAL_RX_INTR, to indicate a breakage of the ABI, should only be done if it is the intention to keep both the current and the new variants of the feature in the DPDK in the future. Otherwise, such a flag should be combined with the standard ABI version indication, so it is clear that this feature belongs to certain versions (i.e. deprecated, current or future).


Med venlig hilsen / kind regards

Morten Brørup
CTO



SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark

Office      +45 70 20 00 93
Direct      +45 89 93 50 22
Mobile      +45 25 40 82 12

mb@smartsharesystems.com
www.smartsharesystems.com
-----Original Message-----
From: announce [mailto:announce-bounces@dpdk.org] On Behalf Of Thomas Monjalon
Sent: 17. juni 2015 01:30
To: announce@dpdk.org
Subject: [dpdk-announce] important design choices - statistics - ABI

Hi all,

Sometimes there are some important discussions about architecture or design which require opinions from several developers. Unfortunately, we cannot read every threads. Maybe that using the announce mailing list will help to bring more audience to these discussions.
Please note that
	- the announce@ ML is moderated to keep a low traffic,
	- every announce email is forwarded to dev@ ML.
In case you want to reply to this email, please use dev@dpdk.org address.

There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
We need to take a decision shortly and discuss (or agree) this proposal:
	http://dpdk.org/ml/archives/dev/2015-June/019461.html

During the development of the release 2.0, there was an agreement to keep ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break should be acknowledged by several developers.
	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
There were some interesting discussions but not a lot of participants:
	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461

During the current development cycle for the release 2.1, the ABI question arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding gap:
	http://dpdk.org/ml/archives/dev/2015-June/019386.html
To support the flow director for VF, there is no proposal yet:
	http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release 2.2:
	http://dpdk.org/ml/archives/dev/2015-June/019225.html
To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
	http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
	http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
	http://dpdk.org/ml/archives/dev/2015-June/019172.html
We must also better document how to remove a deprecated ABI:
	http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand what it means and how to proceed. Even the macros are not yet well documented:
	http://dpdk.org/ml/archives/dev/2015-June/019357.html

Thanks for your attention and your participation in these important choices.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> Hi all,
> 
> Sometimes there are some important discussions about architecture or design
> which require opinions from several developers. Unfortunately, we cannot
> read every threads. Maybe that using the announce mailing list will help
> to bring more audience to these discussions.
> Please note that
> 	- the announce@ ML is moderated to keep a low traffic,
> 	- every announce email is forwarded to dev@ ML.
> In case you want to reply to this email, please use dev@dpdk.org address.
> 
> There were some debates about software statistics disabling.
> Should they be always on or possibly disabled when compiled?
> We need to take a decision shortly and discuss (or agree) this proposal:
> 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> During the development of the release 2.0, there was an agreement to keep
> ABI compatibility or to bring new ABI while keeping old one during one release.
> In case it's not possible to have this transition, the (exceptional) break
> should be acknowledged by several developers.
> 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> There were some interesting discussions but not a lot of participants:
> 	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
> 
> During the current development cycle for the release 2.1, the ABI question
> arises many times in different threads.
> To add the hash key size field, it is proposed to use a struct padding gap:
> 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> To support the flow director for VF, there is no proposal yet:
> 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> To add the speed capability, it is proposed to break ABI in the release 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> To add the interrupt mode, it is proposed to add a build-time option
> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
> 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> To add the packet type, there is a proposal to add a build-time option
> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> We must also better document how to remove a deprecated ABI:
> 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> The ABI compatibility is a new constraint and we need to better understand
> what it means and how to proceed. Even the macros are not yet well documented:
> 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> 
> Thanks for your attention and your participation in these important choices.
> 

Thomas-
	Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions

1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions.  Distributions, by and large, favor
building libraries as DSO's, favoring security and updatability in favor of all
out performance.

2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK exposes
many data structures and instances that cannot be changed without breaking ABI

3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI

4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was introduced
to prevent the loading of an incompatible library with an older application

5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.


It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3).  I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a
result patches are getting delayed, which was never my intent.  Not sure whats
to be done about that, but we should probably address it.  Is use of the
versionnig interface just too hard or convoluted?

Neil

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Richardson, Bruce]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Wednesday, June 17, 2015 11:35 AM
> To: Thomas Monjalon
> Cc: announce@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > Hi all,
> >
> > Sometimes there are some important discussions about architecture or
> > design which require opinions from several developers. Unfortunately,
> > we cannot read every threads. Maybe that using the announce mailing
> > list will help to bring more audience to these discussions.
> > Please note that
> > 	- the announce@ ML is moderated to keep a low traffic,
> > 	- every announce email is forwarded to dev@ ML.
> > In case you want to reply to this email, please use dev@dpdk.org
> address.
> >
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > During the development of the release 2.0, there was an agreement to
> > keep ABI compatibility or to bring new ABI while keeping old one during
> one release.
> > In case it's not possible to have this transition, the (exceptional)
> > break should be acknowledged by several developers.
> > 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> > There were some interesting discussions but not a lot of participants:
> >
> > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84
> > 61
> >
> > During the current development cycle for the release 2.1, the ABI
> > question arises many times in different threads.
> > To add the hash key size field, it is proposed to use a struct padding
> gap:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> > To support the flow director for VF, there is no proposal yet:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> > To add the speed capability, it is proposed to break ABI in the release
> 2.2:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> > To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> > To add the interrupt mode, it is proposed to add a build-time option
> > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
> > 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> > To add the packet type, there is a proposal to add a build-time option
> > CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> > We must also better document how to remove a deprecated ABI:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> > The ABI compatibility is a new constraint and we need to better
> > understand what it means and how to proceed. Even the macros are not yet
> well documented:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> >
> > Thanks for your attention and your participation in these important
> choices.
> >
> 
> Thomas-
> 	Just to re-iterate what you said earlier, and what was discussed in
> the previous ABI discussions
> 
> 1) ABI stability was introduced to promote DPDK's ability to be included
> with various linux and BSD distributions.  Distributions, by and large,
> favor building libraries as DSO's, favoring security and updatability in
> favor of all out performance.
> 
> 2) The desire was to put DPDK developers in a mindset whereby ABI
> stability was something they needed to think about during development, as
> the DPDK exposes many data structures and instances that cannot be changed
> without breaking ABI
> 
> 3) The versioning mechanism was introduced to allow for backward
> compatibility during periods in which we needed to support both an old an
> new ABI
> 
> 4) As Stephan and others point out, its not expected that we will always
> be able to maintain ABI, and as such an easy library versioning mechanism
> was introduced to prevent the loading of an incompatible library with an
> older application
> 
> 5) The ABI policy was introduced to create a method by which new ABI
> facets could be scheduled while allowing distros to prepare their
> downstream users for the upcomming changes.
> 
> 
> It seems to me, looking back over these last few months, that we're
> falling down a bit on our use of (3).  I've seen several people take
> advantage of the ABI scheduled updates, but no one has tried the
> versioning interface, and as a result patches are getting delayed, which
> was never my intent.  Not sure whats to be done about that, but we should
> probably address it.  Is use of the versionnig interface just too hard or
> convoluted?
> 
> Neil


Hi Neil,
 
on my end, some suggestions:

1. the documentation on changing an API function provided in rte_compat.h
is really good, but I don't think this is present in our documentation in
the docs folder or on website is it (apologies if it is and I've missed
it)? This needs to go into programmers guide or some other doc (perhaps
the new doc that the coding style went into).

2. The documentation also needs an example of: this is how you add a new
function and update the map file, and this is how you a) mark a function
as deprecated and b) remove it completely. That way we could have one
guide covering API versioning, how to add, modify and remove functions.

3. This doc should also cover how to use the API checker tool, something I
haven't had the chance to look at yet, but should do in the near future!
:-)

Regards,

/Bruce

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Bruce Richardson]

On Tue, Jun 16, 2015 at 09:36:54PM -0700, Matthew Hall wrote:
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> This goes against the idea I have seen before that we should be moving toward 
> a distro-friendly approach where one copy of DPDK can be used by multiple apps 
> without having to rebuild it. It seems like it is also a bit ABI hostile 
> according to the below goals / discussions.
> 
> Jemalloc is also very high-performance code and still manages to allow 
> enabling and disabling statistics at runtime. Are we sure it's impossible for 
> DPDK or just theorizing?
> 

+1 to this. I think that any compile-time option to disable stats should only
be used when we have a proven performance issue with just disabling them at runtime.
I would assume that apps do not switch on or off stats multiple times per second,
so any code branches to track stats or not would be entirely predictable in the
code - since they always go one way. Therefore, when disabledi, we should be looking
at a very minimal overhead per stat. If there are lots of checks for the same
value in the one path, i.e. lots of stats in a hot path, hopefully the compiler
will be smart enough to make the check just once. If not, we can always do
that in the C code by duplicating the hotpath code for with or without stats cases -
again selectable at runtime.

Also, there is also the case where the stats tracking itself is such low overhead
that its not worth disabling. In that case, neither runtime nor compile-time disabling
should need to be provided. For example, any library that is keeping a track of
bursts of packets should not need to have that stat disable option - one increment
or addition per burst of (32) packets is not going to seriously affect any app. :-)

Regards,
/Bruce

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Panu Matilainen]

(initially accidentally sent to announce, resending to dev)

On 06/17/2015 01:35 PM, Neil Horman wrote:
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
>> Hi all,
>>
>> Sometimes there are some important discussions about architecture or design
>> which require opinions from several developers. Unfortunately, we cannot
>> read every threads. Maybe that using the announce mailing list will help
>> to bring more audience to these discussions.
>> Please note that
>> 	- the announce@ ML is moderated to keep a low traffic,
>> 	- every announce email is forwarded to dev@ ML.
>> In case you want to reply to this email, please use dev@dpdk.org address.
>>
>> There were some debates about software statistics disabling.
>> Should they be always on or possibly disabled when compiled?
>> We need to take a decision shortly and discuss (or agree) this proposal:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
>>
>> During the development of the release 2.0, there was an agreement to keep
>> ABI compatibility or to bring new ABI while keeping old one during one release.
>> In case it's not possible to have this transition, the (exceptional) break
>> should be acknowledged by several developers.
>> 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
>> There were some interesting discussions but not a lot of participants:
>> 	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
>>
>> During the current development cycle for the release 2.1, the ABI question
>> arises many times in different threads.
>> To add the hash key size field, it is proposed to use a struct padding gap:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
>> To support the flow director for VF, there is no proposal yet:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
>> To add the speed capability, it is proposed to break ABI in the release 2.2:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
>> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
>> To add the interrupt mode, it is proposed to add a build-time option
>> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
>> 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
>> To add the packet type, there is a proposal to add a build-time option
>> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
>> We must also better document how to remove a deprecated ABI:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
>> The ABI compatibility is a new constraint and we need to better understand
>> what it means and how to proceed. Even the macros are not yet well documented:
>> 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
>>
>> Thanks for your attention and your participation in these important choices.
>>
>
> Thomas-
> 	Just to re-iterate what you said earlier, and what was discussed in the
> previous ABI discussions
>
> 1) ABI stability was introduced to promote DPDK's ability to be included with
> various linux and BSD distributions.  Distributions, by and large, favor
> building libraries as DSO's, favoring security and updatability in favor of all
> out performance.
>
> 2) The desire was to put DPDK developers in a mindset whereby ABI stability was
> something they needed to think about during development, as the DPDK exposes
> many data structures and instances that cannot be changed without breaking ABI
>
> 3) The versioning mechanism was introduced to allow for backward compatibility
> during periods in which we needed to support both an old an new ABI
>
> 4) As Stephan and others point out, its not expected that we will always be able
> to maintain ABI, and as such an easy library versioning mechanism was introduced
> to prevent the loading of an incompatible library with an older application
>
> 5) The ABI policy was introduced to create a method by which new ABI facets
> could be scheduled while allowing distros to prepare their downstream users for
> the upcomming changes.
>
>
> It seems to me, looking back over these last few months, that we're falling down
> a bit on our use of (3).  I've seen several people take advantage of the ABI
> scheduled updates, but no one has tried the versioning interface, and as a
> result patches are getting delayed, which was never my intent.  Not sure whats
> to be done about that, but we should probably address it.  Is use of the
> versionnig interface just too hard or convoluted?

To me it seems that by far the biggest problem with ABI stability in 
DPDK is features requiring changes to public structs (often directly 
allocated and accessed by apps), which is something the symbol 
versioning doesn't directly help with, you'd need to version the structs 
too.

One only needs to glance at the glibc documentation on how to accomplish 
it [1] to see it gets rather involved. Glibc promises backwards 
compatibility for life, so the effort is justified. However in where 
DPDK we're talking about extending compatibility for a few months by 
minimum requirement, people are unlikely to bother.

[1] https://sourceware.org/glibc/wiki/Development/Versioning_A_Structure

     - Panu -

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Vincent JARDIN]

On 17/06/2015 14:14, Panu Matilainen wrote:
> (initially accidentally sent to announce, resending to dev)
>
> On 06/17/2015 01:35 PM, Neil Horman wrote:
>> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
>>> Hi all,
>>>
>>> Sometimes there are some important discussions about architecture or
>>> design
>>> which require opinions from several developers. Unfortunately, we cannot
>>> read every threads. Maybe that using the announce mailing list will help
>>> to bring more audience to these discussions.
>>> Please note that
>>>     - the announce@ ML is moderated to keep a low traffic,
>>>     - every announce email is forwarded to dev@ ML.
>>> In case you want to reply to this email, please use dev@dpdk.org
>>> address.
>>>
>>> There were some debates about software statistics disabling.
>>> Should they be always on or possibly disabled when compiled?
>>> We need to take a decision shortly and discuss (or agree) this proposal:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019461.html
>>>
>>> During the development of the release 2.0, there was an agreement to
>>> keep
>>> ABI compatibility or to bring new ABI while keeping old one during
>>> one release.
>>> In case it's not possible to have this transition, the (exceptional)
>>> break
>>> should be acknowledged by several developers.
>>>     http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
>>> There were some interesting discussions but not a lot of participants:
>>>     http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
>>>
>>>
>>> During the current development cycle for the release 2.1, the ABI
>>> question
>>> arises many times in different threads.
>>> To add the hash key size field, it is proposed to use a struct
>>> padding gap:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019386.html
>>> To support the flow director for VF, there is no proposal yet:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019343.html
>>> To add the speed capability, it is proposed to break ABI in the
>>> release 2.2:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019225.html
>>> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019443.html
>>> To add the interrupt mode, it is proposed to add a build-time option
>>> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
>>> binary:
>>>     http://dpdk.org/ml/archives/dev/2015-June/018947.html
>>> To add the packet type, there is a proposal to add a build-time option
>>> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019172.html
>>> We must also better document how to remove a deprecated ABI:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019465.html
>>> The ABI compatibility is a new constraint and we need to better
>>> understand
>>> what it means and how to proceed. Even the macros are not yet well
>>> documented:
>>>     http://dpdk.org/ml/archives/dev/2015-June/019357.html
>>>
>>> Thanks for your attention and your participation in these important
>>> choices.
>>>
>>
>> Thomas-
>>     Just to re-iterate what you said earlier, and what was discussed
>> in the
>> previous ABI discussions
>>
>> 1) ABI stability was introduced to promote DPDK's ability to be
>> included with
>> various linux and BSD distributions.  Distributions, by and large, favor
>> building libraries as DSO's, favoring security and updatability in
>> favor of all
>> out performance.
>>
>> 2) The desire was to put DPDK developers in a mindset whereby ABI
>> stability was
>> something they needed to think about during development, as the DPDK
>> exposes
>> many data structures and instances that cannot be changed without
>> breaking ABI
>>
>> 3) The versioning mechanism was introduced to allow for backward
>> compatibility
>> during periods in which we needed to support both an old an new ABI
>>
>> 4) As Stephan and others point out, its not expected that we will
>> always be able
>> to maintain ABI, and as such an easy library versioning mechanism was
>> introduced
>> to prevent the loading of an incompatible library with an older
>> application
>>
>> 5) The ABI policy was introduced to create a method by which new ABI
>> facets
>> could be scheduled while allowing distros to prepare their downstream
>> users for
>> the upcomming changes.
>>
>>
>> It seems to me, looking back over these last few months, that we're
>> falling down
>> a bit on our use of (3).  I've seen several people take advantage of
>> the ABI
>> scheduled updates, but no one has tried the versioning interface, and
>> as a
>> result patches are getting delayed, which was never my intent.  Not
>> sure whats
>> to be done about that, but we should probably address it.  Is use of the
>> versionnig interface just too hard or convoluted?
>
> To me it seems that by far the biggest problem with ABI stability in
> DPDK is features requiring changes to public structs (often directly
> allocated and accessed by apps), which is something the symbol
> versioning doesn't directly help with, you'd need to version the structs
> too.
>
> One only needs to glance at the glibc documentation on how to accomplish
> it [1] to see it gets rather involved. Glibc promises backwards
> compatibility for life, so the effort is justified. However in where
> DPDK we're talking about extending compatibility for a few months by
> minimum requirement, people are unlikely to bother.
>
> [1] https://sourceware.org/glibc/wiki/Development/Versioning_A_Structure

Does it means that it requires a specific engineering so we are back to 
the needs of using two layers:
   a- the direct calls to "non ABI stable layers" (current librte*) for 
those we can/are fine to use it,
   b- the calls thru such layers that guarantee the ABI stabilities (a 
librte_compat?) for those who needs it.

Could both be exposed by the distributions so they can be consumed?

Thank you,
   Vincent

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Zhang, Helin]

Hi Neil

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> Sent: Wednesday, June 17, 2015 6:35 PM
> To: Thomas Monjalon
> Cc: announce@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - statistics -
> ABI
> 
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > Hi all,
> >
> > Sometimes there are some important discussions about architecture or
> > design which require opinions from several developers. Unfortunately,
> > we cannot read every threads. Maybe that using the announce mailing
> > list will help to bring more audience to these discussions.
> > Please note that
> > 	- the announce@ ML is moderated to keep a low traffic,
> > 	- every announce email is forwarded to dev@ ML.
> > In case you want to reply to this email, please use dev@dpdk.org address.
> >
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > During the development of the release 2.0, there was an agreement to
> > keep ABI compatibility or to bring new ABI while keeping old one during one
> release.
> > In case it's not possible to have this transition, the (exceptional)
> > break should be acknowledged by several developers.
> > 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> > There were some interesting discussions but not a lot of participants:
> >
> > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84
> > 61
> >
> > During the current development cycle for the release 2.1, the ABI
> > question arises many times in different threads.
> > To add the hash key size field, it is proposed to use a struct padding gap:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> > To support the flow director for VF, there is no proposal yet:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> > To add the speed capability, it is proposed to break ABI in the release 2.2:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> > To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> > To add the interrupt mode, it is proposed to add a build-time option
> > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
> > 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> > To add the packet type, there is a proposal to add a build-time option
> > CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> > We must also better document how to remove a deprecated ABI:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> > The ABI compatibility is a new constraint and we need to better
> > understand what it means and how to proceed. Even the macros are not yet
> well documented:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> >
> > Thanks for your attention and your participation in these important choices.
> >
> 
> Thomas-
> 	Just to re-iterate what you said earlier, and what was discussed in the
> previous ABI discussions
> 
> 1) ABI stability was introduced to promote DPDK's ability to be included with
> various linux and BSD distributions.  Distributions, by and large, favor building
> libraries as DSO's, favoring security and updatability in favor of all out
> performance.
> 
> 2) The desire was to put DPDK developers in a mindset whereby ABI stability was
> something they needed to think about during development, as the DPDK
> exposes many data structures and instances that cannot be changed without
> breaking ABI
> 
> 3) The versioning mechanism was introduced to allow for backward compatibility
> during periods in which we needed to support both an old an new ABI
> 
> 4) As Stephan and others point out, its not expected that we will always be able
> to maintain ABI, and as such an easy library versioning mechanism was
> introduced to prevent the loading of an incompatible library with an older
> application
> 
> 5) The ABI policy was introduced to create a method by which new ABI facets
> could be scheduled while allowing distros to prepare their downstream users for
> the upcomming changes.
> 
> 
> It seems to me, looking back over these last few months, that we're falling down
> a bit on our use of (3).  I've seen several people take advantage of the ABI
> scheduled updates, but no one has tried the versioning interface, and as a result
> patches are getting delayed, which was never my intent.  Not sure whats to be
> done about that, but we should probably address it.  Is use of the versionnig
> interface just too hard or convoluted?
That means we should try to use versioning mechanism as more as possible?
Are there any rules/instructions to judge what type of changes we should try
versioning mechanism, and what else we shouldn't try?
Is there any good examples of using versioning mechanism for reference?
Even not in DPDK project.

Thanks,
Helin


> 
> Neil

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Morten Brørup
> Sent: Wednesday, June 17, 2015 10:54 AM
> To: Thomas Monjalon
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> Dear Thomas,
> 
> I don't have time to follow the DPDK Developers mailing list, but since you call
> for feedback, I would like to share my thoughts regarding these design
> choices.
> 
> 
> Regarding the statistics discussion:
> 
> 1. The suggested solution assumes that, when statistics is disabled, the cost
> of allocating and maintaining zero-value statistics is negligible. If statistics
> counters are only available through accessor functions, this is probably true.
> 
> However, if statistics counters are directly accessible, e.g. as elements in the
> fast path data structures of a library, maintaining zero-value statistics may a
> have memory and/or performance impact.
> 

Counters are only accessible through API functions.

> Since the compile time flag
> CONFIG_RTE_<LIBRARY_NAME>_STATS_COLLECT already tells the
> application if the statistics are present or not, the application should simply
> use this flag to determine if statistics are accessible or not.
> 
> 2. The suggested solution with only one single flag per library prevents
> implementing statistics with varying granularity for different purposes. E.g. a
> library could have one set of statistics counters for ordinary SNMP purposes,
> and another set of statistics counters for debugging/optimization purposes.
> 
> Multiple flags per library should be possible. A hierarchy of flags per library is
> probably not required.
> 

Morten, thank you for your input. It would be good if you could add your contribution to the of the guidelines documentation patch by replying to the thread that Thomas indicated: http://dpdk.org/ml/archives/dev/2015-June/019461.html.

Our initial stats patch submission had a much finer granularity of stats configuration: per object type instead of per library, but a lot of people on this mailing list are against this, so we are now looking for one configuration flag per library.

> 
> Regarding the PHY speed ABI:
> 
> 1. The Ethernet PHY ABI for speed, duplex, etc. should be common
> throughout the entire DPDK. It might be confusing if some
> structures/functions use a bitmask to indicate PHY
> speed/duplex/personality/etc. and other structures/functions use a
> combination of an unsigned integer, duplex flag, personality enumeration
> etc. (By personality enumeration, I am referring to PHYs with multiple
> electrical interfaces. E.g. a dual personality PHY might have both an RJ45
> copper interface and an SFP module interface, whereof only one can be
> active at any time.)
> 
> 2. The auto-negotiation standard allows the PHY to announce (to its link
> partner) any subset of its capabilities to its link partner. E.g. a standard
> 10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both
> half and full duplex and 1 Gbit/s full duplex) can be configured to announce
> 10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link partner.
> (Of course, more useful combinations are normally announced, but the
> purpose of the example is to show that any combination is possible.)
> 
> The ABI for auto-negotiation should include options to select the list of
> capabilities to announce to the link partner. The Linux PHY ABI only allows
> forcing a selected speed and duplex (thereby disabling auto-negotiation) or
> enabling auto-negotiation (thereby announcing all possible speeds and
> duplex combinations the PHY is capable of). Don't make the same mistake in
> DPDK.
> 
> PS: While working for Vitesse Semiconductors (an Ethernet chip company) a
> long time ago, I actually wrote the API for their line of Ethernet PHYs. So I
> have hands on experience in this area.
> 
> 
> Regarding the discussion about backwards/forwards compatibility in the ABI:
> 
> 1. Sometimes, ABI breakage is required. That is the cost the users pay for
> getting the benefits from upgrading to the latest and greatest version of any
> library. The current solution of requiring acknowledgement from several
> qualified developers is fine - these developers will consider the cost/benefit
> on behalf of all the DPDK users and make a qualified decision.
> 
> 2. It is my general experience that documentation is not always updated to
> reflect the fine details of the source code, and this also applies to release
> notes. For open source software, the primary point of documentation is
> usually the source code itself.
> 
> 2a. It should be clearly visible directly in the DPDK source code (including
> makefiles etc.) which ABI (i.e. functions, macros, type definitions etc.) is the
> current, the deprecated, and the future.
> 
> 2b. When a developer migrates a project using DPDK from a previous version
> of the DPDK, it should be easy for the developer to identify all DPDK ABI
> modifications and variants, e.g. by using a common indicator in the DPDK
> source code, such as LIBAPIVER, that developer can simply search for.
> 
> 3. Adding special feature flags, e.g. CONFIG_RTE_EAL_RX_INTR, to indicate a
> breakage of the ABI, should only be done if it is the intention to keep both
> the current and the new variants of the feature in the DPDK in the future.
> Otherwise, such a flag should be combined with the standard ABI version
> indication, so it is clear that this feature belongs to certain versions (i.e.
> deprecated, current or future).
> 
> 
> Med venlig hilsen / kind regards
> 
> Morten Brørup
> CTO
> 
> 
> 
> SmartShare Systems A/S
> Tonsbakken 16-18
> DK-2740 Skovlunde
> Denmark
> 
> Office      +45 70 20 00 93
> Direct      +45 89 93 50 22
> Mobile      +45 25 40 82 12
> 
> mb@smartsharesystems.com
> www.smartsharesystems.com
> -----Original Message-----
> From: announce [mailto:announce-bounces@dpdk.org] On Behalf Of
> Thomas Monjalon
> Sent: 17. juni 2015 01:30
> To: announce@dpdk.org
> Subject: [dpdk-announce] important design choices - statistics - ABI
> 
> Hi all,
> 
> Sometimes there are some important discussions about architecture or
> design which require opinions from several developers. Unfortunately, we
> cannot read every threads. Maybe that using the announce mailing list will
> help to bring more audience to these discussions.
> Please note that
> 	- the announce@ ML is moderated to keep a low traffic,
> 	- every announce email is forwarded to dev@ ML.
> In case you want to reply to this email, please use dev@dpdk.org address.
> 
> There were some debates about software statistics disabling.
> Should they be always on or possibly disabled when compiled?
> We need to take a decision shortly and discuss (or agree) this proposal:
> 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> During the development of the release 2.0, there was an agreement to keep
> ABI compatibility or to bring new ABI while keeping old one during one
> release.
> In case it's not possible to have this transition, the (exceptional) break should
> be acknowledged by several developers.
> 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> There were some interesting discussions but not a lot of participants:
> 	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367
> /focus=8461
> 
> During the current development cycle for the release 2.1, the ABI question
> arises many times in different threads.
> To add the hash key size field, it is proposed to use a struct padding gap:
> 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> To support the flow director for VF, there is no proposal yet:
> 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> To add the speed capability, it is proposed to break ABI in the release 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> To add the interrupt mode, it is proposed to add a build-time option
> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
> 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> To add the packet type, there is a proposal to add a build-time option
> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> We must also better document how to remove a deprecated ABI:
> 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> The ABI compatibility is a new constraint and we need to better understand
> what it means and how to proceed. Even the macros are not yet well
> documented:
> 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> 
> Thanks for your attention and your participation in these important choices.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Matthew Hall
> Sent: Wednesday, June 17, 2015 5:37 AM
> To: dev@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> This goes against the idea I have seen before that we should be moving
> toward
> a distro-friendly approach where one copy of DPDK can be used by multiple
> apps
> without having to rebuild it. It seems like it is also a bit ABI hostile
> according to the below goals / discussions.

Matthew, thanks for your input. As Thomas also mentions, it would be good to first read the proposal on the guideline (http://dpdk.org/ml/archives/dev/2015-June/019461.html).

In the guideline proposal, we are addressing the topic of preventing the ABI changes due to library statistics support, it would be good to get your opinion on that, I am not sure you saw it. Given DPDK paramount focus on performance, I think probably provides the best solution for ABI compatibility in this case.

> 
> Jemalloc is also very high-performance code and still manages to allow
> enabling and disabling statistics at runtime. Are we sure it's impossible for
> DPDK or just theorizing?

A performance improvement for one environment might still be an overkill for DPDK run-time environment.

Stats cannot be enabled/disabled at run-time without performance impact, as it typically requires testing a persistent flag for incrementing the stats counters, which requires a conditional branch instruction. Sometimes the branches are predicted correctly, sometimes not, and in this case the CPU pipeline needs to be flushed; typical branch misprediction cost is ~14 cycles, which is important given a packet budget of ~200 cycles.

When only a small number of branches are present in the (application + library) code, they are predicted correctly, so I am sure that for a simple test application like l3fw the cost of run-time enabled stats is not visible. The problem is that real applications are much more complex and they typically have a lot of conditional branches, out of which the library stats test is just one of them, so they cannot be all predicted correctly, hence the cost of stats run-time enablement becomes very much visible.

> 
> > During the development of the release 2.0, there was an agreement to
> keep
> > ABI compatibility or to bring new ABI while keeping old one during one
> release.
> > In case it's not possible to have this transition, the (exceptional) break
> > should be acknowledged by several developers.
> 
> Personally to me it seems more important to preserve the ABI on patch
> releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
> 
> > During the current development cycle for the release 2.1, the ABI question
> > arises many times in different threads.
> 
> Most but not all of these examples point to a different issue which
> sometimes
> happens in libraries... often seen as "old-style" versus "new-style" C library
> interface. For example, in old-style like libpcap there are a lot of structs,
> both opaque and non-opaque, which the caller must allocate in order to run
> libpcap.
> 
> However new-style libraries such as libcurl usually just have init functions
> which initialize all the secret structs based on some defaults and some user
> parameters and hide the actual structs from the user. If you want to adjust
> some you call an adjuster function that modifies the actual secret struct
> contents, with some enum saying what field to adjust, and the new value
> you
> want it to have.
> 
> If you want to keep a stable ABI for a non-stable library like DPDK, there's a
> good chance you must begin hiding all these weird device specific structs all
> over the DPDK from the user needing to directly allocate and modify them.
> Otherwise the ABI breaks everytime you have to add adjustments,
> extensions,
> modifications to all these obscure special features.
> 
> Matthew.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Dumitrescu, Cristian]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson
> Sent: Wednesday, June 17, 2015 12:17 PM
> To: Matthew Hall
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> On Tue, Jun 16, 2015 at 09:36:54PM -0700, Matthew Hall wrote:
> > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > > There were some debates about software statistics disabling.
> > > Should they be always on or possibly disabled when compiled?
> > > We need to take a decision shortly and discuss (or agree) this proposal:
> > > 	http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > This goes against the idea I have seen before that we should be moving
> toward
> > a distro-friendly approach where one copy of DPDK can be used by multiple
> apps
> > without having to rebuild it. It seems like it is also a bit ABI hostile
> > according to the below goals / discussions.
> >
> > Jemalloc is also very high-performance code and still manages to allow
> > enabling and disabling statistics at runtime. Are we sure it's impossible for
> > DPDK or just theorizing?
> >
> 
> +1 to this. I think that any compile-time option to disable stats should only
> be used when we have a proven performance issue with just disabling them
> at runtime.
> I would assume that apps do not switch on or off stats multiple times per
> second,
> so any code branches to track stats or not would be entirely predictable in
> the
> code - since they always go one way. Therefore, when disabledi, we should
> be looking
> at a very minimal overhead per stat. If there are lots of checks for the same
> value in the one path, i.e. lots of stats in a hot path, hopefully the compiler
> will be smart enough to make the check just once. If not, we can always do
> that in the C code by duplicating the hotpath code for with or without stats
> cases -
> again selectable at runtime.
> 

I see where you're coming from, but reality is you cannot guarantee that the few conditional branches in the library are going to be predicted correctly simply because the application and the other libraries used by the app have an unknown number of conditional branches themselves. For complex apps, the total number of conditional branches is large and the more there are, the lesser probability of such a branch being predicted correctly is. I agree that for test apps like l3fwd with just a few branches the probability to have library branches being predicted correctly is high, but for me is an incorrect proof point. The cost of ~14 cycles per branch misprediction is important for DPDK packet budgets.

Since we don't control the application, I am feeling very uncomfortable with generic statements about how application is likely to execute and the impact of library conditional branches over the application should be. To me, they sound like we are willing to take chances, and to me this is not the right decision. In my opinion, the right decision for a significant framework like DPDK is to keep all options open for the apps: keep counters always enabled, keep counters always disabled, keep counters enabled first and disabled later. I suggest we should move the focus from WHY arguments like: "I don't think anybody will ever need this this" to the HOW part of making sure that technical solution we pick is correct and keeps all options open.

I think stats are not always equivalent to incrementing a counter. In the guideline proposal, I am providing several examples of more complex statistics logic that is more than just inc one counter. As an extreme case, think about the case where the metric to compute requires complex math like prediction of next packet arrival time based on recent history, etc. When defining a policy, we should consider a broad spectrum of metrics, not just n_pkts_in.


> Also, there is also the case where the stats tracking itself is such low
> overhead
> that its not worth disabling. In that case, neither runtime nor compile-time
> disabling
> should need to be provided. For example, any library that is keeping a track
> of
> bursts of packets should not need to have that stat disable option - one
> increment
> or addition per burst of (32) packets is not going to seriously affect any app. :-
> )
> 
> Regards,
> /Bruce

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[O'Driscoll, Tim]

> -----Original Message-----
> From: announce [mailto:announce-bounces@dpdk.org] On Behalf Of Thomas
> Monjalon
> Sent: Wednesday, June 17, 2015 12:30 AM
> To: announce@dpdk.org
> Subject: [dpdk-announce] important design choices - statistics - ABI
> 
> Hi all,
> 

> During the development of the release 2.0, there was an agreement to
> keep
> ABI compatibility or to bring new ABI while keeping old one during one
> release.
> In case it's not possible to have this transition, the (exceptional)
> break
> should be acknowledged by several developers.
> 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> There were some interesting discussions but not a lot of participants:
> 	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus
> =8461
> 
> During the current development cycle for the release 2.1, the ABI
> question
> arises many times in different threads.
> To add the hash key size field, it is proposed to use a struct padding
> gap:
> 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> To support the flow director for VF, there is no proposal yet:
> 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> To add the speed capability, it is proposed to break ABI in the release
> 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> To add the interrupt mode, it is proposed to add a build-time option
> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
> 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> To add the packet type, there is a proposal to add a build-time option
> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> We must also better document how to remove a deprecated ABI:
> 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> The ABI compatibility is a new constraint and we need to better
> understand
> what it means and how to proceed. Even the macros are not yet well
> documented:
> 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> 
> Thanks for your attention and your participation in these important
> choices.

There's been some good discussion on the ABI policy in various responses to this email. I think we now need to reach a conclusion on how we're going to proceed for the 2.1 release. Then, we can have further discussion on the use of versioning or other methods for avoiding the problem in future.

For the 2.1 release, I think we should agree to make patches that change the ABI controllable via a compile-time option. I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (see http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1.


Tim

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Vincent JARDIN]

On 18/06/2015 18:55, O'Driscoll, Tim wrote:
> I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (seehttp://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1.

Do we have any other options to meet the short deadlines of 2.1?

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > -----Original Message-----
> > From: announce [mailto:announce-bounces@dpdk.org] On Behalf Of Thomas
> > Monjalon
> > Sent: Wednesday, June 17, 2015 12:30 AM
> > To: announce@dpdk.org
> > Subject: [dpdk-announce] important design choices - statistics - ABI
> > 
> > Hi all,
> > 
> 
> > During the development of the release 2.0, there was an agreement to
> > keep
> > ABI compatibility or to bring new ABI while keeping old one during one
> > release.
> > In case it's not possible to have this transition, the (exceptional)
> > break
> > should be acknowledged by several developers.
> > 	http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> > There were some interesting discussions but not a lot of participants:
> > 	http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus
> > =8461
> > 
> > During the current development cycle for the release 2.1, the ABI
> > question
> > arises many times in different threads.
> > To add the hash key size field, it is proposed to use a struct padding
> > gap:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019386.html
> > To support the flow director for VF, there is no proposal yet:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019343.html
> > To add the speed capability, it is proposed to break ABI in the release
> > 2.2:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019225.html
> > To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019443.html
> > To add the interrupt mode, it is proposed to add a build-time option
> > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> > binary:
> > 	http://dpdk.org/ml/archives/dev/2015-June/018947.html
> > To add the packet type, there is a proposal to add a build-time option
> > CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019172.html
> > We must also better document how to remove a deprecated ABI:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019465.html
> > The ABI compatibility is a new constraint and we need to better
> > understand
> > what it means and how to proceed. Even the macros are not yet well
> > documented:
> > 	http://dpdk.org/ml/archives/dev/2015-June/019357.html
> > 
> > Thanks for your attention and your participation in these important
> > choices.
> 
> There's been some good discussion on the ABI policy in various responses to this email. I think we now need to reach a conclusion on how we're going to proceed for the 2.1 release. Then, we can have further discussion on the use of versioning or other methods for avoiding the problem in future.
> 
> For the 2.1 release, I think we should agree to make patches that change the ABI controllable via a compile-time option. I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (see http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1.
> 
This is a bad idea.  Making ABI dependent on compile time options isn't a
maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).  If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.

Neil

> 
> Tim
> 

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Richardson, Bruce
> Sent: Wednesday, June 17, 2015 12:07 PM
> To: Neil Horman; Thomas Monjalon
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> Hi Neil,
> 
> on my end, some suggestions:
> 
> 1. the documentation on changing an API function provided in rte_compat.h
> is really good, but I don't think this is present in our documentation in
> the docs folder or on website is it (apologies if it is and I've missed
> it)? This needs to go into programmers guide or some other doc (perhaps
> the new doc that the coding style went into).
> 
> 2. The documentation also needs an example of: this is how you add a new
> function and update the map file, and this is how you a) mark a function
> as deprecated and b) remove it completely. That way we could have one
> guide covering API versioning, how to add, modify and remove functions.
> 
> 3. This doc should also cover how to use the API checker tool, something I
> haven't had the chance to look at yet, but should do in the near future!
> :-)


+1 on all three. We need better documentation on how to work with the ABI in DPDK. A new document in doc/guides/guidelines/ would be good.

John

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 06:26, Neil Horman:
> On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > For the 2.1 release, I think we should agree to make patches that change
> > the ABI controllable via a compile-time option. I like Olivier's proposal
> > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > changes instead of a separate option per patch set (see
> > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > should rework the affected patch sets to use that approach for 2.1.
> 
> This is a bad idea.  Making ABI dependent on compile time options isn't a
> maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> work (that is to say you make it impossible to really tell what ABI version you
> are building).

The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.

> If you have two compile time options that modify the ABI, you
> have to burn through 4 possible LIBABIVER version values to accomodate all
> possible combinations, and then you need to remember that when you make them
> statically applicable.

The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.

Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> 2015-06-19 06:26, Neil Horman:
> > On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > > For the 2.1 release, I think we should agree to make patches that change
> > > the ABI controllable via a compile-time option. I like Olivier's proposal
> > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > changes instead of a separate option per patch set (see
> > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > should rework the affected patch sets to use that approach for 2.1.
> > 
> > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> > work (that is to say you make it impossible to really tell what ABI version you
> > are building).
> 
> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> So one ABI version number refers always to the same ABI.
> 
> > If you have two compile time options that modify the ABI, you
> > have to burn through 4 possible LIBABIVER version values to accomodate all
> > possible combinations, and then you need to remember that when you make them
> > statically applicable.
> 
> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> 
> Your intent when introducing ABI policy was to allow smooth porting of
> applications from a DPDK version to another. Right?
> The adopted solution was to provide backward compatibility during 1 release.
> But there are cases where it's not possible. So the policy was to notice
> the future change and wait one release cycle to break the ABI (failing
> compatibility goals).
> The compile-time option may provide an alternative DPDK packaging when the
> ABI backward compatibility cannot be provided (case of mbuf changes).
> In such case, it's still possible to upgrade DPDK by providing 2 versions of
> DPDK libs. So the existing apps continue to link with the previous ABI and
> have the possibility of migrating to the new one.
> Another advantage of this approach is that we don't have to wait 1 release
> to integrate the changes.
> The last advantage is to benefit early of these changes with static libraries.
> 


Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.

What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).  Patches that aren't ABI
stable can be put on the next-branch/tree in thier final format.  You can
delcare the branch unstable (thereby reserving your right to rebase it).  People
can use that to preview the next ABI version (complete with the update LIBABIVER
bump), and when you release dpdk-X, the new ABI for dpdk-X+1 is achieved by
simply merging.

Neil

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 09:02, Neil Horman:
> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > 2015-06-19 06:26, Neil Horman:
> > > On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > > > For the 2.1 release, I think we should agree to make patches that change
> > > > the ABI controllable via a compile-time option. I like Olivier's proposal
> > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > > changes instead of a separate option per patch set (see
> > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > > should rework the affected patch sets to use that approach for 2.1.
> > > 
> > > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > > maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> > > work (that is to say you make it impossible to really tell what ABI version you
> > > are building).
> > 
> > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> > So one ABI version number refers always to the same ABI.
> > 
> > > If you have two compile time options that modify the ABI, you
> > > have to burn through 4 possible LIBABIVER version values to accomodate all
> > > possible combinations, and then you need to remember that when you make them
> > > statically applicable.
> > 
> > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > 
> > Your intent when introducing ABI policy was to allow smooth porting of
> > applications from a DPDK version to another. Right?
> > The adopted solution was to provide backward compatibility during 1 release.
> > But there are cases where it's not possible. So the policy was to notice
> > the future change and wait one release cycle to break the ABI (failing
> > compatibility goals).
> > The compile-time option may provide an alternative DPDK packaging when the
> > ABI backward compatibility cannot be provided (case of mbuf changes).
> > In such case, it's still possible to upgrade DPDK by providing 2 versions of
> > DPDK libs. So the existing apps continue to link with the previous ABI and
> > have the possibility of migrating to the new one.
> > Another advantage of this approach is that we don't have to wait 1 release
> > to integrate the changes.
> > The last advantage is to benefit early of these changes with static libraries.
> 
> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> Implementing an ABI preview option like this implies the notion that, after a
> release, you have to remove all the ifdefs that you inserted to create the new
> ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
> large, and is predicated on the centralization of work effort (that is to say
> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
> from the build' patch every release.

It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.

> What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
> the sort that Thomas Herbert proposed a few weeks ago).

This tree was created after Thomas' request:
	http://dpdk.org/browse/next/dpdk-next/

> Patches that aren't ABI stable can be put on the next-branch/tree in thier
> final format.  You can delcare the branch unstable (thereby reserving your
> right to rebase it).  People can use that to preview the next ABI version
> (complete with the update LIBABIVER bump), and when you release dpdk-X,
> the new ABI for dpdk-X+1 is achieved by simply merging.

Having this tree living would be a nice improvement but it won't provide any
stable (and enough validated) releases to rely on.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote:
> 2015-06-19 09:02, Neil Horman:
> > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > > 2015-06-19 06:26, Neil Horman:
> > > > On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > > > > For the 2.1 release, I think we should agree to make patches that change
> > > > > the ABI controllable via a compile-time option. I like Olivier's proposal
> > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > > > changes instead of a separate option per patch set (see
> > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > > > should rework the affected patch sets to use that approach for 2.1.
> > > > 
> > > > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > > > maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> > > > work (that is to say you make it impossible to really tell what ABI version you
> > > > are building).
> > > 
> > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> > > So one ABI version number refers always to the same ABI.
> > > 
> > > > If you have two compile time options that modify the ABI, you
> > > > have to burn through 4 possible LIBABIVER version values to accomodate all
> > > > possible combinations, and then you need to remember that when you make them
> > > > statically applicable.
> > > 
> > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > > 
> > > Your intent when introducing ABI policy was to allow smooth porting of
> > > applications from a DPDK version to another. Right?
> > > The adopted solution was to provide backward compatibility during 1 release.
> > > But there are cases where it's not possible. So the policy was to notice
> > > the future change and wait one release cycle to break the ABI (failing
> > > compatibility goals).
> > > The compile-time option may provide an alternative DPDK packaging when the
> > > ABI backward compatibility cannot be provided (case of mbuf changes).
> > > In such case, it's still possible to upgrade DPDK by providing 2 versions of
> > > DPDK libs. So the existing apps continue to link with the previous ABI and
> > > have the possibility of migrating to the new one.
> > > Another advantage of this approach is that we don't have to wait 1 release
> > > to integrate the changes.
> > > The last advantage is to benefit early of these changes with static libraries.
> > 
> > Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> > Implementing an ABI preview option like this implies the notion that, after a
> > release, you have to remove all the ifdefs that you inserted to create the new
> > ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
> > large, and is predicated on the centralization of work effort (that is to say
> > you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
> > from the build' patch every release.
> 
> It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
> have easy backward compatibility with the compat macros you introduced.
> I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> At the same time, the deprecated API, using the compat macros, will be removed.
> 
I think that is something you can't really predict, as its not an issue of how
stringent we are with its use, but rather a function of how much change
developers want in a given release.  That is to say, if you only reserve it for
the most important/urgently needed changes, thats fine, but if you have a
release in which 50 developers want to make urgent and important changes that
breaks ABI, you still have quite a job on your hands to back out the config
changes.

Not to mention the fact that backing those changes out is a manual process.

> > What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
> > the sort that Thomas Herbert proposed a few weeks ago).
> 
> This tree was created after Thomas' request:
> 	http://dpdk.org/browse/next/dpdk-next/
> 
Awesome, Though I'm not sure thats entirely the right place either.  IIRC that
location was intended to be an early integration site that took unreviewed
patches.  I think this really calls for a branch from the mainline tree that
exclusively accepts reviewed ABI changing patches, that can then be merged after
the next release

> > Patches that aren't ABI stable can be put on the next-branch/tree in thier
> > final format.  You can delcare the branch unstable (thereby reserving your
> > right to rebase it).  People can use that to preview the next ABI version
> > (complete with the update LIBABIVER bump), and when you release dpdk-X,
> > the new ABI for dpdk-X+1 is achieved by simply merging.
> 
> Having this tree living would be a nice improvement but it won't provide any
> stable (and enough validated) releases to rely on.
> 
I'm not sure I follow you entirely here.  If the goal is to find a place to
accept patches that are ABI altering ahead of the main release, why do you need
to provide stable/validated releases?  Just base it off the HEAD of the git tree
during the DPDK release X merge window, any testing done in the base branch
should roughly apply, save for functional changes made by the ABI patches you
add in on the branch.

Neil

> 

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 11:27, Neil Horman:
> On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote:
> > 2015-06-19 09:02, Neil Horman:
> > > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > > > 2015-06-19 06:26, Neil Horman:
> > > > > On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> > > > > > For the 2.1 release, I think we should agree to make patches that change
> > > > > > the ABI controllable via a compile-time option. I like Olivier's proposal
> > > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > > > > changes instead of a separate option per patch set (see
> > > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > > > > should rework the affected patch sets to use that approach for 2.1.
> > > > > 
> > > > > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > > > > maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> > > > > work (that is to say you make it impossible to really tell what ABI version you
> > > > > are building).
> > > > 
> > > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> > > > So one ABI version number refers always to the same ABI.
> > > > 
> > > > > If you have two compile time options that modify the ABI, you
> > > > > have to burn through 4 possible LIBABIVER version values to accomodate all
> > > > > possible combinations, and then you need to remember that when you make them
> > > > > statically applicable.
> > > > 
> > > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > > > 
> > > > Your intent when introducing ABI policy was to allow smooth porting of
> > > > applications from a DPDK version to another. Right?
> > > > The adopted solution was to provide backward compatibility during 1 release.
> > > > But there are cases where it's not possible. So the policy was to notice
> > > > the future change and wait one release cycle to break the ABI (failing
> > > > compatibility goals).
> > > > The compile-time option may provide an alternative DPDK packaging when the
> > > > ABI backward compatibility cannot be provided (case of mbuf changes).
> > > > In such case, it's still possible to upgrade DPDK by providing 2 versions of
> > > > DPDK libs. So the existing apps continue to link with the previous ABI and
> > > > have the possibility of migrating to the new one.
> > > > Another advantage of this approach is that we don't have to wait 1 release
> > > > to integrate the changes.
> > > > The last advantage is to benefit early of these changes with static libraries.
> > > 
> > > Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> > > Implementing an ABI preview option like this implies the notion that, after a
> > > release, you have to remove all the ifdefs that you inserted to create the new
> > > ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
> > > large, and is predicated on the centralization of work effort (that is to say
> > > you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
> > > from the build' patch every release.
> > 
> > It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
> > have easy backward compatibility with the compat macros you introduced.
> > I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> > At the same time, the deprecated API, using the compat macros, will be removed.
> > 
> I think that is something you can't really predict, as its not an issue of how
> stringent we are with its use, but rather a function of how much change
> developers want in a given release.  That is to say, if you only reserve it for
> the most important/urgently needed changes, thats fine, but if you have a
> release in which 50 developers want to make urgent and important changes that
> breaks ABI, you still have quite a job on your hands to back out the config
> changes.
> 
> Not to mention the fact that backing those changes out is a manual process.
> 
> > > What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
> > > the sort that Thomas Herbert proposed a few weeks ago).
> > 
> > This tree was created after Thomas' request:
> > 	http://dpdk.org/browse/next/dpdk-next/
> > 
> Awesome, Though I'm not sure thats entirely the right place either.  IIRC that
> location was intended to be an early integration site that took unreviewed
> patches.  I think this really calls for a branch from the mainline tree that
> exclusively accepts reviewed ABI changing patches, that can then be merged after
> the next release
> 
> > > Patches that aren't ABI stable can be put on the next-branch/tree in thier
> > > final format.  You can delcare the branch unstable (thereby reserving your
> > > right to rebase it).  People can use that to preview the next ABI version
> > > (complete with the update LIBABIVER bump), and when you release dpdk-X,
> > > the new ABI for dpdk-X+1 is achieved by simply merging.
> > 
> > Having this tree living would be a nice improvement but it won't provide any
> > stable (and enough validated) releases to rely on.
> > 
> I'm not sure I follow you entirely here.  If the goal is to find a place to
> accept patches that are ABI altering ahead of the main release, why do you need
> to provide stable/validated releases?  Just base it off the HEAD of the git tree
> during the DPDK release X merge window, any testing done in the base branch
> should roughly apply, save for functional changes made by the ABI patches you
> add in on the branch.

OK, I didn't get you. So you are saying that the changes for the next release
may be prepared in a branch. Yes, it's possible.
But before stating on practical method to execute the policy, we need to agree
on the policy.

This is my proposal:
- The ABI policy must be better explained: what happens to LIBABIVER and .map
when adding a field, adding or removing a function.
- The file doc/guides/rel_notes/abi.rst must contain the ABI notices but the
policy must be moved to doc/guides/guidelines/compat.rst
- The case "backward compatibility broken" must be replaced by the usage of
CONFIG_RTE_NEXT_ABI while describing cases where it can apply.
- The .map files must be generated in order to make it simpler and allows
the use of CONFIG_RTE_NEXT_ABI.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas F Herbert]

On 6/19/15 9:16 AM, Thomas Monjalon wrote:
> 2015-06-19 09:02, Neil Horman:
>> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
>>> 2015-06-19 06:26, Neil Horman:
>>>> On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
>>>>> For the 2.1 release, I think we should agree to make patches that change
>>>>> the ABI controllable via a compile-time option. I like Olivier's proposal
>>>>> on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
>>>>> changes instead of a separate option per patch set (see
>>>>> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
>>>>> should rework the affected patch sets to use that approach for 2.1.
>>>>
>>>> This is a bad idea.  Making ABI dependent on compile time options isn't a
>>>> maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
>>>> work (that is to say you make it impossible to really tell what ABI version you
>>>> are building).
>>>
>>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
>>> So one ABI version number refers always to the same ABI.
>>>
>>>> If you have two compile time options that modify the ABI, you
>>>> have to burn through 4 possible LIBABIVER version values to accomodate all
>>>> possible combinations, and then you need to remember that when you make them
>>>> statically applicable.
>>>
>>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
>>>
>>> Your intent when introducing ABI policy was to allow smooth porting of
>>> applications from a DPDK version to another. Right?
>>> The adopted solution was to provide backward compatibility during 1 release.
>>> But there are cases where it's not possible. So the policy was to notice
>>> the future change and wait one release cycle to break the ABI (failing
>>> compatibility goals).
>>> The compile-time option may provide an alternative DPDK packaging when the
>>> ABI backward compatibility cannot be provided (case of mbuf changes).
>>> In such case, it's still possible to upgrade DPDK by providing 2 versions of
>>> DPDK libs. So the existing apps continue to link with the previous ABI and
>>> have the possibility of migrating to the new one.
>>> Another advantage of this approach is that we don't have to wait 1 release
>>> to integrate the changes.
>>> The last advantage is to benefit early of these changes with static libraries.
>>
>> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
>> Implementing an ABI preview option like this implies the notion that, after a
>> release, you have to remove all the ifdefs that you inserted to create the new
>> ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
>> large, and is predicated on the centralization of work effort (that is to say
>> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
>> from the build' patch every release.
>
> It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
> have easy backward compatibility with the compat macros you introduced.
> I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> At the same time, the deprecated API, using the compat macros, will be removed.
>
>> What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
>> the sort that Thomas Herbert proposed a few weeks ago).
>
> This tree was created after Thomas' request:
> 	http://dpdk.org/browse/next/dpdk-next/

Thomas, I am sorry if I went quiet for awhile but I was on personal 
travel with inconsistent access so I almost missed most of this 
discussion about ABI changes.

My understanding of the purpose of the dpdk-next tree is to validate 
patches by applying and compiling against a "pull" from the main dpdk 
tree. I think a good way to handle ABI change while effectively using 
the dpdk-next might be to do as follows:

Create a specific branch for the new ABI such as 2.X in the main dpdk 
tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X 
branch along with master.

Since, dpdk-next would also have the 2.X branch that is in the main dpdk 
tree, submitted patches could be applied to either the main branch or 
the new-ABI 2.X branch. Providing that patch submitters make it clear 
whether a submitted patch is for the new ABI or the old ABI, dpdk-next 
could continue to validate the patches for either the main branch or the 
new ABI 2.X branch.

>
>> Patches that aren't ABI stable can be put on the next-branch/tree in thier
>> final format.  You can delcare the branch unstable (thereby reserving your
>> right to rebase it).  People can use that to preview the next ABI version
>> (complete with the update LIBABIVER bump), and when you release dpdk-X,
>> the new ABI for dpdk-X+1 is achieved by simply merging.
>
> Having this tree living would be a nice improvement but it won't provide any
> stable (and enough validated) releases to rely on.
>

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 12:13, Thomas F Herbert:
> 
> On 6/19/15 9:16 AM, Thomas Monjalon wrote:
> > 2015-06-19 09:02, Neil Horman:
> >> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> >>> 2015-06-19 06:26, Neil Horman:
> >>>> On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
> >>>>> For the 2.1 release, I think we should agree to make patches that change
> >>>>> the ABI controllable via a compile-time option. I like Olivier's proposal
> >>>>> on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> >>>>> changes instead of a separate option per patch set (see
> >>>>> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> >>>>> should rework the affected patch sets to use that approach for 2.1.
> >>>>
> >>>> This is a bad idea.  Making ABI dependent on compile time options isn't a
> >>>> maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> >>>> work (that is to say you make it impossible to really tell what ABI version you
> >>>> are building).
> >>>
> >>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> >>> So one ABI version number refers always to the same ABI.
> >>>
> >>>> If you have two compile time options that modify the ABI, you
> >>>> have to burn through 4 possible LIBABIVER version values to accomodate all
> >>>> possible combinations, and then you need to remember that when you make them
> >>>> statically applicable.
> >>>
> >>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> >>>
> >>> Your intent when introducing ABI policy was to allow smooth porting of
> >>> applications from a DPDK version to another. Right?
> >>> The adopted solution was to provide backward compatibility during 1 release.
> >>> But there are cases where it's not possible. So the policy was to notice
> >>> the future change and wait one release cycle to break the ABI (failing
> >>> compatibility goals).
> >>> The compile-time option may provide an alternative DPDK packaging when the
> >>> ABI backward compatibility cannot be provided (case of mbuf changes).
> >>> In such case, it's still possible to upgrade DPDK by providing 2 versions of
> >>> DPDK libs. So the existing apps continue to link with the previous ABI and
> >>> have the possibility of migrating to the new one.
> >>> Another advantage of this approach is that we don't have to wait 1 release
> >>> to integrate the changes.
> >>> The last advantage is to benefit early of these changes with static libraries.
> >>
> >> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> >> Implementing an ABI preview option like this implies the notion that, after a
> >> release, you have to remove all the ifdefs that you inserted to create the new
> >> ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
> >> large, and is predicated on the centralization of work effort (that is to say
> >> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
> >> from the build' patch every release.
> >
> > It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
> > have easy backward compatibility with the compat macros you introduced.
> > I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> > At the same time, the deprecated API, using the compat macros, will be removed.
> >
> >> What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
> >> the sort that Thomas Herbert proposed a few weeks ago).
> >
> > This tree was created after Thomas' request:
> > 	http://dpdk.org/browse/next/dpdk-next/
> 
> Thomas, I am sorry if I went quiet for awhile but I was on personal 
> travel with inconsistent access so I almost missed most of this 
> discussion about ABI changes.
> 
> My understanding of the purpose of the dpdk-next tree is to validate 
> patches by applying and compiling against a "pull" from the main dpdk 
> tree. I think a good way to handle ABI change while effectively using 
> the dpdk-next might be to do as follows:
> 
> Create a specific branch for the new ABI such as 2.X in the main dpdk 
> tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X 
> branch along with master.
> 
> Since, dpdk-next would also have the 2.X branch that is in the main dpdk 
> tree, submitted patches could be applied to either the main branch or 
> the new-ABI 2.X branch. Providing that patch submitters make it clear 
> whether a submitted patch is for the new ABI or the old ABI, dpdk-next 
> could continue to validate the patches for either the main branch or the 
> new ABI 2.X branch.

What is the benefit of a new-ABI branch in the -next tree?

The goal of this discussion is to find a consensus on ABI policy to
smoothly integrate new features without forcing users of shared libraries
to re-build their application when upgrading DPDK, and let them do the
transition before the next upgrade.

Re: [dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas F Herbert]

On 6/19/15 1:02 PM, Thomas Monjalon wrote:
> 2015-06-19 12:13, Thomas F Herbert:
>>
>> On 6/19/15 9:16 AM, Thomas Monjalon wrote:
>>> 2015-06-19 09:02, Neil Horman:
>>>> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
>>>>> 2015-06-19 06:26, Neil Horman:
>>>>>> On Thu, Jun 18, 2015 at 04:55:45PM +0000, O'Driscoll, Tim wrote:
>>>>>>> For the 2.1 release, I think we should agree to make patches that change
>>>>>>> the ABI controllable via a compile-time option. I like Olivier's proposal
>>>>>>> on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
>>>>>>> changes instead of a separate option per patch set (see
>>>>>>> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
>>>>>>> should rework the affected patch sets to use that approach for 2.1.
>>>>>>
>>>>>> This is a bad idea.  Making ABI dependent on compile time options isn't a
>>>>>> maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
>>>>>> work (that is to say you make it impossible to really tell what ABI version you
>>>>>> are building).
>>>>>
>>>>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
>>>>> So one ABI version number refers always to the same ABI.
>>>>>
>>>>>> If you have two compile time options that modify the ABI, you
>>>>>> have to burn through 4 possible LIBABIVER version values to accomodate all
>>>>>> possible combinations, and then you need to remember that when you make them
>>>>>> statically applicable.
>>>>>
>>>>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
>>>>>
>>>>> Your intent when introducing ABI policy was to allow smooth porting of
>>>>> applications from a DPDK version to another. Right?
>>>>> The adopted solution was to provide backward compatibility during 1 release.
>>>>> But there are cases where it's not possible. So the policy was to notice
>>>>> the future change and wait one release cycle to break the ABI (failing
>>>>> compatibility goals).
>>>>> The compile-time option may provide an alternative DPDK packaging when the
>>>>> ABI backward compatibility cannot be provided (case of mbuf changes).
>>>>> In such case, it's still possible to upgrade DPDK by providing 2 versions of
>>>>> DPDK libs. So the existing apps continue to link with the previous ABI and
>>>>> have the possibility of migrating to the new one.
>>>>> Another advantage of this approach is that we don't have to wait 1 release
>>>>> to integrate the changes.
>>>>> The last advantage is to benefit early of these changes with static libraries.
>>>>
>>>> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
>>>> Implementing an ABI preview option like this implies the notion that, after a
>>>> release, you have to remove all the ifdefs that you inserted to create the new
>>>> ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
>>>> large, and is predicated on the centralization of work effort (that is to say
>>>> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
>>>> from the build' patch every release.
>>>
>>> It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
>>> have easy backward compatibility with the compat macros you introduced.
>>> I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
>>> At the same time, the deprecated API, using the compat macros, will be removed.
>>>
>>>> What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
>>>> the sort that Thomas Herbert proposed a few weeks ago).
>>>
>>> This tree was created after Thomas' request:
>>> 	http://dpdk.org/browse/next/dpdk-next/
>>
>> Thomas, I am sorry if I went quiet for awhile but I was on personal
>> travel with inconsistent access so I almost missed most of this
>> discussion about ABI changes.
>>
>> My understanding of the purpose of the dpdk-next tree is to validate
>> patches by applying and compiling against a "pull" from the main dpdk
>> tree. I think a good way to handle ABI change while effectively using
>> the dpdk-next might be to do as follows:
>>
>> Create a specific branch for the new ABI such as 2.X in the main dpdk
>> tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X
>> branch along with master.
>>
>> Since, dpdk-next would also have the 2.X branch that is in the main dpdk
>> tree, submitted patches could be applied to either the main branch or
>> the new-ABI 2.X branch. Providing that patch submitters make it clear
>> whether a submitted patch is for the new ABI or the old ABI, dpdk-next
>> could continue to validate the patches for either the main branch or the
>> new ABI 2.X branch.
>
> What is the benefit of a new-ABI branch in the -next tree?
I don't think that there is any specific benefit to an new-ABI branch in 
the dpdk-next tree. I was responding to the suggestion above and perhaps 
I missread it. It sounded like what was being proposed was to use the 
dpdk-next tree specifically for pre-integration of new-ABI. I don't 
think this is of any benefit either.

However if it should be decided to integrate new-ABI patches in a branch 
of dpdk rather then in a separate new-ABI tree, then net-next can 
"mirror" that branch along with the master branch so patches can be 
smoke tested whether they are submitted to the master or to the new-ABI 
branch.
>
> The goal of this discussion is to find a consensus on ABI policy to
> smoothly integrate new features without forcing users of shared libraries
> to re-build their application when upgrading DPDK, and let them do the
> transition before the next upgrade.
I understand this and I think it is a good suggestion to have a 
mechanism to ease the transition.
>