[dpdk-dev] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[dpdk-dev] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[Ray Kinsella]

Folks,

As you may be aware, there was a panel on ABI Stability @ DPDK
Userspace.  There where a number of proposed amendments to the ABI
stability proposal made, as well as a number of points and comments, you
will find all these below. The proposals needs further discussion so
please chime in below.

Thanks to Tim for capturing while I was busy on the stage.

Thanks,

Ray K


Table of Contents
_________________

1. Proposals from the panel discussion.
.. 1. Developer releases (versus User Releases)
.. 2. Core and non-core packaging
.. 3. Approach for public data structures
.. 4. Delaying v19.11
2. Other notes from the panel discussion.
.. 1. Performance as the paramount goal.
.. 2. Length of ABI Stability.
.. 3. Testing ABI Stability
.. 4. Call to action


1 Proposals from the panel discussion.
======================================

1.1 Developer releases (versus User Releases)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  - Summary: Differentiate between "developer" and "user" releases.
    - 1 year is too long to wait to upstream a new feature which breaks
      ABI.
    - Developer releases would be for use by the development community
      only.
    - A proposed compromise was that the .08 release would be the only
      "developer release".


1.2 Core and non-core packaging
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  - Summary: OS packaging doesn't include all libraries.
    - This would create a delta between the community ABI, and the OS
      packaging.
    - OS packagers rational is that some libraries are used very rarely.


1.3 Approach for public data structures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  - Summary: Public/exposed data structures are tricky for ABI
    stability.
    - See discussion @

<http://inbox.dpdk.org/dev/980083c6-130a-9658-f82b-0c9ddc7cc0cc@ashroe.eu/>


1.4 Delaying v19.11
~~~~~~~~~~~~~~~~~~~

  - Summary: push v19.11 to v19.12 to make more time to prepare and wave
    the depreciation process.
    - OVS take Nov LTS release in their Feb release. Delaying 19.11 may
      have an impact on this.
    - Not easy to change release at this stage as many things depend on
      it (OS distros etc.).


2 Other notes from the panel discussion.
========================================

2.1 Performance as the paramount goal.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  - Some users, would trade performance for better readability and
    debug-ability.
  - Skepticism that micro-benchmarks reflect real world performance.


2.2 Length of ABI Stability.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  - Some users, questioned if 1 year would be long enough.
  - It was clarified that the 1 year period, would be reviewed after the
    first year with the intention of lengthening the period.


2.3 Testing ABI Stability
~~~~~~~~~~~~~~~~~~~~~~~~~

  - More work for developers and validation teams. Need to validate
    multiple paths for symbol versioning.


2.4 Call to action
~~~~~~~~~~~~~~~~~~

  - We should do something. So we don’t want to have the same
    conversation again in a year.

Re: [dpdk-dev] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[Kevin Traynor]

On 23/09/2019 18:51, Ray Kinsella wrote:
> Folks,
> 
> As you may be aware, there was a panel on ABI Stability @ DPDK
> Userspace.  There where a number of proposed amendments to the ABI
> stability proposal made, as well as a number of points and comments, you
> will find all these below. The proposals needs further discussion so
> please chime in below.
> 
> Thanks to Tim for capturing while I was busy on the stage.
> 

Thanks for the notes Tim,

> Thanks,
> 
> Ray K
> 
> 
> Table of Contents
> _________________
> 
> 1. Proposals from the panel discussion.
> .. 1. Developer releases (versus User Releases)
> .. 2. Core and non-core packaging
> .. 3. Approach for public data structures
> .. 4. Delaying v19.11
> 2. Other notes from the panel discussion.
> .. 1. Performance as the paramount goal.
> .. 2. Length of ABI Stability.
> .. 3. Testing ABI Stability
> .. 4. Call to action
> 
> 
> 1 Proposals from the panel discussion.
> ======================================
> 
> 1.1 Developer releases (versus User Releases)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>   - Summary: Differentiate between "developer" and "user" releases.
>     - 1 year is too long to wait to upstream a new feature which breaks
>       ABI.
>     - Developer releases would be for use by the development community
>       only.
>     - A proposed compromise was that the .08 release would be the only
>       "developer release".
> 
> 
> 1.2 Core and non-core packaging
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>   - Summary: OS packaging doesn't include all libraries.
>     - This would create a delta between the community ABI, and the OS
>       packaging.
>     - OS packagers rational is that some libraries are used very rarely.
> 
> 
> 1.3 Approach for public data structures
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>   - Summary: Public/exposed data structures are tricky for ABI
>     stability.
>     - See discussion @
> 
> <http://inbox.dpdk.org/dev/980083c6-130a-9658-f82b-0c9ddc7cc0cc@ashroe.eu/>
> 
> 
> 1.4 Delaying v19.11
> ~~~~~~~~~~~~~~~~~~~
> 
>   - Summary: push v19.11 to v19.12 to make more time to prepare and wave
>     the depreciation process.
>     - OVS take Nov LTS release in their Feb release. Delaying 19.11 may
>       have an impact on this.

To give some more info about OVS. tldr there is a soft freeze on Jan 1st
but an update that is being discussed/reviewed could go in until Jan
15th when the OVS 2.13 branch would be created.

Thomas is indeed right in that there is a development branch in OVS that
is intended to keep up with DPDK master with a view to finding any
integration issues early.

More info here
http://docs.openvswitch.org/en/latest/internals/release-process/#release-strategy

>     - Not easy to change release at this stage as many things depend on
>       it (OS distros etc.).
> 

In the short term, based on the feedback at the conference and to give
something concrete to be considered, here is a suggestion,

ABI freeze starts at 20.02 for 9 months, with a review as planned to see
if 20.11 should be frozen 2 years.

pros:
+ Eliminates any need for delaying 19.11 release

+ Allows maintainers to stick to current deprecation policy if they need
to make changes prior to freeze (Based on comment from Hemmant)

+ Not sure if it's worthy of a new bullet or clear from above but I
would add that changing the release cycle/deprecation policy etc 2 weeks
(I think) before RC1 is late to say the least and there is no notice to
users

+ Means that any changes required prior to freeze are not rushed with
usual big LTS release (19.11). Gives more time and maybe during a saner
release cycle (20.02)

cons:
- With view for possible 20.11 freeze, gives 2 releases to tease out
process instead of 3

- Perhaps it is desirable for some users to have the 19.11 LTS ABI
compatible with 20.02/05/08 releases

I've tried to keep them objective, of course people will have different
opinions about starting a freeze now vs. later etc. too.

thanks,
Kevin.

> 
> 2 Other notes from the panel discussion.
> ========================================
> 
> 2.1 Performance as the paramount goal.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>   - Some users, would trade performance for better readability and
>     debug-ability.
>   - Skepticism that micro-benchmarks reflect real world performance.
> 
> 
> 2.2 Length of ABI Stability.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>   - Some users, questioned if 1 year would be long enough.
>   - It was clarified that the 1 year period, would be reviewed after the
>     first year with the intention of lengthening the period.
> 
> 
> 2.3 Testing ABI Stability
> ~~~~~~~~~~~~~~~~~~~~~~~~~
> 
>   - More work for developers and validation teams. Need to validate
>     multiple paths for symbol versioning.
> 
> 
> 2.4 Call to action
> ~~~~~~~~~~~~~~~~~~
> 
>   - We should do something. So we don’t want to have the same
>     conversation again in a year.
> 

Re: [dpdk-dev] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[Ray Kinsella]

> In the short term, based on the feedback at the conference and to give
> something concrete to be considered, here is a suggestion,
> 
> ABI freeze starts at 20.02 for 9 months, with a review as planned to see
> if 20.11 should be frozen 2 years.
> 
> pros:
> + Eliminates any need for delaying 19.11 release
> 
> + Allows maintainers to stick to current deprecation policy if they need
> to make changes prior to freeze (Based on comment from Hemmant)
> 
> + Not sure if it's worthy of a new bullet or clear from above but I
> would add that changing the release cycle/deprecation policy etc 2 weeks
> (I think) before RC1 is late to say the least and there is no notice to
> users
> 
> + Means that any changes required prior to freeze are not rushed with
> usual big LTS release (19.11). Gives more time and maybe during a saner
> release cycle (20.02)
> 
> cons:
> - With view for possible 20.11 freeze, gives 2 releases to tease out
> process instead of 3
> 
> - Perhaps it is desirable for some users to have the 19.11 LTS ABI
> compatible with 20.02/05/08 releases
> 
> I've tried to keep them objective, of course people will have different
> opinions about starting a freeze now vs. later etc. too.
> 
> thanks,
> Kevin.
> 

*interesting*

Another approach, possibly better approach, is to see the LTS as the
final act following an ABI declaration/freeze.

We we declare the v20 ABI in DPDK 20.02, and hold that ABI until 21.02
including the v20.11 LTS. The LTS then becomes the cumulation of the ABI
freeze.

I didn't go this road, because of the community habit of pushing things
in just before the LTS, I thought it would be a bridge too far, and that
it would get considerable push back.

Re: [dpdk-dev] [dpdk-techboard] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[Bruce Richardson]

On Wed, Sep 25, 2019 at 03:29:16PM +0100, Ray Kinsella wrote:
> 
> > In the short term, based on the feedback at the conference and to give
> > something concrete to be considered, here is a suggestion,
> > 
> > ABI freeze starts at 20.02 for 9 months, with a review as planned to see
> > if 20.11 should be frozen 2 years.
> > 
> > pros:
> > + Eliminates any need for delaying 19.11 release
> > 
> > + Allows maintainers to stick to current deprecation policy if they need
> > to make changes prior to freeze (Based on comment from Hemmant)
> > 
> > + Not sure if it's worthy of a new bullet or clear from above but I
> > would add that changing the release cycle/deprecation policy etc 2 weeks
> > (I think) before RC1 is late to say the least and there is no notice to
> > users
> > 
> > + Means that any changes required prior to freeze are not rushed with
> > usual big LTS release (19.11). Gives more time and maybe during a saner
> > release cycle (20.02)
> > 
> > cons:
> > - With view for possible 20.11 freeze, gives 2 releases to tease out
> > process instead of 3
> > 
> > - Perhaps it is desirable for some users to have the 19.11 LTS ABI
> > compatible with 20.02/05/08 releases
> > 
> > I've tried to keep them objective, of course people will have different
> > opinions about starting a freeze now vs. later etc. too.
> > 
> > thanks,
> > Kevin.
> > 
> 
> *interesting*
> 
> Another approach, possibly better approach, is to see the LTS as the
> final act following an ABI declaration/freeze.
> 
> We we declare the v20 ABI in DPDK 20.02, and hold that ABI until 21.02
> including the v20.11 LTS. The LTS then becomes the cumulation of the ABI
> freeze.
> 
> I didn't go this road, because of the community habit of pushing things
> in just before the LTS, I thought it would be a bridge too far, and that
> it would get considerable push back.

I actually think this approach was initially rejected as having an ABI
break immediately after an LTS makes backporting fixes to the LTS more
problematic.

/Bruce

Re: [dpdk-dev] [dpdk-techboard] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[Kevin Traynor]

On 25/09/2019 15:40, Bruce Richardson wrote:
> On Wed, Sep 25, 2019 at 03:29:16PM +0100, Ray Kinsella wrote:
>>
>>> In the short term, based on the feedback at the conference and to give
>>> something concrete to be considered, here is a suggestion,
>>>
>>> ABI freeze starts at 20.02 for 9 months, with a review as planned to see
>>> if 20.11 should be frozen 2 years.
>>>
>>> pros:
>>> + Eliminates any need for delaying 19.11 release
>>>
>>> + Allows maintainers to stick to current deprecation policy if they need
>>> to make changes prior to freeze (Based on comment from Hemmant)
>>>
>>> + Not sure if it's worthy of a new bullet or clear from above but I
>>> would add that changing the release cycle/deprecation policy etc 2 weeks
>>> (I think) before RC1 is late to say the least and there is no notice to
>>> users
>>>
>>> + Means that any changes required prior to freeze are not rushed with
>>> usual big LTS release (19.11). Gives more time and maybe during a saner
>>> release cycle (20.02)
>>>
>>> cons:
>>> - With view for possible 20.11 freeze, gives 2 releases to tease out
>>> process instead of 3
>>>
>>> - Perhaps it is desirable for some users to have the 19.11 LTS ABI
>>> compatible with 20.02/05/08 releases
>>>
>>> I've tried to keep them objective, of course people will have different
>>> opinions about starting a freeze now vs. later etc. too.
>>>
>>> thanks,
>>> Kevin.
>>>
>>
>> *interesting*
>>
>> Another approach, possibly better approach, is to see the LTS as the
>> final act following an ABI declaration/freeze.
>>
>> We we declare the v20 ABI in DPDK 20.02, and hold that ABI until 21.02
>> including the v20.11 LTS. The LTS then becomes the cumulation of the ABI
>> freeze.
>>
>> I didn't go this road, because of the community habit of pushing things
>> in just before the LTS, I thought it would be a bridge too far, and that
>> it would get considerable push back.
> 
> I actually think this approach was initially rejected as having an ABI
> break immediately after an LTS makes backporting fixes to the LTS more
> problematic.
> 

Yeah, it likely would. I guess the freeze cycle or end date of the
freeze trial (if i can call it that) could be discussed further later,
but the start date is the more immediate issue now.

> /Bruce
> 

Re: [dpdk-dev] [dpdk-techboard] [RFC] Proposals and notes from ABI stability panel @ DPDK Userspace

[Ray Kinsella]

On 25/09/2019 15:40, Bruce Richardson wrote:
> On Wed, Sep 25, 2019 at 03:29:16PM +0100, Ray Kinsella wrote:
>>
>>> In the short term, based on the feedback at the conference and to give
>>> something concrete to be considered, here is a suggestion,
>>>
>>> ABI freeze starts at 20.02 for 9 months, with a review as planned to see
>>> if 20.11 should be frozen 2 years.
>>>
>>> pros:
>>> + Eliminates any need for delaying 19.11 release
>>>
>>> + Allows maintainers to stick to current deprecation policy if they need
>>> to make changes prior to freeze (Based on comment from Hemmant)
>>>
>>> + Not sure if it's worthy of a new bullet or clear from above but I
>>> would add that changing the release cycle/deprecation policy etc 2 weeks
>>> (I think) before RC1 is late to say the least and there is no notice to
>>> users
>>>
>>> + Means that any changes required prior to freeze are not rushed with
>>> usual big LTS release (19.11). Gives more time and maybe during a saner
>>> release cycle (20.02)
>>>
>>> cons:
>>> - With view for possible 20.11 freeze, gives 2 releases to tease out
>>> process instead of 3
>>>
>>> - Perhaps it is desirable for some users to have the 19.11 LTS ABI
>>> compatible with 20.02/05/08 releases
>>>
>>> I've tried to keep them objective, of course people will have different
>>> opinions about starting a freeze now vs. later etc. too.
>>>
>>> thanks,
>>> Kevin.
>>>
>>
>> *interesting*
>>
>> Another approach, possibly better approach, is to see the LTS as the
>> final act following an ABI declaration/freeze.
>>
>> We we declare the v20 ABI in DPDK 20.02, and hold that ABI until 21.02
>> including the v20.11 LTS. The LTS then becomes the cumulation of the ABI
>> freeze.
>>
>> I didn't go this road, because of the community habit of pushing things
>> in just before the LTS, I thought it would be a bridge too far, and that
>> it would get considerable push back.
> 
> I actually think this approach was initially rejected as having an ABI
> break immediately after an LTS makes backporting fixes to the LTS more
> problematic.
> 
> /Bruce
> 

That too ..