[dpdk-dev] How to manage new APIs added after major ABI release?

[dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

Hi,

With new process, the major ABI releases will be compatible until it is
deprecated (until next LTS for now),
like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
will be ABI compatible with this version.

But if we introduce a new API after major ABI, say in 20.02 release, are we
allowed to break the ABI for that API before DPDK_20.11?

If we allow it break, following problem will be observed:
Assume an application using .so.20.1 library, and using the new API introduced
in 20.02, lets say foo(),
but when application switches to .so.20.2 (released via DPDK_20.05), application
will fail because of ABI breakage in foo().

I think it is fair that application expects forward compatibility in minor
versions of a shared library.
Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
etc will work fine. I think currently only .so.20.0 is fully forward compatible.

If we all agree on this, we may need to tweak the process a little, but before
diving into implementation details, I would like to be sure we are in same page.


Thanks,
ferruh

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> Hi,
> 
> With new process, the major ABI releases will be compatible until it is
> deprecated (until next LTS for now),
> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> will be ABI compatible with this version.
> 
> But if we introduce a new API after major ABI, say in 20.02 release, are we
> allowed to break the ABI for that API before DPDK_20.11?
> 
> If we allow it break, following problem will be observed:
> Assume an application using .so.20.1 library, and using the new API introduced
> in 20.02, lets say foo(),
> but when application switches to .so.20.2 (released via DPDK_20.05), application
> will fail because of ABI breakage in foo().
> 
> I think it is fair that application expects forward compatibility in minor
> versions of a shared library.
> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> 
> If we all agree on this, we may need to tweak the process a little, but before
> diving into implementation details, I would like to be sure we are in same page.
> 

Well, any new API's generally come in as experimental, in which case
changes are allowed, and breakage can be expected. If they are not
experiemental, then the ABI policy applies to them in that they cannot
change since they are part of the .21 ABI, even if that ABI is not fully
complete yet. For any application only using stable, non-experimental
functions, forward compatibility must be maintained IMHO.

/Bruce

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>> Hi,
>>
>> With new process, the major ABI releases will be compatible until it is
>> deprecated (until next LTS for now),
>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>> will be ABI compatible with this version.
>>
>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>> allowed to break the ABI for that API before DPDK_20.11?
>>
>> If we allow it break, following problem will be observed:
>> Assume an application using .so.20.1 library, and using the new API introduced
>> in 20.02, lets say foo(),
>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>> will fail because of ABI breakage in foo().
>>
>> I think it is fair that application expects forward compatibility in minor
>> versions of a shared library.
>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>
>> If we all agree on this, we may need to tweak the process a little, but before
>> diving into implementation details, I would like to be sure we are in same page.
>>
> 
> Well, any new API's generally come in as experimental, in which case
> changes are allowed, and breakage can be expected. If they are not
> experiemental, then the ABI policy applies to them in that they cannot
> change since they are part of the .21 ABI, even if that ABI is not fully
> complete yet. For any application only using stable, non-experimental
> functions, forward compatibility must be maintained IMHO.
> 

Talking about not experimental APIs, experimental ones free from the process.

And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
it should be supported for following ABI_20.x, instead of calling it ABI_21, and
this minor tweak (and mind shift) in .map files can be our solution.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> >> Hi,
> >>
> >> With new process, the major ABI releases will be compatible until it is
> >> deprecated (until next LTS for now),
> >> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> >> will be ABI compatible with this version.
> >>
> >> But if we introduce a new API after major ABI, say in 20.02 release, are we
> >> allowed to break the ABI for that API before DPDK_20.11?
> >>
> >> If we allow it break, following problem will be observed:
> >> Assume an application using .so.20.1 library, and using the new API introduced
> >> in 20.02, lets say foo(),
> >> but when application switches to .so.20.2 (released via DPDK_20.05), application
> >> will fail because of ABI breakage in foo().
> >>
> >> I think it is fair that application expects forward compatibility in minor
> >> versions of a shared library.
> >> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> >> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> >>
> >> If we all agree on this, we may need to tweak the process a little, but before
> >> diving into implementation details, I would like to be sure we are in same page.
> >>
> > 
> > Well, any new API's generally come in as experimental, in which case
> > changes are allowed, and breakage can be expected. If they are not
> > experiemental, then the ABI policy applies to them in that they cannot
> > change since they are part of the .21 ABI, even if that ABI is not fully
> > complete yet. For any application only using stable, non-experimental
> > functions, forward compatibility must be maintained IMHO.
> > 
> 
> Talking about not experimental APIs, experimental ones free from the process.
> 
> And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
> it should be supported for following ABI_20.x, instead of calling it ABI_21, and
> this minor tweak (and mind shift) in .map files can be our solution.

Related at what to do with adding versions between major ABI versions, when
investigating with Kevin the ABI checking we have made an unpleasant
discovery:

This minor version bumping from 20.0 to 20.1 has apparently already broken
our ABI according to libabigail.

The Gory Details [skip to the end for suggestions to fix]
------------------------------------------------------------

The reason for this is that the soversion encoded in each library - whether
built with meson or make - is the full 20.0 version, not just the major ABI
.20 part. Then when apps link against DPDK, they actually encode the 20.0.

So what this means is that currently - using a make build as an example
here - ldd on the latest head build gives:

 LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
        linux-vdso.so.1 (0x00007fff6813d000)
        librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
        librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
        librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
        librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
        librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
        librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
        librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
        librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
        librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
...

Similarly ldd on a 19.11 checkout gives:

LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
        linux-vdso.so.1 (0x00007ffc2a964000)
        librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
        librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
        librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
        librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
        librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
        librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
        librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
        librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
        librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)

The final check - using the 19.11 compiled testpmd with the library path
set to 20.02 versionned libs:

LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
        linux-vdso.so.1 (0x00007ffc711fc000)
        librte_pmd_bond.so.20.0 => not found
        librte_pmd_dpaa.so.20.0 => not found
        librte_mempool_dpaa.so.20.0 => not found
        librte_pmd_ixgbe.so.20.0 => not found
        librte_pmd_i40e.so.20.0 => not found
        librte_pmd_bnxt.so.20.0 => not found
        librte_pmd_softnic.so.20.0 => not found
        librte_flow_classify.so.0.200 => not found
        librte_pipeline.so.20.0 => not found

Fixing This
-----------

To fix this, we need to ensure that the SONAME remains constant across the
releases. Therefore, I currently see two options:

1. keep 20.0 as the version and soname across all releases in 2020, i.e.
  just revert the ABIVERSION change patch. Trouble there is how to track
  20.02 vs 20.05 etc. etc.

2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
  advantage of keeping the existing planned schemes, but has the really big
  downside of breaking ABI compatibility with anyone who has already
  compiled with 19.11.

Personally, of the two options - unless someone can come up with a third
option - I'd tend towards the second, fixing the builds to remove the .0 in
the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
adoption. Since this ABI stability is new, teething problems may be
expected.

Thoughts and comments?
/Bruce

BTW: For meson, the patch for option 2 is just to remove the so_version
variable and all references to it from lib/meson.build and
drivers/meson.build. Haven't looked into a "make" fix yet.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Luca Boccassi]

On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > Hi,
> > > > 
> > > > With new process, the major ABI releases will be compatible
> > > > until it is
> > > > deprecated (until next LTS for now),
> > > > like current ABI version is 20 in DPDK_19.11 and DPDK versions
> > > > until DPDK_20.11
> > > > will be ABI compatible with this version.
> > > > 
> > > > But if we introduce a new API after major ABI, say in 20.02
> > > > release, are we
> > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > 
> > > > If we allow it break, following problem will be observed:
> > > > Assume an application using .so.20.1 library, and using the new
> > > > API introduced
> > > > in 20.02, lets say foo(),
> > > > but when application switches to .so.20.2 (released via
> > > > DPDK_20.05), application
> > > > will fail because of ABI breakage in foo().
> > > > 
> > > > I think it is fair that application expects forward
> > > > compatibility in minor
> > > > versions of a shared library.
> > > > Like if application linked against .so.20.2, fair to expect
> > > > .so.20.3, .so.20.4
> > > > etc will work fine. I think currently only .so.20.0 is fully
> > > > forward compatible.
> > > > 
> > > > If we all agree on this, we may need to tweak the process a
> > > > little, but before
> > > > diving into implementation details, I would like to be sure we
> > > > are in same page.
> > > > 
> > > 
> > > Well, any new API's generally come in as experimental, in which
> > > case
> > > changes are allowed, and breakage can be expected. If they are
> > > not
> > > experiemental, then the ABI policy applies to them in that they
> > > cannot
> > > change since they are part of the .21 ABI, even if that ABI is
> > > not fully
> > > complete yet. For any application only using stable, non-
> > > experimental
> > > functions, forward compatibility must be maintained IMHO.
> > > 
> > 
> > Talking about not experimental APIs, experimental ones free from
> > the process.
> > 
> > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > ABI_20, because
> > it should be supported for following ABI_20.x, instead of calling
> > it ABI_21, and
> > this minor tweak (and mind shift) in .map files can be our
> > solution.
> 
> Related at what to do with adding versions between major ABI
> versions, when
> investigating with Kevin the ABI checking we have made an unpleasant
> discovery:
> 
> This minor version bumping from 20.0 to 20.1 has apparently already
> broken
> our ABI according to libabigail.
> 
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
> 
> The reason for this is that the soversion encoded in each library -
> whether
> built with meson or make - is the full 20.0 version, not just the
> major ABI
> .20 part. Then when apps link against DPDK, they actually encode the
> 20.0.
> 
> So what this means is that currently - using a make build as an
> example
> here - ldd on the latest head build gives:
> 
>  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> native-linux-gcc/app/testpmd | head
>         linux-vdso.so.1 (0x00007fff6813d000)
>         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
>         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
>         librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
>         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
>         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
>         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
>         librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
>         librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_flow_classify.so.0.201
> (0x00007f36d70b1000)
>         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> ...
> 
> Similarly ldd on a 19.11 checkout gives:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-
> native-linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc2a964000)
>         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> (0x00007fd4dc6b6000)
>         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> (0x00007fd4dc6a3000)
>         librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> (0x00007fd4dc69e000)
>         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> (0x00007fd4dc634000)
>         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> (0x00007fd4dc5a0000)
>         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> (0x00007fd4dc55d000)
>         librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> (0x00007fd4dc531000)
>         librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> (0x00007fd4dc52b000)
>         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> (0x00007fd4dc502000)
> 
> The final check - using the 19.11 compiled testpmd with the library
> path
> set to 20.02 versionned libs:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc711fc000)
>         librte_pmd_bond.so.20.0 => not found
>         librte_pmd_dpaa.so.20.0 => not found
>         librte_mempool_dpaa.so.20.0 => not found
>         librte_pmd_ixgbe.so.20.0 => not found
>         librte_pmd_i40e.so.20.0 => not found
>         librte_pmd_bnxt.so.20.0 => not found
>         librte_pmd_softnic.so.20.0 => not found
>         librte_flow_classify.so.0.200 => not found
>         librte_pipeline.so.20.0 => not found
> 
> Fixing This
> -----------
> 
> To fix this, we need to ensure that the SONAME remains constant
> across the
> releases. Therefore, I currently see two options:
> 
> 1. keep 20.0 as the version and soname across all releases in 2020,
> i.e.
>   just revert the ABIVERSION change patch. Trouble there is how to
> track
>   20.02 vs 20.05 etc. etc.
> 
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This
> has the
>   advantage of keeping the existing planned schemes, but has the
> really big
>   downside of breaking ABI compatibility with anyone who has already
>   compiled with 19.11.
> 
> Personally, of the two options - unless someone can come up with a
> third
> option - I'd tend towards the second, fixing the builds to remove the
> .0 in
> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> widespread
> adoption. Since this ABI stability is new, teething problems may be
> expected.
> 
> Thoughts and comments?
> /Bruce
> 
> BTW: For meson, the patch for option 2 is just to remove the
> so_version
> variable and all references to it from lib/meson.build and
> drivers/meson.build. Haven't looked into a "make" fix yet.

Hi,

With libtool and its (arguably arcane) format, only the first digit is
the ABI current version and gets encoded in the elf header. The other
digits can be used to track compatible minor increments, and are mostly
ignored. On the system a symlink libfoo.so.major ->
libfoo.so.major.minor is added.

Eg:

$ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep SONAME
 0x000000000000000e (SONAME)             Library soname: [libzmq.so.5]
$ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
gnu/libzmq.so.5 -> libzmq.so.5.2.3

Can we do the same? Not sure what the right incantation is for Meson,
but it should be possibly.

Also, we should leave the current at 20.0 - let's not break
compatibility already, please :-)

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/10/2019 2:36 PM, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
>> On 12/10/2019 12:04 PM, Bruce Richardson wrote:
>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>> Hi,
>>>>
>>>> With new process, the major ABI releases will be compatible until it is
>>>> deprecated (until next LTS for now),
>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>> will be ABI compatible with this version.
>>>>
>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>
>>>> If we allow it break, following problem will be observed:
>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>> in 20.02, lets say foo(),
>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>> will fail because of ABI breakage in foo().
>>>>
>>>> I think it is fair that application expects forward compatibility in minor
>>>> versions of a shared library.
>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>
>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>
>>>
>>> Well, any new API's generally come in as experimental, in which case
>>> changes are allowed, and breakage can be expected. If they are not
>>> experiemental, then the ABI policy applies to them in that they cannot
>>> change since they are part of the .21 ABI, even if that ABI is not fully
>>> complete yet. For any application only using stable, non-experimental
>>> functions, forward compatibility must be maintained IMHO.
>>>
>>
>> Talking about not experimental APIs, experimental ones free from the process.
>>
>> And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
>> it should be supported for following ABI_20.x, instead of calling it ABI_21, and
>> this minor tweak (and mind shift) in .map files can be our solution.
> 
> Related at what to do with adding versions between major ABI versions, when
> investigating with Kevin the ABI checking we have made an unpleasant
> discovery:
> 
> This minor version bumping from 20.0 to 20.1 has apparently already broken
> our ABI according to libabigail.
> 
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
> 
> The reason for this is that the soversion encoded in each library - whether
> built with meson or make - is the full 20.0 version, not just the major ABI
> .20 part. Then when apps link against DPDK, they actually encode the 20.0.
> 
> So what this means is that currently - using a make build as an example
> here - ldd on the latest head build gives:
> 
>  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
>         linux-vdso.so.1 (0x00007fff6813d000)
>         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
>         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
>         librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
>         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
>         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
>         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
>         librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
>         librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
>         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> ...
> 
> Similarly ldd on a 19.11 checkout gives:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc2a964000)
>         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
>         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
>         librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
>         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
>         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
>         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
>         librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
>         librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
>         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
> 
> The final check - using the 19.11 compiled testpmd with the library path
> set to 20.02 versionned libs:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc711fc000)
>         librte_pmd_bond.so.20.0 => not found
>         librte_pmd_dpaa.so.20.0 => not found
>         librte_mempool_dpaa.so.20.0 => not found
>         librte_pmd_ixgbe.so.20.0 => not found
>         librte_pmd_i40e.so.20.0 => not found
>         librte_pmd_bnxt.so.20.0 => not found
>         librte_pmd_softnic.so.20.0 => not found
>         librte_flow_classify.so.0.200 => not found
>         librte_pipeline.so.20.0 => not found
> 
> Fixing This
> -----------
> 
> To fix this, we need to ensure that the SONAME remains constant across the
> releases. Therefore, I currently see two options:
> 
> 1. keep 20.0 as the version and soname across all releases in 2020, i.e.
>   just revert the ABIVERSION change patch. Trouble there is how to track
>   20.02 vs 20.05 etc. etc.
> 
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
>   advantage of keeping the existing planned schemes, but has the really big
>   downside of breaking ABI compatibility with anyone who has already
>   compiled with 19.11.
> 
> Personally, of the two options - unless someone can come up with a third
> option - I'd tend towards the second, fixing the builds to remove the .0 in
> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
> adoption. Since this ABI stability is new, teething problems may be
> expected.
> 
> Thoughts and comments?

Arghh :(

I agree that 2) is proper fix and if we want to fix properly we should do a
19.11.1 ASAP.

Can having soname as "20.0" but having file name as .so.20.1 work?
If it does we can "workaround" it by keeping SONAME as "20.0" (instead of 20)
during the life time of the ABI_20 and fix it properly in ABI_21.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/10/2019 3:04 PM, Ferruh Yigit wrote:
> On 12/10/2019 2:36 PM, Bruce Richardson wrote:
>> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
>>> On 12/10/2019 12:04 PM, Bruce Richardson wrote:
>>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>>> Hi,
>>>>>
>>>>> With new process, the major ABI releases will be compatible until it is
>>>>> deprecated (until next LTS for now),
>>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>>> will be ABI compatible with this version.
>>>>>
>>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>>
>>>>> If we allow it break, following problem will be observed:
>>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>>> in 20.02, lets say foo(),
>>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>>> will fail because of ABI breakage in foo().
>>>>>
>>>>> I think it is fair that application expects forward compatibility in minor
>>>>> versions of a shared library.
>>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>>
>>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>>
>>>>
>>>> Well, any new API's generally come in as experimental, in which case
>>>> changes are allowed, and breakage can be expected. If they are not
>>>> experiemental, then the ABI policy applies to them in that they cannot
>>>> change since they are part of the .21 ABI, even if that ABI is not fully
>>>> complete yet. For any application only using stable, non-experimental
>>>> functions, forward compatibility must be maintained IMHO.
>>>>
>>>
>>> Talking about not experimental APIs, experimental ones free from the process.
>>>
>>> And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
>>> it should be supported for following ABI_20.x, instead of calling it ABI_21, and
>>> this minor tweak (and mind shift) in .map files can be our solution.
>>
>> Related at what to do with adding versions between major ABI versions, when
>> investigating with Kevin the ABI checking we have made an unpleasant
>> discovery:
>>
>> This minor version bumping from 20.0 to 20.1 has apparently already broken
>> our ABI according to libabigail.
>>
>> The Gory Details [skip to the end for suggestions to fix]
>> ------------------------------------------------------------
>>
>> The reason for this is that the soversion encoded in each library - whether
>> built with meson or make - is the full 20.0 version, not just the major ABI
>> .20 part. Then when apps link against DPDK, they actually encode the 20.0.
>>
>> So what this means is that currently - using a make build as an example
>> here - ldd on the latest head build gives:
>>
>>  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
>>         linux-vdso.so.1 (0x00007fff6813d000)
>>         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
>>         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
>>         librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
>>         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
>>         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
>>         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
>>         librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
>>         librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
>>         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
>> ...
>>
>> Similarly ldd on a 19.11 checkout gives:
>>
>> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>>         linux-vdso.so.1 (0x00007ffc2a964000)
>>         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
>>         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
>>         librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
>>         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
>>         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
>>         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
>>         librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
>>         librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
>>         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
>>
>> The final check - using the 19.11 compiled testpmd with the library path
>> set to 20.02 versionned libs:
>>
>> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>>         linux-vdso.so.1 (0x00007ffc711fc000)
>>         librte_pmd_bond.so.20.0 => not found
>>         librte_pmd_dpaa.so.20.0 => not found
>>         librte_mempool_dpaa.so.20.0 => not found
>>         librte_pmd_ixgbe.so.20.0 => not found
>>         librte_pmd_i40e.so.20.0 => not found
>>         librte_pmd_bnxt.so.20.0 => not found
>>         librte_pmd_softnic.so.20.0 => not found
>>         librte_flow_classify.so.0.200 => not found
>>         librte_pipeline.so.20.0 => not found
>>
>> Fixing This
>> -----------
>>
>> To fix this, we need to ensure that the SONAME remains constant across the
>> releases. Therefore, I currently see two options:
>>
>> 1. keep 20.0 as the version and soname across all releases in 2020, i.e.
>>   just revert the ABIVERSION change patch. Trouble there is how to track
>>   20.02 vs 20.05 etc. etc.
>>
>> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
>>   advantage of keeping the existing planned schemes, but has the really big
>>   downside of breaking ABI compatibility with anyone who has already
>>   compiled with 19.11.
>>
>> Personally, of the two options - unless someone can come up with a third
>> option - I'd tend towards the second, fixing the builds to remove the .0 in
>> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
>> adoption. Since this ABI stability is new, teething problems may be
>> expected.
>>
>> Thoughts and comments?
> 
> Arghh :(
> 
> I agree that 2) is proper fix and if we want to fix properly we should do a
> 19.11.1 ASAP.
> 
> Can having soname as "20.0" but having file name as .so.20.1 work?

No it is not working, dynamic linker is looking for the .so file that has been
put into soname.

So I am for option 2 and 19.11.1.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Kinsella, Ray]

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Tuesday 10 December 2019 14:37
> To: Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: Kinsella, Ray <ray.kinsella@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>; David Marchand <david.marchand@redhat.com>; Luca
> Boccassi <bluca@debian.org>; Christian Ehrhardt
> <christian.ehrhardt@canonical.com>; Timothy Redaelli
> <tredaelli@redhat.com>; Kevin Traynor <ktraynor@redhat.com>; dpdk-dev
> <dev@dpdk.org>; Laatz, Kevin <kevin.laatz@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; Neil Horman <nhorman@redhat.com>
> Subject: Re: How to manage new APIs added after major ABI release?
> 
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > >> Hi,
> > >>
> > >> With new process, the major ABI releases will be compatible until
> > >> it is deprecated (until next LTS for now), like current ABI
> version
> > >> is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11 will be ABI
> > >> compatible with this version.
> > >>
> > >> But if we introduce a new API after major ABI, say in 20.02
> > >> release, are we allowed to break the ABI for that API before
> DPDK_20.11?
> > >>
> > >> If we allow it break, following problem will be observed:
> > >> Assume an application using .so.20.1 library, and using the new
> API
> > >> introduced in 20.02, lets say foo(), but when application switches
> > >> to .so.20.2 (released via DPDK_20.05), application will fail
> > >> because of ABI breakage in foo().
> > >>
> > >> I think it is fair that application expects forward compatibility
> > >> in minor versions of a shared library.
> > >> Like if application linked against .so.20.2, fair to expect
> > >> .so.20.3, .so.20.4 etc will work fine. I think currently only
> .so.20.0 is fully forward compatible.
> > >>
> > >> If we all agree on this, we may need to tweak the process a
> little,
> > >> but before diving into implementation details, I would like to be
> sure we are in same page.
> > >>
> > >
> > > Well, any new API's generally come in as experimental, in which
> case
> > > changes are allowed, and breakage can be expected. If they are not
> > > experiemental, then the ABI policy applies to them in that they
> > > cannot change since they are part of the .21 ABI, even if that ABI
> > > is not fully complete yet. For any application only using stable,
> > > non-experimental functions, forward compatibility must be
> maintained IMHO.
> > >
> >
> > Talking about not experimental APIs, experimental ones free from the
> process.
> >
> > And when and API added in 20.02 (ABI_20.1) it is kind of still
> ABI_20,
> > because it should be supported for following ABI_20.x, instead of
> > calling it ABI_21, and this minor tweak (and mind shift) in .map
> files can be our solution.
> 
> Related at what to do with adding versions between major ABI versions,
> when investigating with Kevin the ABI checking we have made an
> unpleasant
> discovery:
> 
> This minor version bumping from 20.0 to 20.1 has apparently already
> broken our ABI according to libabigail.
> 
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
> 
> The reason for this is that the soversion encoded in each library -
> whether built with meson or make - is the full 20.0 version, not just
> the major ABI
> .20 part. Then when apps link against DPDK, they actually encode the
> 20.0.
> 
> So what this means is that currently - using a make build as an example
> here - ldd on the latest head build gives:
> 
>  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> linux-gcc/app/testpmd | head
>         linux-vdso.so.1 (0x00007fff6813d000)
>         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
>         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
>         librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
>         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
>         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
>         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
>         librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
>         librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
>         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000) ...
> 
> Similarly ldd on a 19.11 checkout gives:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-
> native-linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc2a964000)
>         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
>         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
>         librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> (0x00007fd4dc69e000)
>         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
>         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
>         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
>         librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> (0x00007fd4dc531000)
>         librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> (0x00007fd4dc52b000)
>         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
> 
> The final check - using the 19.11 compiled testpmd with the library
> path set to 20.02 versionned libs:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc711fc000)
>         librte_pmd_bond.so.20.0 => not found
>         librte_pmd_dpaa.so.20.0 => not found
>         librte_mempool_dpaa.so.20.0 => not found
>         librte_pmd_ixgbe.so.20.0 => not found
>         librte_pmd_i40e.so.20.0 => not found
>         librte_pmd_bnxt.so.20.0 => not found
>         librte_pmd_softnic.so.20.0 => not found
>         librte_flow_classify.so.0.200 => not found
>         librte_pipeline.so.20.0 => not found
> 
> Fixing This
> -----------
> 
> To fix this, we need to ensure that the SONAME remains constant across
> the releases. Therefore, I currently see two options:
> 
> 1. keep 20.0 as the version and soname across all releases in 2020,
> i.e.
>   just revert the ABIVERSION change patch. Trouble there is how to
> track
>   20.02 vs 20.05 etc. etc.
> 
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has
> the
>   advantage of keeping the existing planned schemes, but has the really
> big
>   downside of breaking ABI compatibility with anyone who has already
>   compiled with 19.11.
> 
> Personally, of the two options - unless someone can come up with a
> third option - I'd tend towards the second, fixing the builds to remove
> the .0 in the soname, and releasing that ASAP as 19.11.1 before 19.11
> gets widespread adoption. Since this ABI stability is new, teething
> problems may be expected.
> 
> Thoughts and comments?
> /Bruce
> 
> BTW: For meson, the patch for option 2 is just to remove the so_version
> variable and all references to it from lib/meson.build and
> drivers/meson.build. Haven't looked into a "make" fix yet.

From the DPDK ABI Version Document ...

A library's soname. is typically used to provide backward compatibility information about a given library, describing the lowest common denominator ABI supported by the library. The soname or logical name for the library, is typically comprised of the library's name and major version e.g. librte_eal.so.20

https://doc.dpdk.org/guides/contributing/abi_versioning.html

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > Hi,
> > > > > 
> > > > > With new process, the major ABI releases will be compatible
> > > > > until it is
> > > > > deprecated (until next LTS for now),
> > > > > like current ABI version is 20 in DPDK_19.11 and DPDK versions
> > > > > until DPDK_20.11
> > > > > will be ABI compatible with this version.
> > > > > 
> > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > release, are we
> > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > 
> > > > > If we allow it break, following problem will be observed:
> > > > > Assume an application using .so.20.1 library, and using the new
> > > > > API introduced
> > > > > in 20.02, lets say foo(),
> > > > > but when application switches to .so.20.2 (released via
> > > > > DPDK_20.05), application
> > > > > will fail because of ABI breakage in foo().
> > > > > 
> > > > > I think it is fair that application expects forward
> > > > > compatibility in minor
> > > > > versions of a shared library.
> > > > > Like if application linked against .so.20.2, fair to expect
> > > > > .so.20.3, .so.20.4
> > > > > etc will work fine. I think currently only .so.20.0 is fully
> > > > > forward compatible.
> > > > > 
> > > > > If we all agree on this, we may need to tweak the process a
> > > > > little, but before
> > > > > diving into implementation details, I would like to be sure we
> > > > > are in same page.
> > > > > 
> > > > 
> > > > Well, any new API's generally come in as experimental, in which
> > > > case
> > > > changes are allowed, and breakage can be expected. If they are
> > > > not
> > > > experiemental, then the ABI policy applies to them in that they
> > > > cannot
> > > > change since they are part of the .21 ABI, even if that ABI is
> > > > not fully
> > > > complete yet. For any application only using stable, non-
> > > > experimental
> > > > functions, forward compatibility must be maintained IMHO.
> > > > 
> > > 
> > > Talking about not experimental APIs, experimental ones free from
> > > the process.
> > > 
> > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > ABI_20, because
> > > it should be supported for following ABI_20.x, instead of calling
> > > it ABI_21, and
> > > this minor tweak (and mind shift) in .map files can be our
> > > solution.
> > 
> > Related at what to do with adding versions between major ABI
> > versions, when
> > investigating with Kevin the ABI checking we have made an unpleasant
> > discovery:
> > 
> > This minor version bumping from 20.0 to 20.1 has apparently already
> > broken
> > our ABI according to libabigail.
> > 
> > The Gory Details [skip to the end for suggestions to fix]
> > ------------------------------------------------------------
> > 
> > The reason for this is that the soversion encoded in each library -
> > whether
> > built with meson or make - is the full 20.0 version, not just the
> > major ABI
> > .20 part. Then when apps link against DPDK, they actually encode the
> > 20.0.
> > 
> > So what this means is that currently - using a make build as an
> > example
> > here - ldd on the latest head build gives:
> > 
> >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > native-linux-gcc/app/testpmd | head
> >         linux-vdso.so.1 (0x00007fff6813d000)
> >         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> >         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> >         librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
> >         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
> >         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> >         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> >         librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
> >         librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > (0x00007f36d70b1000)
> >         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > ...
> > 
> > Similarly ldd on a 19.11 checkout gives:
> > 
> > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-
> > native-linux-gcc_v19.11/app/testpmd | head
> >         linux-vdso.so.1 (0x00007ffc2a964000)
> >         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > (0x00007fd4dc6b6000)
> >         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > (0x00007fd4dc6a3000)
> >         librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > (0x00007fd4dc69e000)
> >         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > (0x00007fd4dc634000)
> >         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > (0x00007fd4dc5a0000)
> >         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > (0x00007fd4dc55d000)
> >         librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > (0x00007fd4dc531000)
> >         librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > (0x00007fd4dc52b000)
> >         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > (0x00007fd4dc502000)
> > 
> > The final check - using the 19.11 compiled testpmd with the library
> > path
> > set to 20.02 versionned libs:
> > 
> > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> > linux-gcc_v19.11/app/testpmd | head
> >         linux-vdso.so.1 (0x00007ffc711fc000)
> >         librte_pmd_bond.so.20.0 => not found
> >         librte_pmd_dpaa.so.20.0 => not found
> >         librte_mempool_dpaa.so.20.0 => not found
> >         librte_pmd_ixgbe.so.20.0 => not found
> >         librte_pmd_i40e.so.20.0 => not found
> >         librte_pmd_bnxt.so.20.0 => not found
> >         librte_pmd_softnic.so.20.0 => not found
> >         librte_flow_classify.so.0.200 => not found
> >         librte_pipeline.so.20.0 => not found
> > 
> > Fixing This
> > -----------
> > 
> > To fix this, we need to ensure that the SONAME remains constant
> > across the
> > releases. Therefore, I currently see two options:
> > 
> > 1. keep 20.0 as the version and soname across all releases in 2020,
> > i.e.
> >   just revert the ABIVERSION change patch. Trouble there is how to
> > track
> >   20.02 vs 20.05 etc. etc.
> > 
> > 2. remove the .0, .1 from the SONAMES stored in the libraries. This
> > has the
> >   advantage of keeping the existing planned schemes, but has the
> > really big
> >   downside of breaking ABI compatibility with anyone who has already
> >   compiled with 19.11.
> > 
> > Personally, of the two options - unless someone can come up with a
> > third
> > option - I'd tend towards the second, fixing the builds to remove the
> > .0 in
> > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > widespread
> > adoption. Since this ABI stability is new, teething problems may be
> > expected.
> > 
> > Thoughts and comments?
> > /Bruce
> > 
> > BTW: For meson, the patch for option 2 is just to remove the
> > so_version
> > variable and all references to it from lib/meson.build and
> > drivers/meson.build. Haven't looked into a "make" fix yet.
> 
> Hi,
> 
> With libtool and its (arguably arcane) format, only the first digit is
> the ABI current version and gets encoded in the elf header. The other
> digits can be used to track compatible minor increments, and are mostly
> ignored. On the system a symlink libfoo.so.major ->
> libfoo.so.major.minor is added.
> 
> Eg:
> 
> $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep SONAME
>  0x000000000000000e (SONAME)             Library soname: [libzmq.so.5]
> $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> gnu/libzmq.so.5 -> libzmq.so.5.2.3
> 
> Can we do the same? Not sure what the right incantation is for Meson,
> but it should be possibly.
>

That's essentially option 2, and it's still an ABI break because existing
builds of 19.11 have the soname will the full version number in it. The
default behaviour for meson is exactly how you described it, except that
previously we needed more exact control over the version info (for your
dpdk-specific versions in the sonames) and so overrode the soversion
explicitly. The fix for meson is to remove this overriding i.e. remove
"soversion:" parameter for each shared_library() call.

 
> Also, we should leave the current at 20.0 - let's not break
> compatibility already, please :-)
> 

If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Luca Boccassi]

On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > With new process, the major ABI releases will be compatible
> > > > > > until it is
> > > > > > deprecated (until next LTS for now),
> > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > versions
> > > > > > until DPDK_20.11
> > > > > > will be ABI compatible with this version.
> > > > > > 
> > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > release, are we
> > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > 
> > > > > > If we allow it break, following problem will be observed:
> > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > new
> > > > > > API introduced
> > > > > > in 20.02, lets say foo(),
> > > > > > but when application switches to .so.20.2 (released via
> > > > > > DPDK_20.05), application
> > > > > > will fail because of ABI breakage in foo().
> > > > > > 
> > > > > > I think it is fair that application expects forward
> > > > > > compatibility in minor
> > > > > > versions of a shared library.
> > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > .so.20.3, .so.20.4
> > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > fully
> > > > > > forward compatible.
> > > > > > 
> > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > little, but before
> > > > > > diving into implementation details, I would like to be sure
> > > > > > we
> > > > > > are in same page.
> > > > > > 
> > > > > 
> > > > > Well, any new API's generally come in as experimental, in
> > > > > which
> > > > > case
> > > > > changes are allowed, and breakage can be expected. If they
> > > > > are
> > > > > not
> > > > > experiemental, then the ABI policy applies to them in that
> > > > > they
> > > > > cannot
> > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > is
> > > > > not fully
> > > > > complete yet. For any application only using stable, non-
> > > > > experimental
> > > > > functions, forward compatibility must be maintained IMHO.
> > > > > 
> > > > 
> > > > Talking about not experimental APIs, experimental ones free
> > > > from
> > > > the process.
> > > > 
> > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > ABI_20, because
> > > > it should be supported for following ABI_20.x, instead of
> > > > calling
> > > > it ABI_21, and
> > > > this minor tweak (and mind shift) in .map files can be our
> > > > solution.
> > > 
> > > Related at what to do with adding versions between major ABI
> > > versions, when
> > > investigating with Kevin the ABI checking we have made an
> > > unpleasant
> > > discovery:
> > > 
> > > This minor version bumping from 20.0 to 20.1 has apparently
> > > already
> > > broken
> > > our ABI according to libabigail.
> > > 
> > > The Gory Details [skip to the end for suggestions to fix]
> > > ------------------------------------------------------------
> > > 
> > > The reason for this is that the soversion encoded in each library
> > > -
> > > whether
> > > built with meson or make - is the full 20.0 version, not just the
> > > major ABI
> > > .20 part. Then when apps link against DPDK, they actually encode
> > > the
> > > 20.0.
> > > 
> > > So what this means is that currently - using a make build as an
> > > example
> > > here - ldd on the latest head build gives:
> > > 
> > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > native-linux-gcc/app/testpmd | head
> > >         linux-vdso.so.1 (0x00007fff6813d000)
> > >         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > >         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > >         librte_mempool_dpaa.so.20.1 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > (0x00007f36d7224000)
> > >         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > (0x00007f36d71ba000)
> > >         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > >         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > >         librte_pmd_softnic.so.20.1 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > (0x00007f36d70b7000)
> > >         librte_flow_classify.so.0.201 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > (0x00007f36d70b1000)
> > >         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > ...
> > > 
> > > Similarly ldd on a 19.11 checkout gives:
> > > 
> > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > x86_64-
> > > native-linux-gcc_v19.11/app/testpmd | head
> > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > >         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > (0x00007fd4dc6b6000)
> > >         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > (0x00007fd4dc6a3000)
> > >         librte_mempool_dpaa.so.20.0 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > (0x00007fd4dc69e000)
> > >         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > (0x00007fd4dc634000)
> > >         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > (0x00007fd4dc5a0000)
> > >         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > (0x00007fd4dc55d000)
> > >         librte_pmd_softnic.so.20.0 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > (0x00007fd4dc531000)
> > >         librte_flow_classify.so.0.200 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > (0x00007fd4dc52b000)
> > >         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > (0x00007fd4dc502000)
> > > 
> > > The final check - using the 19.11 compiled testpmd with the
> > > library
> > > path
> > > set to 20.02 versionned libs:
> > > 
> > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > native-
> > > linux-gcc_v19.11/app/testpmd | head
> > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > >         librte_pmd_bond.so.20.0 => not found
> > >         librte_pmd_dpaa.so.20.0 => not found
> > >         librte_mempool_dpaa.so.20.0 => not found
> > >         librte_pmd_ixgbe.so.20.0 => not found
> > >         librte_pmd_i40e.so.20.0 => not found
> > >         librte_pmd_bnxt.so.20.0 => not found
> > >         librte_pmd_softnic.so.20.0 => not found
> > >         librte_flow_classify.so.0.200 => not found
> > >         librte_pipeline.so.20.0 => not found
> > > 
> > > Fixing This
> > > -----------
> > > 
> > > To fix this, we need to ensure that the SONAME remains constant
> > > across the
> > > releases. Therefore, I currently see two options:
> > > 
> > > 1. keep 20.0 as the version and soname across all releases in
> > > 2020,
> > > i.e.
> > >   just revert the ABIVERSION change patch. Trouble there is how
> > > to
> > > track
> > >   20.02 vs 20.05 etc. etc.
> > > 
> > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > This
> > > has the
> > >   advantage of keeping the existing planned schemes, but has the
> > > really big
> > >   downside of breaking ABI compatibility with anyone who has
> > > already
> > >   compiled with 19.11.
> > > 
> > > Personally, of the two options - unless someone can come up with
> > > a
> > > third
> > > option - I'd tend towards the second, fixing the builds to remove
> > > the
> > > .0 in
> > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > widespread
> > > adoption. Since this ABI stability is new, teething problems may
> > > be
> > > expected.
> > > 
> > > Thoughts and comments?
> > > /Bruce
> > > 
> > > BTW: For meson, the patch for option 2 is just to remove the
> > > so_version
> > > variable and all references to it from lib/meson.build and
> > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > 
> > Hi,
> > 
> > With libtool and its (arguably arcane) format, only the first digit
> > is
> > the ABI current version and gets encoded in the elf header. The
> > other
> > digits can be used to track compatible minor increments, and are
> > mostly
> > ignored. On the system a symlink libfoo.so.major ->
> > libfoo.so.major.minor is added.
> > 
> > Eg:
> > 
> > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > SONAME
> >  0x000000000000000e (SONAME)             Library soname:
> > [libzmq.so.5]
> > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > 
> > Can we do the same? Not sure what the right incantation is for
> > Meson,
> > but it should be possibly.
> > 
> 
> That's essentially option 2, and it's still an ABI break because
> existing
> builds of 19.11 have the soname will the full version number in it.
> The
> default behaviour for meson is exactly how you described it, except
> that
> previously we needed more exact control over the version info (for
> your
> dpdk-specific versions in the sonames) and so overrode the soversion
> explicitly. The fix for meson is to remove this overriding i.e.
> remove
> "soversion:" parameter for each shared_library() call.
> 
>  
> > Also, we should leave the current at 20.0 - let's not break
> > compatibility already, please :-)
> > 
> 
> If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?

Yes, that's what I meant - IMHO we should just take the hit and use the
slightly weird 20.0 format until next year, and add a third digit for
compatible updates. Then for v21 we can drop it.

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > With new process, the major ABI releases will be compatible
> > > > > > > until it is
> > > > > > > deprecated (until next LTS for now),
> > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > versions
> > > > > > > until DPDK_20.11
> > > > > > > will be ABI compatible with this version.
> > > > > > > 
> > > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > > release, are we
> > > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > > 
> > > > > > > If we allow it break, following problem will be observed:
> > > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > > new
> > > > > > > API introduced
> > > > > > > in 20.02, lets say foo(),
> > > > > > > but when application switches to .so.20.2 (released via
> > > > > > > DPDK_20.05), application
> > > > > > > will fail because of ABI breakage in foo().
> > > > > > > 
> > > > > > > I think it is fair that application expects forward
> > > > > > > compatibility in minor
> > > > > > > versions of a shared library.
> > > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > > .so.20.3, .so.20.4
> > > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > > fully
> > > > > > > forward compatible.
> > > > > > > 
> > > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > > little, but before
> > > > > > > diving into implementation details, I would like to be sure
> > > > > > > we
> > > > > > > are in same page.
> > > > > > > 
> > > > > > 
> > > > > > Well, any new API's generally come in as experimental, in
> > > > > > which
> > > > > > case
> > > > > > changes are allowed, and breakage can be expected. If they
> > > > > > are
> > > > > > not
> > > > > > experiemental, then the ABI policy applies to them in that
> > > > > > they
> > > > > > cannot
> > > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > > is
> > > > > > not fully
> > > > > > complete yet. For any application only using stable, non-
> > > > > > experimental
> > > > > > functions, forward compatibility must be maintained IMHO.
> > > > > > 
> > > > > 
> > > > > Talking about not experimental APIs, experimental ones free
> > > > > from
> > > > > the process.
> > > > > 
> > > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > > ABI_20, because
> > > > > it should be supported for following ABI_20.x, instead of
> > > > > calling
> > > > > it ABI_21, and
> > > > > this minor tweak (and mind shift) in .map files can be our
> > > > > solution.
> > > > 
> > > > Related at what to do with adding versions between major ABI
> > > > versions, when
> > > > investigating with Kevin the ABI checking we have made an
> > > > unpleasant
> > > > discovery:
> > > > 
> > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > already
> > > > broken
> > > > our ABI according to libabigail.
> > > > 
> > > > The Gory Details [skip to the end for suggestions to fix]
> > > > ------------------------------------------------------------
> > > > 
> > > > The reason for this is that the soversion encoded in each library
> > > > -
> > > > whether
> > > > built with meson or make - is the full 20.0 version, not just the
> > > > major ABI
> > > > .20 part. Then when apps link against DPDK, they actually encode
> > > > the
> > > > 20.0.
> > > > 
> > > > So what this means is that currently - using a make build as an
> > > > example
> > > > here - ldd on the latest head build gives:
> > > > 
> > > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-linux-gcc/app/testpmd | head
> > > >         linux-vdso.so.1 (0x00007fff6813d000)
> > > >         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > >         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > >         librte_mempool_dpaa.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > (0x00007f36d7224000)
> > > >         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > (0x00007f36d71ba000)
> > > >         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > >         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > >         librte_pmd_softnic.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > (0x00007f36d70b7000)
> > > >         librte_flow_classify.so.0.201 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > (0x00007f36d70b1000)
> > > >         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > > ...
> > > > 
> > > > Similarly ldd on a 19.11 checkout gives:
> > > > 
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > x86_64-
> > > > native-linux-gcc_v19.11/app/testpmd | head
> > > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > > >         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > (0x00007fd4dc6b6000)
> > > >         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > (0x00007fd4dc6a3000)
> > > >         librte_mempool_dpaa.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > (0x00007fd4dc69e000)
> > > >         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > (0x00007fd4dc634000)
> > > >         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > (0x00007fd4dc5a0000)
> > > >         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > (0x00007fd4dc55d000)
> > > >         librte_pmd_softnic.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > (0x00007fd4dc531000)
> > > >         librte_flow_classify.so.0.200 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > (0x00007fd4dc52b000)
> > > >         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > (0x00007fd4dc502000)
> > > > 
> > > > The final check - using the 19.11 compiled testpmd with the
> > > > library
> > > > path
> > > > set to 20.02 versionned libs:
> > > > 
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-
> > > > linux-gcc_v19.11/app/testpmd | head
> > > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > > >         librte_pmd_bond.so.20.0 => not found
> > > >         librte_pmd_dpaa.so.20.0 => not found
> > > >         librte_mempool_dpaa.so.20.0 => not found
> > > >         librte_pmd_ixgbe.so.20.0 => not found
> > > >         librte_pmd_i40e.so.20.0 => not found
> > > >         librte_pmd_bnxt.so.20.0 => not found
> > > >         librte_pmd_softnic.so.20.0 => not found
> > > >         librte_flow_classify.so.0.200 => not found
> > > >         librte_pipeline.so.20.0 => not found
> > > > 
> > > > Fixing This
> > > > -----------
> > > > 
> > > > To fix this, we need to ensure that the SONAME remains constant
> > > > across the
> > > > releases. Therefore, I currently see two options:
> > > > 
> > > > 1. keep 20.0 as the version and soname across all releases in
> > > > 2020,
> > > > i.e.
> > > >   just revert the ABIVERSION change patch. Trouble there is how
> > > > to
> > > > track
> > > >   20.02 vs 20.05 etc. etc.
> > > > 
> > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > This
> > > > has the
> > > >   advantage of keeping the existing planned schemes, but has the
> > > > really big
> > > >   downside of breaking ABI compatibility with anyone who has
> > > > already
> > > >   compiled with 19.11.
> > > > 
> > > > Personally, of the two options - unless someone can come up with
> > > > a
> > > > third
> > > > option - I'd tend towards the second, fixing the builds to remove
> > > > the
> > > > .0 in
> > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > widespread
> > > > adoption. Since this ABI stability is new, teething problems may
> > > > be
> > > > expected.
> > > > 
> > > > Thoughts and comments?
> > > > /Bruce
> > > > 
> > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > so_version
> > > > variable and all references to it from lib/meson.build and
> > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > 
> > > Hi,
> > > 
> > > With libtool and its (arguably arcane) format, only the first digit
> > > is
> > > the ABI current version and gets encoded in the elf header. The
> > > other
> > > digits can be used to track compatible minor increments, and are
> > > mostly
> > > ignored. On the system a symlink libfoo.so.major ->
> > > libfoo.so.major.minor is added.
> > > 
> > > Eg:
> > > 
> > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > SONAME
> > >  0x000000000000000e (SONAME)             Library soname:
> > > [libzmq.so.5]
> > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > 
> > > Can we do the same? Not sure what the right incantation is for
> > > Meson,
> > > but it should be possibly.
> > > 
> > 
> > That's essentially option 2, and it's still an ABI break because
> > existing
> > builds of 19.11 have the soname will the full version number in it.
> > The
> > default behaviour for meson is exactly how you described it, except
> > that
> > previously we needed more exact control over the version info (for
> > your
> > dpdk-specific versions in the sonames) and so overrode the soversion
> > explicitly. The fix for meson is to remove this overriding i.e.
> > remove
> > "soversion:" parameter for each shared_library() call.
> > 
> >  
> > > Also, we should leave the current at 20.0 - let's not break
> > > compatibility already, please :-)
> > > 
> > 
> > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> 
> Yes, that's what I meant - IMHO we should just take the hit and use the
> slightly weird 20.0 format until next year, and add a third digit for
> compatible updates. Then for v21 we can drop it.
>
My concern with that is us forgetting, because we'll put in place hacks to
have the soversion be the first two numbers of the version. Then we need to
remember to remove those hacks before 20.11 goes out or we'll end up with
21.0 being the soversion again.

For that reason, I'd rather see us fix it now before 19.11 gets widely
adopted.

/Bruce

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Thomas Monjalon]

10/12/2019 17:20, Luca Boccassi:
> On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > With new process, the major ABI releases will be compatible
> > > > > > > until it is
> > > > > > > deprecated (until next LTS for now),
> > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > versions
> > > > > > > until DPDK_20.11
> > > > > > > will be ABI compatible with this version.
> > > > > > > 
> > > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > > release, are we
> > > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > > 
> > > > > > > If we allow it break, following problem will be observed:
> > > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > > new
> > > > > > > API introduced
> > > > > > > in 20.02, lets say foo(),
> > > > > > > but when application switches to .so.20.2 (released via
> > > > > > > DPDK_20.05), application
> > > > > > > will fail because of ABI breakage in foo().
> > > > > > > 
> > > > > > > I think it is fair that application expects forward
> > > > > > > compatibility in minor
> > > > > > > versions of a shared library.
> > > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > > .so.20.3, .so.20.4
> > > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > > fully
> > > > > > > forward compatible.
> > > > > > > 
> > > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > > little, but before
> > > > > > > diving into implementation details, I would like to be sure
> > > > > > > we
> > > > > > > are in same page.
> > > > > > > 
> > > > > > 
> > > > > > Well, any new API's generally come in as experimental, in
> > > > > > which
> > > > > > case
> > > > > > changes are allowed, and breakage can be expected. If they
> > > > > > are
> > > > > > not
> > > > > > experiemental, then the ABI policy applies to them in that
> > > > > > they
> > > > > > cannot
> > > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > > is
> > > > > > not fully
> > > > > > complete yet. For any application only using stable, non-
> > > > > > experimental
> > > > > > functions, forward compatibility must be maintained IMHO.
> > > > > > 
> > > > > 
> > > > > Talking about not experimental APIs, experimental ones free
> > > > > from
> > > > > the process.
> > > > > 
> > > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > > ABI_20, because
> > > > > it should be supported for following ABI_20.x, instead of
> > > > > calling
> > > > > it ABI_21, and
> > > > > this minor tweak (and mind shift) in .map files can be our
> > > > > solution.
> > > > 
> > > > Related at what to do with adding versions between major ABI
> > > > versions, when
> > > > investigating with Kevin the ABI checking we have made an
> > > > unpleasant
> > > > discovery:
> > > > 
> > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > already
> > > > broken
> > > > our ABI according to libabigail.
> > > > 
> > > > The Gory Details [skip to the end for suggestions to fix]
> > > > ------------------------------------------------------------
> > > > 
> > > > The reason for this is that the soversion encoded in each library
> > > > -
> > > > whether
> > > > built with meson or make - is the full 20.0 version, not just the
> > > > major ABI
> > > > .20 part. Then when apps link against DPDK, they actually encode
> > > > the
> > > > 20.0.
> > > > 
> > > > So what this means is that currently - using a make build as an
> > > > example
> > > > here - ldd on the latest head build gives:
> > > > 
> > > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-linux-gcc/app/testpmd | head
> > > >         linux-vdso.so.1 (0x00007fff6813d000)
> > > >         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > >         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > >         librte_mempool_dpaa.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > (0x00007f36d7224000)
> > > >         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > (0x00007f36d71ba000)
> > > >         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > >         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > >         librte_pmd_softnic.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > (0x00007f36d70b7000)
> > > >         librte_flow_classify.so.0.201 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > (0x00007f36d70b1000)
> > > >         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > > ...
> > > > 
> > > > Similarly ldd on a 19.11 checkout gives:
> > > > 
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > x86_64-
> > > > native-linux-gcc_v19.11/app/testpmd | head
> > > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > > >         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > (0x00007fd4dc6b6000)
> > > >         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > (0x00007fd4dc6a3000)
> > > >         librte_mempool_dpaa.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > (0x00007fd4dc69e000)
> > > >         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > (0x00007fd4dc634000)
> > > >         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > (0x00007fd4dc5a0000)
> > > >         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > (0x00007fd4dc55d000)
> > > >         librte_pmd_softnic.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > (0x00007fd4dc531000)
> > > >         librte_flow_classify.so.0.200 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > (0x00007fd4dc52b000)
> > > >         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > (0x00007fd4dc502000)
> > > > 
> > > > The final check - using the 19.11 compiled testpmd with the
> > > > library
> > > > path
> > > > set to 20.02 versionned libs:
> > > > 
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-
> > > > linux-gcc_v19.11/app/testpmd | head
> > > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > > >         librte_pmd_bond.so.20.0 => not found
> > > >         librte_pmd_dpaa.so.20.0 => not found
> > > >         librte_mempool_dpaa.so.20.0 => not found
> > > >         librte_pmd_ixgbe.so.20.0 => not found
> > > >         librte_pmd_i40e.so.20.0 => not found
> > > >         librte_pmd_bnxt.so.20.0 => not found
> > > >         librte_pmd_softnic.so.20.0 => not found
> > > >         librte_flow_classify.so.0.200 => not found
> > > >         librte_pipeline.so.20.0 => not found
> > > > 
> > > > Fixing This
> > > > -----------
> > > > 
> > > > To fix this, we need to ensure that the SONAME remains constant
> > > > across the
> > > > releases. Therefore, I currently see two options:
> > > > 
> > > > 1. keep 20.0 as the version and soname across all releases in
> > > > 2020,
> > > > i.e.
> > > >   just revert the ABIVERSION change patch. Trouble there is how
> > > > to
> > > > track
> > > >   20.02 vs 20.05 etc. etc.
> > > > 
> > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > This
> > > > has the
> > > >   advantage of keeping the existing planned schemes, but has the
> > > > really big
> > > >   downside of breaking ABI compatibility with anyone who has
> > > > already
> > > >   compiled with 19.11.
> > > > 
> > > > Personally, of the two options - unless someone can come up with
> > > > a
> > > > third
> > > > option - I'd tend towards the second, fixing the builds to remove
> > > > the
> > > > .0 in
> > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > widespread
> > > > adoption. Since this ABI stability is new, teething problems may
> > > > be
> > > > expected.
> > > > 
> > > > Thoughts and comments?
> > > > /Bruce
> > > > 
> > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > so_version
> > > > variable and all references to it from lib/meson.build and
> > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > 
> > > Hi,
> > > 
> > > With libtool and its (arguably arcane) format, only the first digit
> > > is
> > > the ABI current version and gets encoded in the elf header. The
> > > other
> > > digits can be used to track compatible minor increments, and are
> > > mostly
> > > ignored. On the system a symlink libfoo.so.major ->
> > > libfoo.so.major.minor is added.
> > > 
> > > Eg:
> > > 
> > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > SONAME
> > >  0x000000000000000e (SONAME)             Library soname:
> > > [libzmq.so.5]
> > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > 
> > > Can we do the same? Not sure what the right incantation is for
> > > Meson,
> > > but it should be possibly.
> > > 
> > 
> > That's essentially option 2, and it's still an ABI break because
> > existing
> > builds of 19.11 have the soname will the full version number in it.
> > The
> > default behaviour for meson is exactly how you described it, except
> > that
> > previously we needed more exact control over the version info (for
> > your
> > dpdk-specific versions in the sonames) and so overrode the soversion
> > explicitly. The fix for meson is to remove this overriding i.e.
> > remove
> > "soversion:" parameter for each shared_library() call.
> > 
> >  
> > > Also, we should leave the current at 20.0 - let's not break
> > > compatibility already, please :-)
> > > 
> > 
> > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> 
> Yes, that's what I meant - IMHO we should just take the hit and use the
> slightly weird 20.0 format until next year, and add a third digit for
> compatible updates. Then for v21 we can drop it.

I am not sure to understand everything above.
My understanding is that the soname must include major and minor:
	libfoo.major.minor
and the package system must install 2 symlinks:
	libfoo.major points to latest installed minor
	libfoo points to latest major

We are missing the symlink libfoo.major in 19.11.0.
I suggest to just add this symlink in 19.11.1 and 20.02.

The other question is to know what the internal apps should link?
Is there any sense linking testpmd to libfoo.major?

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Tue, Dec 10, 2019 at 05:39:00PM +0100, Thomas Monjalon wrote:
> 10/12/2019 17:20, Luca Boccassi:
> > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > 
> > > > > Related at what to do with adding versions between major ABI
> > > > > versions, when
> > > > > investigating with Kevin the ABI checking we have made an
> > > > > unpleasant
> > > > > discovery:
> > > > > 
> > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > already
> > > > > broken
> > > > > our ABI according to libabigail.
> > > > > 
> > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > ------------------------------------------------------------
<snip>
> > > > > 
> > > > > Fixing This
> > > > > -----------
> > > > > 
> > > > > To fix this, we need to ensure that the SONAME remains constant
> > > > > across the
> > > > > releases. Therefore, I currently see two options:
> > > > > 
> > > > > 1. keep 20.0 as the version and soname across all releases in
> > > > > 2020,
> > > > > i.e.
> > > > >   just revert the ABIVERSION change patch. Trouble there is how
> > > > > to
> > > > > track
> > > > >   20.02 vs 20.05 etc. etc.
> > > > > 
> > > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > > This
> > > > > has the
> > > > >   advantage of keeping the existing planned schemes, but has the
> > > > > really big
> > > > >   downside of breaking ABI compatibility with anyone who has
> > > > > already
> > > > >   compiled with 19.11.
> > > > > 
> > > > > Personally, of the two options - unless someone can come up with
> > > > > a
> > > > > third
> > > > > option - I'd tend towards the second, fixing the builds to remove
> > > > > the
> > > > > .0 in
> > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > > widespread
> > > > > adoption. Since this ABI stability is new, teething problems may
> > > > > be
> > > > > expected.
> > > > > 
> > > > > Thoughts and comments?
> > > > > /Bruce
> > > > > 
> > > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > > so_version
> > > > > variable and all references to it from lib/meson.build and
> > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > > 
> > > > Hi,
> > > > 
> > > > With libtool and its (arguably arcane) format, only the first digit
> > > > is
> > > > the ABI current version and gets encoded in the elf header. The
> > > > other
> > > > digits can be used to track compatible minor increments, and are
> > > > mostly
> > > > ignored. On the system a symlink libfoo.so.major ->
> > > > libfoo.so.major.minor is added.
> > > > 
> > > > Eg:
> > > > 
> > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > SONAME
> > > >  0x000000000000000e (SONAME)             Library soname:
> > > > [libzmq.so.5]
> > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > > 
> > > > Can we do the same? Not sure what the right incantation is for
> > > > Meson,
> > > > but it should be possibly.
> > > > 
> > > 
> > > That's essentially option 2, and it's still an ABI break because
> > > existing
> > > builds of 19.11 have the soname will the full version number in it.
> > > The
> > > default behaviour for meson is exactly how you described it, except
> > > that
> > > previously we needed more exact control over the version info (for
> > > your
> > > dpdk-specific versions in the sonames) and so overrode the soversion
> > > explicitly. The fix for meson is to remove this overriding i.e.
> > > remove
> > > "soversion:" parameter for each shared_library() call.
> > > 
> > >  
> > > > Also, we should leave the current at 20.0 - let's not break
> > > > compatibility already, please :-)
> > > > 
> > > 
> > > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> > 
> > Yes, that's what I meant - IMHO we should just take the hit and use the
> > slightly weird 20.0 format until next year, and add a third digit for
> > compatible updates. Then for v21 we can drop it.
> 
> I am not sure to understand everything above.
> My understanding is that the soname must include major and minor:
> 	libfoo.major.minor

The filename should have, but the soname should only have the major ABI
version. It's the soname recorded in the elf file that is used by the
linker, which ignores any version suffixes on the .so.

> and the package system must install 2 symlinks:
> 	libfoo.major points to latest installed minor
> 	libfoo points to latest major
> 
> We are missing the symlink libfoo.major in 19.11.0.
> I suggest to just add this symlink in 19.11.1 and 20.02.
> 

Symlinks won't actually help here, as the suffixes are ignored. In a normal
case, where libfoo.so and libfoo.so.x both point to libfoo.so.x.y, linking
against any one of the three filenames will result in the exact same
linkage info in the binary, based on the soname in libfoo.so.x.y. Ideally,
that should be "libfoo.so.x", which remains constant for all values of "y".

> The other question is to know what the internal apps should link?
> Is there any sense linking testpmd to libfoo.major?
> 
Nothing we can do to control this, except ensure correct soname info on
linking the .so.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Kinsella, Ray]

First, good find, much kudos.
To fix, I tend to agree with Bruce.

The SONAME should be comprised of the library name and the major version number, period.
As described in the DPDK documentation.

I too would be concerned about our forgetting to make the change in 20.11.

Also, what would happen when DPDK 20.05 roles around.
In DPDK 20.05's case the SONAME would still be 20.0
However the library file name would be stamped libeal_20.2.so ... 

Better to drop the minor version to avoid any ambiguity.
I would suggest a respin of DPDK 19.11 to fix.

Ray K

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Tuesday 10 December 2019 16:32
> To: Luca Boccassi <bluca@debian.org>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Kinsella, Ray
> <ray.kinsella@intel.com>; Thomas Monjalon <thomas@monjalon.net>; David
> Marchand <david.marchand@redhat.com>; Christian Ehrhardt
> <christian.ehrhardt@canonical.com>; Timothy Redaelli
> <tredaelli@redhat.com>; Kevin Traynor <ktraynor@redhat.com>; dpdk-dev
> <dev@dpdk.org>; Laatz, Kevin <kevin.laatz@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; Neil Horman <nhorman@redhat.com>
> Subject: Re: [dpdk-dev] How to manage new APIs added after major ABI
> release?
> 
> On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit
> wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > With new process, the major ABI releases will be
> > > > > > > > compatible until it is deprecated (until next LTS for
> > > > > > > > now), like current ABI version is 20 in DPDK_19.11 and
> > > > > > > > DPDK versions until DPDK_20.11 will be ABI compatible
> with
> > > > > > > > this version.
> > > > > > > >
> > > > > > > > But if we introduce a new API after major ABI, say in
> > > > > > > > 20.02 release, are we allowed to break the ABI for that
> > > > > > > > API before DPDK_20.11?
> > > > > > > >
> > > > > > > > If we allow it break, following problem will be observed:
> > > > > > > > Assume an application using .so.20.1 library, and using
> > > > > > > > the new API introduced in 20.02, lets say foo(), but when
> > > > > > > > application switches to .so.20.2 (released via
> > > > > > > > DPDK_20.05), application will fail because of ABI
> breakage
> > > > > > > > in foo().
> > > > > > > >
> > > > > > > > I think it is fair that application expects forward
> > > > > > > > compatibility in minor versions of a shared library.
> > > > > > > > Like if application linked against .so.20.2, fair to
> > > > > > > > expect .so.20.3, .so.20.4 etc will work fine. I think
> > > > > > > > currently only .so.20.0 is fully forward compatible.
> > > > > > > >
> > > > > > > > If we all agree on this, we may need to tweak the process
> > > > > > > > a little, but before diving into implementation details,
> I
> > > > > > > > would like to be sure we are in same page.
> > > > > > > >
> > > > > > >
> > > > > > > Well, any new API's generally come in as experimental, in
> > > > > > > which case changes are allowed, and breakage can be
> > > > > > > expected. If they are not experiemental, then the ABI
> policy
> > > > > > > applies to them in that they cannot change since they are
> > > > > > > part of the .21 ABI, even if that ABI is not fully complete
> > > > > > > yet. For any application only using stable, non-
> > > > > > > experimental functions, forward compatibility must be
> > > > > > > maintained IMHO.
> > > > > > >
> > > > > >
> > > > > > Talking about not experimental APIs, experimental ones free
> > > > > > from the process.
> > > > > >
> > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of
> still
> > > > > > ABI_20, because it should be supported for following
> ABI_20.x,
> > > > > > instead of calling it ABI_21, and this minor tweak (and mind
> > > > > > shift) in .map files can be our solution.
> > > > >
> > > > > Related at what to do with adding versions between major ABI
> > > > > versions, when investigating with Kevin the ABI checking we
> have
> > > > > made an unpleasant
> > > > > discovery:
> > > > >
> > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > already broken our ABI according to libabigail.
> > > > >
> > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > ------------------------------------------------------------
> > > > >
> > > > > The reason for this is that the soversion encoded in each
> > > > > library
> > > > > -
> > > > > whether
> > > > > built with meson or make - is the full 20.0 version, not just
> > > > > the major ABI
> > > > > .20 part. Then when apps link against DPDK, they actually
> encode
> > > > > the 20.0.
> > > > >
> > > > > So what this means is that currently - using a make build as an
> > > > > example here - ldd on the latest head build gives:
> > > > >
> > > > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-linux-gcc/app/testpmd | head
> > > > >         linux-vdso.so.1 (0x00007fff6813d000)
> > > > >         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1
> (0x00007f36d723c000)
> > > > >         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1
> (0x00007f36d7229000)
> > > > >         librte_mempool_dpaa.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > (0x00007f36d7224000)
> > > > >         librte_pmd_ixgbe.so.20.1 =>
> /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > (0x00007f36d71ba000)
> > > > >         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1
> (0x00007f36d7126000)
> > > > >         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1
> (0x00007f36d70e5000)
> > > > >         librte_pmd_softnic.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > (0x00007f36d70b7000)
> > > > >         librte_flow_classify.so.0.201 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > (0x00007f36d70b1000)
> > > > >         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pipeline.so.20.1
> > > > > (0x00007f36d7088000) ...
> > > > >
> > > > > Similarly ldd on a 19.11 checkout gives:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > > x86_64-
> > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > > > >         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > (0x00007fd4dc6b6000)
> > > > >         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > (0x00007fd4dc6a3000)
> > > > >         librte_mempool_dpaa.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > (0x00007fd4dc69e000)
> > > > >         librte_pmd_ixgbe.so.20.0 =>
> /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > (0x00007fd4dc634000)
> > > > >         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > (0x00007fd4dc5a0000)
> > > > >         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > (0x00007fd4dc55d000)
> > > > >         librte_pmd_softnic.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > (0x00007fd4dc531000)
> > > > >         librte_flow_classify.so.0.200 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > (0x00007fd4dc52b000)
> > > > >         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > (0x00007fd4dc502000)
> > > > >
> > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > library path set to 20.02 versionned libs:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-
> > > > > linux-gcc_v19.11/app/testpmd | head
> > > > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > > > >         librte_pmd_bond.so.20.0 => not found
> > > > >         librte_pmd_dpaa.so.20.0 => not found
> > > > >         librte_mempool_dpaa.so.20.0 => not found
> > > > >         librte_pmd_ixgbe.so.20.0 => not found
> > > > >         librte_pmd_i40e.so.20.0 => not found
> > > > >         librte_pmd_bnxt.so.20.0 => not found
> > > > >         librte_pmd_softnic.so.20.0 => not found
> > > > >         librte_flow_classify.so.0.200 => not found
> > > > >         librte_pipeline.so.20.0 => not found
> > > > >
> > > > > Fixing This
> > > > > -----------
> > > > >
> > > > > To fix this, we need to ensure that the SONAME remains constant
> > > > > across the releases. Therefore, I currently see two options:
> > > > >
> > > > > 1. keep 20.0 as the version and soname across all releases in
> > > > > 2020, i.e.
> > > > >   just revert the ABIVERSION change patch. Trouble there is how
> > > > > to track
> > > > >   20.02 vs 20.05 etc. etc.
> > > > >
> > > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > > This
> > > > > has the
> > > > >   advantage of keeping the existing planned schemes, but has
> the
> > > > > really big
> > > > >   downside of breaking ABI compatibility with anyone who has
> > > > > already
> > > > >   compiled with 19.11.
> > > > >
> > > > > Personally, of the two options - unless someone can come up
> with
> > > > > a third option - I'd tend towards the second, fixing the builds
> > > > > to remove the
> > > > > .0 in
> > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11
> gets
> > > > > widespread adoption. Since this ABI stability is new, teething
> > > > > problems may be expected.
> > > > >
> > > > > Thoughts and comments?
> > > > > /Bruce
> > > > >
> > > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > > so_version variable and all references to it from
> > > > > lib/meson.build and drivers/meson.build. Haven't looked into a
> > > > > "make" fix yet.
> > > >
> > > > Hi,
> > > >
> > > > With libtool and its (arguably arcane) format, only the first
> > > > digit is the ABI current version and gets encoded in the elf
> > > > header. The other digits can be used to track compatible minor
> > > > increments, and are mostly ignored. On the system a symlink
> > > > libfoo.so.major -> libfoo.so.major.minor is added.
> > > >
> > > > Eg:
> > > >
> > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > SONAME
> > > >  0x000000000000000e (SONAME)             Library soname:
> > > > [libzmq.so.5]
> > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > >
> > > > Can we do the same? Not sure what the right incantation is for
> > > > Meson, but it should be possibly.
> > > >
> > >
> > > That's essentially option 2, and it's still an ABI break because
> > > existing
> > > builds of 19.11 have the soname will the full version number in it.
> > > The
> > > default behaviour for meson is exactly how you described it, except
> > > that
> > > previously we needed more exact control over the version info (for
> > > your
> > > dpdk-specific versions in the sonames) and so overrode the
> soversion
> > > explicitly. The fix for meson is to remove this overriding i.e.
> > > remove
> > > "soversion:" parameter for each shared_library() call.
> > >
> > >
> > > > Also, we should leave the current at 20.0 - let's not break
> > > > compatibility already, please :-)
> > > >
> > >
> > > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> >
> > Yes, that's what I meant - IMHO we should just take the hit and use
> the
> > slightly weird 20.0 format until next year, and add a third digit for
> > compatible updates. Then for v21 we can drop it.
> >
> My concern with that is us forgetting, because we'll put in place hacks
> to
> have the soversion be the first two numbers of the version. Then we
> need to
> remember to remove those hacks before 20.11 goes out or we'll end up
> with
> 21.0 being the soversion again.
> 
> For that reason, I'd rather see us fix it now before 19.11 gets widely
> adopted.
> 
> /Bruce

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Thomas Monjalon]

10/12/2019 17:32, Bruce Richardson:
> On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > With new process, the major ABI releases will be compatible
> > > > > > > > until it is
> > > > > > > > deprecated (until next LTS for now),
> > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > > versions
> > > > > > > > until DPDK_20.11
> > > > > > > > will be ABI compatible with this version.
> > > > > > > > 
> > > > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > > > release, are we
> > > > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > > > 
> > > > > > > > If we allow it break, following problem will be observed:
> > > > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > > > new
> > > > > > > > API introduced
> > > > > > > > in 20.02, lets say foo(),
> > > > > > > > but when application switches to .so.20.2 (released via
> > > > > > > > DPDK_20.05), application
> > > > > > > > will fail because of ABI breakage in foo().
> > > > > > > > 
> > > > > > > > I think it is fair that application expects forward
> > > > > > > > compatibility in minor
> > > > > > > > versions of a shared library.
> > > > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > > > .so.20.3, .so.20.4
> > > > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > > > fully
> > > > > > > > forward compatible.
> > > > > > > > 
> > > > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > > > little, but before
> > > > > > > > diving into implementation details, I would like to be sure
> > > > > > > > we
> > > > > > > > are in same page.
> > > > > > > > 
> > > > > > > 
> > > > > > > Well, any new API's generally come in as experimental, in
> > > > > > > which
> > > > > > > case
> > > > > > > changes are allowed, and breakage can be expected. If they
> > > > > > > are
> > > > > > > not
> > > > > > > experiemental, then the ABI policy applies to them in that
> > > > > > > they
> > > > > > > cannot
> > > > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > > > is
> > > > > > > not fully
> > > > > > > complete yet. For any application only using stable, non-
> > > > > > > experimental
> > > > > > > functions, forward compatibility must be maintained IMHO.
> > > > > > > 
> > > > > > 
> > > > > > Talking about not experimental APIs, experimental ones free
> > > > > > from
> > > > > > the process.
> > > > > > 
> > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > > > ABI_20, because
> > > > > > it should be supported for following ABI_20.x, instead of
> > > > > > calling
> > > > > > it ABI_21, and
> > > > > > this minor tweak (and mind shift) in .map files can be our
> > > > > > solution.
> > > > > 
> > > > > Related at what to do with adding versions between major ABI
> > > > > versions, when
> > > > > investigating with Kevin the ABI checking we have made an
> > > > > unpleasant
> > > > > discovery:
> > > > > 
> > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > already
> > > > > broken
> > > > > our ABI according to libabigail.
> > > > > 
> > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > ------------------------------------------------------------
> > > > > 
> > > > > The reason for this is that the soversion encoded in each library
> > > > > -
> > > > > whether
> > > > > built with meson or make - is the full 20.0 version, not just the
> > > > > major ABI
> > > > > .20 part. Then when apps link against DPDK, they actually encode
> > > > > the
> > > > > 20.0.
> > > > > 
> > > > > So what this means is that currently - using a make build as an
> > > > > example
> > > > > here - ldd on the latest head build gives:
> > > > > 
> > > > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-linux-gcc/app/testpmd | head
> > > > >         linux-vdso.so.1 (0x00007fff6813d000)
> > > > >         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > > >         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > > >         librte_mempool_dpaa.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > (0x00007f36d7224000)
> > > > >         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > (0x00007f36d71ba000)
> > > > >         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > > >         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > > >         librte_pmd_softnic.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > (0x00007f36d70b7000)
> > > > >         librte_flow_classify.so.0.201 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > (0x00007f36d70b1000)
> > > > >         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > > > ...
> > > > > 
> > > > > Similarly ldd on a 19.11 checkout gives:
> > > > > 
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > > x86_64-
> > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > > > >         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > (0x00007fd4dc6b6000)
> > > > >         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > (0x00007fd4dc6a3000)
> > > > >         librte_mempool_dpaa.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > (0x00007fd4dc69e000)
> > > > >         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > (0x00007fd4dc634000)
> > > > >         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > (0x00007fd4dc5a0000)
> > > > >         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > (0x00007fd4dc55d000)
> > > > >         librte_pmd_softnic.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > (0x00007fd4dc531000)
> > > > >         librte_flow_classify.so.0.200 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > (0x00007fd4dc52b000)
> > > > >         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > (0x00007fd4dc502000)
> > > > > 
> > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > library
> > > > > path
> > > > > set to 20.02 versionned libs:
> > > > > 
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-
> > > > > linux-gcc_v19.11/app/testpmd | head
> > > > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > > > >         librte_pmd_bond.so.20.0 => not found
> > > > >         librte_pmd_dpaa.so.20.0 => not found
> > > > >         librte_mempool_dpaa.so.20.0 => not found
> > > > >         librte_pmd_ixgbe.so.20.0 => not found
> > > > >         librte_pmd_i40e.so.20.0 => not found
> > > > >         librte_pmd_bnxt.so.20.0 => not found
> > > > >         librte_pmd_softnic.so.20.0 => not found
> > > > >         librte_flow_classify.so.0.200 => not found
> > > > >         librte_pipeline.so.20.0 => not found
> > > > > 
> > > > > Fixing This
> > > > > -----------
> > > > > 
> > > > > To fix this, we need to ensure that the SONAME remains constant
> > > > > across the
> > > > > releases. Therefore, I currently see two options:
> > > > > 
> > > > > 1. keep 20.0 as the version and soname across all releases in
> > > > > 2020,
> > > > > i.e.
> > > > >   just revert the ABIVERSION change patch. Trouble there is how
> > > > > to
> > > > > track
> > > > >   20.02 vs 20.05 etc. etc.
> > > > > 
> > > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > > This
> > > > > has the
> > > > >   advantage of keeping the existing planned schemes, but has the
> > > > > really big
> > > > >   downside of breaking ABI compatibility with anyone who has
> > > > > already
> > > > >   compiled with 19.11.
> > > > > 
> > > > > Personally, of the two options - unless someone can come up with
> > > > > a
> > > > > third
> > > > > option - I'd tend towards the second, fixing the builds to remove
> > > > > the
> > > > > .0 in
> > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > > widespread
> > > > > adoption. Since this ABI stability is new, teething problems may
> > > > > be
> > > > > expected.
> > > > > 
> > > > > Thoughts and comments?
> > > > > /Bruce
> > > > > 
> > > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > > so_version
> > > > > variable and all references to it from lib/meson.build and
> > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > > 
> > > > Hi,
> > > > 
> > > > With libtool and its (arguably arcane) format, only the first digit
> > > > is
> > > > the ABI current version and gets encoded in the elf header. The
> > > > other
> > > > digits can be used to track compatible minor increments, and are
> > > > mostly
> > > > ignored. On the system a symlink libfoo.so.major ->
> > > > libfoo.so.major.minor is added.
> > > > 
> > > > Eg:
> > > > 
> > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > SONAME
> > > >  0x000000000000000e (SONAME)             Library soname:
> > > > [libzmq.so.5]
> > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > > 
> > > > Can we do the same? Not sure what the right incantation is for
> > > > Meson,
> > > > but it should be possibly.
> > > > 
> > > 
> > > That's essentially option 2, and it's still an ABI break because
> > > existing
> > > builds of 19.11 have the soname will the full version number in it.
> > > The
> > > default behaviour for meson is exactly how you described it, except
> > > that
> > > previously we needed more exact control over the version info (for
> > > your
> > > dpdk-specific versions in the sonames) and so overrode the soversion
> > > explicitly. The fix for meson is to remove this overriding i.e.
> > > remove
> > > "soversion:" parameter for each shared_library() call.
> > > 
> > >  
> > > > Also, we should leave the current at 20.0 - let's not break
> > > > compatibility already, please :-)
> > > > 
> > > 
> > > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> > 
> > Yes, that's what I meant - IMHO we should just take the hit and use the
> > slightly weird 20.0 format until next year, and add a third digit for
> > compatible updates. Then for v21 we can drop it.
> >
> My concern with that is us forgetting, because we'll put in place hacks to
> have the soversion be the first two numbers of the version. Then we need to
> remember to remove those hacks before 20.11 goes out or we'll end up with
> 21.0 being the soversion again.
> 
> For that reason, I'd rather see us fix it now before 19.11 gets widely
> adopted.

Me too I vote for fixing soname as 20 in a small 19.11.1
and release it quickly.

We are supposed to experience tooling and scheme adoption during this
first year of new ABI policy. I think we should stick to the planned
versionning, in order to avoid any surprise for the next major ABI.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Luca Boccassi]

On Tue, 2019-12-10 at 18:04 +0100, Thomas Monjalon wrote:
> 10/12/2019 17:32, Bruce Richardson:
> > On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit
> > > > > > wrote:
> > > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit
> > > > > > > > wrote:
> > > > > > > > > Hi,
> > > > > > > > > 
> > > > > > > > > With new process, the major ABI releases will be
> > > > > > > > > compatible
> > > > > > > > > until it is
> > > > > > > > > deprecated (until next LTS for now),
> > > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > > > versions
> > > > > > > > > until DPDK_20.11
> > > > > > > > > will be ABI compatible with this version.
> > > > > > > > > 
> > > > > > > > > But if we introduce a new API after major ABI, say in
> > > > > > > > > 20.02
> > > > > > > > > release, are we
> > > > > > > > > allowed to break the ABI for that API before
> > > > > > > > > DPDK_20.11?
> > > > > > > > > 
> > > > > > > > > If we allow it break, following problem will be
> > > > > > > > > observed:
> > > > > > > > > Assume an application using .so.20.1 library, and
> > > > > > > > > using the
> > > > > > > > > new
> > > > > > > > > API introduced
> > > > > > > > > in 20.02, lets say foo(),
> > > > > > > > > but when application switches to .so.20.2 (released
> > > > > > > > > via
> > > > > > > > > DPDK_20.05), application
> > > > > > > > > will fail because of ABI breakage in foo().
> > > > > > > > > 
> > > > > > > > > I think it is fair that application expects forward
> > > > > > > > > compatibility in minor
> > > > > > > > > versions of a shared library.
> > > > > > > > > Like if application linked against .so.20.2, fair to
> > > > > > > > > expect
> > > > > > > > > .so.20.3, .so.20.4
> > > > > > > > > etc will work fine. I think currently only .so.20.0
> > > > > > > > > is
> > > > > > > > > fully
> > > > > > > > > forward compatible.
> > > > > > > > > 
> > > > > > > > > If we all agree on this, we may need to tweak the
> > > > > > > > > process a
> > > > > > > > > little, but before
> > > > > > > > > diving into implementation details, I would like to
> > > > > > > > > be sure
> > > > > > > > > we
> > > > > > > > > are in same page.
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Well, any new API's generally come in as experimental,
> > > > > > > > in
> > > > > > > > which
> > > > > > > > case
> > > > > > > > changes are allowed, and breakage can be expected. If
> > > > > > > > they
> > > > > > > > are
> > > > > > > > not
> > > > > > > > experiemental, then the ABI policy applies to them in
> > > > > > > > that
> > > > > > > > they
> > > > > > > > cannot
> > > > > > > > change since they are part of the .21 ABI, even if that
> > > > > > > > ABI
> > > > > > > > is
> > > > > > > > not fully
> > > > > > > > complete yet. For any application only using stable,
> > > > > > > > non-
> > > > > > > > experimental
> > > > > > > > functions, forward compatibility must be maintained
> > > > > > > > IMHO.
> > > > > > > > 
> > > > > > > 
> > > > > > > Talking about not experimental APIs, experimental ones
> > > > > > > free
> > > > > > > from
> > > > > > > the process.
> > > > > > > 
> > > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of
> > > > > > > still
> > > > > > > ABI_20, because
> > > > > > > it should be supported for following ABI_20.x, instead of
> > > > > > > calling
> > > > > > > it ABI_21, and
> > > > > > > this minor tweak (and mind shift) in .map files can be
> > > > > > > our
> > > > > > > solution.
> > > > > > 
> > > > > > Related at what to do with adding versions between major
> > > > > > ABI
> > > > > > versions, when
> > > > > > investigating with Kevin the ABI checking we have made an
> > > > > > unpleasant
> > > > > > discovery:
> > > > > > 
> > > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > > already
> > > > > > broken
> > > > > > our ABI according to libabigail.
> > > > > > 
> > > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > > ---------------------------------------------------------
> > > > > > ---
> > > > > > 
> > > > > > The reason for this is that the soversion encoded in each
> > > > > > library
> > > > > > -
> > > > > > whether
> > > > > > built with meson or make - is the full 20.0 version, not
> > > > > > just the
> > > > > > major ABI
> > > > > > .20 part. Then when apps link against DPDK, they actually
> > > > > > encode
> > > > > > the
> > > > > > 20.0.
> > > > > > 
> > > > > > So what this means is that currently - using a make build
> > > > > > as an
> > > > > > example
> > > > > > here - ldd on the latest head build gives:
> > > > > > 
> > > > > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > x86_64-
> > > > > > native-linux-gcc/app/testpmd | head
> > > > > >         linux-vdso.so.1 (0x00007fff6813d000)
> > > > > >         librte_pmd_bond.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1
> > > > > > (0x00007f36d723c000)
> > > > > >         librte_pmd_dpaa.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1
> > > > > > (0x00007f36d7229000)
> > > > > >         librte_mempool_dpaa.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > > (0x00007f36d7224000)
> > > > > >         librte_pmd_ixgbe.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > > (0x00007f36d71ba000)
> > > > > >         librte_pmd_i40e.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1
> > > > > > (0x00007f36d7126000)
> > > > > >         librte_pmd_bnxt.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1
> > > > > > (0x00007f36d70e5000)
> > > > > >         librte_pmd_softnic.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > > (0x00007f36d70b7000)
> > > > > >         librte_flow_classify.so.0.201 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > > (0x00007f36d70b1000)
> > > > > >         librte_pipeline.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pipeline.so.20.1
> > > > > > (0x00007f36d7088000)
> > > > > > ...
> > > > > > 
> > > > > > Similarly ldd on a 19.11 checkout gives:
> > > > > > 
> > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib
> > > > > > ldd
> > > > > > x86_64-
> > > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > > > > >         librte_pmd_bond.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > > (0x00007fd4dc6b6000)
> > > > > >         librte_pmd_dpaa.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > > (0x00007fd4dc6a3000)
> > > > > >         librte_mempool_dpaa.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > > (0x00007fd4dc69e000)
> > > > > >         librte_pmd_ixgbe.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > > (0x00007fd4dc634000)
> > > > > >         librte_pmd_i40e.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > > (0x00007fd4dc5a0000)
> > > > > >         librte_pmd_bnxt.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > > (0x00007fd4dc55d000)
> > > > > >         librte_pmd_softnic.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > > (0x00007fd4dc531000)
> > > > > >         librte_flow_classify.so.0.200 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > > (0x00007fd4dc52b000)
> > > > > >         librte_pipeline.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > > (0x00007fd4dc502000)
> > > > > > 
> > > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > > library
> > > > > > path
> > > > > > set to 20.02 versionned libs:
> > > > > > 
> > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > x86_64-
> > > > > > native-
> > > > > > linux-gcc_v19.11/app/testpmd | head
> > > > > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > > > > >         librte_pmd_bond.so.20.0 => not found
> > > > > >         librte_pmd_dpaa.so.20.0 => not found
> > > > > >         librte_mempool_dpaa.so.20.0 => not found
> > > > > >         librte_pmd_ixgbe.so.20.0 => not found
> > > > > >         librte_pmd_i40e.so.20.0 => not found
> > > > > >         librte_pmd_bnxt.so.20.0 => not found
> > > > > >         librte_pmd_softnic.so.20.0 => not found
> > > > > >         librte_flow_classify.so.0.200 => not found
> > > > > >         librte_pipeline.so.20.0 => not found
> > > > > > 
> > > > > > Fixing This
> > > > > > -----------
> > > > > > 
> > > > > > To fix this, we need to ensure that the SONAME remains
> > > > > > constant
> > > > > > across the
> > > > > > releases. Therefore, I currently see two options:
> > > > > > 
> > > > > > 1. keep 20.0 as the version and soname across all releases
> > > > > > in
> > > > > > 2020,
> > > > > > i.e.
> > > > > >   just revert the ABIVERSION change patch. Trouble there is
> > > > > > how
> > > > > > to
> > > > > > track
> > > > > >   20.02 vs 20.05 etc. etc.
> > > > > > 
> > > > > > 2. remove the .0, .1 from the SONAMES stored in the
> > > > > > libraries.
> > > > > > This
> > > > > > has the
> > > > > >   advantage of keeping the existing planned schemes, but
> > > > > > has the
> > > > > > really big
> > > > > >   downside of breaking ABI compatibility with anyone who
> > > > > > has
> > > > > > already
> > > > > >   compiled with 19.11.
> > > > > > 
> > > > > > Personally, of the two options - unless someone can come up
> > > > > > with
> > > > > > a
> > > > > > third
> > > > > > option - I'd tend towards the second, fixing the builds to
> > > > > > remove
> > > > > > the
> > > > > > .0 in
> > > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11
> > > > > > gets
> > > > > > widespread
> > > > > > adoption. Since this ABI stability is new, teething
> > > > > > problems may
> > > > > > be
> > > > > > expected.
> > > > > > 
> > > > > > Thoughts and comments?
> > > > > > /Bruce
> > > > > > 
> > > > > > BTW: For meson, the patch for option 2 is just to remove
> > > > > > the
> > > > > > so_version
> > > > > > variable and all references to it from lib/meson.build and
> > > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > > > 
> > > > > Hi,
> > > > > 
> > > > > With libtool and its (arguably arcane) format, only the first
> > > > > digit
> > > > > is
> > > > > the ABI current version and gets encoded in the elf header.
> > > > > The
> > > > > other
> > > > > digits can be used to track compatible minor increments, and
> > > > > are
> > > > > mostly
> > > > > ignored. On the system a symlink libfoo.so.major ->
> > > > > libfoo.so.major.minor is added.
> > > > > 
> > > > > Eg:
> > > > > 
> > > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > > SONAME
> > > > >  0x000000000000000e (SONAME)             Library soname:
> > > > > [libzmq.so.5]
> > > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > > > 
> > > > > Can we do the same? Not sure what the right incantation is
> > > > > for
> > > > > Meson,
> > > > > but it should be possibly.
> > > > > 
> > > > 
> > > > That's essentially option 2, and it's still an ABI break
> > > > because
> > > > existing
> > > > builds of 19.11 have the soname will the full version number in
> > > > it.
> > > > The
> > > > default behaviour for meson is exactly how you described it,
> > > > except
> > > > that
> > > > previously we needed more exact control over the version info
> > > > (for
> > > > your
> > > > dpdk-specific versions in the sonames) and so overrode the
> > > > soversion
> > > > explicitly. The fix for meson is to remove this overriding i.e.
> > > > remove
> > > > "soversion:" parameter for each shared_library() call.
> > > > 
> > > >  
> > > > > Also, we should leave the current at 20.0 - let's not break
> > > > > compatibility already, please :-)
> > > > > 
> > > > 
> > > > If we do this, maybe we can use 20.0.1 and 20.0.2 version
> > > > numbers?
> > > 
> > > Yes, that's what I meant - IMHO we should just take the hit and
> > > use the
> > > slightly weird 20.0 format until next year, and add a third digit
> > > for
> > > compatible updates. Then for v21 we can drop it.
> > > 
> > 
> > My concern with that is us forgetting, because we'll put in place
> > hacks to
> > have the soversion be the first two numbers of the version. Then we
> > need to
> > remember to remove those hacks before 20.11 goes out or we'll end
> > up with
> > 21.0 being the soversion again.
> > 
> > For that reason, I'd rather see us fix it now before 19.11 gets
> > widely
> > adopted.
> 
> Me too I vote for fixing soname as 20 in a small 19.11.1
> and release it quickly.
> 
> We are supposed to experience tooling and scheme adoption during this
> first year of new ABI policy. I think we should stick to the planned
> versionning, in order to avoid any surprise for the next major ABI.

If the concern is forgetting, I'll happily set a reminder in my
calendar and then nag everybody :-)

In my opinion declaring stability and compatibility, with press
releases and articles, and then immediately breaking it in the first
point release does not send the right message. Not because there was a
bug, which happens all the time, but because mantaining backward
compatibility sometimes means having to carry less-than-ideal or
downright ugly hacks for a while.

If we are not ready to maintain compatibility by carrying an ugly
workaround when the implications are minor or non-existant (I'm pretty
sure few would really cares what format the soname has, it's all
handled by tools), what happens next time when the implications are
that a broken feature stays broken, or that lower performance stays
lower, and so on?

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Tue, Dec 10, 2019 at 06:22:36PM +0000, Luca Boccassi wrote:
> On Tue, 2019-12-10 at 18:04 +0100, Thomas Monjalon wrote:
> > 10/12/2019 17:32, Bruce Richardson:
> > > On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit
> > > > > > > wrote:
> > > > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit
> > > > > > > > > wrote:
> > > > > > > > > > Hi,
> > > > > > > > > > 
> > > > > > > > > > With new process, the major ABI releases will be
> > > > > > > > > > compatible
> > > > > > > > > > until it is
> > > > > > > > > > deprecated (until next LTS for now),
> > > > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > > > > versions
> > > > > > > > > > until DPDK_20.11
> > > > > > > > > > will be ABI compatible with this version.
> > > > > > > > > > 
> > > > > > > > > > But if we introduce a new API after major ABI, say in
> > > > > > > > > > 20.02
> > > > > > > > > > release, are we
> > > > > > > > > > allowed to break the ABI for that API before
> > > > > > > > > > DPDK_20.11?
> > > > > > > > > > 
> > > > > > > > > > If we allow it break, following problem will be
> > > > > > > > > > observed:
> > > > > > > > > > Assume an application using .so.20.1 library, and
> > > > > > > > > > using the
> > > > > > > > > > new
> > > > > > > > > > API introduced
> > > > > > > > > > in 20.02, lets say foo(),
> > > > > > > > > > but when application switches to .so.20.2 (released
> > > > > > > > > > via
> > > > > > > > > > DPDK_20.05), application
> > > > > > > > > > will fail because of ABI breakage in foo().
> > > > > > > > > > 
> > > > > > > > > > I think it is fair that application expects forward
> > > > > > > > > > compatibility in minor
> > > > > > > > > > versions of a shared library.
> > > > > > > > > > Like if application linked against .so.20.2, fair to
> > > > > > > > > > expect
> > > > > > > > > > .so.20.3, .so.20.4
> > > > > > > > > > etc will work fine. I think currently only .so.20.0
> > > > > > > > > > is
> > > > > > > > > > fully
> > > > > > > > > > forward compatible.
> > > > > > > > > > 
> > > > > > > > > > If we all agree on this, we may need to tweak the
> > > > > > > > > > process a
> > > > > > > > > > little, but before
> > > > > > > > > > diving into implementation details, I would like to
> > > > > > > > > > be sure
> > > > > > > > > > we
> > > > > > > > > > are in same page.
> > > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Well, any new API's generally come in as experimental,
> > > > > > > > > in
> > > > > > > > > which
> > > > > > > > > case
> > > > > > > > > changes are allowed, and breakage can be expected. If
> > > > > > > > > they
> > > > > > > > > are
> > > > > > > > > not
> > > > > > > > > experiemental, then the ABI policy applies to them in
> > > > > > > > > that
> > > > > > > > > they
> > > > > > > > > cannot
> > > > > > > > > change since they are part of the .21 ABI, even if that
> > > > > > > > > ABI
> > > > > > > > > is
> > > > > > > > > not fully
> > > > > > > > > complete yet. For any application only using stable,
> > > > > > > > > non-
> > > > > > > > > experimental
> > > > > > > > > functions, forward compatibility must be maintained
> > > > > > > > > IMHO.
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Talking about not experimental APIs, experimental ones
> > > > > > > > free
> > > > > > > > from
> > > > > > > > the process.
> > > > > > > > 
> > > > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of
> > > > > > > > still
> > > > > > > > ABI_20, because
> > > > > > > > it should be supported for following ABI_20.x, instead of
> > > > > > > > calling
> > > > > > > > it ABI_21, and
> > > > > > > > this minor tweak (and mind shift) in .map files can be
> > > > > > > > our
> > > > > > > > solution.
> > > > > > > 
> > > > > > > Related at what to do with adding versions between major
> > > > > > > ABI
> > > > > > > versions, when
> > > > > > > investigating with Kevin the ABI checking we have made an
> > > > > > > unpleasant
> > > > > > > discovery:
> > > > > > > 
> > > > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > > > already
> > > > > > > broken
> > > > > > > our ABI according to libabigail.
> > > > > > > 
> > > > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > > > ---------------------------------------------------------
> > > > > > > ---
> > > > > > > 
> > > > > > > The reason for this is that the soversion encoded in each
> > > > > > > library
> > > > > > > -
> > > > > > > whether
> > > > > > > built with meson or make - is the full 20.0 version, not
> > > > > > > just the
> > > > > > > major ABI
> > > > > > > .20 part. Then when apps link against DPDK, they actually
> > > > > > > encode
> > > > > > > the
> > > > > > > 20.0.
> > > > > > > 
> > > > > > > So what this means is that currently - using a make build
> > > > > > > as an
> > > > > > > example
> > > > > > > here - ldd on the latest head build gives:
> > > > > > > 
> > > > > > >  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > > x86_64-
> > > > > > > native-linux-gcc/app/testpmd | head
> > > > > > >         linux-vdso.so.1 (0x00007fff6813d000)
> > > > > > >         librte_pmd_bond.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1
> > > > > > > (0x00007f36d723c000)
> > > > > > >         librte_pmd_dpaa.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1
> > > > > > > (0x00007f36d7229000)
> > > > > > >         librte_mempool_dpaa.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > > > (0x00007f36d7224000)
> > > > > > >         librte_pmd_ixgbe.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > > > (0x00007f36d71ba000)
> > > > > > >         librte_pmd_i40e.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1
> > > > > > > (0x00007f36d7126000)
> > > > > > >         librte_pmd_bnxt.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1
> > > > > > > (0x00007f36d70e5000)
> > > > > > >         librte_pmd_softnic.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > > > (0x00007f36d70b7000)
> > > > > > >         librte_flow_classify.so.0.201 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > > > (0x00007f36d70b1000)
> > > > > > >         librte_pipeline.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pipeline.so.20.1
> > > > > > > (0x00007f36d7088000)
> > > > > > > ...
> > > > > > > 
> > > > > > > Similarly ldd on a 19.11 checkout gives:
> > > > > > > 
> > > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib
> > > > > > > ldd
> > > > > > > x86_64-
> > > > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > > > >         linux-vdso.so.1 (0x00007ffc2a964000)
> > > > > > >         librte_pmd_bond.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > > > (0x00007fd4dc6b6000)
> > > > > > >         librte_pmd_dpaa.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > > > (0x00007fd4dc6a3000)
> > > > > > >         librte_mempool_dpaa.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > > > (0x00007fd4dc69e000)
> > > > > > >         librte_pmd_ixgbe.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > > > (0x00007fd4dc634000)
> > > > > > >         librte_pmd_i40e.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > > > (0x00007fd4dc5a0000)
> > > > > > >         librte_pmd_bnxt.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > > > (0x00007fd4dc55d000)
> > > > > > >         librte_pmd_softnic.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > > > (0x00007fd4dc531000)
> > > > > > >         librte_flow_classify.so.0.200 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > > > (0x00007fd4dc52b000)
> > > > > > >         librte_pipeline.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > > > (0x00007fd4dc502000)
> > > > > > > 
> > > > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > > > library
> > > > > > > path
> > > > > > > set to 20.02 versionned libs:
> > > > > > > 
> > > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > > x86_64-
> > > > > > > native-
> > > > > > > linux-gcc_v19.11/app/testpmd | head
> > > > > > >         linux-vdso.so.1 (0x00007ffc711fc000)
> > > > > > >         librte_pmd_bond.so.20.0 => not found
> > > > > > >         librte_pmd_dpaa.so.20.0 => not found
> > > > > > >         librte_mempool_dpaa.so.20.0 => not found
> > > > > > >         librte_pmd_ixgbe.so.20.0 => not found
> > > > > > >         librte_pmd_i40e.so.20.0 => not found
> > > > > > >         librte_pmd_bnxt.so.20.0 => not found
> > > > > > >         librte_pmd_softnic.so.20.0 => not found
> > > > > > >         librte_flow_classify.so.0.200 => not found
> > > > > > >         librte_pipeline.so.20.0 => not found
> > > > > > > 
> > > > > > > Fixing This
> > > > > > > -----------
> > > > > > > 
> > > > > > > To fix this, we need to ensure that the SONAME remains
> > > > > > > constant
> > > > > > > across the
> > > > > > > releases. Therefore, I currently see two options:
> > > > > > > 
> > > > > > > 1. keep 20.0 as the version and soname across all releases
> > > > > > > in
> > > > > > > 2020,
> > > > > > > i.e.
> > > > > > >   just revert the ABIVERSION change patch. Trouble there is
> > > > > > > how
> > > > > > > to
> > > > > > > track
> > > > > > >   20.02 vs 20.05 etc. etc.
> > > > > > > 
> > > > > > > 2. remove the .0, .1 from the SONAMES stored in the
> > > > > > > libraries.
> > > > > > > This
> > > > > > > has the
> > > > > > >   advantage of keeping the existing planned schemes, but
> > > > > > > has the
> > > > > > > really big
> > > > > > >   downside of breaking ABI compatibility with anyone who
> > > > > > > has
> > > > > > > already
> > > > > > >   compiled with 19.11.
> > > > > > > 
> > > > > > > Personally, of the two options - unless someone can come up
> > > > > > > with
> > > > > > > a
> > > > > > > third
> > > > > > > option - I'd tend towards the second, fixing the builds to
> > > > > > > remove
> > > > > > > the
> > > > > > > .0 in
> > > > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11
> > > > > > > gets
> > > > > > > widespread
> > > > > > > adoption. Since this ABI stability is new, teething
> > > > > > > problems may
> > > > > > > be
> > > > > > > expected.
> > > > > > > 
> > > > > > > Thoughts and comments?
> > > > > > > /Bruce
> > > > > > > 
> > > > > > > BTW: For meson, the patch for option 2 is just to remove
> > > > > > > the
> > > > > > > so_version
> > > > > > > variable and all references to it from lib/meson.build and
> > > > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > With libtool and its (arguably arcane) format, only the first
> > > > > > digit
> > > > > > is
> > > > > > the ABI current version and gets encoded in the elf header.
> > > > > > The
> > > > > > other
> > > > > > digits can be used to track compatible minor increments, and
> > > > > > are
> > > > > > mostly
> > > > > > ignored. On the system a symlink libfoo.so.major ->
> > > > > > libfoo.so.major.minor is added.
> > > > > > 
> > > > > > Eg:
> > > > > > 
> > > > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > > > SONAME
> > > > > >  0x000000000000000e (SONAME)             Library soname:
> > > > > > [libzmq.so.5]
> > > > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > > > lrwxrwxrwx 1 root root 15 Dec 31  2014 /usr/lib/x86_64-linux-
> > > > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > > > > 
> > > > > > Can we do the same? Not sure what the right incantation is
> > > > > > for
> > > > > > Meson,
> > > > > > but it should be possibly.
> > > > > > 
> > > > > 
> > > > > That's essentially option 2, and it's still an ABI break
> > > > > because
> > > > > existing
> > > > > builds of 19.11 have the soname will the full version number in
> > > > > it.
> > > > > The
> > > > > default behaviour for meson is exactly how you described it,
> > > > > except
> > > > > that
> > > > > previously we needed more exact control over the version info
> > > > > (for
> > > > > your
> > > > > dpdk-specific versions in the sonames) and so overrode the
> > > > > soversion
> > > > > explicitly. The fix for meson is to remove this overriding i.e.
> > > > > remove
> > > > > "soversion:" parameter for each shared_library() call.
> > > > > 
> > > > >  
> > > > > > Also, we should leave the current at 20.0 - let's not break
> > > > > > compatibility already, please :-)
> > > > > > 
> > > > > 
> > > > > If we do this, maybe we can use 20.0.1 and 20.0.2 version
> > > > > numbers?
> > > > 
> > > > Yes, that's what I meant - IMHO we should just take the hit and
> > > > use the
> > > > slightly weird 20.0 format until next year, and add a third digit
> > > > for
> > > > compatible updates. Then for v21 we can drop it.
> > > > 
> > > 
> > > My concern with that is us forgetting, because we'll put in place
> > > hacks to
> > > have the soversion be the first two numbers of the version. Then we
> > > need to
> > > remember to remove those hacks before 20.11 goes out or we'll end
> > > up with
> > > 21.0 being the soversion again.
> > > 
> > > For that reason, I'd rather see us fix it now before 19.11 gets
> > > widely
> > > adopted.
> > 
> > Me too I vote for fixing soname as 20 in a small 19.11.1
> > and release it quickly.
> > 
> > We are supposed to experience tooling and scheme adoption during this
> > first year of new ABI policy. I think we should stick to the planned
> > versionning, in order to avoid any surprise for the next major ABI.
> 
> If the concern is forgetting, I'll happily set a reminder in my
> calendar and then nag everybody :-)
> 
> In my opinion declaring stability and compatibility, with press
> releases and articles, and then immediately breaking it in the first
> point release does not send the right message. Not because there was a
> bug, which happens all the time, but because mantaining backward
> compatibility sometimes means having to carry less-than-ideal or
> downright ugly hacks for a while.
> 
> If we are not ready to maintain compatibility by carrying an ugly
> workaround when the implications are minor or non-existant (I'm pretty
> sure few would really cares what format the soname has, it's all
> handled by tools), what happens next time when the implications are
> that a broken feature stays broken, or that lower performance stays
> lower, and so on?
> 
Valid points. I've already posted one pair of patches for the quick-break
solution that most folks preferred, but it's probably as well to
investigate the alternative too a bit more. I'll try and come up with
something for it tomorrow.

/Bruce

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Neil Horman]

On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> Hi,
> 
> With new process, the major ABI releases will be compatible until it is
> deprecated (until next LTS for now),
> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> will be ABI compatible with this version.
> 
> But if we introduce a new API after major ABI, say in 20.02 release, are we
> allowed to break the ABI for that API before DPDK_20.11?
> 
> If we allow it break, following problem will be observed:
> Assume an application using .so.20.1 library, and using the new API introduced
> in 20.02, lets say foo(),
> but when application switches to .so.20.2 (released via DPDK_20.05), application
> will fail because of ABI breakage in foo().
> 
> I think it is fair that application expects forward compatibility in minor
> versions of a shared library.
> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> 
> If we all agree on this, we may need to tweak the process a little, but before
> diving into implementation details, I would like to be sure we are in same page.
> 
Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
with all future minor releases subsequent to the fixing of that ABI, until the
next major update.  That is to say, once you release ABI_20, all minor updates
20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
released.

Neil

> 
> Thanks,
> ferruh
> 

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Thomas Monjalon]

11/12/2019 14:11, Neil Horman:
> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > Hi,
> > 
> > With new process, the major ABI releases will be compatible until it is
> > deprecated (until next LTS for now),
> > like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> > will be ABI compatible with this version.
> > 
> > But if we introduce a new API after major ABI, say in 20.02 release, are we
> > allowed to break the ABI for that API before DPDK_20.11?
> > 
> > If we allow it break, following problem will be observed:
> > Assume an application using .so.20.1 library, and using the new API introduced
> > in 20.02, lets say foo(),
> > but when application switches to .so.20.2 (released via DPDK_20.05), application
> > will fail because of ABI breakage in foo().
> > 
> > I think it is fair that application expects forward compatibility in minor
> > versions of a shared library.
> > Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> > etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> > 
> > If we all agree on this, we may need to tweak the process a little, but before
> > diving into implementation details, I would like to be sure we are in same page.
> > 
> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> with all future minor releases subsequent to the fixing of that ABI, until the
> next major update.  That is to say, once you release ABI_20, all minor updates
> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> released.

The question of Ferruh was about compatibility of 20.2 vs 20.1,
given both are compatible with 20.0 of course.
The question makes sense if a new symbol is added in 20.1.
And yes I think the symbol added in a minor version must be kept
until the next major ABI.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/11/2019 1:11 PM, Neil Horman wrote:
> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>> Hi,
>>
>> With new process, the major ABI releases will be compatible until it is
>> deprecated (until next LTS for now),
>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>> will be ABI compatible with this version.
>>
>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>> allowed to break the ABI for that API before DPDK_20.11?
>>
>> If we allow it break, following problem will be observed:
>> Assume an application using .so.20.1 library, and using the new API introduced
>> in 20.02, lets say foo(),
>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>> will fail because of ABI breakage in foo().
>>
>> I think it is fair that application expects forward compatibility in minor
>> versions of a shared library.
>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>
>> If we all agree on this, we may need to tweak the process a little, but before
>> diving into implementation details, I would like to be sure we are in same page.
>>
> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> with all future minor releases subsequent to the fixing of that ABI, until the
> next major update.  That is to say, once you release ABI_20, all minor updates
> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> released.

There is a slight difference. All minor versions already compatible with ABI_20,
like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).

Question is if 20.03 should be compatible with 20.02?

This can happen if a new API is introduced in 20.2 and ABI has broken for that
API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
meanwhile both are compatible with ABI_20.

I can see two options:
a) New APIs are introduced only when we switch to new major ABI version. But if
we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
wait up to two years to have (non experimental) APIs.

b) APIs added in minor version will be part of ABI_20 after that point and same
rules will apply to them. Like if and API has introduced in 20.2, it is not
allowed to be broken until next major ABI version.

Thanks,
ferruh

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Neil Horman]

On Tue, Dec 10, 2019 at 02:36:43PM +0000, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > >> Hi,
> > >>
> > >> With new process, the major ABI releases will be compatible until it is
> > >> deprecated (until next LTS for now),
> > >> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> > >> will be ABI compatible with this version.
> > >>
> > >> But if we introduce a new API after major ABI, say in 20.02 release, are we
> > >> allowed to break the ABI for that API before DPDK_20.11?
> > >>
> > >> If we allow it break, following problem will be observed:
> > >> Assume an application using .so.20.1 library, and using the new API introduced
> > >> in 20.02, lets say foo(),
> > >> but when application switches to .so.20.2 (released via DPDK_20.05), application
> > >> will fail because of ABI breakage in foo().
> > >>
> > >> I think it is fair that application expects forward compatibility in minor
> > >> versions of a shared library.
> > >> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> > >> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> > >>
> > >> If we all agree on this, we may need to tweak the process a little, but before
> > >> diving into implementation details, I would like to be sure we are in same page.
> > >>
> > > 
> > > Well, any new API's generally come in as experimental, in which case
> > > changes are allowed, and breakage can be expected. If they are not
> > > experiemental, then the ABI policy applies to them in that they cannot
> > > change since they are part of the .21 ABI, even if that ABI is not fully
> > > complete yet. For any application only using stable, non-experimental
> > > functions, forward compatibility must be maintained IMHO.
> > > 
> > 
> > Talking about not experimental APIs, experimental ones free from the process.
> > 
> > And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
> > it should be supported for following ABI_20.x, instead of calling it ABI_21, and
> > this minor tweak (and mind shift) in .map files can be our solution.
> 
> Related at what to do with adding versions between major ABI versions, when
> investigating with Kevin the ABI checking we have made an unpleasant
> discovery:
> 
> This minor version bumping from 20.0 to 20.1 has apparently already broken
> our ABI according to libabigail.
> 
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
> 
> The reason for this is that the soversion encoded in each library - whether
> built with meson or make - is the full 20.0 version, not just the major ABI
> .20 part. Then when apps link against DPDK, they actually encode the 20.0.
> 
> So what this means is that currently - using a make build as an example
> here - ldd on the latest head build gives:
> 
>  LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
>         linux-vdso.so.1 (0x00007fff6813d000)
>         librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
>         librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
>         librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
>         librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
>         librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
>         librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
>         librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
>         librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
>         librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> ...
> 
> Similarly ldd on a 19.11 checkout gives:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc2a964000)
>         librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
>         librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
>         librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
>         librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
>         librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
>         librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
>         librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
>         librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
>         librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
> 
Whats going on here?  A 19.11 checkout should have DT_NEEDED encoded in
the scheme lib*.so.19.x, no?

> The final check - using the 19.11 compiled testpmd with the library path
> set to 20.02 versionned libs:
> 
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>         linux-vdso.so.1 (0x00007ffc711fc000)
>         librte_pmd_bond.so.20.0 => not found
>         librte_pmd_dpaa.so.20.0 => not found
>         librte_mempool_dpaa.so.20.0 => not found
>         librte_pmd_ixgbe.so.20.0 => not found
>         librte_pmd_i40e.so.20.0 => not found
>         librte_pmd_bnxt.so.20.0 => not found
>         librte_pmd_softnic.so.20.0 => not found
>         librte_flow_classify.so.0.200 => not found
>         librte_pipeline.so.20.0 => not found
> 
Shouldn't ldconfig fix that?  i.e. when you install libraries, you run ldconfig
and the various minor release libraries should be symlinks to the latest version
of the minor release.  That is to say, your library patch for, for example,
librte_pnd_bnxt should look like:
librte_pmd_bnxt.so.20.02
librte_pmd_bnxt.so.20 => librete_pmd_bnxt.so.20
librte_pmd_bnxt.so => librte_pmd_bnxt.so.20.02

Which will allow older built binaries to link at run time.

> Fixing This
> -----------
> 
> To fix this, we need to ensure that the SONAME remains constant across the
> releases. Therefore, I currently see two options:
> 
> 1. keep 20.0 as the version and soname across all releases in 2020, i.e.
>   just revert the ABIVERSION change patch. Trouble there is how to track
>   20.02 vs 20.05 etc. etc.
> 
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
>   advantage of keeping the existing planned schemes, but has the really big
>   downside of breaking ABI compatibility with anyone who has already
>   compiled with 19.11.
> 
> Personally, of the two options - unless someone can come up with a third
> option - I'd tend towards the second, fixing the builds to remove the .0 in
> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
> adoption. Since this ABI stability is new, teething problems may be
> expected.
> 
> Thoughts and comments?
> /Bruce
> 
> BTW: For meson, the patch for option 2 is just to remove the so_version
> variable and all references to it from lib/meson.build and
> drivers/meson.build. Haven't looked into a "make" fix yet.
> 
> 

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Neil Horman]

On Wed, Dec 11, 2019 at 01:30:07PM +0000, Ferruh Yigit wrote:
> On 12/11/2019 1:11 PM, Neil Horman wrote:
> > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> >> Hi,
> >>
> >> With new process, the major ABI releases will be compatible until it is
> >> deprecated (until next LTS for now),
> >> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> >> will be ABI compatible with this version.
> >>
> >> But if we introduce a new API after major ABI, say in 20.02 release, are we
> >> allowed to break the ABI for that API before DPDK_20.11?
> >>
> >> If we allow it break, following problem will be observed:
> >> Assume an application using .so.20.1 library, and using the new API introduced
> >> in 20.02, lets say foo(),
> >> but when application switches to .so.20.2 (released via DPDK_20.05), application
> >> will fail because of ABI breakage in foo().
> >>
> >> I think it is fair that application expects forward compatibility in minor
> >> versions of a shared library.
> >> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> >> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> >>
> >> If we all agree on this, we may need to tweak the process a little, but before
> >> diving into implementation details, I would like to be sure we are in same page.
> >>
> > Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> > with all future minor releases subsequent to the fixing of that ABI, until the
> > next major update.  That is to say, once you release ABI_20, all minor updates
> > 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> > released.
> 
> There is a slight difference. All minor versions already compatible with ABI_20,
> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
> 
> Question is if 20.03 should be compatible with 20.02?
> 
Yes, as long as that new API was _not_ introduced with the experimental tag,
then its part of the ABI.

Its less about defining ABI levels, and more about customer compatibility in my
mind.

Regardless of what policy we want to set, if we publish a symbol in a library,
unless we clearly mark it as being experimental/unstable, consumers of the
library might use it, and will expect it to be stable for the duragion of that
libraries major version.

Thats how consumers expect this to work.  For a given major release, all minor
releases should function in a simmilar fashion.  If we introduce a new feature
in a minor release, thats fine, but all subsequent minor releases need to
maintain that.

Neil

> This can happen if a new API is introduced in 20.2 and ABI has broken for that
> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
> meanwhile both are compatible with ABI_20.
> 
> I can see two options:
> a) New APIs are introduced only when we switch to new major ABI version. But if
> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
> wait up to two years to have (non experimental) APIs.
> 
> b) APIs added in minor version will be part of ABI_20 after that point and same
> rules will apply to them. Like if and API has introduced in 20.2, it is not
> allowed to be broken until next major ABI version.
> 
> Thanks,
> ferruh
> 
> 

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Thomas Monjalon]

11/12/2019 14:30, Ferruh Yigit:
> On 12/11/2019 1:11 PM, Neil Horman wrote:
> > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> >> Hi,
> >>
> >> With new process, the major ABI releases will be compatible until it is
> >> deprecated (until next LTS for now),
> >> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> >> will be ABI compatible with this version.
> >>
> >> But if we introduce a new API after major ABI, say in 20.02 release, are we
> >> allowed to break the ABI for that API before DPDK_20.11?
> >>
> >> If we allow it break, following problem will be observed:
> >> Assume an application using .so.20.1 library, and using the new API introduced
> >> in 20.02, lets say foo(),
> >> but when application switches to .so.20.2 (released via DPDK_20.05), application
> >> will fail because of ABI breakage in foo().
> >>
> >> I think it is fair that application expects forward compatibility in minor
> >> versions of a shared library.
> >> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> >> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> >>
> >> If we all agree on this, we may need to tweak the process a little, but before
> >> diving into implementation details, I would like to be sure we are in same page.
> >>
> > Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> > with all future minor releases subsequent to the fixing of that ABI, until the
> > next major update.  That is to say, once you release ABI_20, all minor updates
> > 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> > released.
> 
> There is a slight difference. All minor versions already compatible with ABI_20,
> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
> 
> Question is if 20.03 should be compatible with 20.02?
> 
> This can happen if a new API is introduced in 20.2 and ABI has broken for that
> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
> meanwhile both are compatible with ABI_20.
> 
> I can see two options:
> a) New APIs are introduced only when we switch to new major ABI version. But if
> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
> wait up to two years to have (non experimental) APIs.

I agree we should allow to add a new stable API in the middle of an ABI lifecycle.

> b) APIs added in minor version will be part of ABI_20 after that point and same
> rules will apply to them. Like if and API has introduced in 20.2, it is not
> allowed to be broken until next major ABI version.

Yes I think it is compliant with the agreed policy.
Note that an app linked with ABI 20.2 won't be compatible with ABI 20.1,
though the reverse works.

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Wed, Dec 11, 2019 at 04:02:16PM +0100, Thomas Monjalon wrote:
> 11/12/2019 14:30, Ferruh Yigit:
> > On 12/11/2019 1:11 PM, Neil Horman wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > >> Hi,
> > >>
> > >> With new process, the major ABI releases will be compatible until it is
> > >> deprecated (until next LTS for now),
> > >> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> > >> will be ABI compatible with this version.
> > >>
> > >> But if we introduce a new API after major ABI, say in 20.02 release, are we
> > >> allowed to break the ABI for that API before DPDK_20.11?
> > >>
> > >> If we allow it break, following problem will be observed:
> > >> Assume an application using .so.20.1 library, and using the new API introduced
> > >> in 20.02, lets say foo(),
> > >> but when application switches to .so.20.2 (released via DPDK_20.05), application
> > >> will fail because of ABI breakage in foo().
> > >>
> > >> I think it is fair that application expects forward compatibility in minor
> > >> versions of a shared library.
> > >> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> > >> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> > >>
> > >> If we all agree on this, we may need to tweak the process a little, but before
> > >> diving into implementation details, I would like to be sure we are in same page.
> > >>
> > > Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> > > with all future minor releases subsequent to the fixing of that ABI, until the
> > > next major update.  That is to say, once you release ABI_20, all minor updates
> > > 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> > > released.
> > 
> > There is a slight difference. All minor versions already compatible with ABI_20,
> > like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
> > 
> > Question is if 20.03 should be compatible with 20.02?
> > 
> > This can happen if a new API is introduced in 20.2 and ABI has broken for that
> > API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
> > meanwhile both are compatible with ABI_20.
> > 
> > I can see two options:
> > a) New APIs are introduced only when we switch to new major ABI version. But if
> > we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
> > wait up to two years to have (non experimental) APIs.
> 
> I agree we should allow to add a new stable API in the middle of an ABI lifecycle.
> 
> > b) APIs added in minor version will be part of ABI_20 after that point and same
> > rules will apply to them. Like if and API has introduced in 20.2, it is not
> > allowed to be broken until next major ABI version.
> 
> Yes I think it is compliant with the agreed policy.
> Note that an app linked with ABI 20.2 won't be compatible with ABI 20.1,
> though the reverse works.
>
Which I think is fine, as it is the way most people would expect it to
work. 

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/11/2019 2:34 PM, Neil Horman wrote:
> On Wed, Dec 11, 2019 at 01:30:07PM +0000, Ferruh Yigit wrote:
>> On 12/11/2019 1:11 PM, Neil Horman wrote:
>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>> Hi,
>>>>
>>>> With new process, the major ABI releases will be compatible until it is
>>>> deprecated (until next LTS for now),
>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>> will be ABI compatible with this version.
>>>>
>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>
>>>> If we allow it break, following problem will be observed:
>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>> in 20.02, lets say foo(),
>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>> will fail because of ABI breakage in foo().
>>>>
>>>> I think it is fair that application expects forward compatibility in minor
>>>> versions of a shared library.
>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>
>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>
>>> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
>>> with all future minor releases subsequent to the fixing of that ABI, until the
>>> next major update.  That is to say, once you release ABI_20, all minor updates
>>> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
>>> released.
>>
>> There is a slight difference. All minor versions already compatible with ABI_20,
>> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
>>
>> Question is if 20.03 should be compatible with 20.02?
>>
> Yes, as long as that new API was _not_ introduced with the experimental tag,
> then its part of the ABI.
> 
> Its less about defining ABI levels, and more about customer compatibility in my
> mind.
> 
> Regardless of what policy we want to set, if we publish a symbol in a library,
> unless we clearly mark it as being experimental/unstable, consumers of the
> library might use it, and will expect it to be stable for the duragion of that
> libraries major version.
> 
> Thats how consumers expect this to work.  For a given major release, all minor
> releases should function in a simmilar fashion.  If we introduce a new feature
> in a minor release, thats fine, but all subsequent minor releases need to
> maintain that.

+1, thanks.

> 
> Neil
> 
>> This can happen if a new API is introduced in 20.2 and ABI has broken for that
>> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
>> meanwhile both are compatible with ABI_20.
>>
>> I can see two options:
>> a) New APIs are introduced only when we switch to new major ABI version. But if
>> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
>> wait up to two years to have (non experimental) APIs.
>>
>> b) APIs added in minor version will be part of ABI_20 after that point and same
>> rules will apply to them. Like if and API has introduced in 20.2, it is not
>> allowed to be broken until next major ABI version.
>>
>> Thanks,
>> ferruh
>>
>>

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/11/2019 3:02 PM, Thomas Monjalon wrote:
> 11/12/2019 14:30, Ferruh Yigit:
>> On 12/11/2019 1:11 PM, Neil Horman wrote:
>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>> Hi,
>>>>
>>>> With new process, the major ABI releases will be compatible until it is
>>>> deprecated (until next LTS for now),
>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>> will be ABI compatible with this version.
>>>>
>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>
>>>> If we allow it break, following problem will be observed:
>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>> in 20.02, lets say foo(),
>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>> will fail because of ABI breakage in foo().
>>>>
>>>> I think it is fair that application expects forward compatibility in minor
>>>> versions of a shared library.
>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>
>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>
>>> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
>>> with all future minor releases subsequent to the fixing of that ABI, until the
>>> next major update.  That is to say, once you release ABI_20, all minor updates
>>> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
>>> released.
>>
>> There is a slight difference. All minor versions already compatible with ABI_20,
>> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
>>
>> Question is if 20.03 should be compatible with 20.02?
>>
>> This can happen if a new API is introduced in 20.2 and ABI has broken for that
>> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
>> meanwhile both are compatible with ABI_20.
>>
>> I can see two options:
>> a) New APIs are introduced only when we switch to new major ABI version. But if
>> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
>> wait up to two years to have (non experimental) APIs.
> 
> I agree we should allow to add a new stable API in the middle of an ABI lifecycle.
> 
>> b) APIs added in minor version will be part of ABI_20 after that point and same
>> rules will apply to them. Like if and API has introduced in 20.2, it is not
>> allowed to be broken until next major ABI version.
> 
> Yes I think it is compliant with the agreed policy.

So I think two minor changes are required in the process to reflect this,
1) In ABI policy [1], it mentions in minor release both ABI_20 and ABI_21 can
exist together, also in graph it says for minor versions:
"v21 symbols are added and v20 symbols are modified, support for v20 ABI continues."
I am not sure if we can call the symbols added in minor versions as v21 ABI,
because it implies ABI compatibility is not required for them.

2) In ABI versioning [2], documented as .map files will have 'DPDK_20' and
'DPDK_21' blocks.
But instead, I think they should have 'DPDK_20.0', 'DPDK_20.1', ... blocks, and
when major ABI version changed they all can be flattened to 'DPDK_21.0'.
For example we can't do ABI versioning between 20.2 & 20.3 if we don't have
these blocks.
Current block names in .map files are already defined as 'DPDK_20.0', what we
need to do is update the document to use 'DPDK_20.x' for the symbols added in
minor version and follow that process.


[1]
https://doc.dpdk.org/guides/contributing/abi_policy.html#the-dpdk-abi-policy

[2]
https://doc.dpdk.org/guides/contributing/abi_versioning.html#examples-of-abi-macro-use

> Note that an app linked with ABI 20.2 won't be compatible with ABI 20.1,
> though the reverse works.
> 
> 
> 

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Wed, Dec 11, 2019 at 03:46:10PM +0000, Ferruh Yigit wrote:
> On 12/11/2019 3:02 PM, Thomas Monjalon wrote:
> > 11/12/2019 14:30, Ferruh Yigit:
> >> On 12/11/2019 1:11 PM, Neil Horman wrote:
> >>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> >>>> Hi,
> >>>>
> >>>> With new process, the major ABI releases will be compatible until it is
> >>>> deprecated (until next LTS for now),
> >>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> >>>> will be ABI compatible with this version.
> >>>>
> >>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
> >>>> allowed to break the ABI for that API before DPDK_20.11?
> >>>>
> >>>> If we allow it break, following problem will be observed:
> >>>> Assume an application using .so.20.1 library, and using the new API introduced
> >>>> in 20.02, lets say foo(),
> >>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
> >>>> will fail because of ABI breakage in foo().
> >>>>
> >>>> I think it is fair that application expects forward compatibility in minor
> >>>> versions of a shared library.
> >>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> >>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> >>>>
> >>>> If we all agree on this, we may need to tweak the process a little, but before
> >>>> diving into implementation details, I would like to be sure we are in same page.
> >>>>
> >>> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> >>> with all future minor releases subsequent to the fixing of that ABI, until the
> >>> next major update.  That is to say, once you release ABI_20, all minor updates
> >>> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> >>> released.
> >>
> >> There is a slight difference. All minor versions already compatible with ABI_20,
> >> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
> >>
> >> Question is if 20.03 should be compatible with 20.02?
> >>
> >> This can happen if a new API is introduced in 20.2 and ABI has broken for that
> >> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
> >> meanwhile both are compatible with ABI_20.
> >>
> >> I can see two options:
> >> a) New APIs are introduced only when we switch to new major ABI version. But if
> >> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
> >> wait up to two years to have (non experimental) APIs.
> > 
> > I agree we should allow to add a new stable API in the middle of an ABI lifecycle.
> > 
> >> b) APIs added in minor version will be part of ABI_20 after that point and same
> >> rules will apply to them. Like if and API has introduced in 20.2, it is not
> >> allowed to be broken until next major ABI version.
> > 
> > Yes I think it is compliant with the agreed policy.
> 
> So I think two minor changes are required in the process to reflect this,
> 1) In ABI policy [1], it mentions in minor release both ABI_20 and ABI_21 can
> exist together, also in graph it says for minor versions:
> "v21 symbols are added and v20 symbols are modified, support for v20 ABI continues."
> I am not sure if we can call the symbols added in minor versions as v21 ABI,
> because it implies ABI compatibility is not required for them.
> 
> 2) In ABI versioning [2], documented as .map files will have 'DPDK_20' and
> 'DPDK_21' blocks.
> But instead, I think they should have 'DPDK_20.0', 'DPDK_20.1', ... blocks, and
> when major ABI version changed they all can be flattened to 'DPDK_21.0'.
> For example we can't do ABI versioning between 20.2 & 20.3 if we don't have
> these blocks.
> Current block names in .map files are already defined as 'DPDK_20.0', what we
> need to do is update the document to use 'DPDK_20.x' for the symbols added in
> minor version and follow that process.
> 

What do we really gain from making such a change from the policy? I think
it will work fine as-is, with putting all new symbols in the DPDK_21
section. Whatever way you look at it, the functions will be forward but not
backward compatible, which is all that really matters.

/Bruce

Re: [dpdk-dev] How to manage new APIs added after major ABI release?

[Ferruh Yigit]

On 12/11/2019 3:55 PM, Bruce Richardson wrote:
> On Wed, Dec 11, 2019 at 03:46:10PM +0000, Ferruh Yigit wrote:
>> On 12/11/2019 3:02 PM, Thomas Monjalon wrote:
>>> 11/12/2019 14:30, Ferruh Yigit:
>>>> On 12/11/2019 1:11 PM, Neil Horman wrote:
>>>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>>>> Hi,
>>>>>>
>>>>>> With new process, the major ABI releases will be compatible until it is
>>>>>> deprecated (until next LTS for now),
>>>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>>>> will be ABI compatible with this version.
>>>>>>
>>>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>>>
>>>>>> If we allow it break, following problem will be observed:
>>>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>>>> in 20.02, lets say foo(),
>>>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>>>> will fail because of ABI breakage in foo().
>>>>>>
>>>>>> I think it is fair that application expects forward compatibility in minor
>>>>>> versions of a shared library.
>>>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>>>
>>>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>>>
>>>>> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
>>>>> with all future minor releases subsequent to the fixing of that ABI, until the
>>>>> next major update.  That is to say, once you release ABI_20, all minor updates
>>>>> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
>>>>> released.
>>>>
>>>> There is a slight difference. All minor versions already compatible with ABI_20,
>>>> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
>>>>
>>>> Question is if 20.03 should be compatible with 20.02?
>>>>
>>>> This can happen if a new API is introduced in 20.2 and ABI has broken for that
>>>> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
>>>> meanwhile both are compatible with ABI_20.
>>>>
>>>> I can see two options:
>>>> a) New APIs are introduced only when we switch to new major ABI version. But if
>>>> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
>>>> wait up to two years to have (non experimental) APIs.
>>>
>>> I agree we should allow to add a new stable API in the middle of an ABI lifecycle.
>>>
>>>> b) APIs added in minor version will be part of ABI_20 after that point and same
>>>> rules will apply to them. Like if and API has introduced in 20.2, it is not
>>>> allowed to be broken until next major ABI version.
>>>
>>> Yes I think it is compliant with the agreed policy.
>>
>> So I think two minor changes are required in the process to reflect this,
>> 1) In ABI policy [1], it mentions in minor release both ABI_20 and ABI_21 can
>> exist together, also in graph it says for minor versions:
>> "v21 symbols are added and v20 symbols are modified, support for v20 ABI continues."
>> I am not sure if we can call the symbols added in minor versions as v21 ABI,
>> because it implies ABI compatibility is not required for them.
>>
>> 2) In ABI versioning [2], documented as .map files will have 'DPDK_20' and
>> 'DPDK_21' blocks.
>> But instead, I think they should have 'DPDK_20.0', 'DPDK_20.1', ... blocks, and
>> when major ABI version changed they all can be flattened to 'DPDK_21.0'.
>> For example we can't do ABI versioning between 20.2 & 20.3 if we don't have
>> these blocks.
>> Current block names in .map files are already defined as 'DPDK_20.0', what we
>> need to do is update the document to use 'DPDK_20.x' for the symbols added in
>> minor version and follow that process.
>>
> 
> What do we really gain from making such a change from the policy? I think
> it will work fine as-is, with putting all new symbols in the DPDK_21
> section. Whatever way you look at it, the functions will be forward but not
> backward compatible, which is all that really matters.
> 

As mentioned above it allows us ABI versioning between minor versions.

Also clarifies the intention, how current process puts can be read as ABI_21
APIs can be changed until 21 becomes supported ABI version. (because only ABI_20
is ABI compatible), so the question I asked in this thread no needed to be asked
and will be obvious.