* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 14:36 ` Bruce Richardson
@ 2019-12-10 15:03 ` Luca Boccassi
2019-12-10 15:46 ` Bruce Richardson
2019-12-10 15:04 ` Ferruh Yigit
` (2 subsequent siblings)
3 siblings, 1 reply; 28+ messages in thread
From: Luca Boccassi @ 2019-12-10 15:03 UTC (permalink / raw)
To: Bruce Richardson, Ferruh Yigit
Cc: Kinsella, Ray, Thomas Monjalon, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > Hi,
> > > >
> > > > With new process, the major ABI releases will be compatible
> > > > until it is
> > > > deprecated (until next LTS for now),
> > > > like current ABI version is 20 in DPDK_19.11 and DPDK versions
> > > > until DPDK_20.11
> > > > will be ABI compatible with this version.
> > > >
> > > > But if we introduce a new API after major ABI, say in 20.02
> > > > release, are we
> > > > allowed to break the ABI for that API before DPDK_20.11?
> > > >
> > > > If we allow it break, following problem will be observed:
> > > > Assume an application using .so.20.1 library, and using the new
> > > > API introduced
> > > > in 20.02, lets say foo(),
> > > > but when application switches to .so.20.2 (released via
> > > > DPDK_20.05), application
> > > > will fail because of ABI breakage in foo().
> > > >
> > > > I think it is fair that application expects forward
> > > > compatibility in minor
> > > > versions of a shared library.
> > > > Like if application linked against .so.20.2, fair to expect
> > > > .so.20.3, .so.20.4
> > > > etc will work fine. I think currently only .so.20.0 is fully
> > > > forward compatible.
> > > >
> > > > If we all agree on this, we may need to tweak the process a
> > > > little, but before
> > > > diving into implementation details, I would like to be sure we
> > > > are in same page.
> > > >
> > >
> > > Well, any new API's generally come in as experimental, in which
> > > case
> > > changes are allowed, and breakage can be expected. If they are
> > > not
> > > experiemental, then the ABI policy applies to them in that they
> > > cannot
> > > change since they are part of the .21 ABI, even if that ABI is
> > > not fully
> > > complete yet. For any application only using stable, non-
> > > experimental
> > > functions, forward compatibility must be maintained IMHO.
> > >
> >
> > Talking about not experimental APIs, experimental ones free from
> > the process.
> >
> > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > ABI_20, because
> > it should be supported for following ABI_20.x, instead of calling
> > it ABI_21, and
> > this minor tweak (and mind shift) in .map files can be our
> > solution.
>
> Related at what to do with adding versions between major ABI
> versions, when
> investigating with Kevin the ABI checking we have made an unpleasant
> discovery:
>
> This minor version bumping from 20.0 to 20.1 has apparently already
> broken
> our ABI according to libabigail.
>
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
>
> The reason for this is that the soversion encoded in each library -
> whether
> built with meson or make - is the full 20.0 version, not just the
> major ABI
> .20 part. Then when apps link against DPDK, they actually encode the
> 20.0.
>
> So what this means is that currently - using a make build as an
> example
> here - ldd on the latest head build gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> native-linux-gcc/app/testpmd | head
> linux-vdso.so.1 (0x00007fff6813d000)
> librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
> librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
> librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
> librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_flow_classify.so.0.201
> (0x00007f36d70b1000)
> librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> ...
>
> Similarly ldd on a 19.11 checkout gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-
> native-linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc2a964000)
> librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> (0x00007fd4dc6b6000)
> librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> (0x00007fd4dc6a3000)
> librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> (0x00007fd4dc69e000)
> librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> (0x00007fd4dc634000)
> librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> (0x00007fd4dc5a0000)
> librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> (0x00007fd4dc55d000)
> librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> (0x00007fd4dc531000)
> librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> (0x00007fd4dc52b000)
> librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> (0x00007fd4dc502000)
>
> The final check - using the 19.11 compiled testpmd with the library
> path
> set to 20.02 versionned libs:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc711fc000)
> librte_pmd_bond.so.20.0 => not found
> librte_pmd_dpaa.so.20.0 => not found
> librte_mempool_dpaa.so.20.0 => not found
> librte_pmd_ixgbe.so.20.0 => not found
> librte_pmd_i40e.so.20.0 => not found
> librte_pmd_bnxt.so.20.0 => not found
> librte_pmd_softnic.so.20.0 => not found
> librte_flow_classify.so.0.200 => not found
> librte_pipeline.so.20.0 => not found
>
> Fixing This
> -----------
>
> To fix this, we need to ensure that the SONAME remains constant
> across the
> releases. Therefore, I currently see two options:
>
> 1. keep 20.0 as the version and soname across all releases in 2020,
> i.e.
> just revert the ABIVERSION change patch. Trouble there is how to
> track
> 20.02 vs 20.05 etc. etc.
>
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This
> has the
> advantage of keeping the existing planned schemes, but has the
> really big
> downside of breaking ABI compatibility with anyone who has already
> compiled with 19.11.
>
> Personally, of the two options - unless someone can come up with a
> third
> option - I'd tend towards the second, fixing the builds to remove the
> .0 in
> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> widespread
> adoption. Since this ABI stability is new, teething problems may be
> expected.
>
> Thoughts and comments?
> /Bruce
>
> BTW: For meson, the patch for option 2 is just to remove the
> so_version
> variable and all references to it from lib/meson.build and
> drivers/meson.build. Haven't looked into a "make" fix yet.
Hi,
With libtool and its (arguably arcane) format, only the first digit is
the ABI current version and gets encoded in the elf header. The other
digits can be used to track compatible minor increments, and are mostly
ignored. On the system a symlink libfoo.so.major ->
libfoo.so.major.minor is added.
Eg:
$ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep SONAME
0x000000000000000e (SONAME) Library soname: [libzmq.so.5]
$ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
gnu/libzmq.so.5 -> libzmq.so.5.2.3
Can we do the same? Not sure what the right incantation is for Meson,
but it should be possibly.
Also, we should leave the current at 20.0 - let's not break
compatibility already, please :-)
--
Kind regards,
Luca Boccassi
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 15:03 ` Luca Boccassi
@ 2019-12-10 15:46 ` Bruce Richardson
2019-12-10 16:20 ` Luca Boccassi
0 siblings, 1 reply; 28+ messages in thread
From: Bruce Richardson @ 2019-12-10 15:46 UTC (permalink / raw)
To: Luca Boccassi
Cc: Ferruh Yigit, Kinsella, Ray, Thomas Monjalon, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > Hi,
> > > > >
> > > > > With new process, the major ABI releases will be compatible
> > > > > until it is
> > > > > deprecated (until next LTS for now),
> > > > > like current ABI version is 20 in DPDK_19.11 and DPDK versions
> > > > > until DPDK_20.11
> > > > > will be ABI compatible with this version.
> > > > >
> > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > release, are we
> > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > >
> > > > > If we allow it break, following problem will be observed:
> > > > > Assume an application using .so.20.1 library, and using the new
> > > > > API introduced
> > > > > in 20.02, lets say foo(),
> > > > > but when application switches to .so.20.2 (released via
> > > > > DPDK_20.05), application
> > > > > will fail because of ABI breakage in foo().
> > > > >
> > > > > I think it is fair that application expects forward
> > > > > compatibility in minor
> > > > > versions of a shared library.
> > > > > Like if application linked against .so.20.2, fair to expect
> > > > > .so.20.3, .so.20.4
> > > > > etc will work fine. I think currently only .so.20.0 is fully
> > > > > forward compatible.
> > > > >
> > > > > If we all agree on this, we may need to tweak the process a
> > > > > little, but before
> > > > > diving into implementation details, I would like to be sure we
> > > > > are in same page.
> > > > >
> > > >
> > > > Well, any new API's generally come in as experimental, in which
> > > > case
> > > > changes are allowed, and breakage can be expected. If they are
> > > > not
> > > > experiemental, then the ABI policy applies to them in that they
> > > > cannot
> > > > change since they are part of the .21 ABI, even if that ABI is
> > > > not fully
> > > > complete yet. For any application only using stable, non-
> > > > experimental
> > > > functions, forward compatibility must be maintained IMHO.
> > > >
> > >
> > > Talking about not experimental APIs, experimental ones free from
> > > the process.
> > >
> > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > ABI_20, because
> > > it should be supported for following ABI_20.x, instead of calling
> > > it ABI_21, and
> > > this minor tweak (and mind shift) in .map files can be our
> > > solution.
> >
> > Related at what to do with adding versions between major ABI
> > versions, when
> > investigating with Kevin the ABI checking we have made an unpleasant
> > discovery:
> >
> > This minor version bumping from 20.0 to 20.1 has apparently already
> > broken
> > our ABI according to libabigail.
> >
> > The Gory Details [skip to the end for suggestions to fix]
> > ------------------------------------------------------------
> >
> > The reason for this is that the soversion encoded in each library -
> > whether
> > built with meson or make - is the full 20.0 version, not just the
> > major ABI
> > .20 part. Then when apps link against DPDK, they actually encode the
> > 20.0.
> >
> > So what this means is that currently - using a make build as an
> > example
> > here - ldd on the latest head build gives:
> >
> > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > native-linux-gcc/app/testpmd | head
> > linux-vdso.so.1 (0x00007fff6813d000)
> > librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
> > librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
> > librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
> > librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > (0x00007f36d70b1000)
> > librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > ...
> >
> > Similarly ldd on a 19.11 checkout gives:
> >
> > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-
> > native-linux-gcc_v19.11/app/testpmd | head
> > linux-vdso.so.1 (0x00007ffc2a964000)
> > librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > (0x00007fd4dc6b6000)
> > librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > (0x00007fd4dc6a3000)
> > librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > (0x00007fd4dc69e000)
> > librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > (0x00007fd4dc634000)
> > librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > (0x00007fd4dc5a0000)
> > librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > (0x00007fd4dc55d000)
> > librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > (0x00007fd4dc531000)
> > librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > (0x00007fd4dc52b000)
> > librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > (0x00007fd4dc502000)
> >
> > The final check - using the 19.11 compiled testpmd with the library
> > path
> > set to 20.02 versionned libs:
> >
> > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> > linux-gcc_v19.11/app/testpmd | head
> > linux-vdso.so.1 (0x00007ffc711fc000)
> > librte_pmd_bond.so.20.0 => not found
> > librte_pmd_dpaa.so.20.0 => not found
> > librte_mempool_dpaa.so.20.0 => not found
> > librte_pmd_ixgbe.so.20.0 => not found
> > librte_pmd_i40e.so.20.0 => not found
> > librte_pmd_bnxt.so.20.0 => not found
> > librte_pmd_softnic.so.20.0 => not found
> > librte_flow_classify.so.0.200 => not found
> > librte_pipeline.so.20.0 => not found
> >
> > Fixing This
> > -----------
> >
> > To fix this, we need to ensure that the SONAME remains constant
> > across the
> > releases. Therefore, I currently see two options:
> >
> > 1. keep 20.0 as the version and soname across all releases in 2020,
> > i.e.
> > just revert the ABIVERSION change patch. Trouble there is how to
> > track
> > 20.02 vs 20.05 etc. etc.
> >
> > 2. remove the .0, .1 from the SONAMES stored in the libraries. This
> > has the
> > advantage of keeping the existing planned schemes, but has the
> > really big
> > downside of breaking ABI compatibility with anyone who has already
> > compiled with 19.11.
> >
> > Personally, of the two options - unless someone can come up with a
> > third
> > option - I'd tend towards the second, fixing the builds to remove the
> > .0 in
> > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > widespread
> > adoption. Since this ABI stability is new, teething problems may be
> > expected.
> >
> > Thoughts and comments?
> > /Bruce
> >
> > BTW: For meson, the patch for option 2 is just to remove the
> > so_version
> > variable and all references to it from lib/meson.build and
> > drivers/meson.build. Haven't looked into a "make" fix yet.
>
> Hi,
>
> With libtool and its (arguably arcane) format, only the first digit is
> the ABI current version and gets encoded in the elf header. The other
> digits can be used to track compatible minor increments, and are mostly
> ignored. On the system a symlink libfoo.so.major ->
> libfoo.so.major.minor is added.
>
> Eg:
>
> $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep SONAME
> 0x000000000000000e (SONAME) Library soname: [libzmq.so.5]
> $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> gnu/libzmq.so.5 -> libzmq.so.5.2.3
>
> Can we do the same? Not sure what the right incantation is for Meson,
> but it should be possibly.
>
That's essentially option 2, and it's still an ABI break because existing
builds of 19.11 have the soname will the full version number in it. The
default behaviour for meson is exactly how you described it, except that
previously we needed more exact control over the version info (for your
dpdk-specific versions in the sonames) and so overrode the soversion
explicitly. The fix for meson is to remove this overriding i.e. remove
"soversion:" parameter for each shared_library() call.
> Also, we should leave the current at 20.0 - let's not break
> compatibility already, please :-)
>
If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 15:46 ` Bruce Richardson
@ 2019-12-10 16:20 ` Luca Boccassi
2019-12-10 16:32 ` Bruce Richardson
2019-12-10 16:39 ` Thomas Monjalon
0 siblings, 2 replies; 28+ messages in thread
From: Luca Boccassi @ 2019-12-10 16:20 UTC (permalink / raw)
To: Bruce Richardson
Cc: Ferruh Yigit, Kinsella, Ray, Thomas Monjalon, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > Hi,
> > > > > >
> > > > > > With new process, the major ABI releases will be compatible
> > > > > > until it is
> > > > > > deprecated (until next LTS for now),
> > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > versions
> > > > > > until DPDK_20.11
> > > > > > will be ABI compatible with this version.
> > > > > >
> > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > release, are we
> > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > >
> > > > > > If we allow it break, following problem will be observed:
> > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > new
> > > > > > API introduced
> > > > > > in 20.02, lets say foo(),
> > > > > > but when application switches to .so.20.2 (released via
> > > > > > DPDK_20.05), application
> > > > > > will fail because of ABI breakage in foo().
> > > > > >
> > > > > > I think it is fair that application expects forward
> > > > > > compatibility in minor
> > > > > > versions of a shared library.
> > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > .so.20.3, .so.20.4
> > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > fully
> > > > > > forward compatible.
> > > > > >
> > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > little, but before
> > > > > > diving into implementation details, I would like to be sure
> > > > > > we
> > > > > > are in same page.
> > > > > >
> > > > >
> > > > > Well, any new API's generally come in as experimental, in
> > > > > which
> > > > > case
> > > > > changes are allowed, and breakage can be expected. If they
> > > > > are
> > > > > not
> > > > > experiemental, then the ABI policy applies to them in that
> > > > > they
> > > > > cannot
> > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > is
> > > > > not fully
> > > > > complete yet. For any application only using stable, non-
> > > > > experimental
> > > > > functions, forward compatibility must be maintained IMHO.
> > > > >
> > > >
> > > > Talking about not experimental APIs, experimental ones free
> > > > from
> > > > the process.
> > > >
> > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > ABI_20, because
> > > > it should be supported for following ABI_20.x, instead of
> > > > calling
> > > > it ABI_21, and
> > > > this minor tweak (and mind shift) in .map files can be our
> > > > solution.
> > >
> > > Related at what to do with adding versions between major ABI
> > > versions, when
> > > investigating with Kevin the ABI checking we have made an
> > > unpleasant
> > > discovery:
> > >
> > > This minor version bumping from 20.0 to 20.1 has apparently
> > > already
> > > broken
> > > our ABI according to libabigail.
> > >
> > > The Gory Details [skip to the end for suggestions to fix]
> > > ------------------------------------------------------------
> > >
> > > The reason for this is that the soversion encoded in each library
> > > -
> > > whether
> > > built with meson or make - is the full 20.0 version, not just the
> > > major ABI
> > > .20 part. Then when apps link against DPDK, they actually encode
> > > the
> > > 20.0.
> > >
> > > So what this means is that currently - using a make build as an
> > > example
> > > here - ldd on the latest head build gives:
> > >
> > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > native-linux-gcc/app/testpmd | head
> > > linux-vdso.so.1 (0x00007fff6813d000)
> > > librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > librte_mempool_dpaa.so.20.1 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > (0x00007f36d7224000)
> > > librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > (0x00007f36d71ba000)
> > > librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > librte_pmd_softnic.so.20.1 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > (0x00007f36d70b7000)
> > > librte_flow_classify.so.0.201 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > (0x00007f36d70b1000)
> > > librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > ...
> > >
> > > Similarly ldd on a 19.11 checkout gives:
> > >
> > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > x86_64-
> > > native-linux-gcc_v19.11/app/testpmd | head
> > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > (0x00007fd4dc6b6000)
> > > librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > (0x00007fd4dc6a3000)
> > > librte_mempool_dpaa.so.20.0 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > (0x00007fd4dc69e000)
> > > librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > (0x00007fd4dc634000)
> > > librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > (0x00007fd4dc5a0000)
> > > librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > (0x00007fd4dc55d000)
> > > librte_pmd_softnic.so.20.0 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > (0x00007fd4dc531000)
> > > librte_flow_classify.so.0.200 =>
> > > /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > (0x00007fd4dc52b000)
> > > librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > (0x00007fd4dc502000)
> > >
> > > The final check - using the 19.11 compiled testpmd with the
> > > library
> > > path
> > > set to 20.02 versionned libs:
> > >
> > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > native-
> > > linux-gcc_v19.11/app/testpmd | head
> > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > librte_pmd_bond.so.20.0 => not found
> > > librte_pmd_dpaa.so.20.0 => not found
> > > librte_mempool_dpaa.so.20.0 => not found
> > > librte_pmd_ixgbe.so.20.0 => not found
> > > librte_pmd_i40e.so.20.0 => not found
> > > librte_pmd_bnxt.so.20.0 => not found
> > > librte_pmd_softnic.so.20.0 => not found
> > > librte_flow_classify.so.0.200 => not found
> > > librte_pipeline.so.20.0 => not found
> > >
> > > Fixing This
> > > -----------
> > >
> > > To fix this, we need to ensure that the SONAME remains constant
> > > across the
> > > releases. Therefore, I currently see two options:
> > >
> > > 1. keep 20.0 as the version and soname across all releases in
> > > 2020,
> > > i.e.
> > > just revert the ABIVERSION change patch. Trouble there is how
> > > to
> > > track
> > > 20.02 vs 20.05 etc. etc.
> > >
> > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > This
> > > has the
> > > advantage of keeping the existing planned schemes, but has the
> > > really big
> > > downside of breaking ABI compatibility with anyone who has
> > > already
> > > compiled with 19.11.
> > >
> > > Personally, of the two options - unless someone can come up with
> > > a
> > > third
> > > option - I'd tend towards the second, fixing the builds to remove
> > > the
> > > .0 in
> > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > widespread
> > > adoption. Since this ABI stability is new, teething problems may
> > > be
> > > expected.
> > >
> > > Thoughts and comments?
> > > /Bruce
> > >
> > > BTW: For meson, the patch for option 2 is just to remove the
> > > so_version
> > > variable and all references to it from lib/meson.build and
> > > drivers/meson.build. Haven't looked into a "make" fix yet.
> >
> > Hi,
> >
> > With libtool and its (arguably arcane) format, only the first digit
> > is
> > the ABI current version and gets encoded in the elf header. The
> > other
> > digits can be used to track compatible minor increments, and are
> > mostly
> > ignored. On the system a symlink libfoo.so.major ->
> > libfoo.so.major.minor is added.
> >
> > Eg:
> >
> > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > SONAME
> > 0x000000000000000e (SONAME) Library soname:
> > [libzmq.so.5]
> > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> >
> > Can we do the same? Not sure what the right incantation is for
> > Meson,
> > but it should be possibly.
> >
>
> That's essentially option 2, and it's still an ABI break because
> existing
> builds of 19.11 have the soname will the full version number in it.
> The
> default behaviour for meson is exactly how you described it, except
> that
> previously we needed more exact control over the version info (for
> your
> dpdk-specific versions in the sonames) and so overrode the soversion
> explicitly. The fix for meson is to remove this overriding i.e.
> remove
> "soversion:" parameter for each shared_library() call.
>
>
> > Also, we should leave the current at 20.0 - let's not break
> > compatibility already, please :-)
> >
>
> If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
Yes, that's what I meant - IMHO we should just take the hit and use the
slightly weird 20.0 format until next year, and add a third digit for
compatible updates. Then for v21 we can drop it.
--
Kind regards,
Luca Boccassi
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 16:20 ` Luca Boccassi
@ 2019-12-10 16:32 ` Bruce Richardson
2019-12-10 17:01 ` Kinsella, Ray
2019-12-10 17:04 ` Thomas Monjalon
2019-12-10 16:39 ` Thomas Monjalon
1 sibling, 2 replies; 28+ messages in thread
From: Bruce Richardson @ 2019-12-10 16:32 UTC (permalink / raw)
To: Luca Boccassi
Cc: Ferruh Yigit, Kinsella, Ray, Thomas Monjalon, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > With new process, the major ABI releases will be compatible
> > > > > > > until it is
> > > > > > > deprecated (until next LTS for now),
> > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > versions
> > > > > > > until DPDK_20.11
> > > > > > > will be ABI compatible with this version.
> > > > > > >
> > > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > > release, are we
> > > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > >
> > > > > > > If we allow it break, following problem will be observed:
> > > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > > new
> > > > > > > API introduced
> > > > > > > in 20.02, lets say foo(),
> > > > > > > but when application switches to .so.20.2 (released via
> > > > > > > DPDK_20.05), application
> > > > > > > will fail because of ABI breakage in foo().
> > > > > > >
> > > > > > > I think it is fair that application expects forward
> > > > > > > compatibility in minor
> > > > > > > versions of a shared library.
> > > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > > .so.20.3, .so.20.4
> > > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > > fully
> > > > > > > forward compatible.
> > > > > > >
> > > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > > little, but before
> > > > > > > diving into implementation details, I would like to be sure
> > > > > > > we
> > > > > > > are in same page.
> > > > > > >
> > > > > >
> > > > > > Well, any new API's generally come in as experimental, in
> > > > > > which
> > > > > > case
> > > > > > changes are allowed, and breakage can be expected. If they
> > > > > > are
> > > > > > not
> > > > > > experiemental, then the ABI policy applies to them in that
> > > > > > they
> > > > > > cannot
> > > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > > is
> > > > > > not fully
> > > > > > complete yet. For any application only using stable, non-
> > > > > > experimental
> > > > > > functions, forward compatibility must be maintained IMHO.
> > > > > >
> > > > >
> > > > > Talking about not experimental APIs, experimental ones free
> > > > > from
> > > > > the process.
> > > > >
> > > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > > ABI_20, because
> > > > > it should be supported for following ABI_20.x, instead of
> > > > > calling
> > > > > it ABI_21, and
> > > > > this minor tweak (and mind shift) in .map files can be our
> > > > > solution.
> > > >
> > > > Related at what to do with adding versions between major ABI
> > > > versions, when
> > > > investigating with Kevin the ABI checking we have made an
> > > > unpleasant
> > > > discovery:
> > > >
> > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > already
> > > > broken
> > > > our ABI according to libabigail.
> > > >
> > > > The Gory Details [skip to the end for suggestions to fix]
> > > > ------------------------------------------------------------
> > > >
> > > > The reason for this is that the soversion encoded in each library
> > > > -
> > > > whether
> > > > built with meson or make - is the full 20.0 version, not just the
> > > > major ABI
> > > > .20 part. Then when apps link against DPDK, they actually encode
> > > > the
> > > > 20.0.
> > > >
> > > > So what this means is that currently - using a make build as an
> > > > example
> > > > here - ldd on the latest head build gives:
> > > >
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-linux-gcc/app/testpmd | head
> > > > linux-vdso.so.1 (0x00007fff6813d000)
> > > > librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > > librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > > librte_mempool_dpaa.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > (0x00007f36d7224000)
> > > > librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > (0x00007f36d71ba000)
> > > > librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > > librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > > librte_pmd_softnic.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > (0x00007f36d70b7000)
> > > > librte_flow_classify.so.0.201 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > (0x00007f36d70b1000)
> > > > librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > > ...
> > > >
> > > > Similarly ldd on a 19.11 checkout gives:
> > > >
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > x86_64-
> > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > > librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > (0x00007fd4dc6b6000)
> > > > librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > (0x00007fd4dc6a3000)
> > > > librte_mempool_dpaa.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > (0x00007fd4dc69e000)
> > > > librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > (0x00007fd4dc634000)
> > > > librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > (0x00007fd4dc5a0000)
> > > > librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > (0x00007fd4dc55d000)
> > > > librte_pmd_softnic.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > (0x00007fd4dc531000)
> > > > librte_flow_classify.so.0.200 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > (0x00007fd4dc52b000)
> > > > librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > (0x00007fd4dc502000)
> > > >
> > > > The final check - using the 19.11 compiled testpmd with the
> > > > library
> > > > path
> > > > set to 20.02 versionned libs:
> > > >
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-
> > > > linux-gcc_v19.11/app/testpmd | head
> > > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > > librte_pmd_bond.so.20.0 => not found
> > > > librte_pmd_dpaa.so.20.0 => not found
> > > > librte_mempool_dpaa.so.20.0 => not found
> > > > librte_pmd_ixgbe.so.20.0 => not found
> > > > librte_pmd_i40e.so.20.0 => not found
> > > > librte_pmd_bnxt.so.20.0 => not found
> > > > librte_pmd_softnic.so.20.0 => not found
> > > > librte_flow_classify.so.0.200 => not found
> > > > librte_pipeline.so.20.0 => not found
> > > >
> > > > Fixing This
> > > > -----------
> > > >
> > > > To fix this, we need to ensure that the SONAME remains constant
> > > > across the
> > > > releases. Therefore, I currently see two options:
> > > >
> > > > 1. keep 20.0 as the version and soname across all releases in
> > > > 2020,
> > > > i.e.
> > > > just revert the ABIVERSION change patch. Trouble there is how
> > > > to
> > > > track
> > > > 20.02 vs 20.05 etc. etc.
> > > >
> > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > This
> > > > has the
> > > > advantage of keeping the existing planned schemes, but has the
> > > > really big
> > > > downside of breaking ABI compatibility with anyone who has
> > > > already
> > > > compiled with 19.11.
> > > >
> > > > Personally, of the two options - unless someone can come up with
> > > > a
> > > > third
> > > > option - I'd tend towards the second, fixing the builds to remove
> > > > the
> > > > .0 in
> > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > widespread
> > > > adoption. Since this ABI stability is new, teething problems may
> > > > be
> > > > expected.
> > > >
> > > > Thoughts and comments?
> > > > /Bruce
> > > >
> > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > so_version
> > > > variable and all references to it from lib/meson.build and
> > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > >
> > > Hi,
> > >
> > > With libtool and its (arguably arcane) format, only the first digit
> > > is
> > > the ABI current version and gets encoded in the elf header. The
> > > other
> > > digits can be used to track compatible minor increments, and are
> > > mostly
> > > ignored. On the system a symlink libfoo.so.major ->
> > > libfoo.so.major.minor is added.
> > >
> > > Eg:
> > >
> > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > SONAME
> > > 0x000000000000000e (SONAME) Library soname:
> > > [libzmq.so.5]
> > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > >
> > > Can we do the same? Not sure what the right incantation is for
> > > Meson,
> > > but it should be possibly.
> > >
> >
> > That's essentially option 2, and it's still an ABI break because
> > existing
> > builds of 19.11 have the soname will the full version number in it.
> > The
> > default behaviour for meson is exactly how you described it, except
> > that
> > previously we needed more exact control over the version info (for
> > your
> > dpdk-specific versions in the sonames) and so overrode the soversion
> > explicitly. The fix for meson is to remove this overriding i.e.
> > remove
> > "soversion:" parameter for each shared_library() call.
> >
> >
> > > Also, we should leave the current at 20.0 - let's not break
> > > compatibility already, please :-)
> > >
> >
> > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
>
> Yes, that's what I meant - IMHO we should just take the hit and use the
> slightly weird 20.0 format until next year, and add a third digit for
> compatible updates. Then for v21 we can drop it.
>
My concern with that is us forgetting, because we'll put in place hacks to
have the soversion be the first two numbers of the version. Then we need to
remember to remove those hacks before 20.11 goes out or we'll end up with
21.0 being the soversion again.
For that reason, I'd rather see us fix it now before 19.11 gets widely
adopted.
/Bruce
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 16:32 ` Bruce Richardson
@ 2019-12-10 17:01 ` Kinsella, Ray
2019-12-10 17:04 ` Thomas Monjalon
1 sibling, 0 replies; 28+ messages in thread
From: Kinsella, Ray @ 2019-12-10 17:01 UTC (permalink / raw)
To: Richardson, Bruce, Luca Boccassi
Cc: Yigit, Ferruh, Thomas Monjalon, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
First, good find, much kudos.
To fix, I tend to agree with Bruce.
The SONAME should be comprised of the library name and the major version number, period.
As described in the DPDK documentation.
I too would be concerned about our forgetting to make the change in 20.11.
Also, what would happen when DPDK 20.05 roles around.
In DPDK 20.05's case the SONAME would still be 20.0
However the library file name would be stamped libeal_20.2.so ...
Better to drop the minor version to avoid any ambiguity.
I would suggest a respin of DPDK 19.11 to fix.
Ray K
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Tuesday 10 December 2019 16:32
> To: Luca Boccassi <bluca@debian.org>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Kinsella, Ray
> <ray.kinsella@intel.com>; Thomas Monjalon <thomas@monjalon.net>; David
> Marchand <david.marchand@redhat.com>; Christian Ehrhardt
> <christian.ehrhardt@canonical.com>; Timothy Redaelli
> <tredaelli@redhat.com>; Kevin Traynor <ktraynor@redhat.com>; dpdk-dev
> <dev@dpdk.org>; Laatz, Kevin <kevin.laatz@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; Neil Horman <nhorman@redhat.com>
> Subject: Re: [dpdk-dev] How to manage new APIs added after major ABI
> release?
>
> On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit
> wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > With new process, the major ABI releases will be
> > > > > > > > compatible until it is deprecated (until next LTS for
> > > > > > > > now), like current ABI version is 20 in DPDK_19.11 and
> > > > > > > > DPDK versions until DPDK_20.11 will be ABI compatible
> with
> > > > > > > > this version.
> > > > > > > >
> > > > > > > > But if we introduce a new API after major ABI, say in
> > > > > > > > 20.02 release, are we allowed to break the ABI for that
> > > > > > > > API before DPDK_20.11?
> > > > > > > >
> > > > > > > > If we allow it break, following problem will be observed:
> > > > > > > > Assume an application using .so.20.1 library, and using
> > > > > > > > the new API introduced in 20.02, lets say foo(), but when
> > > > > > > > application switches to .so.20.2 (released via
> > > > > > > > DPDK_20.05), application will fail because of ABI
> breakage
> > > > > > > > in foo().
> > > > > > > >
> > > > > > > > I think it is fair that application expects forward
> > > > > > > > compatibility in minor versions of a shared library.
> > > > > > > > Like if application linked against .so.20.2, fair to
> > > > > > > > expect .so.20.3, .so.20.4 etc will work fine. I think
> > > > > > > > currently only .so.20.0 is fully forward compatible.
> > > > > > > >
> > > > > > > > If we all agree on this, we may need to tweak the process
> > > > > > > > a little, but before diving into implementation details,
> I
> > > > > > > > would like to be sure we are in same page.
> > > > > > > >
> > > > > > >
> > > > > > > Well, any new API's generally come in as experimental, in
> > > > > > > which case changes are allowed, and breakage can be
> > > > > > > expected. If they are not experiemental, then the ABI
> policy
> > > > > > > applies to them in that they cannot change since they are
> > > > > > > part of the .21 ABI, even if that ABI is not fully complete
> > > > > > > yet. For any application only using stable, non-
> > > > > > > experimental functions, forward compatibility must be
> > > > > > > maintained IMHO.
> > > > > > >
> > > > > >
> > > > > > Talking about not experimental APIs, experimental ones free
> > > > > > from the process.
> > > > > >
> > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of
> still
> > > > > > ABI_20, because it should be supported for following
> ABI_20.x,
> > > > > > instead of calling it ABI_21, and this minor tweak (and mind
> > > > > > shift) in .map files can be our solution.
> > > > >
> > > > > Related at what to do with adding versions between major ABI
> > > > > versions, when investigating with Kevin the ABI checking we
> have
> > > > > made an unpleasant
> > > > > discovery:
> > > > >
> > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > already broken our ABI according to libabigail.
> > > > >
> > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > ------------------------------------------------------------
> > > > >
> > > > > The reason for this is that the soversion encoded in each
> > > > > library
> > > > > -
> > > > > whether
> > > > > built with meson or make - is the full 20.0 version, not just
> > > > > the major ABI
> > > > > .20 part. Then when apps link against DPDK, they actually
> encode
> > > > > the 20.0.
> > > > >
> > > > > So what this means is that currently - using a make build as an
> > > > > example here - ldd on the latest head build gives:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-linux-gcc/app/testpmd | head
> > > > > linux-vdso.so.1 (0x00007fff6813d000)
> > > > > librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1
> (0x00007f36d723c000)
> > > > > librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1
> (0x00007f36d7229000)
> > > > > librte_mempool_dpaa.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > (0x00007f36d7224000)
> > > > > librte_pmd_ixgbe.so.20.1 =>
> /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > (0x00007f36d71ba000)
> > > > > librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1
> (0x00007f36d7126000)
> > > > > librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1
> (0x00007f36d70e5000)
> > > > > librte_pmd_softnic.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > (0x00007f36d70b7000)
> > > > > librte_flow_classify.so.0.201 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > (0x00007f36d70b1000)
> > > > > librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pipeline.so.20.1
> > > > > (0x00007f36d7088000) ...
> > > > >
> > > > > Similarly ldd on a 19.11 checkout gives:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > > x86_64-
> > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > > > librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > (0x00007fd4dc6b6000)
> > > > > librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > (0x00007fd4dc6a3000)
> > > > > librte_mempool_dpaa.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > (0x00007fd4dc69e000)
> > > > > librte_pmd_ixgbe.so.20.0 =>
> /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > (0x00007fd4dc634000)
> > > > > librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > (0x00007fd4dc5a0000)
> > > > > librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > (0x00007fd4dc55d000)
> > > > > librte_pmd_softnic.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > (0x00007fd4dc531000)
> > > > > librte_flow_classify.so.0.200 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > (0x00007fd4dc52b000)
> > > > > librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > (0x00007fd4dc502000)
> > > > >
> > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > library path set to 20.02 versionned libs:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-
> > > > > linux-gcc_v19.11/app/testpmd | head
> > > > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > > > librte_pmd_bond.so.20.0 => not found
> > > > > librte_pmd_dpaa.so.20.0 => not found
> > > > > librte_mempool_dpaa.so.20.0 => not found
> > > > > librte_pmd_ixgbe.so.20.0 => not found
> > > > > librte_pmd_i40e.so.20.0 => not found
> > > > > librte_pmd_bnxt.so.20.0 => not found
> > > > > librte_pmd_softnic.so.20.0 => not found
> > > > > librte_flow_classify.so.0.200 => not found
> > > > > librte_pipeline.so.20.0 => not found
> > > > >
> > > > > Fixing This
> > > > > -----------
> > > > >
> > > > > To fix this, we need to ensure that the SONAME remains constant
> > > > > across the releases. Therefore, I currently see two options:
> > > > >
> > > > > 1. keep 20.0 as the version and soname across all releases in
> > > > > 2020, i.e.
> > > > > just revert the ABIVERSION change patch. Trouble there is how
> > > > > to track
> > > > > 20.02 vs 20.05 etc. etc.
> > > > >
> > > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > > This
> > > > > has the
> > > > > advantage of keeping the existing planned schemes, but has
> the
> > > > > really big
> > > > > downside of breaking ABI compatibility with anyone who has
> > > > > already
> > > > > compiled with 19.11.
> > > > >
> > > > > Personally, of the two options - unless someone can come up
> with
> > > > > a third option - I'd tend towards the second, fixing the builds
> > > > > to remove the
> > > > > .0 in
> > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11
> gets
> > > > > widespread adoption. Since this ABI stability is new, teething
> > > > > problems may be expected.
> > > > >
> > > > > Thoughts and comments?
> > > > > /Bruce
> > > > >
> > > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > > so_version variable and all references to it from
> > > > > lib/meson.build and drivers/meson.build. Haven't looked into a
> > > > > "make" fix yet.
> > > >
> > > > Hi,
> > > >
> > > > With libtool and its (arguably arcane) format, only the first
> > > > digit is the ABI current version and gets encoded in the elf
> > > > header. The other digits can be used to track compatible minor
> > > > increments, and are mostly ignored. On the system a symlink
> > > > libfoo.so.major -> libfoo.so.major.minor is added.
> > > >
> > > > Eg:
> > > >
> > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > SONAME
> > > > 0x000000000000000e (SONAME) Library soname:
> > > > [libzmq.so.5]
> > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > >
> > > > Can we do the same? Not sure what the right incantation is for
> > > > Meson, but it should be possibly.
> > > >
> > >
> > > That's essentially option 2, and it's still an ABI break because
> > > existing
> > > builds of 19.11 have the soname will the full version number in it.
> > > The
> > > default behaviour for meson is exactly how you described it, except
> > > that
> > > previously we needed more exact control over the version info (for
> > > your
> > > dpdk-specific versions in the sonames) and so overrode the
> soversion
> > > explicitly. The fix for meson is to remove this overriding i.e.
> > > remove
> > > "soversion:" parameter for each shared_library() call.
> > >
> > >
> > > > Also, we should leave the current at 20.0 - let's not break
> > > > compatibility already, please :-)
> > > >
> > >
> > > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> >
> > Yes, that's what I meant - IMHO we should just take the hit and use
> the
> > slightly weird 20.0 format until next year, and add a third digit for
> > compatible updates. Then for v21 we can drop it.
> >
> My concern with that is us forgetting, because we'll put in place hacks
> to
> have the soversion be the first two numbers of the version. Then we
> need to
> remember to remove those hacks before 20.11 goes out or we'll end up
> with
> 21.0 being the soversion again.
>
> For that reason, I'd rather see us fix it now before 19.11 gets widely
> adopted.
>
> /Bruce
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 16:32 ` Bruce Richardson
2019-12-10 17:01 ` Kinsella, Ray
@ 2019-12-10 17:04 ` Thomas Monjalon
2019-12-10 18:22 ` Luca Boccassi
1 sibling, 1 reply; 28+ messages in thread
From: Thomas Monjalon @ 2019-12-10 17:04 UTC (permalink / raw)
To: Bruce Richardson
Cc: Luca Boccassi, Ferruh Yigit, Kinsella, Ray, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
10/12/2019 17:32, Bruce Richardson:
> On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > With new process, the major ABI releases will be compatible
> > > > > > > > until it is
> > > > > > > > deprecated (until next LTS for now),
> > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > > versions
> > > > > > > > until DPDK_20.11
> > > > > > > > will be ABI compatible with this version.
> > > > > > > >
> > > > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > > > release, are we
> > > > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > > >
> > > > > > > > If we allow it break, following problem will be observed:
> > > > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > > > new
> > > > > > > > API introduced
> > > > > > > > in 20.02, lets say foo(),
> > > > > > > > but when application switches to .so.20.2 (released via
> > > > > > > > DPDK_20.05), application
> > > > > > > > will fail because of ABI breakage in foo().
> > > > > > > >
> > > > > > > > I think it is fair that application expects forward
> > > > > > > > compatibility in minor
> > > > > > > > versions of a shared library.
> > > > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > > > .so.20.3, .so.20.4
> > > > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > > > fully
> > > > > > > > forward compatible.
> > > > > > > >
> > > > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > > > little, but before
> > > > > > > > diving into implementation details, I would like to be sure
> > > > > > > > we
> > > > > > > > are in same page.
> > > > > > > >
> > > > > > >
> > > > > > > Well, any new API's generally come in as experimental, in
> > > > > > > which
> > > > > > > case
> > > > > > > changes are allowed, and breakage can be expected. If they
> > > > > > > are
> > > > > > > not
> > > > > > > experiemental, then the ABI policy applies to them in that
> > > > > > > they
> > > > > > > cannot
> > > > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > > > is
> > > > > > > not fully
> > > > > > > complete yet. For any application only using stable, non-
> > > > > > > experimental
> > > > > > > functions, forward compatibility must be maintained IMHO.
> > > > > > >
> > > > > >
> > > > > > Talking about not experimental APIs, experimental ones free
> > > > > > from
> > > > > > the process.
> > > > > >
> > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > > > ABI_20, because
> > > > > > it should be supported for following ABI_20.x, instead of
> > > > > > calling
> > > > > > it ABI_21, and
> > > > > > this minor tweak (and mind shift) in .map files can be our
> > > > > > solution.
> > > > >
> > > > > Related at what to do with adding versions between major ABI
> > > > > versions, when
> > > > > investigating with Kevin the ABI checking we have made an
> > > > > unpleasant
> > > > > discovery:
> > > > >
> > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > already
> > > > > broken
> > > > > our ABI according to libabigail.
> > > > >
> > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > ------------------------------------------------------------
> > > > >
> > > > > The reason for this is that the soversion encoded in each library
> > > > > -
> > > > > whether
> > > > > built with meson or make - is the full 20.0 version, not just the
> > > > > major ABI
> > > > > .20 part. Then when apps link against DPDK, they actually encode
> > > > > the
> > > > > 20.0.
> > > > >
> > > > > So what this means is that currently - using a make build as an
> > > > > example
> > > > > here - ldd on the latest head build gives:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-linux-gcc/app/testpmd | head
> > > > > linux-vdso.so.1 (0x00007fff6813d000)
> > > > > librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > > > librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > > > librte_mempool_dpaa.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > (0x00007f36d7224000)
> > > > > librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > (0x00007f36d71ba000)
> > > > > librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > > > librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > > > librte_pmd_softnic.so.20.1 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > (0x00007f36d70b7000)
> > > > > librte_flow_classify.so.0.201 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > (0x00007f36d70b1000)
> > > > > librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > > > ...
> > > > >
> > > > > Similarly ldd on a 19.11 checkout gives:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > > x86_64-
> > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > > > librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > (0x00007fd4dc6b6000)
> > > > > librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > (0x00007fd4dc6a3000)
> > > > > librte_mempool_dpaa.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > (0x00007fd4dc69e000)
> > > > > librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > (0x00007fd4dc634000)
> > > > > librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > (0x00007fd4dc5a0000)
> > > > > librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > (0x00007fd4dc55d000)
> > > > > librte_pmd_softnic.so.20.0 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > (0x00007fd4dc531000)
> > > > > librte_flow_classify.so.0.200 =>
> > > > > /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > (0x00007fd4dc52b000)
> > > > > librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > (0x00007fd4dc502000)
> > > > >
> > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > library
> > > > > path
> > > > > set to 20.02 versionned libs:
> > > > >
> > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > > native-
> > > > > linux-gcc_v19.11/app/testpmd | head
> > > > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > > > librte_pmd_bond.so.20.0 => not found
> > > > > librte_pmd_dpaa.so.20.0 => not found
> > > > > librte_mempool_dpaa.so.20.0 => not found
> > > > > librte_pmd_ixgbe.so.20.0 => not found
> > > > > librte_pmd_i40e.so.20.0 => not found
> > > > > librte_pmd_bnxt.so.20.0 => not found
> > > > > librte_pmd_softnic.so.20.0 => not found
> > > > > librte_flow_classify.so.0.200 => not found
> > > > > librte_pipeline.so.20.0 => not found
> > > > >
> > > > > Fixing This
> > > > > -----------
> > > > >
> > > > > To fix this, we need to ensure that the SONAME remains constant
> > > > > across the
> > > > > releases. Therefore, I currently see two options:
> > > > >
> > > > > 1. keep 20.0 as the version and soname across all releases in
> > > > > 2020,
> > > > > i.e.
> > > > > just revert the ABIVERSION change patch. Trouble there is how
> > > > > to
> > > > > track
> > > > > 20.02 vs 20.05 etc. etc.
> > > > >
> > > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > > This
> > > > > has the
> > > > > advantage of keeping the existing planned schemes, but has the
> > > > > really big
> > > > > downside of breaking ABI compatibility with anyone who has
> > > > > already
> > > > > compiled with 19.11.
> > > > >
> > > > > Personally, of the two options - unless someone can come up with
> > > > > a
> > > > > third
> > > > > option - I'd tend towards the second, fixing the builds to remove
> > > > > the
> > > > > .0 in
> > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > > widespread
> > > > > adoption. Since this ABI stability is new, teething problems may
> > > > > be
> > > > > expected.
> > > > >
> > > > > Thoughts and comments?
> > > > > /Bruce
> > > > >
> > > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > > so_version
> > > > > variable and all references to it from lib/meson.build and
> > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > >
> > > > Hi,
> > > >
> > > > With libtool and its (arguably arcane) format, only the first digit
> > > > is
> > > > the ABI current version and gets encoded in the elf header. The
> > > > other
> > > > digits can be used to track compatible minor increments, and are
> > > > mostly
> > > > ignored. On the system a symlink libfoo.so.major ->
> > > > libfoo.so.major.minor is added.
> > > >
> > > > Eg:
> > > >
> > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > SONAME
> > > > 0x000000000000000e (SONAME) Library soname:
> > > > [libzmq.so.5]
> > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > >
> > > > Can we do the same? Not sure what the right incantation is for
> > > > Meson,
> > > > but it should be possibly.
> > > >
> > >
> > > That's essentially option 2, and it's still an ABI break because
> > > existing
> > > builds of 19.11 have the soname will the full version number in it.
> > > The
> > > default behaviour for meson is exactly how you described it, except
> > > that
> > > previously we needed more exact control over the version info (for
> > > your
> > > dpdk-specific versions in the sonames) and so overrode the soversion
> > > explicitly. The fix for meson is to remove this overriding i.e.
> > > remove
> > > "soversion:" parameter for each shared_library() call.
> > >
> > >
> > > > Also, we should leave the current at 20.0 - let's not break
> > > > compatibility already, please :-)
> > > >
> > >
> > > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> >
> > Yes, that's what I meant - IMHO we should just take the hit and use the
> > slightly weird 20.0 format until next year, and add a third digit for
> > compatible updates. Then for v21 we can drop it.
> >
> My concern with that is us forgetting, because we'll put in place hacks to
> have the soversion be the first two numbers of the version. Then we need to
> remember to remove those hacks before 20.11 goes out or we'll end up with
> 21.0 being the soversion again.
>
> For that reason, I'd rather see us fix it now before 19.11 gets widely
> adopted.
Me too I vote for fixing soname as 20 in a small 19.11.1
and release it quickly.
We are supposed to experience tooling and scheme adoption during this
first year of new ABI policy. I think we should stick to the planned
versionning, in order to avoid any surprise for the next major ABI.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 17:04 ` Thomas Monjalon
@ 2019-12-10 18:22 ` Luca Boccassi
2019-12-10 23:34 ` Bruce Richardson
0 siblings, 1 reply; 28+ messages in thread
From: Luca Boccassi @ 2019-12-10 18:22 UTC (permalink / raw)
To: Thomas Monjalon, Bruce Richardson
Cc: Ferruh Yigit, Kinsella, Ray, David Marchand, Christian Ehrhardt,
Timothy Redaelli, Kevin Traynor, dpdk-dev, Laatz, Kevin,
Andrew Rybchenko, Neil Horman
On Tue, 2019-12-10 at 18:04 +0100, Thomas Monjalon wrote:
> 10/12/2019 17:32, Bruce Richardson:
> > On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit
> > > > > > wrote:
> > > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit
> > > > > > > > wrote:
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > With new process, the major ABI releases will be
> > > > > > > > > compatible
> > > > > > > > > until it is
> > > > > > > > > deprecated (until next LTS for now),
> > > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > > > versions
> > > > > > > > > until DPDK_20.11
> > > > > > > > > will be ABI compatible with this version.
> > > > > > > > >
> > > > > > > > > But if we introduce a new API after major ABI, say in
> > > > > > > > > 20.02
> > > > > > > > > release, are we
> > > > > > > > > allowed to break the ABI for that API before
> > > > > > > > > DPDK_20.11?
> > > > > > > > >
> > > > > > > > > If we allow it break, following problem will be
> > > > > > > > > observed:
> > > > > > > > > Assume an application using .so.20.1 library, and
> > > > > > > > > using the
> > > > > > > > > new
> > > > > > > > > API introduced
> > > > > > > > > in 20.02, lets say foo(),
> > > > > > > > > but when application switches to .so.20.2 (released
> > > > > > > > > via
> > > > > > > > > DPDK_20.05), application
> > > > > > > > > will fail because of ABI breakage in foo().
> > > > > > > > >
> > > > > > > > > I think it is fair that application expects forward
> > > > > > > > > compatibility in minor
> > > > > > > > > versions of a shared library.
> > > > > > > > > Like if application linked against .so.20.2, fair to
> > > > > > > > > expect
> > > > > > > > > .so.20.3, .so.20.4
> > > > > > > > > etc will work fine. I think currently only .so.20.0
> > > > > > > > > is
> > > > > > > > > fully
> > > > > > > > > forward compatible.
> > > > > > > > >
> > > > > > > > > If we all agree on this, we may need to tweak the
> > > > > > > > > process a
> > > > > > > > > little, but before
> > > > > > > > > diving into implementation details, I would like to
> > > > > > > > > be sure
> > > > > > > > > we
> > > > > > > > > are in same page.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Well, any new API's generally come in as experimental,
> > > > > > > > in
> > > > > > > > which
> > > > > > > > case
> > > > > > > > changes are allowed, and breakage can be expected. If
> > > > > > > > they
> > > > > > > > are
> > > > > > > > not
> > > > > > > > experiemental, then the ABI policy applies to them in
> > > > > > > > that
> > > > > > > > they
> > > > > > > > cannot
> > > > > > > > change since they are part of the .21 ABI, even if that
> > > > > > > > ABI
> > > > > > > > is
> > > > > > > > not fully
> > > > > > > > complete yet. For any application only using stable,
> > > > > > > > non-
> > > > > > > > experimental
> > > > > > > > functions, forward compatibility must be maintained
> > > > > > > > IMHO.
> > > > > > > >
> > > > > > >
> > > > > > > Talking about not experimental APIs, experimental ones
> > > > > > > free
> > > > > > > from
> > > > > > > the process.
> > > > > > >
> > > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of
> > > > > > > still
> > > > > > > ABI_20, because
> > > > > > > it should be supported for following ABI_20.x, instead of
> > > > > > > calling
> > > > > > > it ABI_21, and
> > > > > > > this minor tweak (and mind shift) in .map files can be
> > > > > > > our
> > > > > > > solution.
> > > > > >
> > > > > > Related at what to do with adding versions between major
> > > > > > ABI
> > > > > > versions, when
> > > > > > investigating with Kevin the ABI checking we have made an
> > > > > > unpleasant
> > > > > > discovery:
> > > > > >
> > > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > > already
> > > > > > broken
> > > > > > our ABI according to libabigail.
> > > > > >
> > > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > > ---------------------------------------------------------
> > > > > > ---
> > > > > >
> > > > > > The reason for this is that the soversion encoded in each
> > > > > > library
> > > > > > -
> > > > > > whether
> > > > > > built with meson or make - is the full 20.0 version, not
> > > > > > just the
> > > > > > major ABI
> > > > > > .20 part. Then when apps link against DPDK, they actually
> > > > > > encode
> > > > > > the
> > > > > > 20.0.
> > > > > >
> > > > > > So what this means is that currently - using a make build
> > > > > > as an
> > > > > > example
> > > > > > here - ldd on the latest head build gives:
> > > > > >
> > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > x86_64-
> > > > > > native-linux-gcc/app/testpmd | head
> > > > > > linux-vdso.so.1 (0x00007fff6813d000)
> > > > > > librte_pmd_bond.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1
> > > > > > (0x00007f36d723c000)
> > > > > > librte_pmd_dpaa.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1
> > > > > > (0x00007f36d7229000)
> > > > > > librte_mempool_dpaa.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > > (0x00007f36d7224000)
> > > > > > librte_pmd_ixgbe.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > > (0x00007f36d71ba000)
> > > > > > librte_pmd_i40e.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1
> > > > > > (0x00007f36d7126000)
> > > > > > librte_pmd_bnxt.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1
> > > > > > (0x00007f36d70e5000)
> > > > > > librte_pmd_softnic.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > > (0x00007f36d70b7000)
> > > > > > librte_flow_classify.so.0.201 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > > (0x00007f36d70b1000)
> > > > > > librte_pipeline.so.20.1 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc/lib/librte_pipeline.so.20.1
> > > > > > (0x00007f36d7088000)
> > > > > > ...
> > > > > >
> > > > > > Similarly ldd on a 19.11 checkout gives:
> > > > > >
> > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib
> > > > > > ldd
> > > > > > x86_64-
> > > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > > > > librte_pmd_bond.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > > (0x00007fd4dc6b6000)
> > > > > > librte_pmd_dpaa.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > > (0x00007fd4dc6a3000)
> > > > > > librte_mempool_dpaa.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > > (0x00007fd4dc69e000)
> > > > > > librte_pmd_ixgbe.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > > (0x00007fd4dc634000)
> > > > > > librte_pmd_i40e.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > > (0x00007fd4dc5a0000)
> > > > > > librte_pmd_bnxt.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > > (0x00007fd4dc55d000)
> > > > > > librte_pmd_softnic.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > > (0x00007fd4dc531000)
> > > > > > librte_flow_classify.so.0.200 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > > (0x00007fd4dc52b000)
> > > > > > librte_pipeline.so.20.0 =>
> > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > > (0x00007fd4dc502000)
> > > > > >
> > > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > > library
> > > > > > path
> > > > > > set to 20.02 versionned libs:
> > > > > >
> > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > x86_64-
> > > > > > native-
> > > > > > linux-gcc_v19.11/app/testpmd | head
> > > > > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > > > > librte_pmd_bond.so.20.0 => not found
> > > > > > librte_pmd_dpaa.so.20.0 => not found
> > > > > > librte_mempool_dpaa.so.20.0 => not found
> > > > > > librte_pmd_ixgbe.so.20.0 => not found
> > > > > > librte_pmd_i40e.so.20.0 => not found
> > > > > > librte_pmd_bnxt.so.20.0 => not found
> > > > > > librte_pmd_softnic.so.20.0 => not found
> > > > > > librte_flow_classify.so.0.200 => not found
> > > > > > librte_pipeline.so.20.0 => not found
> > > > > >
> > > > > > Fixing This
> > > > > > -----------
> > > > > >
> > > > > > To fix this, we need to ensure that the SONAME remains
> > > > > > constant
> > > > > > across the
> > > > > > releases. Therefore, I currently see two options:
> > > > > >
> > > > > > 1. keep 20.0 as the version and soname across all releases
> > > > > > in
> > > > > > 2020,
> > > > > > i.e.
> > > > > > just revert the ABIVERSION change patch. Trouble there is
> > > > > > how
> > > > > > to
> > > > > > track
> > > > > > 20.02 vs 20.05 etc. etc.
> > > > > >
> > > > > > 2. remove the .0, .1 from the SONAMES stored in the
> > > > > > libraries.
> > > > > > This
> > > > > > has the
> > > > > > advantage of keeping the existing planned schemes, but
> > > > > > has the
> > > > > > really big
> > > > > > downside of breaking ABI compatibility with anyone who
> > > > > > has
> > > > > > already
> > > > > > compiled with 19.11.
> > > > > >
> > > > > > Personally, of the two options - unless someone can come up
> > > > > > with
> > > > > > a
> > > > > > third
> > > > > > option - I'd tend towards the second, fixing the builds to
> > > > > > remove
> > > > > > the
> > > > > > .0 in
> > > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11
> > > > > > gets
> > > > > > widespread
> > > > > > adoption. Since this ABI stability is new, teething
> > > > > > problems may
> > > > > > be
> > > > > > expected.
> > > > > >
> > > > > > Thoughts and comments?
> > > > > > /Bruce
> > > > > >
> > > > > > BTW: For meson, the patch for option 2 is just to remove
> > > > > > the
> > > > > > so_version
> > > > > > variable and all references to it from lib/meson.build and
> > > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > > >
> > > > > Hi,
> > > > >
> > > > > With libtool and its (arguably arcane) format, only the first
> > > > > digit
> > > > > is
> > > > > the ABI current version and gets encoded in the elf header.
> > > > > The
> > > > > other
> > > > > digits can be used to track compatible minor increments, and
> > > > > are
> > > > > mostly
> > > > > ignored. On the system a symlink libfoo.so.major ->
> > > > > libfoo.so.major.minor is added.
> > > > >
> > > > > Eg:
> > > > >
> > > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > > SONAME
> > > > > 0x000000000000000e (SONAME) Library soname:
> > > > > [libzmq.so.5]
> > > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > > >
> > > > > Can we do the same? Not sure what the right incantation is
> > > > > for
> > > > > Meson,
> > > > > but it should be possibly.
> > > > >
> > > >
> > > > That's essentially option 2, and it's still an ABI break
> > > > because
> > > > existing
> > > > builds of 19.11 have the soname will the full version number in
> > > > it.
> > > > The
> > > > default behaviour for meson is exactly how you described it,
> > > > except
> > > > that
> > > > previously we needed more exact control over the version info
> > > > (for
> > > > your
> > > > dpdk-specific versions in the sonames) and so overrode the
> > > > soversion
> > > > explicitly. The fix for meson is to remove this overriding i.e.
> > > > remove
> > > > "soversion:" parameter for each shared_library() call.
> > > >
> > > >
> > > > > Also, we should leave the current at 20.0 - let's not break
> > > > > compatibility already, please :-)
> > > > >
> > > >
> > > > If we do this, maybe we can use 20.0.1 and 20.0.2 version
> > > > numbers?
> > >
> > > Yes, that's what I meant - IMHO we should just take the hit and
> > > use the
> > > slightly weird 20.0 format until next year, and add a third digit
> > > for
> > > compatible updates. Then for v21 we can drop it.
> > >
> >
> > My concern with that is us forgetting, because we'll put in place
> > hacks to
> > have the soversion be the first two numbers of the version. Then we
> > need to
> > remember to remove those hacks before 20.11 goes out or we'll end
> > up with
> > 21.0 being the soversion again.
> >
> > For that reason, I'd rather see us fix it now before 19.11 gets
> > widely
> > adopted.
>
> Me too I vote for fixing soname as 20 in a small 19.11.1
> and release it quickly.
>
> We are supposed to experience tooling and scheme adoption during this
> first year of new ABI policy. I think we should stick to the planned
> versionning, in order to avoid any surprise for the next major ABI.
If the concern is forgetting, I'll happily set a reminder in my
calendar and then nag everybody :-)
In my opinion declaring stability and compatibility, with press
releases and articles, and then immediately breaking it in the first
point release does not send the right message. Not because there was a
bug, which happens all the time, but because mantaining backward
compatibility sometimes means having to carry less-than-ideal or
downright ugly hacks for a while.
If we are not ready to maintain compatibility by carrying an ugly
workaround when the implications are minor or non-existant (I'm pretty
sure few would really cares what format the soname has, it's all
handled by tools), what happens next time when the implications are
that a broken feature stays broken, or that lower performance stays
lower, and so on?
--
Kind regards,
Luca Boccassi
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 18:22 ` Luca Boccassi
@ 2019-12-10 23:34 ` Bruce Richardson
0 siblings, 0 replies; 28+ messages in thread
From: Bruce Richardson @ 2019-12-10 23:34 UTC (permalink / raw)
To: Luca Boccassi
Cc: Thomas Monjalon, Ferruh Yigit, Kinsella, Ray, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On Tue, Dec 10, 2019 at 06:22:36PM +0000, Luca Boccassi wrote:
> On Tue, 2019-12-10 at 18:04 +0100, Thomas Monjalon wrote:
> > 10/12/2019 17:32, Bruce Richardson:
> > > On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit
> > > > > > > wrote:
> > > > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit
> > > > > > > > > wrote:
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > With new process, the major ABI releases will be
> > > > > > > > > > compatible
> > > > > > > > > > until it is
> > > > > > > > > > deprecated (until next LTS for now),
> > > > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > > > > versions
> > > > > > > > > > until DPDK_20.11
> > > > > > > > > > will be ABI compatible with this version.
> > > > > > > > > >
> > > > > > > > > > But if we introduce a new API after major ABI, say in
> > > > > > > > > > 20.02
> > > > > > > > > > release, are we
> > > > > > > > > > allowed to break the ABI for that API before
> > > > > > > > > > DPDK_20.11?
> > > > > > > > > >
> > > > > > > > > > If we allow it break, following problem will be
> > > > > > > > > > observed:
> > > > > > > > > > Assume an application using .so.20.1 library, and
> > > > > > > > > > using the
> > > > > > > > > > new
> > > > > > > > > > API introduced
> > > > > > > > > > in 20.02, lets say foo(),
> > > > > > > > > > but when application switches to .so.20.2 (released
> > > > > > > > > > via
> > > > > > > > > > DPDK_20.05), application
> > > > > > > > > > will fail because of ABI breakage in foo().
> > > > > > > > > >
> > > > > > > > > > I think it is fair that application expects forward
> > > > > > > > > > compatibility in minor
> > > > > > > > > > versions of a shared library.
> > > > > > > > > > Like if application linked against .so.20.2, fair to
> > > > > > > > > > expect
> > > > > > > > > > .so.20.3, .so.20.4
> > > > > > > > > > etc will work fine. I think currently only .so.20.0
> > > > > > > > > > is
> > > > > > > > > > fully
> > > > > > > > > > forward compatible.
> > > > > > > > > >
> > > > > > > > > > If we all agree on this, we may need to tweak the
> > > > > > > > > > process a
> > > > > > > > > > little, but before
> > > > > > > > > > diving into implementation details, I would like to
> > > > > > > > > > be sure
> > > > > > > > > > we
> > > > > > > > > > are in same page.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Well, any new API's generally come in as experimental,
> > > > > > > > > in
> > > > > > > > > which
> > > > > > > > > case
> > > > > > > > > changes are allowed, and breakage can be expected. If
> > > > > > > > > they
> > > > > > > > > are
> > > > > > > > > not
> > > > > > > > > experiemental, then the ABI policy applies to them in
> > > > > > > > > that
> > > > > > > > > they
> > > > > > > > > cannot
> > > > > > > > > change since they are part of the .21 ABI, even if that
> > > > > > > > > ABI
> > > > > > > > > is
> > > > > > > > > not fully
> > > > > > > > > complete yet. For any application only using stable,
> > > > > > > > > non-
> > > > > > > > > experimental
> > > > > > > > > functions, forward compatibility must be maintained
> > > > > > > > > IMHO.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Talking about not experimental APIs, experimental ones
> > > > > > > > free
> > > > > > > > from
> > > > > > > > the process.
> > > > > > > >
> > > > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of
> > > > > > > > still
> > > > > > > > ABI_20, because
> > > > > > > > it should be supported for following ABI_20.x, instead of
> > > > > > > > calling
> > > > > > > > it ABI_21, and
> > > > > > > > this minor tweak (and mind shift) in .map files can be
> > > > > > > > our
> > > > > > > > solution.
> > > > > > >
> > > > > > > Related at what to do with adding versions between major
> > > > > > > ABI
> > > > > > > versions, when
> > > > > > > investigating with Kevin the ABI checking we have made an
> > > > > > > unpleasant
> > > > > > > discovery:
> > > > > > >
> > > > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > > > already
> > > > > > > broken
> > > > > > > our ABI according to libabigail.
> > > > > > >
> > > > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > > > ---------------------------------------------------------
> > > > > > > ---
> > > > > > >
> > > > > > > The reason for this is that the soversion encoded in each
> > > > > > > library
> > > > > > > -
> > > > > > > whether
> > > > > > > built with meson or make - is the full 20.0 version, not
> > > > > > > just the
> > > > > > > major ABI
> > > > > > > .20 part. Then when apps link against DPDK, they actually
> > > > > > > encode
> > > > > > > the
> > > > > > > 20.0.
> > > > > > >
> > > > > > > So what this means is that currently - using a make build
> > > > > > > as an
> > > > > > > example
> > > > > > > here - ldd on the latest head build gives:
> > > > > > >
> > > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > > x86_64-
> > > > > > > native-linux-gcc/app/testpmd | head
> > > > > > > linux-vdso.so.1 (0x00007fff6813d000)
> > > > > > > librte_pmd_bond.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1
> > > > > > > (0x00007f36d723c000)
> > > > > > > librte_pmd_dpaa.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1
> > > > > > > (0x00007f36d7229000)
> > > > > > > librte_mempool_dpaa.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > > > > (0x00007f36d7224000)
> > > > > > > librte_pmd_ixgbe.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > > > > (0x00007f36d71ba000)
> > > > > > > librte_pmd_i40e.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1
> > > > > > > (0x00007f36d7126000)
> > > > > > > librte_pmd_bnxt.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1
> > > > > > > (0x00007f36d70e5000)
> > > > > > > librte_pmd_softnic.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > > > > (0x00007f36d70b7000)
> > > > > > > librte_flow_classify.so.0.201 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > > > > (0x00007f36d70b1000)
> > > > > > > librte_pipeline.so.20.1 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc/lib/librte_pipeline.so.20.1
> > > > > > > (0x00007f36d7088000)
> > > > > > > ...
> > > > > > >
> > > > > > > Similarly ldd on a 19.11 checkout gives:
> > > > > > >
> > > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib
> > > > > > > ldd
> > > > > > > x86_64-
> > > > > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > > > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > > > > > librte_pmd_bond.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > > > > (0x00007fd4dc6b6000)
> > > > > > > librte_pmd_dpaa.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > > > > (0x00007fd4dc6a3000)
> > > > > > > librte_mempool_dpaa.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > > > > (0x00007fd4dc69e000)
> > > > > > > librte_pmd_ixgbe.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > > > > (0x00007fd4dc634000)
> > > > > > > librte_pmd_i40e.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > > > > (0x00007fd4dc5a0000)
> > > > > > > librte_pmd_bnxt.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > > > > (0x00007fd4dc55d000)
> > > > > > > librte_pmd_softnic.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > > > > (0x00007fd4dc531000)
> > > > > > > librte_flow_classify.so.0.200 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > > > > (0x00007fd4dc52b000)
> > > > > > > librte_pipeline.so.20.0 =>
> > > > > > > /home/bruce/dpdk.org/x86_64-
> > > > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > > > > (0x00007fd4dc502000)
> > > > > > >
> > > > > > > The final check - using the 19.11 compiled testpmd with the
> > > > > > > library
> > > > > > > path
> > > > > > > set to 20.02 versionned libs:
> > > > > > >
> > > > > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd
> > > > > > > x86_64-
> > > > > > > native-
> > > > > > > linux-gcc_v19.11/app/testpmd | head
> > > > > > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > > > > > librte_pmd_bond.so.20.0 => not found
> > > > > > > librte_pmd_dpaa.so.20.0 => not found
> > > > > > > librte_mempool_dpaa.so.20.0 => not found
> > > > > > > librte_pmd_ixgbe.so.20.0 => not found
> > > > > > > librte_pmd_i40e.so.20.0 => not found
> > > > > > > librte_pmd_bnxt.so.20.0 => not found
> > > > > > > librte_pmd_softnic.so.20.0 => not found
> > > > > > > librte_flow_classify.so.0.200 => not found
> > > > > > > librte_pipeline.so.20.0 => not found
> > > > > > >
> > > > > > > Fixing This
> > > > > > > -----------
> > > > > > >
> > > > > > > To fix this, we need to ensure that the SONAME remains
> > > > > > > constant
> > > > > > > across the
> > > > > > > releases. Therefore, I currently see two options:
> > > > > > >
> > > > > > > 1. keep 20.0 as the version and soname across all releases
> > > > > > > in
> > > > > > > 2020,
> > > > > > > i.e.
> > > > > > > just revert the ABIVERSION change patch. Trouble there is
> > > > > > > how
> > > > > > > to
> > > > > > > track
> > > > > > > 20.02 vs 20.05 etc. etc.
> > > > > > >
> > > > > > > 2. remove the .0, .1 from the SONAMES stored in the
> > > > > > > libraries.
> > > > > > > This
> > > > > > > has the
> > > > > > > advantage of keeping the existing planned schemes, but
> > > > > > > has the
> > > > > > > really big
> > > > > > > downside of breaking ABI compatibility with anyone who
> > > > > > > has
> > > > > > > already
> > > > > > > compiled with 19.11.
> > > > > > >
> > > > > > > Personally, of the two options - unless someone can come up
> > > > > > > with
> > > > > > > a
> > > > > > > third
> > > > > > > option - I'd tend towards the second, fixing the builds to
> > > > > > > remove
> > > > > > > the
> > > > > > > .0 in
> > > > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11
> > > > > > > gets
> > > > > > > widespread
> > > > > > > adoption. Since this ABI stability is new, teething
> > > > > > > problems may
> > > > > > > be
> > > > > > > expected.
> > > > > > >
> > > > > > > Thoughts and comments?
> > > > > > > /Bruce
> > > > > > >
> > > > > > > BTW: For meson, the patch for option 2 is just to remove
> > > > > > > the
> > > > > > > so_version
> > > > > > > variable and all references to it from lib/meson.build and
> > > > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > With libtool and its (arguably arcane) format, only the first
> > > > > > digit
> > > > > > is
> > > > > > the ABI current version and gets encoded in the elf header.
> > > > > > The
> > > > > > other
> > > > > > digits can be used to track compatible minor increments, and
> > > > > > are
> > > > > > mostly
> > > > > > ignored. On the system a symlink libfoo.so.major ->
> > > > > > libfoo.so.major.minor is added.
> > > > > >
> > > > > > Eg:
> > > > > >
> > > > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > > > SONAME
> > > > > > 0x000000000000000e (SONAME) Library soname:
> > > > > > [libzmq.so.5]
> > > > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > > > >
> > > > > > Can we do the same? Not sure what the right incantation is
> > > > > > for
> > > > > > Meson,
> > > > > > but it should be possibly.
> > > > > >
> > > > >
> > > > > That's essentially option 2, and it's still an ABI break
> > > > > because
> > > > > existing
> > > > > builds of 19.11 have the soname will the full version number in
> > > > > it.
> > > > > The
> > > > > default behaviour for meson is exactly how you described it,
> > > > > except
> > > > > that
> > > > > previously we needed more exact control over the version info
> > > > > (for
> > > > > your
> > > > > dpdk-specific versions in the sonames) and so overrode the
> > > > > soversion
> > > > > explicitly. The fix for meson is to remove this overriding i.e.
> > > > > remove
> > > > > "soversion:" parameter for each shared_library() call.
> > > > >
> > > > >
> > > > > > Also, we should leave the current at 20.0 - let's not break
> > > > > > compatibility already, please :-)
> > > > > >
> > > > >
> > > > > If we do this, maybe we can use 20.0.1 and 20.0.2 version
> > > > > numbers?
> > > >
> > > > Yes, that's what I meant - IMHO we should just take the hit and
> > > > use the
> > > > slightly weird 20.0 format until next year, and add a third digit
> > > > for
> > > > compatible updates. Then for v21 we can drop it.
> > > >
> > >
> > > My concern with that is us forgetting, because we'll put in place
> > > hacks to
> > > have the soversion be the first two numbers of the version. Then we
> > > need to
> > > remember to remove those hacks before 20.11 goes out or we'll end
> > > up with
> > > 21.0 being the soversion again.
> > >
> > > For that reason, I'd rather see us fix it now before 19.11 gets
> > > widely
> > > adopted.
> >
> > Me too I vote for fixing soname as 20 in a small 19.11.1
> > and release it quickly.
> >
> > We are supposed to experience tooling and scheme adoption during this
> > first year of new ABI policy. I think we should stick to the planned
> > versionning, in order to avoid any surprise for the next major ABI.
>
> If the concern is forgetting, I'll happily set a reminder in my
> calendar and then nag everybody :-)
>
> In my opinion declaring stability and compatibility, with press
> releases and articles, and then immediately breaking it in the first
> point release does not send the right message. Not because there was a
> bug, which happens all the time, but because mantaining backward
> compatibility sometimes means having to carry less-than-ideal or
> downright ugly hacks for a while.
>
> If we are not ready to maintain compatibility by carrying an ugly
> workaround when the implications are minor or non-existant (I'm pretty
> sure few would really cares what format the soname has, it's all
> handled by tools), what happens next time when the implications are
> that a broken feature stays broken, or that lower performance stays
> lower, and so on?
>
Valid points. I've already posted one pair of patches for the quick-break
solution that most folks preferred, but it's probably as well to
investigate the alternative too a bit more. I'll try and come up with
something for it tomorrow.
/Bruce
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 16:20 ` Luca Boccassi
2019-12-10 16:32 ` Bruce Richardson
@ 2019-12-10 16:39 ` Thomas Monjalon
2019-12-10 17:00 ` Bruce Richardson
1 sibling, 1 reply; 28+ messages in thread
From: Thomas Monjalon @ 2019-12-10 16:39 UTC (permalink / raw)
To: Luca Boccassi, Bruce Richardson, Ferruh Yigit
Cc: Kinsella, Ray, David Marchand, Christian Ehrhardt,
Timothy Redaelli, Kevin Traynor, dpdk-dev, Laatz, Kevin,
Andrew Rybchenko, Neil Horman
10/12/2019 17:20, Luca Boccassi:
> On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > With new process, the major ABI releases will be compatible
> > > > > > > until it is
> > > > > > > deprecated (until next LTS for now),
> > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK
> > > > > > > versions
> > > > > > > until DPDK_20.11
> > > > > > > will be ABI compatible with this version.
> > > > > > >
> > > > > > > But if we introduce a new API after major ABI, say in 20.02
> > > > > > > release, are we
> > > > > > > allowed to break the ABI for that API before DPDK_20.11?
> > > > > > >
> > > > > > > If we allow it break, following problem will be observed:
> > > > > > > Assume an application using .so.20.1 library, and using the
> > > > > > > new
> > > > > > > API introduced
> > > > > > > in 20.02, lets say foo(),
> > > > > > > but when application switches to .so.20.2 (released via
> > > > > > > DPDK_20.05), application
> > > > > > > will fail because of ABI breakage in foo().
> > > > > > >
> > > > > > > I think it is fair that application expects forward
> > > > > > > compatibility in minor
> > > > > > > versions of a shared library.
> > > > > > > Like if application linked against .so.20.2, fair to expect
> > > > > > > .so.20.3, .so.20.4
> > > > > > > etc will work fine. I think currently only .so.20.0 is
> > > > > > > fully
> > > > > > > forward compatible.
> > > > > > >
> > > > > > > If we all agree on this, we may need to tweak the process a
> > > > > > > little, but before
> > > > > > > diving into implementation details, I would like to be sure
> > > > > > > we
> > > > > > > are in same page.
> > > > > > >
> > > > > >
> > > > > > Well, any new API's generally come in as experimental, in
> > > > > > which
> > > > > > case
> > > > > > changes are allowed, and breakage can be expected. If they
> > > > > > are
> > > > > > not
> > > > > > experiemental, then the ABI policy applies to them in that
> > > > > > they
> > > > > > cannot
> > > > > > change since they are part of the .21 ABI, even if that ABI
> > > > > > is
> > > > > > not fully
> > > > > > complete yet. For any application only using stable, non-
> > > > > > experimental
> > > > > > functions, forward compatibility must be maintained IMHO.
> > > > > >
> > > > >
> > > > > Talking about not experimental APIs, experimental ones free
> > > > > from
> > > > > the process.
> > > > >
> > > > > And when and API added in 20.02 (ABI_20.1) it is kind of still
> > > > > ABI_20, because
> > > > > it should be supported for following ABI_20.x, instead of
> > > > > calling
> > > > > it ABI_21, and
> > > > > this minor tweak (and mind shift) in .map files can be our
> > > > > solution.
> > > >
> > > > Related at what to do with adding versions between major ABI
> > > > versions, when
> > > > investigating with Kevin the ABI checking we have made an
> > > > unpleasant
> > > > discovery:
> > > >
> > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > already
> > > > broken
> > > > our ABI according to libabigail.
> > > >
> > > > The Gory Details [skip to the end for suggestions to fix]
> > > > ------------------------------------------------------------
> > > >
> > > > The reason for this is that the soversion encoded in each library
> > > > -
> > > > whether
> > > > built with meson or make - is the full 20.0 version, not just the
> > > > major ABI
> > > > .20 part. Then when apps link against DPDK, they actually encode
> > > > the
> > > > 20.0.
> > > >
> > > > So what this means is that currently - using a make build as an
> > > > example
> > > > here - ldd on the latest head build gives:
> > > >
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-linux-gcc/app/testpmd | head
> > > > linux-vdso.so.1 (0x00007fff6813d000)
> > > > librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> > > > librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> > > > librte_mempool_dpaa.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1
> > > > (0x00007f36d7224000)
> > > > librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1
> > > > (0x00007f36d71ba000)
> > > > librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> > > > librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> > > > librte_pmd_softnic.so.20.1 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1
> > > > (0x00007f36d70b7000)
> > > > librte_flow_classify.so.0.201 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_flow_classify.so.0.201
> > > > (0x00007f36d70b1000)
> > > > librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> > > > ...
> > > >
> > > > Similarly ldd on a 19.11 checkout gives:
> > > >
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd
> > > > x86_64-
> > > > native-linux-gcc_v19.11/app/testpmd | head
> > > > linux-vdso.so.1 (0x00007ffc2a964000)
> > > > librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0
> > > > (0x00007fd4dc6b6000)
> > > > librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0
> > > > (0x00007fd4dc6a3000)
> > > > librte_mempool_dpaa.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> > > > (0x00007fd4dc69e000)
> > > > librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0
> > > > (0x00007fd4dc634000)
> > > > librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0
> > > > (0x00007fd4dc5a0000)
> > > > librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0
> > > > (0x00007fd4dc55d000)
> > > > librte_pmd_softnic.so.20.0 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> > > > (0x00007fd4dc531000)
> > > > librte_flow_classify.so.0.200 =>
> > > > /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> > > > (0x00007fd4dc52b000)
> > > > librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-
> > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0
> > > > (0x00007fd4dc502000)
> > > >
> > > > The final check - using the 19.11 compiled testpmd with the
> > > > library
> > > > path
> > > > set to 20.02 versionned libs:
> > > >
> > > > LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-
> > > > native-
> > > > linux-gcc_v19.11/app/testpmd | head
> > > > linux-vdso.so.1 (0x00007ffc711fc000)
> > > > librte_pmd_bond.so.20.0 => not found
> > > > librte_pmd_dpaa.so.20.0 => not found
> > > > librte_mempool_dpaa.so.20.0 => not found
> > > > librte_pmd_ixgbe.so.20.0 => not found
> > > > librte_pmd_i40e.so.20.0 => not found
> > > > librte_pmd_bnxt.so.20.0 => not found
> > > > librte_pmd_softnic.so.20.0 => not found
> > > > librte_flow_classify.so.0.200 => not found
> > > > librte_pipeline.so.20.0 => not found
> > > >
> > > > Fixing This
> > > > -----------
> > > >
> > > > To fix this, we need to ensure that the SONAME remains constant
> > > > across the
> > > > releases. Therefore, I currently see two options:
> > > >
> > > > 1. keep 20.0 as the version and soname across all releases in
> > > > 2020,
> > > > i.e.
> > > > just revert the ABIVERSION change patch. Trouble there is how
> > > > to
> > > > track
> > > > 20.02 vs 20.05 etc. etc.
> > > >
> > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > This
> > > > has the
> > > > advantage of keeping the existing planned schemes, but has the
> > > > really big
> > > > downside of breaking ABI compatibility with anyone who has
> > > > already
> > > > compiled with 19.11.
> > > >
> > > > Personally, of the two options - unless someone can come up with
> > > > a
> > > > third
> > > > option - I'd tend towards the second, fixing the builds to remove
> > > > the
> > > > .0 in
> > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > widespread
> > > > adoption. Since this ABI stability is new, teething problems may
> > > > be
> > > > expected.
> > > >
> > > > Thoughts and comments?
> > > > /Bruce
> > > >
> > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > so_version
> > > > variable and all references to it from lib/meson.build and
> > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > >
> > > Hi,
> > >
> > > With libtool and its (arguably arcane) format, only the first digit
> > > is
> > > the ABI current version and gets encoded in the elf header. The
> > > other
> > > digits can be used to track compatible minor increments, and are
> > > mostly
> > > ignored. On the system a symlink libfoo.so.major ->
> > > libfoo.so.major.minor is added.
> > >
> > > Eg:
> > >
> > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > SONAME
> > > 0x000000000000000e (SONAME) Library soname:
> > > [libzmq.so.5]
> > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > >
> > > Can we do the same? Not sure what the right incantation is for
> > > Meson,
> > > but it should be possibly.
> > >
> >
> > That's essentially option 2, and it's still an ABI break because
> > existing
> > builds of 19.11 have the soname will the full version number in it.
> > The
> > default behaviour for meson is exactly how you described it, except
> > that
> > previously we needed more exact control over the version info (for
> > your
> > dpdk-specific versions in the sonames) and so overrode the soversion
> > explicitly. The fix for meson is to remove this overriding i.e.
> > remove
> > "soversion:" parameter for each shared_library() call.
> >
> >
> > > Also, we should leave the current at 20.0 - let's not break
> > > compatibility already, please :-)
> > >
> >
> > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
>
> Yes, that's what I meant - IMHO we should just take the hit and use the
> slightly weird 20.0 format until next year, and add a third digit for
> compatible updates. Then for v21 we can drop it.
I am not sure to understand everything above.
My understanding is that the soname must include major and minor:
libfoo.major.minor
and the package system must install 2 symlinks:
libfoo.major points to latest installed minor
libfoo points to latest major
We are missing the symlink libfoo.major in 19.11.0.
I suggest to just add this symlink in 19.11.1 and 20.02.
The other question is to know what the internal apps should link?
Is there any sense linking testpmd to libfoo.major?
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 16:39 ` Thomas Monjalon
@ 2019-12-10 17:00 ` Bruce Richardson
0 siblings, 0 replies; 28+ messages in thread
From: Bruce Richardson @ 2019-12-10 17:00 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Luca Boccassi, Ferruh Yigit, Kinsella, Ray, David Marchand,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On Tue, Dec 10, 2019 at 05:39:00PM +0100, Thomas Monjalon wrote:
> 10/12/2019 17:20, Luca Boccassi:
> > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote:
> > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote:
> > > > >
> > > > > Related at what to do with adding versions between major ABI
> > > > > versions, when
> > > > > investigating with Kevin the ABI checking we have made an
> > > > > unpleasant
> > > > > discovery:
> > > > >
> > > > > This minor version bumping from 20.0 to 20.1 has apparently
> > > > > already
> > > > > broken
> > > > > our ABI according to libabigail.
> > > > >
> > > > > The Gory Details [skip to the end for suggestions to fix]
> > > > > ------------------------------------------------------------
<snip>
> > > > >
> > > > > Fixing This
> > > > > -----------
> > > > >
> > > > > To fix this, we need to ensure that the SONAME remains constant
> > > > > across the
> > > > > releases. Therefore, I currently see two options:
> > > > >
> > > > > 1. keep 20.0 as the version and soname across all releases in
> > > > > 2020,
> > > > > i.e.
> > > > > just revert the ABIVERSION change patch. Trouble there is how
> > > > > to
> > > > > track
> > > > > 20.02 vs 20.05 etc. etc.
> > > > >
> > > > > 2. remove the .0, .1 from the SONAMES stored in the libraries.
> > > > > This
> > > > > has the
> > > > > advantage of keeping the existing planned schemes, but has the
> > > > > really big
> > > > > downside of breaking ABI compatibility with anyone who has
> > > > > already
> > > > > compiled with 19.11.
> > > > >
> > > > > Personally, of the two options - unless someone can come up with
> > > > > a
> > > > > third
> > > > > option - I'd tend towards the second, fixing the builds to remove
> > > > > the
> > > > > .0 in
> > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 gets
> > > > > widespread
> > > > > adoption. Since this ABI stability is new, teething problems may
> > > > > be
> > > > > expected.
> > > > >
> > > > > Thoughts and comments?
> > > > > /Bruce
> > > > >
> > > > > BTW: For meson, the patch for option 2 is just to remove the
> > > > > so_version
> > > > > variable and all references to it from lib/meson.build and
> > > > > drivers/meson.build. Haven't looked into a "make" fix yet.
> > > >
> > > > Hi,
> > > >
> > > > With libtool and its (arguably arcane) format, only the first digit
> > > > is
> > > > the ABI current version and gets encoded in the elf header. The
> > > > other
> > > > digits can be used to track compatible minor increments, and are
> > > > mostly
> > > > ignored. On the system a symlink libfoo.so.major ->
> > > > libfoo.so.major.minor is added.
> > > >
> > > > Eg:
> > > >
> > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep
> > > > SONAME
> > > > 0x000000000000000e (SONAME) Library soname:
> > > > [libzmq.so.5]
> > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5
> > > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux-
> > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3
> > > >
> > > > Can we do the same? Not sure what the right incantation is for
> > > > Meson,
> > > > but it should be possibly.
> > > >
> > >
> > > That's essentially option 2, and it's still an ABI break because
> > > existing
> > > builds of 19.11 have the soname will the full version number in it.
> > > The
> > > default behaviour for meson is exactly how you described it, except
> > > that
> > > previously we needed more exact control over the version info (for
> > > your
> > > dpdk-specific versions in the sonames) and so overrode the soversion
> > > explicitly. The fix for meson is to remove this overriding i.e.
> > > remove
> > > "soversion:" parameter for each shared_library() call.
> > >
> > >
> > > > Also, we should leave the current at 20.0 - let's not break
> > > > compatibility already, please :-)
> > > >
> > >
> > > If we do this, maybe we can use 20.0.1 and 20.0.2 version numbers?
> >
> > Yes, that's what I meant - IMHO we should just take the hit and use the
> > slightly weird 20.0 format until next year, and add a third digit for
> > compatible updates. Then for v21 we can drop it.
>
> I am not sure to understand everything above.
> My understanding is that the soname must include major and minor:
> libfoo.major.minor
The filename should have, but the soname should only have the major ABI
version. It's the soname recorded in the elf file that is used by the
linker, which ignores any version suffixes on the .so.
> and the package system must install 2 symlinks:
> libfoo.major points to latest installed minor
> libfoo points to latest major
>
> We are missing the symlink libfoo.major in 19.11.0.
> I suggest to just add this symlink in 19.11.1 and 20.02.
>
Symlinks won't actually help here, as the suffixes are ignored. In a normal
case, where libfoo.so and libfoo.so.x both point to libfoo.so.x.y, linking
against any one of the three filenames will result in the exact same
linkage info in the binary, based on the soname in libfoo.so.x.y. Ideally,
that should be "libfoo.so.x", which remains constant for all values of "y".
> The other question is to know what the internal apps should link?
> Is there any sense linking testpmd to libfoo.major?
>
Nothing we can do to control this, except ensure correct soname info on
linking the .so.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 14:36 ` Bruce Richardson
2019-12-10 15:03 ` Luca Boccassi
@ 2019-12-10 15:04 ` Ferruh Yigit
2019-12-10 15:37 ` Ferruh Yigit
2019-12-10 15:40 ` Kinsella, Ray
2019-12-11 13:32 ` Neil Horman
3 siblings, 1 reply; 28+ messages in thread
From: Ferruh Yigit @ 2019-12-10 15:04 UTC (permalink / raw)
To: Bruce Richardson
Cc: Kinsella, Ray, Thomas Monjalon, David Marchand, Luca Boccassi,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On 12/10/2019 2:36 PM, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
>> On 12/10/2019 12:04 PM, Bruce Richardson wrote:
>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>> Hi,
>>>>
>>>> With new process, the major ABI releases will be compatible until it is
>>>> deprecated (until next LTS for now),
>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>> will be ABI compatible with this version.
>>>>
>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>
>>>> If we allow it break, following problem will be observed:
>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>> in 20.02, lets say foo(),
>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>> will fail because of ABI breakage in foo().
>>>>
>>>> I think it is fair that application expects forward compatibility in minor
>>>> versions of a shared library.
>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>
>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>
>>>
>>> Well, any new API's generally come in as experimental, in which case
>>> changes are allowed, and breakage can be expected. If they are not
>>> experiemental, then the ABI policy applies to them in that they cannot
>>> change since they are part of the .21 ABI, even if that ABI is not fully
>>> complete yet. For any application only using stable, non-experimental
>>> functions, forward compatibility must be maintained IMHO.
>>>
>>
>> Talking about not experimental APIs, experimental ones free from the process.
>>
>> And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
>> it should be supported for following ABI_20.x, instead of calling it ABI_21, and
>> this minor tweak (and mind shift) in .map files can be our solution.
>
> Related at what to do with adding versions between major ABI versions, when
> investigating with Kevin the ABI checking we have made an unpleasant
> discovery:
>
> This minor version bumping from 20.0 to 20.1 has apparently already broken
> our ABI according to libabigail.
>
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
>
> The reason for this is that the soversion encoded in each library - whether
> built with meson or make - is the full 20.0 version, not just the major ABI
> .20 part. Then when apps link against DPDK, they actually encode the 20.0.
>
> So what this means is that currently - using a make build as an example
> here - ldd on the latest head build gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
> linux-vdso.so.1 (0x00007fff6813d000)
> librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
> librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
> librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
> librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
> librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> ...
>
> Similarly ldd on a 19.11 checkout gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc2a964000)
> librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
> librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
> librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
> librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
> librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
> librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
> librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
> librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
> librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
>
> The final check - using the 19.11 compiled testpmd with the library path
> set to 20.02 versionned libs:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc711fc000)
> librte_pmd_bond.so.20.0 => not found
> librte_pmd_dpaa.so.20.0 => not found
> librte_mempool_dpaa.so.20.0 => not found
> librte_pmd_ixgbe.so.20.0 => not found
> librte_pmd_i40e.so.20.0 => not found
> librte_pmd_bnxt.so.20.0 => not found
> librte_pmd_softnic.so.20.0 => not found
> librte_flow_classify.so.0.200 => not found
> librte_pipeline.so.20.0 => not found
>
> Fixing This
> -----------
>
> To fix this, we need to ensure that the SONAME remains constant across the
> releases. Therefore, I currently see two options:
>
> 1. keep 20.0 as the version and soname across all releases in 2020, i.e.
> just revert the ABIVERSION change patch. Trouble there is how to track
> 20.02 vs 20.05 etc. etc.
>
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
> advantage of keeping the existing planned schemes, but has the really big
> downside of breaking ABI compatibility with anyone who has already
> compiled with 19.11.
>
> Personally, of the two options - unless someone can come up with a third
> option - I'd tend towards the second, fixing the builds to remove the .0 in
> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
> adoption. Since this ABI stability is new, teething problems may be
> expected.
>
> Thoughts and comments?
Arghh :(
I agree that 2) is proper fix and if we want to fix properly we should do a
19.11.1 ASAP.
Can having soname as "20.0" but having file name as .so.20.1 work?
If it does we can "workaround" it by keeping SONAME as "20.0" (instead of 20)
during the life time of the ABI_20 and fix it properly in ABI_21.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 15:04 ` Ferruh Yigit
@ 2019-12-10 15:37 ` Ferruh Yigit
0 siblings, 0 replies; 28+ messages in thread
From: Ferruh Yigit @ 2019-12-10 15:37 UTC (permalink / raw)
To: Bruce Richardson
Cc: Kinsella, Ray, Thomas Monjalon, David Marchand, Luca Boccassi,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
On 12/10/2019 3:04 PM, Ferruh Yigit wrote:
> On 12/10/2019 2:36 PM, Bruce Richardson wrote:
>> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
>>> On 12/10/2019 12:04 PM, Bruce Richardson wrote:
>>>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
>>>>> Hi,
>>>>>
>>>>> With new process, the major ABI releases will be compatible until it is
>>>>> deprecated (until next LTS for now),
>>>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
>>>>> will be ABI compatible with this version.
>>>>>
>>>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
>>>>> allowed to break the ABI for that API before DPDK_20.11?
>>>>>
>>>>> If we allow it break, following problem will be observed:
>>>>> Assume an application using .so.20.1 library, and using the new API introduced
>>>>> in 20.02, lets say foo(),
>>>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
>>>>> will fail because of ABI breakage in foo().
>>>>>
>>>>> I think it is fair that application expects forward compatibility in minor
>>>>> versions of a shared library.
>>>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
>>>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
>>>>>
>>>>> If we all agree on this, we may need to tweak the process a little, but before
>>>>> diving into implementation details, I would like to be sure we are in same page.
>>>>>
>>>>
>>>> Well, any new API's generally come in as experimental, in which case
>>>> changes are allowed, and breakage can be expected. If they are not
>>>> experiemental, then the ABI policy applies to them in that they cannot
>>>> change since they are part of the .21 ABI, even if that ABI is not fully
>>>> complete yet. For any application only using stable, non-experimental
>>>> functions, forward compatibility must be maintained IMHO.
>>>>
>>>
>>> Talking about not experimental APIs, experimental ones free from the process.
>>>
>>> And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
>>> it should be supported for following ABI_20.x, instead of calling it ABI_21, and
>>> this minor tweak (and mind shift) in .map files can be our solution.
>>
>> Related at what to do with adding versions between major ABI versions, when
>> investigating with Kevin the ABI checking we have made an unpleasant
>> discovery:
>>
>> This minor version bumping from 20.0 to 20.1 has apparently already broken
>> our ABI according to libabigail.
>>
>> The Gory Details [skip to the end for suggestions to fix]
>> ------------------------------------------------------------
>>
>> The reason for this is that the soversion encoded in each library - whether
>> built with meson or make - is the full 20.0 version, not just the major ABI
>> .20 part. Then when apps link against DPDK, they actually encode the 20.0.
>>
>> So what this means is that currently - using a make build as an example
>> here - ldd on the latest head build gives:
>>
>> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
>> linux-vdso.so.1 (0x00007fff6813d000)
>> librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
>> librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
>> librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
>> librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
>> librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
>> librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
>> librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
>> librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
>> librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
>> ...
>>
>> Similarly ldd on a 19.11 checkout gives:
>>
>> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>> linux-vdso.so.1 (0x00007ffc2a964000)
>> librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
>> librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
>> librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
>> librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
>> librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
>> librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
>> librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
>> librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
>> librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
>>
>> The final check - using the 19.11 compiled testpmd with the library path
>> set to 20.02 versionned libs:
>>
>> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
>> linux-vdso.so.1 (0x00007ffc711fc000)
>> librte_pmd_bond.so.20.0 => not found
>> librte_pmd_dpaa.so.20.0 => not found
>> librte_mempool_dpaa.so.20.0 => not found
>> librte_pmd_ixgbe.so.20.0 => not found
>> librte_pmd_i40e.so.20.0 => not found
>> librte_pmd_bnxt.so.20.0 => not found
>> librte_pmd_softnic.so.20.0 => not found
>> librte_flow_classify.so.0.200 => not found
>> librte_pipeline.so.20.0 => not found
>>
>> Fixing This
>> -----------
>>
>> To fix this, we need to ensure that the SONAME remains constant across the
>> releases. Therefore, I currently see two options:
>>
>> 1. keep 20.0 as the version and soname across all releases in 2020, i.e.
>> just revert the ABIVERSION change patch. Trouble there is how to track
>> 20.02 vs 20.05 etc. etc.
>>
>> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
>> advantage of keeping the existing planned schemes, but has the really big
>> downside of breaking ABI compatibility with anyone who has already
>> compiled with 19.11.
>>
>> Personally, of the two options - unless someone can come up with a third
>> option - I'd tend towards the second, fixing the builds to remove the .0 in
>> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
>> adoption. Since this ABI stability is new, teething problems may be
>> expected.
>>
>> Thoughts and comments?
>
> Arghh :(
>
> I agree that 2) is proper fix and if we want to fix properly we should do a
> 19.11.1 ASAP.
>
> Can having soname as "20.0" but having file name as .so.20.1 work?
No it is not working, dynamic linker is looking for the .so file that has been
put into soname.
So I am for option 2 and 19.11.1.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 14:36 ` Bruce Richardson
2019-12-10 15:03 ` Luca Boccassi
2019-12-10 15:04 ` Ferruh Yigit
@ 2019-12-10 15:40 ` Kinsella, Ray
2019-12-11 13:32 ` Neil Horman
3 siblings, 0 replies; 28+ messages in thread
From: Kinsella, Ray @ 2019-12-10 15:40 UTC (permalink / raw)
To: Richardson, Bruce, Yigit, Ferruh
Cc: Thomas Monjalon, David Marchand, Luca Boccassi,
Christian Ehrhardt, Timothy Redaelli, Kevin Traynor, dpdk-dev,
Laatz, Kevin, Andrew Rybchenko, Neil Horman
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Tuesday 10 December 2019 14:37
> To: Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: Kinsella, Ray <ray.kinsella@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>; David Marchand <david.marchand@redhat.com>; Luca
> Boccassi <bluca@debian.org>; Christian Ehrhardt
> <christian.ehrhardt@canonical.com>; Timothy Redaelli
> <tredaelli@redhat.com>; Kevin Traynor <ktraynor@redhat.com>; dpdk-dev
> <dev@dpdk.org>; Laatz, Kevin <kevin.laatz@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; Neil Horman <nhorman@redhat.com>
> Subject: Re: How to manage new APIs added after major ABI release?
>
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > >> Hi,
> > >>
> > >> With new process, the major ABI releases will be compatible until
> > >> it is deprecated (until next LTS for now), like current ABI
> version
> > >> is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11 will be ABI
> > >> compatible with this version.
> > >>
> > >> But if we introduce a new API after major ABI, say in 20.02
> > >> release, are we allowed to break the ABI for that API before
> DPDK_20.11?
> > >>
> > >> If we allow it break, following problem will be observed:
> > >> Assume an application using .so.20.1 library, and using the new
> API
> > >> introduced in 20.02, lets say foo(), but when application switches
> > >> to .so.20.2 (released via DPDK_20.05), application will fail
> > >> because of ABI breakage in foo().
> > >>
> > >> I think it is fair that application expects forward compatibility
> > >> in minor versions of a shared library.
> > >> Like if application linked against .so.20.2, fair to expect
> > >> .so.20.3, .so.20.4 etc will work fine. I think currently only
> .so.20.0 is fully forward compatible.
> > >>
> > >> If we all agree on this, we may need to tweak the process a
> little,
> > >> but before diving into implementation details, I would like to be
> sure we are in same page.
> > >>
> > >
> > > Well, any new API's generally come in as experimental, in which
> case
> > > changes are allowed, and breakage can be expected. If they are not
> > > experiemental, then the ABI policy applies to them in that they
> > > cannot change since they are part of the .21 ABI, even if that ABI
> > > is not fully complete yet. For any application only using stable,
> > > non-experimental functions, forward compatibility must be
> maintained IMHO.
> > >
> >
> > Talking about not experimental APIs, experimental ones free from the
> process.
> >
> > And when and API added in 20.02 (ABI_20.1) it is kind of still
> ABI_20,
> > because it should be supported for following ABI_20.x, instead of
> > calling it ABI_21, and this minor tweak (and mind shift) in .map
> files can be our solution.
>
> Related at what to do with adding versions between major ABI versions,
> when investigating with Kevin the ABI checking we have made an
> unpleasant
> discovery:
>
> This minor version bumping from 20.0 to 20.1 has apparently already
> broken our ABI according to libabigail.
>
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
>
> The reason for this is that the soversion encoded in each library -
> whether built with meson or make - is the full 20.0 version, not just
> the major ABI
> .20 part. Then when apps link against DPDK, they actually encode the
> 20.0.
>
> So what this means is that currently - using a make build as an example
> here - ldd on the latest head build gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> linux-gcc/app/testpmd | head
> linux-vdso.so.1 (0x00007fff6813d000)
> librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
> librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
> librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
> librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
> librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000) ...
>
> Similarly ldd on a 19.11 checkout gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-
> native-linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc2a964000)
> librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
> librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
> librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0
> (0x00007fd4dc69e000)
> librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
> librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
> librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
> librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0
> (0x00007fd4dc531000)
> librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-
> native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200
> (0x00007fd4dc52b000)
> librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-
> linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
>
> The final check - using the 19.11 compiled testpmd with the library
> path set to 20.02 versionned libs:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-
> linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc711fc000)
> librte_pmd_bond.so.20.0 => not found
> librte_pmd_dpaa.so.20.0 => not found
> librte_mempool_dpaa.so.20.0 => not found
> librte_pmd_ixgbe.so.20.0 => not found
> librte_pmd_i40e.so.20.0 => not found
> librte_pmd_bnxt.so.20.0 => not found
> librte_pmd_softnic.so.20.0 => not found
> librte_flow_classify.so.0.200 => not found
> librte_pipeline.so.20.0 => not found
>
> Fixing This
> -----------
>
> To fix this, we need to ensure that the SONAME remains constant across
> the releases. Therefore, I currently see two options:
>
> 1. keep 20.0 as the version and soname across all releases in 2020,
> i.e.
> just revert the ABIVERSION change patch. Trouble there is how to
> track
> 20.02 vs 20.05 etc. etc.
>
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has
> the
> advantage of keeping the existing planned schemes, but has the really
> big
> downside of breaking ABI compatibility with anyone who has already
> compiled with 19.11.
>
> Personally, of the two options - unless someone can come up with a
> third option - I'd tend towards the second, fixing the builds to remove
> the .0 in the soname, and releasing that ASAP as 19.11.1 before 19.11
> gets widespread adoption. Since this ABI stability is new, teething
> problems may be expected.
>
> Thoughts and comments?
> /Bruce
>
> BTW: For meson, the patch for option 2 is just to remove the so_version
> variable and all references to it from lib/meson.build and
> drivers/meson.build. Haven't looked into a "make" fix yet.
From the DPDK ABI Version Document ...
A library's soname. is typically used to provide backward compatibility information about a given library, describing the lowest common denominator ABI supported by the library. The soname or logical name for the library, is typically comprised of the library's name and major version e.g. librte_eal.so.20
https://doc.dpdk.org/guides/contributing/abi_versioning.html
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [dpdk-dev] How to manage new APIs added after major ABI release?
2019-12-10 14:36 ` Bruce Richardson
` (2 preceding siblings ...)
2019-12-10 15:40 ` Kinsella, Ray
@ 2019-12-11 13:32 ` Neil Horman
3 siblings, 0 replies; 28+ messages in thread
From: Neil Horman @ 2019-12-11 13:32 UTC (permalink / raw)
To: Bruce Richardson
Cc: Ferruh Yigit, Kinsella, Ray, Thomas Monjalon, David Marchand,
Luca Boccassi, Christian Ehrhardt, Timothy Redaelli,
Kevin Traynor, dpdk-dev, Laatz, Kevin, Andrew Rybchenko,
Neil Horman
On Tue, Dec 10, 2019 at 02:36:43PM +0000, Bruce Richardson wrote:
> On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit wrote:
> > On 12/10/2019 12:04 PM, Bruce Richardson wrote:
> > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> > >> Hi,
> > >>
> > >> With new process, the major ABI releases will be compatible until it is
> > >> deprecated (until next LTS for now),
> > >> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> > >> will be ABI compatible with this version.
> > >>
> > >> But if we introduce a new API after major ABI, say in 20.02 release, are we
> > >> allowed to break the ABI for that API before DPDK_20.11?
> > >>
> > >> If we allow it break, following problem will be observed:
> > >> Assume an application using .so.20.1 library, and using the new API introduced
> > >> in 20.02, lets say foo(),
> > >> but when application switches to .so.20.2 (released via DPDK_20.05), application
> > >> will fail because of ABI breakage in foo().
> > >>
> > >> I think it is fair that application expects forward compatibility in minor
> > >> versions of a shared library.
> > >> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> > >> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> > >>
> > >> If we all agree on this, we may need to tweak the process a little, but before
> > >> diving into implementation details, I would like to be sure we are in same page.
> > >>
> > >
> > > Well, any new API's generally come in as experimental, in which case
> > > changes are allowed, and breakage can be expected. If they are not
> > > experiemental, then the ABI policy applies to them in that they cannot
> > > change since they are part of the .21 ABI, even if that ABI is not fully
> > > complete yet. For any application only using stable, non-experimental
> > > functions, forward compatibility must be maintained IMHO.
> > >
> >
> > Talking about not experimental APIs, experimental ones free from the process.
> >
> > And when and API added in 20.02 (ABI_20.1) it is kind of still ABI_20, because
> > it should be supported for following ABI_20.x, instead of calling it ABI_21, and
> > this minor tweak (and mind shift) in .map files can be our solution.
>
> Related at what to do with adding versions between major ABI versions, when
> investigating with Kevin the ABI checking we have made an unpleasant
> discovery:
>
> This minor version bumping from 20.0 to 20.1 has apparently already broken
> our ABI according to libabigail.
>
> The Gory Details [skip to the end for suggestions to fix]
> ------------------------------------------------------------
>
> The reason for this is that the soversion encoded in each library - whether
> built with meson or make - is the full 20.0 version, not just the major ABI
> .20 part. Then when apps link against DPDK, they actually encode the 20.0.
>
> So what this means is that currently - using a make build as an example
> here - ldd on the latest head build gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc/app/testpmd | head
> linux-vdso.so.1 (0x00007fff6813d000)
> librte_pmd_bond.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bond.so.20.1 (0x00007f36d723c000)
> librte_pmd_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 (0x00007f36d7229000)
> librte_mempool_dpaa.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 (0x00007f36d7224000)
> librte_pmd_ixgbe.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 (0x00007f36d71ba000)
> librte_pmd_i40e.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_i40e.so.20.1 (0x00007f36d7126000)
> librte_pmd_bnxt.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 (0x00007f36d70e5000)
> librte_pmd_softnic.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pmd_softnic.so.20.1 (0x00007f36d70b7000)
> librte_flow_classify.so.0.201 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_flow_classify.so.0.201 (0x00007f36d70b1000)
> librte_pipeline.so.20.1 => /home/bruce/dpdk.org/x86_64-native-linux-gcc/lib/librte_pipeline.so.20.1 (0x00007f36d7088000)
> ...
>
> Similarly ldd on a 19.11 checkout gives:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc_v19.11/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc2a964000)
> librte_pmd_bond.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 (0x00007fd4dc6b6000)
> librte_pmd_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 (0x00007fd4dc6a3000)
> librte_mempool_dpaa.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 (0x00007fd4dc69e000)
> librte_pmd_ixgbe.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 (0x00007fd4dc634000)
> librte_pmd_i40e.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 (0x00007fd4dc5a0000)
> librte_pmd_bnxt.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 (0x00007fd4dc55d000)
> librte_pmd_softnic.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 (0x00007fd4dc531000)
> librte_flow_classify.so.0.200 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 (0x00007fd4dc52b000)
> librte_pipeline.so.20.0 => /home/bruce/dpdk.org/x86_64-native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 (0x00007fd4dc502000)
>
Whats going on here? A 19.11 checkout should have DT_NEEDED encoded in
the scheme lib*.so.19.x, no?
> The final check - using the 19.11 compiled testpmd with the library path
> set to 20.02 versionned libs:
>
> LD_LIBRARY_PATH=$(pwd)/x86_64-native-linux-gcc/lib ldd x86_64-native-linux-gcc_v19.11/app/testpmd | head
> linux-vdso.so.1 (0x00007ffc711fc000)
> librte_pmd_bond.so.20.0 => not found
> librte_pmd_dpaa.so.20.0 => not found
> librte_mempool_dpaa.so.20.0 => not found
> librte_pmd_ixgbe.so.20.0 => not found
> librte_pmd_i40e.so.20.0 => not found
> librte_pmd_bnxt.so.20.0 => not found
> librte_pmd_softnic.so.20.0 => not found
> librte_flow_classify.so.0.200 => not found
> librte_pipeline.so.20.0 => not found
>
Shouldn't ldconfig fix that? i.e. when you install libraries, you run ldconfig
and the various minor release libraries should be symlinks to the latest version
of the minor release. That is to say, your library patch for, for example,
librte_pnd_bnxt should look like:
librte_pmd_bnxt.so.20.02
librte_pmd_bnxt.so.20 => librete_pmd_bnxt.so.20
librte_pmd_bnxt.so => librte_pmd_bnxt.so.20.02
Which will allow older built binaries to link at run time.
> Fixing This
> -----------
>
> To fix this, we need to ensure that the SONAME remains constant across the
> releases. Therefore, I currently see two options:
>
> 1. keep 20.0 as the version and soname across all releases in 2020, i.e.
> just revert the ABIVERSION change patch. Trouble there is how to track
> 20.02 vs 20.05 etc. etc.
>
> 2. remove the .0, .1 from the SONAMES stored in the libraries. This has the
> advantage of keeping the existing planned schemes, but has the really big
> downside of breaking ABI compatibility with anyone who has already
> compiled with 19.11.
>
> Personally, of the two options - unless someone can come up with a third
> option - I'd tend towards the second, fixing the builds to remove the .0 in
> the soname, and releasing that ASAP as 19.11.1 before 19.11 gets widespread
> adoption. Since this ABI stability is new, teething problems may be
> expected.
>
> Thoughts and comments?
> /Bruce
>
> BTW: For meson, the patch for option 2 is just to remove the so_version
> variable and all references to it from lib/meson.build and
> drivers/meson.build. Haven't looked into a "make" fix yet.
>
>
^ permalink raw reply [flat|nested] 28+ messages in thread