From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60680A04F0; Tue, 10 Dec 2019 19:22:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 26B6823D; Tue, 10 Dec 2019 19:22:39 +0100 (CET) Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by dpdk.org (Postfix) with ESMTP id 3FBAF1F5 for ; Tue, 10 Dec 2019 19:22:38 +0100 (CET) Received: by mail-wm1-f68.google.com with SMTP id c20so4322212wmb.0 for ; Tue, 10 Dec 2019 10:22:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:content-transfer-encoding:user-agent:mime-version; bh=BE8fU6GiRpl1Uj+zF92SAgBQr6oIVZA2A6xEox6p6UA=; b=r1BxMvMmCEOSPWBHkf7Z3qfspmI5LlJFrA/5Fypxnb8JDhnozQxSql+YlaCUWWKq64 9WkDSScqf6YHJYTeWf2D9UEjUlr4PJ+WFPxcsINPDXHEPYEL/80ZFWh9morBtLlKIe7x yYToJfvQLYG1W99gvrrzKSKNl5qnpAzzGb1EAR4GKSwg8vxWSccmCun6+/KjKIzc/Gbt +WQvSQTzMYW1CbLAP4Ln/joXWbK0/cnfe2tekztjxA5pqg2qiXMA776bASIEUQP5y7RH lZqtLVFbP6k+sySq6MWq56zad8k3L7abJCmBNHvoEwTa4Z9g8WzKIEK348j53RqtbIVJ uP7w== X-Gm-Message-State: APjAAAVieqp5c4PQJedpWOq0rWejy8yGiF7MBwGGRGEpfHdhjA1oFTUp t+J6aVz6CFWHqjc61rLzuBKDKy8lhbY= X-Google-Smtp-Source: APXvYqw7FWXNYbKUNeMZlU7UNEn5mHQwY4z0JvgE9RFHFN/fdWkL/AtlF4Nhxw0lS+CISAJqvW83/A== X-Received: by 2002:a7b:ce98:: with SMTP id q24mr6300880wmj.41.1576002157557; Tue, 10 Dec 2019 10:22:37 -0800 (PST) Received: from localhost ([2a01:4b00:f419:6f00:7a8e:ed70:5c52:ea3]) by smtp.gmail.com with ESMTPSA id n188sm4395126wme.14.2019.12.10.10.22.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2019 10:22:36 -0800 (PST) Message-ID: <7b8e361498c22d53acac2a33f13ecc47c7901a55.camel@debian.org> From: Luca Boccassi To: Thomas Monjalon , Bruce Richardson Cc: Ferruh Yigit , "Kinsella, Ray" , David Marchand , Christian Ehrhardt , Timothy Redaelli , Kevin Traynor , dpdk-dev , "Laatz, Kevin" , Andrew Rybchenko , Neil Horman Date: Tue, 10 Dec 2019 18:22:36 +0000 In-Reply-To: <2575985.8OLF5JM4Jj@xps> References: <5df1a33b-b338-bde1-6834-e8b5fbe65a04@intel.com> <8ce12dcd655b77fe20e35237684afedb11a51445.camel@debian.org> <20191210163216.GA119@bricha3-MOBL.ger.corp.intel.com> <2575985.8OLF5JM4Jj@xps> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.30.5-1.1 MIME-Version: 1.0 Subject: Re: [dpdk-dev] How to manage new APIs added after major ABI release? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, 2019-12-10 at 18:04 +0100, Thomas Monjalon wrote: > 10/12/2019 17:32, Bruce Richardson: > > On Tue, Dec 10, 2019 at 04:20:41PM +0000, Luca Boccassi wrote: > > > On Tue, 2019-12-10 at 15:46 +0000, Bruce Richardson wrote: > > > > On Tue, Dec 10, 2019 at 03:03:51PM +0000, Luca Boccassi wrote: > > > > > On Tue, 2019-12-10 at 14:36 +0000, Bruce Richardson wrote: > > > > > > On Tue, Dec 10, 2019 at 12:40:53PM +0000, Ferruh Yigit > > > > > > wrote: > > > > > > > On 12/10/2019 12:04 PM, Bruce Richardson wrote: > > > > > > > > On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit > > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > >=20 > > > > > > > > > With new process, the major ABI releases will be > > > > > > > > > compatible > > > > > > > > > until it is > > > > > > > > > deprecated (until next LTS for now), > > > > > > > > > like current ABI version is 20 in DPDK_19.11 and DPDK > > > > > > > > > versions > > > > > > > > > until DPDK_20.11 > > > > > > > > > will be ABI compatible with this version. > > > > > > > > >=20 > > > > > > > > > But if we introduce a new API after major ABI, say in > > > > > > > > > 20.02 > > > > > > > > > release, are we > > > > > > > > > allowed to break the ABI for that API before > > > > > > > > > DPDK_20.11? > > > > > > > > >=20 > > > > > > > > > If we allow it break, following problem will be > > > > > > > > > observed: > > > > > > > > > Assume an application using .so.20.1 library, and > > > > > > > > > using the > > > > > > > > > new > > > > > > > > > API introduced > > > > > > > > > in 20.02, lets say foo(), > > > > > > > > > but when application switches to .so.20.2 (released > > > > > > > > > via > > > > > > > > > DPDK_20.05), application > > > > > > > > > will fail because of ABI breakage in foo(). > > > > > > > > >=20 > > > > > > > > > I think it is fair that application expects forward > > > > > > > > > compatibility in minor > > > > > > > > > versions of a shared library. > > > > > > > > > Like if application linked against .so.20.2, fair to > > > > > > > > > expect > > > > > > > > > .so.20.3, .so.20.4 > > > > > > > > > etc will work fine. I think currently only .so.20.0 > > > > > > > > > is > > > > > > > > > fully > > > > > > > > > forward compatible. > > > > > > > > >=20 > > > > > > > > > If we all agree on this, we may need to tweak the > > > > > > > > > process a > > > > > > > > > little, but before > > > > > > > > > diving into implementation details, I would like to > > > > > > > > > be sure > > > > > > > > > we > > > > > > > > > are in same page. > > > > > > > > >=20 > > > > > > > >=20 > > > > > > > > Well, any new API's generally come in as experimental, > > > > > > > > in > > > > > > > > which > > > > > > > > case > > > > > > > > changes are allowed, and breakage can be expected. If > > > > > > > > they > > > > > > > > are > > > > > > > > not > > > > > > > > experiemental, then the ABI policy applies to them in > > > > > > > > that > > > > > > > > they > > > > > > > > cannot > > > > > > > > change since they are part of the .21 ABI, even if that > > > > > > > > ABI > > > > > > > > is > > > > > > > > not fully > > > > > > > > complete yet. For any application only using stable, > > > > > > > > non- > > > > > > > > experimental > > > > > > > > functions, forward compatibility must be maintained > > > > > > > > IMHO. > > > > > > > >=20 > > > > > > >=20 > > > > > > > Talking about not experimental APIs, experimental ones > > > > > > > free > > > > > > > from > > > > > > > the process. > > > > > > >=20 > > > > > > > And when and API added in 20.02 (ABI_20.1) it is kind of > > > > > > > still > > > > > > > ABI_20, because > > > > > > > it should be supported for following ABI_20.x, instead of > > > > > > > calling > > > > > > > it ABI_21, and > > > > > > > this minor tweak (and mind shift) in .map files can be > > > > > > > our > > > > > > > solution. > > > > > >=20 > > > > > > Related at what to do with adding versions between major > > > > > > ABI > > > > > > versions, when > > > > > > investigating with Kevin the ABI checking we have made an > > > > > > unpleasant > > > > > > discovery: > > > > > >=20 > > > > > > This minor version bumping from 20.0 to 20.1 has apparently > > > > > > already > > > > > > broken > > > > > > our ABI according to libabigail. > > > > > >=20 > > > > > > The Gory Details [skip to the end for suggestions to fix] > > > > > > --------------------------------------------------------- > > > > > > --- > > > > > >=20 > > > > > > The reason for this is that the soversion encoded in each > > > > > > library > > > > > > - > > > > > > whether > > > > > > built with meson or make - is the full 20.0 version, not > > > > > > just the > > > > > > major ABI > > > > > > .20 part. Then when apps link against DPDK, they actually > > > > > > encode > > > > > > the > > > > > > 20.0. > > > > > >=20 > > > > > > So what this means is that currently - using a make build > > > > > > as an > > > > > > example > > > > > > here - ldd on the latest head build gives: > > > > > >=20 > > > > > > LD_LIBRARY_PATH=3D$(pwd)/x86_64-native-linux-gcc/lib ldd > > > > > > x86_64- > > > > > > native-linux-gcc/app/testpmd | head > > > > > > linux-vdso.so.1 (0x00007fff6813d000) > > > > > > librte_pmd_bond.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pmd_bond.so.20.1 > > > > > > (0x00007f36d723c000) > > > > > > librte_pmd_dpaa.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pmd_dpaa.so.20.1 > > > > > > (0x00007f36d7229000) > > > > > > librte_mempool_dpaa.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_mempool_dpaa.so.20.1 > > > > > > (0x00007f36d7224000) > > > > > > librte_pmd_ixgbe.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pmd_ixgbe.so.20.1 > > > > > > (0x00007f36d71ba000) > > > > > > librte_pmd_i40e.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pmd_i40e.so.20.1 > > > > > > (0x00007f36d7126000) > > > > > > librte_pmd_bnxt.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pmd_bnxt.so.20.1 > > > > > > (0x00007f36d70e5000) > > > > > > librte_pmd_softnic.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pmd_softnic.so.20.1 > > > > > > (0x00007f36d70b7000) > > > > > > librte_flow_classify.so.0.201 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_flow_classify.so.0.201 > > > > > > (0x00007f36d70b1000) > > > > > > librte_pipeline.so.20.1 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc/lib/librte_pipeline.so.20.1 > > > > > > (0x00007f36d7088000) > > > > > > ... > > > > > >=20 > > > > > > Similarly ldd on a 19.11 checkout gives: > > > > > >=20 > > > > > > LD_LIBRARY_PATH=3D$(pwd)/x86_64-native-linux-gcc_v19.11/lib > > > > > > ldd > > > > > > x86_64- > > > > > > native-linux-gcc_v19.11/app/testpmd | head > > > > > > linux-vdso.so.1 (0x00007ffc2a964000) > > > > > > librte_pmd_bond.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bond.so.20.0 > > > > > > (0x00007fd4dc6b6000) > > > > > > librte_pmd_dpaa.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_dpaa.so.20.0 > > > > > > (0x00007fd4dc6a3000) > > > > > > librte_mempool_dpaa.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_mempool_dpaa.so.20.0 > > > > > > (0x00007fd4dc69e000) > > > > > > librte_pmd_ixgbe.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_ixgbe.so.20.0 > > > > > > (0x00007fd4dc634000) > > > > > > librte_pmd_i40e.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_i40e.so.20.0 > > > > > > (0x00007fd4dc5a0000) > > > > > > librte_pmd_bnxt.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_bnxt.so.20.0 > > > > > > (0x00007fd4dc55d000) > > > > > > librte_pmd_softnic.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pmd_softnic.so.20.0 > > > > > > (0x00007fd4dc531000) > > > > > > librte_flow_classify.so.0.200 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_flow_classify.so.0.200 > > > > > > (0x00007fd4dc52b000) > > > > > > librte_pipeline.so.20.0 =3D> > > > > > > /home/bruce/dpdk.org/x86_64- > > > > > > native-linux-gcc_v19.11/lib/librte_pipeline.so.20.0 > > > > > > (0x00007fd4dc502000) > > > > > >=20 > > > > > > The final check - using the 19.11 compiled testpmd with the > > > > > > library > > > > > > path > > > > > > set to 20.02 versionned libs: > > > > > >=20 > > > > > > LD_LIBRARY_PATH=3D$(pwd)/x86_64-native-linux-gcc/lib ldd > > > > > > x86_64- > > > > > > native- > > > > > > linux-gcc_v19.11/app/testpmd | head > > > > > > linux-vdso.so.1 (0x00007ffc711fc000) > > > > > > librte_pmd_bond.so.20.0 =3D> not found > > > > > > librte_pmd_dpaa.so.20.0 =3D> not found > > > > > > librte_mempool_dpaa.so.20.0 =3D> not found > > > > > > librte_pmd_ixgbe.so.20.0 =3D> not found > > > > > > librte_pmd_i40e.so.20.0 =3D> not found > > > > > > librte_pmd_bnxt.so.20.0 =3D> not found > > > > > > librte_pmd_softnic.so.20.0 =3D> not found > > > > > > librte_flow_classify.so.0.200 =3D> not found > > > > > > librte_pipeline.so.20.0 =3D> not found > > > > > >=20 > > > > > > Fixing This > > > > > > ----------- > > > > > >=20 > > > > > > To fix this, we need to ensure that the SONAME remains > > > > > > constant > > > > > > across the > > > > > > releases. Therefore, I currently see two options: > > > > > >=20 > > > > > > 1. keep 20.0 as the version and soname across all releases > > > > > > in > > > > > > 2020, > > > > > > i.e. > > > > > > just revert the ABIVERSION change patch. Trouble there is > > > > > > how > > > > > > to > > > > > > track > > > > > > 20.02 vs 20.05 etc. etc. > > > > > >=20 > > > > > > 2. remove the .0, .1 from the SONAMES stored in the > > > > > > libraries. > > > > > > This > > > > > > has the > > > > > > advantage of keeping the existing planned schemes, but > > > > > > has the > > > > > > really big > > > > > > downside of breaking ABI compatibility with anyone who > > > > > > has > > > > > > already > > > > > > compiled with 19.11. > > > > > >=20 > > > > > > Personally, of the two options - unless someone can come up > > > > > > with > > > > > > a > > > > > > third > > > > > > option - I'd tend towards the second, fixing the builds to > > > > > > remove > > > > > > the > > > > > > .0 in > > > > > > the soname, and releasing that ASAP as 19.11.1 before 19.11 > > > > > > gets > > > > > > widespread > > > > > > adoption. Since this ABI stability is new, teething > > > > > > problems may > > > > > > be > > > > > > expected. > > > > > >=20 > > > > > > Thoughts and comments? > > > > > > /Bruce > > > > > >=20 > > > > > > BTW: For meson, the patch for option 2 is just to remove > > > > > > the > > > > > > so_version > > > > > > variable and all references to it from lib/meson.build and > > > > > > drivers/meson.build. Haven't looked into a "make" fix yet. > > > > >=20 > > > > > Hi, > > > > >=20 > > > > > With libtool and its (arguably arcane) format, only the first > > > > > digit > > > > > is > > > > > the ABI current version and gets encoded in the elf header. > > > > > The > > > > > other > > > > > digits can be used to track compatible minor increments, and > > > > > are > > > > > mostly > > > > > ignored. On the system a symlink libfoo.so.major -> > > > > > libfoo.so.major.minor is added. > > > > >=20 > > > > > Eg: > > > > >=20 > > > > > $ readelf -d /usr/lib/x86_64-linux-gnu/libzmq.so.5.2.3 | grep > > > > > SONAME > > > > > 0x000000000000000e (SONAME) Library soname: > > > > > [libzmq.so.5] > > > > > $ ls -l /usr/lib/x86_64-linux-gnu/libzmq.so.5 > > > > > lrwxrwxrwx 1 root root 15 Dec 31 2014 /usr/lib/x86_64-linux- > > > > > gnu/libzmq.so.5 -> libzmq.so.5.2.3 > > > > >=20 > > > > > Can we do the same? Not sure what the right incantation is > > > > > for > > > > > Meson, > > > > > but it should be possibly. > > > > >=20 > > > >=20 > > > > That's essentially option 2, and it's still an ABI break > > > > because > > > > existing > > > > builds of 19.11 have the soname will the full version number in > > > > it. > > > > The > > > > default behaviour for meson is exactly how you described it, > > > > except > > > > that > > > > previously we needed more exact control over the version info > > > > (for > > > > your > > > > dpdk-specific versions in the sonames) and so overrode the > > > > soversion > > > > explicitly. The fix for meson is to remove this overriding i.e. > > > > remove > > > > "soversion:" parameter for each shared_library() call. > > > >=20 > > > > =20 > > > > > Also, we should leave the current at 20.0 - let's not break > > > > > compatibility already, please :-) > > > > >=20 > > > >=20 > > > > If we do this, maybe we can use 20.0.1 and 20.0.2 version > > > > numbers? > > >=20 > > > Yes, that's what I meant - IMHO we should just take the hit and > > > use the > > > slightly weird 20.0 format until next year, and add a third digit > > > for > > > compatible updates. Then for v21 we can drop it. > > >=20 > >=20 > > My concern with that is us forgetting, because we'll put in place > > hacks to > > have the soversion be the first two numbers of the version. Then we > > need to > > remember to remove those hacks before 20.11 goes out or we'll end > > up with > > 21.0 being the soversion again. > >=20 > > For that reason, I'd rather see us fix it now before 19.11 gets > > widely > > adopted. >=20 > Me too I vote for fixing soname as 20 in a small 19.11.1 > and release it quickly. >=20 > We are supposed to experience tooling and scheme adoption during this > first year of new ABI policy. I think we should stick to the planned > versionning, in order to avoid any surprise for the next major ABI. If the concern is forgetting, I'll happily set a reminder in my calendar and then nag everybody :-) In my opinion declaring stability and compatibility, with press releases and articles, and then immediately breaking it in the first point release does not send the right message. Not because there was a bug, which happens all the time, but because mantaining backward compatibility sometimes means having to carry less-than-ideal or downright ugly hacks for a while. If we are not ready to maintain compatibility by carrying an ugly workaround when the implications are minor or non-existant (I'm pretty sure few would really cares what format the soname has, it's all handled by tools), what happens next time when the implications are that a broken feature stays broken, or that lower performance stays lower, and so on? --=20 Kind regards, Luca Boccassi