From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id C91D49E7 for ; Tue, 14 Feb 2017 17:19:30 +0100 (CET) Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Feb 2017 08:19:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,161,1484035200"; d="scan'208";a="64619962" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.61]) by orsmga005.jf.intel.com with SMTP; 14 Feb 2017 08:19:26 -0800 Received: by (sSMTP sendmail emulation); Tue, 14 Feb 2017 16:19:25 +0000 Date: Tue, 14 Feb 2017 16:19:25 +0000 From: Bruce Richardson To: Christian Ehrhardt Cc: dev , "cjcollier@linuxfoundation.org" , ricardo.salveti@linaro.org, Luca Boccassi Message-ID: <20170214161925.GA41772@bricha3-MOBL3.ger.corp.intel.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.7.1 (2016-10-04) Subject: Re: [dpdk-dev] Further fun with ABI tracking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2017 16:19:31 -0000 On Tue, Feb 14, 2017 at 11:52:00AM +0100, Christian Ehrhardt wrote: > Hi, > when moving to DPDK 16.11 Debian/Ubuntu packaging of DPDK has hit a new > twist on the (it seems reoccurring) topic of DPDK ABI tracking. > > I have found, ... well I don't want to call it solution ..., let's say a > crutch to get around it for the moment. But I wanted to use the example I > had to share a few thoughts on it and to kick off a wider discussion. > > > *## In library cross-dependencies plus partial ABI bumps ##* > > Since the day moving away from the combined shared library we had several > improvements on tracking the ABI versions. These days [1] we have LIBABIVER > per library and it gets bumped to reflect it is breaking with former > versions e.g. removing symbols. > > Now in the 16.11 release the ABIs for cryptodev, eal and ethdev got bumped > by [2] and [3]. > > OTOH please remember that in general two versions of a shared library in > the usual sense are meant to be able to stay alongside on a system without > hurting each other. I picked a random one on my system. > Package Library > libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160 > libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160.0.0 > libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95 > libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95.5.0 > Some link against the new, some against the old library - all fine. > Usually most programs can just be rebuilt against the new library and after > some time the old one can be dropped. That mechanism gives downstream > distributions a way to handle transitions and consumers of libraries which > might not all be ready for the same version every time. > And since the per lib versioning with LIBABIVER and and the version maps we > are good - in fact we qualify for all common cases on [4]. > > Now in DPDK of those libraries that got an ABI bump eal and ethdev are part > of those which most of us consider "core libraries" and most other libs and > pmds link to them. > And here DPDK continues to be special, due to that inter-dependency with > old and new libraries installed on the same system the following happens on > openvswitch built for an older version of dpdk: > ovs-vswitchd-dpdk > librte_eal.so.2 => /usr/lib/x86_64-linux-gnu/librte_eal.so.2 > librte_pdump.so.1 => /usr/lib/x86_64-linux-gnu/librte_pdump.so.1 > librte_eal.so.3 => /usr/lib/x86_64-linux-gnu/librte_eal.so.3 > > You can see that Openvswitch itself depends on the "old" librte_eal.so.2. > But because librte_pdump.so.1 did not get an ABI bump it got upgraded to > the newer version from DPDK 16.11. > But since the "new" pdump got built with the new DPDK 16.11 it depends on > the "new" librte_eal.so.3. > And having both in the same executable space at the same time causes > segfaults and pain. > > As I said for now I have passed the issue with a crutch that I'm not proud > of and I'd like to avoid in the future. For that I'm reaching out to you > with several suggestions to discuss. > > > *## Thoughts ##* > None of these seems like a perfect solution to me yet, but clearly good to > start discussions on them. > > Options that were in discussion so far and that we might adopt next cycle > (some of these are upstream changes, some downstream, some require both to > change - but any of them should have an ack upstream so that we are > agreeing how to proceed with those cases). > > 1. Downstreams to insert Major version into soname > Distributions could insert the DPDK major version (like 16.11) into the > soname and package names. A common example of this is libboost [5]. > That would perfectly allow 16.07. to coexist with > 16.11. even if for a given library LIBABIVER did not change. > Yet it would mean that anything depending on the old library will have to > be recompiled to pick up the new code, even if it depends on an ABI that is > still present in the new release. > Also - not a technical reason - but it is clearly more work to force update > all dependencies and clean out old packages for every release. > > > 2. ABI Ranges > One could argue that due to the detailed tracking of functions DPDK is > already close to track not ABI levels but actually ABI ranges. DPDK could > track LIBABIVERMIN and LIBABIVER. > Every time functionality is added LIBABIVER would get bumped, but > LIBABIVERMIN only gets moved to the OLDEST still supported ABI when things > are dropped. > So on a given library librte_foo you could have LIBABIVER=5 and > LIBABIVERMIN=3. The make install would then install the shared lib as: > librte_foo.so.5 > and additionally links for all compatible versions: > librte_foo.so.3 -> librte_foo.so.5 > librte_foo.so.4 -> librte_foo.so.5 > Yet, while is has some nice attributes this might make DPDK even more > special and cause ABI level proliferation over time. > Also even with this in place, changes moving LIBABIVERMIN "too fast" (too > fast is different for each downstream) could still cause an issue like the > one I initially described. > > > 3. A lot of conflicts > In packaging one can declare a package to conflict with another package [6]. > Now we could declare e.g. librte_eal3 to conflict with librte_eal2 (and the > same for all other bumps). > That would make them not coinstallable, and working on a new release would > mean that all former consumers would become not installable as well and > have to be rebuilt before they all could migrate [7] together. > That "works" in some sense, but it denies the whole purpose of versioned > library packages (to be coninstallable, to allow different library > consumers to depent on different versions) > > > 4. ABI bump is infecting > Another way might be to also bump any dependent DPDK library. > So when core libs like eal are ABI bumped likely all libs would get a bump. > If only e.g. mempool gets a bump only those other parts using it would be > bumped as well. > To some extend this might still proliferate ABI versions more than one > would like. > Also it surely is hard to track if not automated - think of dependencies > that are existing only in certain config cases. > > 5. back to single ABI > For the sake of giving everybody a chance to re-open old wounds I wanted to > mention that DPDK could also decide to go back to a single ABI again. > This could (but doesn't have to!) be combined with having a single .so file > again. > To decide for this might be a much cleaner and easier to track way to #4. > > 6. More > I'm sure there are more approaches to this, feel free to come up with more. > > I'm sure my five suggestions alone will make the thread messy, Maybe we do > this in two rounds, sorting out the insane and identifying the preferred > ones to then in a second run focus on discussing and maybe implementing the > details of what we like. > > Of the 5 options you propose, No 4 looks most appealing to me. If it does cause problems with different config cases, then that looks a good reason to cut down on the allowed configs. :-) /Bruce > [1]: http://dpdk.org/browse/dpdk/tree/doc/guides/contributing/versioning.rst > [2]: http://dpdk.org/browse/dpdk/commit/?id=d7e61ad3ae36 > [3]: http://dpdk.org/browse/dpdk/commit/?id=6ba1affa54108 > [4]: https://wiki.debian.org/TransitionBestPractices > [5]: https://packages.debian.org/sid/libboost1.62-dev > [6]: > https://www.debian.org/doc/debian-policy/ch-relationships.html#s-conflicts > [7]: https://wiki.ubuntu.com/ProposedMigration > > P.S. I beg a pardon for the wall of text > > -- > Christian Ehrhardt > Software Engineer, Ubuntu Server > Canonical Ltd