From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw0-f177.google.com (mail-yw0-f177.google.com [209.85.161.177]) by dpdk.org (Postfix) with ESMTP id 567BE2A5E for ; Tue, 14 Feb 2017 11:52:31 +0100 (CET) Received: by mail-yw0-f177.google.com with SMTP id v200so63891309ywc.3 for ; Tue, 14 Feb 2017 02:52:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc; bh=K8GJley5ddAhfQFcD3h2vkZMVtuicmJvZEtLuOdehOM=; b=q4MYm+s8sTnmZ+mif0nUf0OiaZS2Fa4hhQmQ8s6ZfCPCma/npykZDQAsz/mGq0k49w 4gd0enpX9vHBTFh5Hkvp7gcAKLW1GQXoWd/0HJNPSWA2heWutAL1bfpcGulgHe+yX9/O W5my9juO8JPNm/Zipg1+94X6iWF6jfoLPtPAfpp6ZjQZ9XzuymmtBsTI4H8p/vzQ+qT7 bhous9OE25pXBOfT6R1kGXG4OFWozAyFNA2Gl9w+9DW5WCaQ2vXEsIr59g5QfU1EkatG cPmTge24ALLQkWUmBocm53SS0BZSw2PJT5nKs4KEx7Xuta5WsHDvHScypsLtwmhUrS6z 5BaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=K8GJley5ddAhfQFcD3h2vkZMVtuicmJvZEtLuOdehOM=; b=B356mQfBClKwKMs9Rf6xeuvGK24A+qj6fwf4W1nm7VuUTyKGEA3Cni7AO13jeiWKM6 36LNbhiEWzwhMgvXPk2vDqwss3cp5p7rTBw5CF70kUzWLWgDdp0OlvhMB8CxFcvs2osV +foaKBMEdUg/hqBBK5dDUCikWSYASGaADOvA7+Kv5yT3ruMvrAA2uS1k16jp2fF2z6iZ UsjuptrTfDbYKjOL8TjhEFAceQtYwbMrSG0Pc3wd09zNm1LMnDnhX8lXKmxZR5mftYoP TIbRh4EIKQ/4pvQg/3v9G1n3yFDeWuigLejc6wb3HxioUCNsFw4l/hOolCZI1UOnehQz nIlw== X-Gm-Message-State: AMke39mo3c0Gj0Fa+PTiyiVnZgGrnoEfXo6k+hdiv81pQvzQ3D2nyp6W2wtKO+ojKOWHHH49cmipPxSaf8RQJz4S X-Received: by 10.129.108.215 with SMTP id h206mr19868287ywc.164.1487069550559; Tue, 14 Feb 2017 02:52:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.59.11 with HTTP; Tue, 14 Feb 2017 02:52:00 -0800 (PST) From: Christian Ehrhardt Date: Tue, 14 Feb 2017 11:52:00 +0100 Message-ID: To: dev Cc: "cjcollier@linuxfoundation.org" , ricardo.salveti@linaro.org, Luca Boccassi Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Further fun with ABI tracking X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2017 10:52:31 -0000 Hi, when moving to DPDK 16.11 Debian/Ubuntu packaging of DPDK has hit a new twist on the (it seems reoccurring) topic of DPDK ABI tracking. I have found, ... well I don't want to call it solution ..., let's say a crutch to get around it for the moment. But I wanted to use the example I had to share a few thoughts on it and to kick off a wider discussion. *## In library cross-dependencies plus partial ABI bumps ##* Since the day moving away from the combined shared library we had several improvements on tracking the ABI versions. These days [1] we have LIBABIVER per library and it gets bumped to reflect it is breaking with former versions e.g. removing symbols. Now in the 16.11 release the ABIs for cryptodev, eal and ethdev got bumped by [2] and [3]. OTOH please remember that in general two versions of a shared library in the usual sense are meant to be able to stay alongside on a system without hurting each other. I picked a random one on my system. Package Library libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160 libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160.0.0 libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95 libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95.5.0 Some link against the new, some against the old library - all fine. Usually most programs can just be rebuilt against the new library and after some time the old one can be dropped. That mechanism gives downstream distributions a way to handle transitions and consumers of libraries which might not all be ready for the same version every time. And since the per lib versioning with LIBABIVER and and the version maps we are good - in fact we qualify for all common cases on [4]. Now in DPDK of those libraries that got an ABI bump eal and ethdev are part of those which most of us consider "core libraries" and most other libs and pmds link to them. And here DPDK continues to be special, due to that inter-dependency with old and new libraries installed on the same system the following happens on openvswitch built for an older version of dpdk: ovs-vswitchd-dpdk librte_eal.so.2 => /usr/lib/x86_64-linux-gnu/librte_eal.so.2 librte_pdump.so.1 => /usr/lib/x86_64-linux-gnu/librte_pdump.so.1 librte_eal.so.3 => /usr/lib/x86_64-linux-gnu/librte_eal.so.3 You can see that Openvswitch itself depends on the "old" librte_eal.so.2. But because librte_pdump.so.1 did not get an ABI bump it got upgraded to the newer version from DPDK 16.11. But since the "new" pdump got built with the new DPDK 16.11 it depends on the "new" librte_eal.so.3. And having both in the same executable space at the same time causes segfaults and pain. As I said for now I have passed the issue with a crutch that I'm not proud of and I'd like to avoid in the future. For that I'm reaching out to you with several suggestions to discuss. *## Thoughts ##* None of these seems like a perfect solution to me yet, but clearly good to start discussions on them. Options that were in discussion so far and that we might adopt next cycle (some of these are upstream changes, some downstream, some require both to change - but any of them should have an ack upstream so that we are agreeing how to proceed with those cases). 1. Downstreams to insert Major version into soname Distributions could insert the DPDK major version (like 16.11) into the soname and package names. A common example of this is libboost [5]. That would perfectly allow 16.07. to coexist with 16.11. even if for a given library LIBABIVER did not change. Yet it would mean that anything depending on the old library will have to be recompiled to pick up the new code, even if it depends on an ABI that is still present in the new release. Also - not a technical reason - but it is clearly more work to force update all dependencies and clean out old packages for every release. 2. ABI Ranges One could argue that due to the detailed tracking of functions DPDK is already close to track not ABI levels but actually ABI ranges. DPDK could track LIBABIVERMIN and LIBABIVER. Every time functionality is added LIBABIVER would get bumped, but LIBABIVERMIN only gets moved to the OLDEST still supported ABI when things are dropped. So on a given library librte_foo you could have LIBABIVER=5 and LIBABIVERMIN=3. The make install would then install the shared lib as: librte_foo.so.5 and additionally links for all compatible versions: librte_foo.so.3 -> librte_foo.so.5 librte_foo.so.4 -> librte_foo.so.5 Yet, while is has some nice attributes this might make DPDK even more special and cause ABI level proliferation over time. Also even with this in place, changes moving LIBABIVERMIN "too fast" (too fast is different for each downstream) could still cause an issue like the one I initially described. 3. A lot of conflicts In packaging one can declare a package to conflict with another package [6]. Now we could declare e.g. librte_eal3 to conflict with librte_eal2 (and the same for all other bumps). That would make them not coinstallable, and working on a new release would mean that all former consumers would become not installable as well and have to be rebuilt before they all could migrate [7] together. That "works" in some sense, but it denies the whole purpose of versioned library packages (to be coninstallable, to allow different library consumers to depent on different versions) 4. ABI bump is infecting Another way might be to also bump any dependent DPDK library. So when core libs like eal are ABI bumped likely all libs would get a bump. If only e.g. mempool gets a bump only those other parts using it would be bumped as well. To some extend this might still proliferate ABI versions more than one would like. Also it surely is hard to track if not automated - think of dependencies that are existing only in certain config cases. 5. back to single ABI For the sake of giving everybody a chance to re-open old wounds I wanted to mention that DPDK could also decide to go back to a single ABI again. This could (but doesn't have to!) be combined with having a single .so file again. To decide for this might be a much cleaner and easier to track way to #4. 6. More I'm sure there are more approaches to this, feel free to come up with more. I'm sure my five suggestions alone will make the thread messy, Maybe we do this in two rounds, sorting out the insane and identifying the preferred ones to then in a second run focus on discussing and maybe implementing the details of what we like. [1]: http://dpdk.org/browse/dpdk/tree/doc/guides/contributing/versioning.rst [2]: http://dpdk.org/browse/dpdk/commit/?id=d7e61ad3ae36 [3]: http://dpdk.org/browse/dpdk/commit/?id=6ba1affa54108 [4]: https://wiki.debian.org/TransitionBestPractices [5]: https://packages.debian.org/sid/libboost1.62-dev [6]: https://www.debian.org/doc/debian-policy/ch-relationships.html#s-conflicts [7]: https://wiki.ubuntu.com/ProposedMigration P.S. I beg a pardon for the wall of text -- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd