From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id 08537590C for ; Wed, 9 Dec 2015 15:16:21 +0100 (CET) Received: from hmsreliant.think-freely.org ([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1a6fXK-0001gA-Dz; Wed, 09 Dec 2015 09:16:12 -0500 Date: Wed, 9 Dec 2015 09:16:00 -0500 From: Neil Horman To: Robie Basak Message-ID: <20151209141600.GA28945@hmsreliant.think-freely.org> References: <20151203145924.GA29392@hmsreliant.think-freely.org> <20151202114419.GB22216@hmsreliant.think-freely.org> <20151201133043.GB26189@hmsreliant.think-freely.org> <20151208170326.GN28475@mal.justgohome.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151208170326.GN28475@mal.justgohome.co.uk> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Score: -1.0 (-) X-Spam-Status: No Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH] mk: fix the combined library problems by replacing it with a linker script X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2015 14:16:21 -0000 On Tue, Dec 08, 2015 at 05:03:26PM +0000, Robie Basak wrote: > On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote: > > Theres nothing "complex" about the simple fact that a project builds lots of > > libraries. Its extreemely common. Any graphic window manager has exactly the > > same situation, as do any number of tools that have multiple hardware backends > > impelmented in user space (v4l, sane, iptables, to name just a few). > > > > > Before I go into details, it would be nice if someone could please > > > explain why DPDK has to be "special" in needing to do this? I don't > > Its not special, see above. Not saying the build environment cant be improved, > > but the fact that there are multiple libraries is pretty straightforward. > > It's fine in principle for an upstream to ship multiple shared > libraries, but it is extra and unnecessary work unless there's a > *reason* to have multiple shared libraries. What are the reasons for > DPDK? > Separation of functionality. There is no need to build a library that supports 10 different NICS when a given application is only using one. Likewise with other discrete functionality, e.g. you don't link against the ACL library if you dont' want to use ACL's. > > > In Debian and Ubuntu, we manage a library transition (an ABI bump in a > > > library together with all dependencies moving to use the new ABI) by > > > concurrently packaging both the old and new libraries at once. This > > > works well with the norm for libraries. We ship one binary package per > > > soname, with the major version as part of the package name. This allows > > > a system to have two (or more) ABIs installed simultaneously. For a > > > library transition, we just package the new version and then that can > > > land and work concurrently as we then individually update every > > > dependent (library-consuming) package. > > > So thats, a distribution choice, not an upstream problem. > > No, that's how shared libraries work. By design, multiple ABI versions > can be co-installed. That's why sonames have the ABI major version > inside them and the filenames reflect the sonames. > We're talking about different things. The DPDK support ABI versioning in their sonames, if you would look at the makefiles and output, you would clearly see that. Theres no reason that multiple versions of DPDK can't be co-installed, thats easy, the question here is weather it should support multiple discrete libraries or only a single large library. It seems from your comments that a single monolithic library should be the only option, and thats clearly the less flexible one. > It is a distribution choice to exploit this capability. But it is an > upstream problem if this capability is broken. > Its not broken. In what way do you think soname versioning is broken in DPDK? And in what way is it broken that the only solution is to merge all the libraries into a single monolithic one? > By shipping multiple shared libraries, DPDK isn't breaking this > capability per se. But if the upstream expectation is that it's no > additional work for distributions because the multiple libraries can > just be bundled together into a single distribution package, then _this_ > is what breaks the capability. > > Instead DPDK needs to acknowledge that splitting libraries _does_ cause > additional packaging work for any distribution that wants to use the > multiple co-installed ABI feature of shared libraries as they are > designed. > Very well, allowing multiple separate libraries causes some extra work for packaging. Specifically it requires that distributions carry a patch that instantiate a specific library ABI major version number that is incremented ni lock step for every library in a given build (which is admittedly not what upstream currently does). I don't see that as overly hard to do, but to each their own. However, the solution there is to propose a patch that defines a single ABI variable in the makefile structure that is applied to every library on a symbol version bump. The answer is _not_ to somehow entangle that with the idea that we have to have a single monolithic library. Their separate issues, and you can solve the problem that you are complaining about without throwing the proverbial baby out with the bathwater. > Then, it becomes for upstream a question of the trade-off: does the > benefit of split libraries outweigh the extra work this creates on > packagers? To understand this, first I need to understand the rationale > for shipping multiple shared libaries specifically in DPDK, and I feel > that you (well, Red Hat) have yet to present a case. > Some of the reasons I've mentioned above. Regardless however, the solution your advocating to the problem you describe is orthogonal to the complaints you're making there. If your goal is to allow multiple ABI versions in a given package, then propose that ABI versions be incremented monolithically in the upstream build. Even if a given library hasn't changed, it doesn't hurt to bump the version number - that is to say, as a distribution, if an application links against library A version 2, it will also link against library B version 2 (assuming it uses library B), even if library B has no ABI changes in it (thats simply an artifact of how packaging works, you dont' typically mix and match libraries from separate builds). So...just bump the soname version number, and while you're at it, make it a global build setting, not a per library build setting. Then you can use it to append a version number to the combined library (script) as well What you shouldn't do is assume that each library _has_ to have an independent ABI version number, and use that to bootstrap an argument that the only solution is a single combined library. > > And it seems like a > > problem you should have already solved (note the examples above). If you feel > > like you need to package multiple ABI versions in the same library, you can, > > just update the LIBABIVER of all the libraries, instead of the ones that truly > > change, so that each library is guaranteed a newer so version, to make the > > library file name unique. Yes you have to make a small change from upstream, > > but thats part of the work that distribution maintainers do. > Yes, this makes good sense. > If it makes sense for upstream, it would be better for all if the code > was maintained in once place rather than fragmented across distribution > patches. My argument here is that _does_ make sense for upstream, which > is why I took the question to this list before we uploaded our first > patched version to Ubuntu. > yes, thats fine. So, it seems like perhaps we're talking about the same solution here. If we have a single LIBABIVER variable that is applied to each DSO, why do we then further need to only allow a single combined library? Neil