From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <sergio.gonzalez.monroy@intel.com> Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 9AE56B46E for <dev@dpdk.org>; Fri, 13 Feb 2015 12:08:30 +0100 (CET) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP; 13 Feb 2015 03:08:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,570,1418112000"; d="scan'208";a="527078752" Received: from smonroyx-mobl.ger.corp.intel.com (HELO [10.237.221.33]) ([10.237.221.33]) by orsmga003.jf.intel.com with ESMTP; 13 Feb 2015 02:59:47 -0800 Message-ID: <54DDDB12.3090100@intel.com> Date: Fri, 13 Feb 2015 11:08:02 +0000 From: "Gonzalez Monroy, Sergio" <sergio.gonzalez.monroy@intel.com> User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Panu Matilainen <pmatilai@redhat.com> References: <20150129194539.GG1999@hmsreliant.think-freely.org> <91383E96CE459D47BCE92EFBF5CE73B004F453D7@IRSMSX108.ger.corp.intel.com> <20150130140507.GA2664@hmsreliant.think-freely.org> <91383E96CE459D47BCE92EFBF5CE73B004F45534@IRSMSX108.ger.corp.intel.com> <20150130181249.GC2664@hmsreliant.think-freely.org> <91383E96CE459D47BCE92EFBF5CE73B004F4AB9B@IRSMSX108.ger.corp.intel.com> <54DC70F3.4020902@redhat.com> <54DC7A87.1090208@intel.com> <20150212122354.GB8729@neilslaptop.think-freely.org> <54DCB3B6.1010204@redhat.com> <20150212155225.GB4634@neilslaptop.think-freely.org> <54DDCE68.7090400@redhat.com> In-Reply-To: <54DDCE68.7090400@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" <dev@dpdk.org> Subject: Re: [dpdk-dev] [PATCH 0/8] Improve build process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Fri, 13 Feb 2015 11:08:31 -0000 On 13/02/2015 10:14, Panu Matilainen wrote: > On 02/12/2015 05:52 PM, Neil Horman wrote: >> On Thu, Feb 12, 2015 at 04:07:50PM +0200, Panu Matilainen wrote: >>> On 02/12/2015 02:23 PM, Neil Horman wrote: > [...snip...] >>>>>>>> >>>>>>> So I just realized that I was not having into account a possible >>>>>>> scenario, where >>>>>>> we have an app built with static dpdk libs then loading a dso >>>>>>> with -d >>>>>>> option. >>>>>>> >>>>>>> In such case, because the pmd would have DT_NEEDED entries, >>>>>>> dlopen will >>>>>>> fail. >>>>>>> So to enable such scenario we would need to build PMDs without >>>>>>> DT_NEEDED >>>>>>> entries. >>>>>> >>>>>> Hmm, for that to be a problem you'd need to have the PMD built >>>>>> against >>>>>> shared dpdk libs and while the application is built against >>>>>> static dpdk >>>>>> libs. I dont think that's a supportable scenario in any case. >>>>>> >>>>>> Or is there some other scenario that I'm not seeing? >>>>>> >>>>>> - Panu - >>>>>> >>>>> I agree with you. I suppose it comes down to, do we want to >>>>> support such >>>>> scenario? >>>>> >>>>> From what I can see, it seems that we do currently support such >>>>> scenario by >>>>> building dpdk apps against all static dpdk libs using >>>>> --whole-archive (all >>>>> libs and not only PMDs). >>>>> http://dpdk.org/browse/dpdk/commit/?id=20afd76a504155e947c770783ef5023e87136ad8 >>>>> >>>>> >>>>> Am I misunderstanding this? >>>>> >>>> Shoot, you're right, I missed the static build aspect to this. >>>> Yes, if we do the following: >>>> >>>> 1) Build the DPDK as a static library >>>> 2) Link an application against (1) >>>> 3) Use the dlopen mechanism to load a PMD built as a DSO >>>> >>>> Then the DT_NEEDED entries in the DSO will go unsatisfied, because >>>> the shared >>>> objects on which it (the PMD) depends will not exist in the file >>>> system. >>> >>> I think its even more twisty: >>> >>> 1) Build the DPDK as a static library >>> 2) Link an application against (1) >>> 3) Do another build of DPDK as a shared library >>> 4) In app 2), use the dlopen mechanism to load a PMD built as a part >>> of or >>> against 3) >>> >>> Somehow I doubt this would work very well. >>> >> Ideally it should, presuming the ABI is preserved between (1) and >> (3), though I >> agree, up until recently, that was an assumption that was unreliable. > > Versioning is a big and important step towards reliability but there > are more issues to solve. This of course getting pretty far from the > original topic, but at least one such issue is that there are some > cases where a config value affects what are apparently public structs > (rte_mbuf wrt RTE_MBUF_REFCNT for example), which really is a no-go. > Agree, the RTE_MBUF_REFCNT is something that needs to be dealt with asap. I'll look into it. >>>> >>>> I think the problem is a little bit orthogonal to the libdpdk_core >>>> problem you >>>> were initially addressing. That is to say, this problem of >>>> dlopen-ed PMD's >>>> exists regardless of weather you build the DPDK as part of a static >>>> or dynamic >>>> library. The problems just happen to intersect in their >>>> manipulation of the >>>> DT_NEEDED entries. >>>> >>>> Ok, so, given the above, I would say your approach is likely >>>> correct, just >>>> prevent DT_NEEDED entries from getting added to PMD's. Doing so >>>> will sidestep >>>> loading issue for libraries that may not exist in the filesystem, >>>> but thats ok, >>>> because by all rights, the symbols codified in those needed >>>> libraries should >>>> already be present in the running application (either made >>>> available by the >>>> application having statically linked them, or having the linker >>>> load them from >>>> the proper libraries at run time). >>> >>> My 5c is that I'd much rather see the common case (all static or all >>> shared) >>> be simple and reliable, which in case of DSOs includes no lying >>> (whether by >>> omission or otherwise) about DT_NEEDED, ever. That way the issue is >>> dealt >>> once where it belongs. If somebody wants to go down the rabbit hole >>> of mixed >>> shared + static linkage, let them dig the hole by themselves :) >>> >> This is a fair point. Can DT_NEEDED sections be stripped via tools >> like objcopy >> after the build is complete? If so, end users can hack this corner >> case to work >> as needed. > > Patchelf (http://nixos.org/patchelf.html) appears to support that, but > given that source is available it'd be easier to just modify the > makefiles if that's really needed. > I think we agree on the issue. So I'll be sending a patch to add DT_NEEDED entries to all libraries and PMDs. The only exception would be librte_eal, which would not have proper NEEDED entries. Do we bother adding a linker script for librte_eal that would include dependent libraries? Sergio