Re: [dpdk-dev] [PATCH 0/8] Improve build process

[Panu Matilainen <pmatilai@redhat.com>]

On 02/12/2015 02:23 PM, Neil Horman wrote:
> On Thu, Feb 12, 2015 at 10:03:51AM +0000, Gonzalez Monroy, Sergio wrote:
>> On 12/02/2015 09:22, Panu Matilainen wrote:
>>> On 02/11/2015 01:11 PM, Gonzalez Monroy, Sergio wrote:
>>>>> From: Neil Horman [mailto:nhorman@tuxdriver.com]
>>>>> Sent: Friday, January 30, 2015 6:13 PM
>>>>> To: Gonzalez Monroy, Sergio
>>>>> Cc: Thomas Monjalon; dev@dpdk.org
>>>>> Subject: Re: [dpdk-dev] [PATCH 0/8] Improve build process
>>>>>
>>>>> On Fri, Jan 30, 2015 at 05:38:49PM +0000, Gonzalez Monroy, Sergio
>>>>> wrote:
>>>>
>>>> [snip]
>>>>
>>>>>>
>>>>>> So would it be reasonable to add DT_NEEDED entries to all DPDK
>>>>>> libraries
>>>>> but EAL?
>>>>>> If I understood what you were saying right, we could enforce the
>>>>>> 'dependency' in the linker script with something like this:
>>>>>> $ cat  librte_eal.so
>>>>>> INPUT( librte_eal.so.1 -lrte_mempool -lrte_malloc) We could have such
>>>>>> linker script for librte_eal.so instead of the soft link once
>>>>>> versioning is in place.
>>>>>>
>>>>> Correct.
>>>>>
>>>>>> Things that would be missing versus the proposed patch:
>>>>>>   - As I have mention in previous post, ldd info for EAL library
>>>>>> would not
>>>>> reflect
>>>>>>     its dependency to other DPDK libs.
>>>>> librte_eal.so would no show those dependencies, as far as I know
>>>>> (though I
>>>>> haven't explicitly checked).  The subordunate libraries included in
>>>>> the input
>>>>> line, may or may not show dependencies among themselves, depending on
>>>>> your build setup (and the use of --no-as-needed and -l when linking the
>>>>> individual .so libraries.
>>>>>
>>>>>>   - I was enforcing resolving all references when building the
>>>>>> libraries (-z
>>>>> defs), so
>>>>>>     we either remove it altogether or skip eal.
>>>>> I think thats correct, yes.
>>>>>
>>>>>>   - All apps would show DT_NEEDED entries for a set of DPDK
>>>>>> libraries that
>>>>>>     in most cases are required (eal, mempool, malloc, mbuf, ring VS
>>>>>> dpdk_core)
>>>>>>
>>>>> I think apps linked to libdpdk_core would have DT_NEEDED entries for
>>>>> libdpdk_core, not the subordonate libraries (though check me on that
>>>>> to be
>>>>> sure).
>>>>>
>>>> Just checked on this and they do link against the subordinate libraries,
>>>> although
>>>> It does not really matter as we are dropping the 'core' library approach
>>>> anyway.
>>>>
>>>>>> I think that the linker script approach is reasonable if we prefer to
>>>>>> go that way instead of creating a core library.
>>>>>>
>>>>> I think it would make sense from a build environment point of view, in
>>>>> that it
>>>>> allows library specific flags to be incorporated properly.  I think
>>>>> the only
>>>>> downside is that the individual libraries still need to be carried
>>>>> around
>>>>> (though they can be ignored from an application build/run standpoint).
>>>>> You're question should probably be asked of people using COMBINED_LIBS
>>>>> currently to make sure that meets their needs, though I think it will.
>>>>>
>>>>> Neil
>>>>>
>>>> So I just realized that I was not having into account a possible
>>>> scenario, where
>>>> we have an app built with static dpdk libs then loading a dso with -d
>>>> option.
>>>>
>>>> In such case, because the pmd would have DT_NEEDED entries, dlopen will
>>>> fail.
>>>> So to enable such scenario we would need to build PMDs without DT_NEEDED
>>>> entries.
>>>
>>> Hmm, for that to be a problem you'd need to have the PMD built against
>>> shared dpdk libs and while the application is built against static dpdk
>>> libs. I dont think that's a supportable scenario in any case.
>>>
>>> Or is there some other scenario that I'm not seeing?
>>>
>>>     - Panu -
>>>
>> I agree with you. I suppose it comes down to, do we want to support such
>> scenario?
>>
>>  From what I can see, it seems that we do currently support such scenario by
>> building dpdk apps against all static dpdk libs using --whole-archive (all
>> libs and not only PMDs).
>> http://dpdk.org/browse/dpdk/commit/?id=20afd76a504155e947c770783ef5023e87136ad8
>>
>> Am I misunderstanding this?
>>
> Shoot, you're right, I missed the static build aspect to this.  Yes, if we do the following:
>
> 1) Build the DPDK as a static library
> 2) Link an application against (1)
> 3) Use the dlopen mechanism to load a PMD built as a DSO
>
> Then the DT_NEEDED entries in the DSO will go unsatisfied, because the shared
> objects on which it (the PMD) depends will not exist in the file system.

I think its even more twisty:

1) Build the DPDK as a static library
2) Link an application against (1)
3) Do another build of DPDK as a shared library
4) In app 2), use the dlopen mechanism to load a PMD built as a part of 
or against 3)

Somehow I doubt this would work very well.

>
> I think the problem is a little bit orthogonal to the libdpdk_core problem you
> were initially addressing.  That is to say, this problem of dlopen-ed PMD's
> exists regardless of weather you build the DPDK as part of a static or dynamic
> library.  The problems just happen to intersect in their manipulation of the
> DT_NEEDED entries.
>
> Ok, so, given the above, I would say your approach is likely correct, just
> prevent DT_NEEDED entries from getting added to PMD's.  Doing so will sidestep
> loading issue for libraries that may not exist in the filesystem, but thats ok,
> because by all rights, the symbols codified in those needed libraries should
> already be present in the running application (either made available by the
> application having statically linked them, or having the linker load them from
> the proper libraries at run time).

My 5c is that I'd much rather see the common case (all static or all 
shared) be simple and reliable, which in case of DSOs includes no lying 
(whether by omission or otherwise) about DT_NEEDED, ever. That way the 
issue is dealt once where it belongs. If somebody wants to go down the 
rabbit hole of mixed shared + static linkage, let them dig the hole by 
themselves :)

	- Panu -

> Regards
> Neil
>
>> Regards,
>> Sergio
>>