From: Christian Ehrhardt <christian.ehrhardt@canonical.com>
To: Panu Matilainen <pmatilai@redhat.com>
Cc: Thomas Monjalon <thomas.monjalon@6wind.com>, dev <dev@dpdk.org>
Subject: Re: [dpdk-dev] Underlinked libs and overlinked applications - an issue?
Date: Fri, 20 May 2016 14:41:53 +0200 [thread overview]
Message-ID: <CAATJJ0KBTA1KPdb-ZaSFGLP6UCKORLZ-O75euo3PhnbKiAG0DQ@mail.gmail.com> (raw)
In-Reply-To: <f3625f86-5a8b-ae1d-c3a4-6ecc8563b2ad@redhat.com>
On Fri, May 20, 2016 at 1:01 PM, Panu Matilainen <pmatilai@redhat.com>
wrote:
> On 05/19/2016 09:17 PM, Thomas Monjalon wrote:
>
>> 2016-05-19 17:38, Christian Ehrhardt:
>>
>>> Hi,
>>> I was working on the new 16.04 build system to adapt deb packaging to it.
>>> I remember somewhen back in the DPDK 2.2 and shared+combined library
>>> days I
>>> had some issues with over/underlinking - but it seems those are still
>>> existent or came back.
>>>
>>
>> I would say it has always been like that.
>> Thanks for raising the issue.
>>
>> After a build in almost default config (just disabled the kernel modules)
>>> and set RTE_MACHINE to default I find the following.
>>>
>>> #1
>>> The libraries are all only linked against external things - even clearly
>>> using internal structures:
>>> ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
>>> linux-vdso.so.1 => (0x00007fff7e7a5000)
>>> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
>>> /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)
>>>
>>
>> The DT_NEEDED entries are created only for external dependencies.
>> Probably we should create ones for internal dependencies based on the
>> variable DEPDIRS-y.
>>
>
> Yes, DT_NEEDED for internal dependencies are needed too, see eg recent
> commits c6417ce61f83 and 8180554d82b3. That was part of Sergios' build
> system improvement patch series last spring or so (but since then
> abandoned), I picked up the work and been doing it in smaller pieces. Phase
> 1 was the external dependencies, phase 2 is internal dependencies which I'm
> working on. Unfortunately health issues have stalled my work a lot recently
> :-/
Since I work on 16.04 as released currently I almost missed those, but I at
least remember seeing the first on the list.
I had almost the same change - adopting yours since it is accepted.
Maybe I can provide a few more baby steps on that path - or at least a few
more pieces for discussion and review.
I hope you get well soon Panu!
> #2
>>> The Application then seem to try to make up for that by realizing all
>>> that
>>> is missing.
>>> But looking at the app alone it seems overlinked by that - it is not
>>> using
>>> all of these on its own.
>>> ldd usr/bin/cmdline_test
>>> linux-vdso.so.1 => (0x00007ffeec9ea000)
>>> librte_distributor.so.1 => not found
>>> librte_reorder.so.1 => not found
>>> [...]
>>> librte_jobstats.so.1 => not found
>>> [...]
>>> And for example none of the librte_jobstats.so.1 symbols are used
>>> "directly" in there.
>>>
>>
>> Yes every libraries are put for every apps in rte.app.mk.
>> Probably that we could better use DEPDIRS-y for apps.
>>
>>
> Another trick from Sergios' original patch series is to utilize
> AS_NEEDED() in the linker script, so it ends up looking something like this
> for shared library config (static would remain as it is now):
>
> INPUT ( librte_eal librte_mempool librte_ring
> AS_NEEDED ( librte_acl librte_timer librte_vhost <...> )
> )
>
> ...and then use the linker script for linking the apps, which should
> simplify rte.app.mk considerably. Or so the theory goes. Anyway, the
> above requires DT_NEEDED for the internal libraries to be present first,
> but once all these pieces are in place, dynamically linked apps will only
> link to the libraries they actually need.
>
Interesting approach, I wasn't even working on DPDK back then - that could
really simplify the mk there.
I was experimenting with just dropping the "no-as-needed" in
mk/exec-env/linuxapp/rte.vars.mk, but that seems to shoot too far - haven't
had the time to look deeper.
Interested if your approach could help.
But the other part of it - getting the proper DT_NEEDED into the libs (I
also consider underlinking worse than overlinking) seems to be just missing
a bunch of -l - my last test build looked fine.
There is the underlinking issue of librte_eal left that Sergio mentioned
back in 6d25d90c, but it seems to be the next baby step to solve most else
of the underlinking.
I'll adapt it to git level and at least throw it out for discussion later
on.
next prev parent reply other threads:[~2016-05-20 12:42 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-19 15:38 Christian Ehrhardt
2016-05-19 18:17 ` Thomas Monjalon
2016-05-20 11:01 ` Panu Matilainen
2016-05-20 12:41 ` Christian Ehrhardt [this message]
2016-05-20 16:43 ` Christian Ehrhardt
2016-05-20 16:50 ` [dpdk-dev] [PATCH] mk: fix underlinking issues of most librte libraries Christian Ehrhardt
2016-05-20 17:08 ` Thomas Monjalon
2016-05-24 9:56 ` Panu Matilainen
2016-05-24 10:11 ` Christian Ehrhardt
2016-05-26 13:13 ` Ferruh Yigit
2016-05-27 5:59 ` Panu Matilainen
2016-05-27 16:15 ` Ferruh Yigit
2016-05-27 16:48 ` [dpdk-dev] [PATCH 1/2] mk: prevent overlinking in applications Ferruh Yigit
2016-05-27 16:48 ` [dpdk-dev] [PATCH 2/2] mk: reduce scope of whole-archive to pmd libraries Ferruh Yigit
2016-06-10 9:03 ` Thomas Monjalon
2016-06-10 9:57 ` Ferruh Yigit
2016-06-10 10:18 ` Thomas Monjalon
2016-06-10 11:06 ` Ferruh Yigit
2016-06-10 12:21 ` Thomas Monjalon
2016-06-10 13:35 ` Ferruh Yigit
2016-06-09 10:10 ` [dpdk-dev] [PATCH 1/2] mk: prevent overlinking in applications Thomas Monjalon
2016-06-09 13:05 ` Ferruh Yigit
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 0/6] reduce " Thomas Monjalon
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 1/6] mk: sort drivers in static application link list Thomas Monjalon
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 2/6] mk: fix driver dependencies order for static application Thomas Monjalon
2016-06-10 14:49 ` Ferruh Yigit
2016-06-10 14:54 ` Thomas Monjalon
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 3/6] mk: remove library grouping during application linking Thomas Monjalon
2016-06-10 14:54 ` Ferruh Yigit
2016-06-10 15:14 ` Thomas Monjalon
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 4/6] mk: prevent overlinking in applications Thomas Monjalon
2016-06-10 14:56 ` Ferruh Yigit
2016-06-10 15:15 ` Thomas Monjalon
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 5/6] mk: sort libraries in level order when linking Thomas Monjalon
2016-06-10 13:19 ` [dpdk-dev] [PATCH v2 6/6] mk: reduce scope of whole-archive static linking Thomas Monjalon
2016-06-10 15:03 ` Ferruh Yigit
2016-06-10 15:18 ` Thomas Monjalon
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 0/6] reduce overlinking in applications Ferruh Yigit
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 1/6] mk: sort drivers in static application link list Ferruh Yigit
2016-06-10 19:38 ` Wiles, Keith
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 2/6] mk: fix driver dependencies order for static application Ferruh Yigit
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 3/6] mk: sort libraries when linking, move pmd libs to higher level Ferruh Yigit
2016-06-13 9:29 ` Thomas Monjalon
2016-06-13 10:04 ` Ferruh Yigit
2016-06-13 10:21 ` Thomas Monjalon
2016-06-13 10:29 ` Ferruh Yigit
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 4/6] mk: sort libraries when linking, move external libs to lower level Ferruh Yigit
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 5/6] mk: remove library grouping during application linking Ferruh Yigit
2016-06-10 18:32 ` [dpdk-dev] [PATCH v3 6/6] mk: prevent overlinking in applications Ferruh Yigit
2016-06-11 6:34 ` Thomas Monjalon
2016-06-13 9:05 ` Ferruh Yigit
2016-06-13 9:24 ` Christian Ehrhardt
2016-06-13 9:48 ` [dpdk-dev] [PATCH] mk: fix missing DEPDIRS to avoid libarary underlinking Christian Ehrhardt
2016-06-13 10:08 ` Thomas Monjalon
2016-06-13 10:20 ` Christian Ehrhardt
2016-06-13 11:06 ` [dpdk-dev] [PATCH v2 0/2] mk: fix more library underlinking Christian Ehrhardt
2016-06-13 11:06 ` [dpdk-dev] [PATCH v2 1/2] mk: fix missing vhost dependency to pthread to avoid libarary underlinking Christian Ehrhardt
2016-06-13 11:06 ` [dpdk-dev] [PATCH v2 2/2] mk: fix missing DEPDIRS " Christian Ehrhardt
2016-06-13 14:16 ` [dpdk-dev] [PATCH v2 0/2] mk: fix more library underlinking Thomas Monjalon
2016-06-13 14:14 ` [dpdk-dev] [PATCH v2 0/6] reduce overlinking in applications Thomas Monjalon
2016-06-07 8:23 ` [dpdk-dev] [PATCH] mk: fix underlinking issues of most librte libraries Thomas Monjalon
2016-06-07 9:25 ` Panu Matilainen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAATJJ0KBTA1KPdb-ZaSFGLP6UCKORLZ-O75euo3PhnbKiAG0DQ@mail.gmail.com \
--to=christian.ehrhardt@canonical.com \
--cc=dev@dpdk.org \
--cc=pmatilai@redhat.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).