From: Ruifeng Wang <Ruifeng.Wang@arm.com>
To: David Marchand <david.marchand@redhat.com>,
"Laatz, Kevin" <kevin.laatz@intel.com>
Cc: Bruce Richardson <bruce.richardson@intel.com>,
Aaron Conole <aconole@redhat.com>,
Michael Santana <maicolgabriel@hotmail.com>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"Yigit, Ferruh" <ferruh.yigit@intel.com>,
Andrew Rybchenko <arybchenko@solarflare.com>, dev <dev@dpdk.org>,
Gavin Hu <Gavin.Hu@arm.com>,
Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
nd <nd@arm.com>, nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files
Date: Thu, 19 Dec 2019 03:14:52 +0000 [thread overview]
Message-ID: <AM0PR08MB39863F949BCD07BE2563F5FE9E520@AM0PR08MB3986.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <CAJFAV8y9+D-dMWmOgmWJ4wkQMvwbi-bpYyqAtE_n2hvLRbhE=Q@mail.gmail.com>
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, December 18, 2019 23:32
> To: Laatz, Kevin <kevin.laatz@intel.com>
> Cc: Ruifeng Wang <Ruifeng.Wang@arm.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Aaron Conole <aconole@redhat.com>;
> Michael Santana <maicolgabriel@hotmail.com>; thomas@monjalon.net; Yigit,
> Ferruh <ferruh.yigit@intel.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>; dev <dev@dpdk.org>; Gavin Hu
> <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared
> object files
>
> On Wed, Dec 18, 2019 at 2:43 PM Laatz, Kevin <kevin.laatz@intel.com> wrote:
> >
> > On 18/12/2019 08:23, David Marchand wrote:
> > > On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang <ruifeng.wang@arm.com>
> wrote:
> > >> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder.
> > >> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and
> > >> make use of these shared libraries.
> > >>
> > >> Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> > >> ---
> > >> devtools/test-null.sh | 2 +-
> > >> 1 file changed, 1 insertion(+), 1 deletion(-)
> > >>
> > >> diff --git a/devtools/test-null.sh b/devtools/test-null.sh index
> > >> f39af2c06..548de8113 100755
> > >> --- a/devtools/test-null.sh
> > >> +++ b/devtools/test-null.sh
> > >> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then
> > >> fi
> > >>
> > >> if ldd $testpmd | grep -q librte_ ; then
> > >> - export LD_LIBRARY_PATH=$build/lib:$LD_LIBRARY_PATH
> > >> + export
> > >> + LD_LIBRARY_PATH=$build/drivers:$build/lib:$LD_LIBRARY_PATH
> > >> libs='-d librte_mempool_ring.so -d librte_pmd_null.so'
> > >> else
> > >> libs=
> > >> --
> > >> 2.17.1
> > >>
> > > I'm surprised to see this.
> > > So far, the CI ran fine without it, so something is different in the
> > > environment.
> > >
> > > I can see that the RPATH entry disappeared from the testpmd binary.
> > > Xenial:
> > > # readelf -d build/app/dpdk-testpmd |grep RPATH
> > > 0x000000000000000f (RPATH) Library rpath:
> > > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
> > >
> > > (not sure what XXXX purpose is, but different topic)
> > >
> > > Bionic:
> > > # readelf -d build/app/dpdk-testpmd |grep RPATH
> >
> > It looks like RPATH just changed to be named RUNPATH in Bionic:
> >
> > # readelf -d build/app/dpdk-testpmd | grep R.*PATH
> > 0x000000000000001d (RUNPATH) Library runpath:
> > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX]
>
> Did some experiment with some test program and .so of mine.
> TL;DR, if I understand correctly, RPATH on the binary applies to all lookups,
> even in a subsequent .so code.
> But RUNPATH only applies to the current ELF, meaning that the dlopen() in
> my intermediate .so does not get it.
>
> dlopen() is called from librte_eal.so, and RUNPATH on testpmd is not enough.
>
Thanks for your experiment and analysis. Really happy to know more around the issue.
>
> Details:
> [dmarchan@dmarchan plop]$ cat main.c
> extern void loader(void);
>
> int main(int argc, char *argv[])
> {
> loader();
> return 0;
> }
> [dmarchan@dmarchan plop]$ cat loader/loader.c #include <dlfcn.h>
> #include <stdio.h>
>
> void loader(void)
> {
> if (dlopen("lib.so", RTLD_NOW) == NULL)
> fprintf(stderr, "%s\n", dlerror()); }
>
> [dmarchan@dmarchan plop]$ cat lib/lib.c
> #include <stdio.h>
>
> __attribute__((constructor))
> static void plop(void)
> {
> fprintf(stdout, "plop\n");
> }
>
>
> # no rpath/runpath
> [dmarchan@dmarchan plop]$ gcc -o lib/lib.so -Wall -Werror -shared -fPIC
> lib/lib.c [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -
> shared -fPIC loader/loader.c -ldl [dmarchan@dmarchan plop]$ gcc -o main -
> Wall -Werror main.c loader/loader.so [dmarchan@dmarchan plop]$ readelf -
> d main |grep R.*PATH [dmarchan@dmarchan plop]$ ./main
> lib.so: cannot open shared object file: No such file or directory
>
> # using rpath on final binary
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-rpath,loader:lib [dmarchan@dmarchan plop]$ readelf -
> d main |grep R.*PATH
> 0x000000000000000f (RPATH) Library rpath: [loader:lib]
> [dmarchan@dmarchan plop]$ ./main
> plop
>
> # using runpath on final binary
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-enable-new-dtag,-rpath,loader:lib
> [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
> 0x000000000000001d (RUNPATH) Library runpath: [loader:lib]
> [dmarchan@dmarchan plop]$ ./main
> lib.so: cannot open shared object file: No such file or directory
>
> # using runpath on loader
> [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -shared -
> fPIC loader/loader.c -ldl -Wl,-enable-new-dtag,-rpath,lib
> [dmarchan@dmarchan plop]$ readelf -d loader/loader.so |grep R.*PATH
> 0x000000000000001d (RUNPATH) Library runpath: [lib]
> [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c
> loader/loader.so -Wl,-enable-new-dtag,-rpath,loader
> [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH
> 0x000000000000001d (RUNPATH) Library runpath: [loader]
> [dmarchan@dmarchan plop]$ ./main
> plop
>
>
> > > Adding Bruce and Kevin, as I think this is the same issue than:
> > > http://mails.dpdk.org/archives/dev/2019-December/153627.html
> > > Could it be a change in meson?
> >
> > Yes, looks like the same error to me.
> >
> > I'm not sure this is solely a meson issue, I often need to pass -d
> > librte_mempool_ring.so when using make builds too. Maybe I'm missing
> > something here?
>
> In a make build directory, all libraries ends up in the same lib/ directory and
> the test-null.sh script work with the current LD_LIBRARY_PATH.
>
>
> Ruifeng patch seems valid to me, but I would love some explanations in the
> commitlog.
>
Sure. I will re-spin to add explanations.
> --
> David Marchand
next prev parent reply other threads:[~2019-12-19 3:15 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-18 5:39 [dpdk-dev] [PATCH 0/2] add travis ci support for aarch64 Ruifeng Wang
2019-12-18 5:39 ` [dpdk-dev] [PATCH 1/2] ci: " Ruifeng Wang
2019-12-18 5:39 ` [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files Ruifeng Wang
2019-12-18 8:23 ` David Marchand
2019-12-18 13:43 ` Laatz, Kevin
2019-12-18 15:32 ` David Marchand
2019-12-18 16:00 ` Richardson, Bruce
2019-12-19 3:14 ` Ruifeng Wang [this message]
2019-12-20 9:37 ` [dpdk-dev] [PATCH v2 0/2] add travis ci support for aarch64 Ruifeng Wang
2019-12-20 9:37 ` [dpdk-dev] [PATCH v2 1/2] ci: " Ruifeng Wang
2020-01-06 20:17 ` dwilder
2020-01-07 6:42 ` Ruifeng Wang
2019-12-20 9:37 ` [dpdk-dev] [PATCH v2 2/2] devtools: add path to additional shared object files Ruifeng Wang
2019-12-20 13:57 ` [dpdk-dev] [PATCH v2 0/2] add travis ci support for aarch64 David Marchand
2019-12-23 7:08 ` [dpdk-dev] [PATCH v3 " Ruifeng Wang
2019-12-23 7:08 ` [dpdk-dev] [PATCH v3 1/2] devtools: add path to additional shared object files Ruifeng Wang
2019-12-23 7:08 ` [dpdk-dev] [PATCH v3 2/2] ci: add travis ci support for aarch64 Ruifeng Wang
2020-01-06 13:34 ` Aaron Conole
2020-01-07 6:24 ` Ruifeng Wang
2020-01-07 14:40 ` Honnappa Nagarahalli
2020-01-08 16:06 ` Aaron Conole
2020-01-08 16:05 ` Aaron Conole
2020-01-08 17:37 ` Bruce Richardson
2020-01-09 7:00 ` Ruifeng Wang
2020-01-09 15:50 ` Honnappa Nagarahalli
2020-01-09 17:45 ` Aaron Conole
2020-01-10 9:53 ` [dpdk-dev] [PATCH v4 0/2] " Ruifeng Wang
2020-01-10 9:53 ` [dpdk-dev] [PATCH v4 1/2] devtools: add path to additional shared object files Ruifeng Wang
2020-01-10 15:03 ` Aaron Conole
2020-01-10 9:53 ` [dpdk-dev] [PATCH v4 2/2] ci: add travis ci support for aarch64 Ruifeng Wang
2020-01-10 15:13 ` Aaron Conole
2020-01-13 6:09 ` Ruifeng Wang
2020-01-13 6:26 ` [dpdk-dev] [PATCH v5 0/2] add travis ci support for native aarch64 Ruifeng Wang
2020-01-13 6:26 ` [dpdk-dev] [PATCH v5 1/2] devtools: add path to additional shared object files Ruifeng Wang
2020-01-13 6:26 ` [dpdk-dev] [PATCH v5 2/2] ci: add travis ci support for native aarch64 Ruifeng Wang
2020-01-14 14:03 ` [dpdk-dev] [PATCH v5 0/2] " David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AM0PR08MB39863F949BCD07BE2563F5FE9E520@AM0PR08MB3986.eurprd08.prod.outlook.com \
--to=ruifeng.wang@arm.com \
--cc=Gavin.Hu@arm.com \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=aconole@redhat.com \
--cc=arybchenko@solarflare.com \
--cc=bruce.richardson@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=kevin.laatz@intel.com \
--cc=maicolgabriel@hotmail.com \
--cc=nd@arm.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).