From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84556A0510; Wed, 18 Dec 2019 16:32:39 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B0F131C01; Wed, 18 Dec 2019 16:32:38 +0100 (CET) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 5B542F90 for ; Wed, 18 Dec 2019 16:32:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576683156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O/5CXB9Hy7q9BUFEx59zGeystDzvt1COlDSj9OBuB8g=; b=K7AXHpdFcb9N55glq43FWRRkGgwQUt+/Uww2fp3JhkFi4JYjKecZ45Qicshch9lgw9V+Uq 0P13PmGKWFnSWtPeSkl1RuZPhZmgUiHnoeHR34BZGL20CqMc7xAzjKGpXweiF0UY2NUTps PZlAHA8IwvagXWIMOSmbuUIrheV+ZxI= Received: from mail-ua1-f71.google.com (mail-ua1-f71.google.com [209.85.222.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-207-trID9DVjN_a30g9OWB5_Dg-1; Wed, 18 Dec 2019 10:32:34 -0500 Received: by mail-ua1-f71.google.com with SMTP id s22so479768uar.17 for ; Wed, 18 Dec 2019 07:32:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SUmI8+sIPMn2YS/gYHHmxY4LnhJRL+fhr1D3FwKJFik=; b=eRELE3axCNbm5Ar6BRnHPxBA8ITwOmuVdJyBfiONmlcixj3ZVMPXDhXQ7VPVzry3w/ wSEvVTDAnh5tiLbc4My3pgRHEZ2h6z6SrKtpdIfeXdVrf+jAaWw70zx1IEzIb0Li7N8M vFHE8QLwrKU9+eDxP39f+l7DgJr4XhpuMbyTLpjeD8pGbY2VlacNnzPNkNdCUt0HDw1d J8XQbfhgG4o5z/XaootRbJ0yrqoEFLcET+jRQofU25bF9iiMmnndHzipDZ+rw9W7SShH zoue5/zgZmoIuvWxYH6F1UMqYgYQWkerE+3ngnPqGxBv1c112x8Y3nP4KID6LcVGjHH7 FWYA== X-Gm-Message-State: APjAAAX2HERTxv2sPiJc1be4jqOg9JuVdjbGc88Lzuk8CgELdJ8W81eE S7KbeVP2c+oKUv7rVoyvFBHoH4ptQ5UuFvPqzQ5ppW7qeEfqZ5fDThlBOyaCBUZAfQkFbIbA2Hs KjIHa5TrQK6XImeQmdKo= X-Received: by 2002:a9f:372c:: with SMTP id z41mr1771400uad.87.1576683154049; Wed, 18 Dec 2019 07:32:34 -0800 (PST) X-Google-Smtp-Source: APXvYqxvnXLQbNOHnY5qHKllPJLKXU3pTk0l0I+jQInDzILTJR47edQJk+MBrXn+swpr0sBBRETgcVuqvZtn4XTADIA= X-Received: by 2002:a9f:372c:: with SMTP id z41mr1771358uad.87.1576683153555; Wed, 18 Dec 2019 07:32:33 -0800 (PST) MIME-Version: 1.0 References: <20191218053902.193417-1-ruifeng.wang@arm.com> <20191218053902.193417-3-ruifeng.wang@arm.com> In-Reply-To: From: David Marchand Date: Wed, 18 Dec 2019 16:32:22 +0100 Message-ID: To: "Laatz, Kevin" Cc: Ruifeng Wang , Bruce Richardson , Aaron Conole , Michael Santana , Thomas Monjalon , "Yigit, Ferruh" , Andrew Rybchenko , dev , Gavin Hu , Honnappa Nagarahalli , nd X-MC-Unique: trID9DVjN_a30g9OWB5_Dg-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH 2/2] devtools: add path to additional shared object files X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Dec 18, 2019 at 2:43 PM Laatz, Kevin wrote: > > On 18/12/2019 08:23, David Marchand wrote: > > On Wed, Dec 18, 2019 at 6:39 AM Ruifeng Wang wro= te: > >> librte_mempool_ring.so and librte_pmd_null.so are in 'drivers' folder. > >> Add 'drivers' into LD_LIBRARY_PATH so that testpmd can find and make > >> use of these shared libraries. > >> > >> Signed-off-by: Ruifeng Wang > >> Reviewed-by: Gavin Hu > >> --- > >> devtools/test-null.sh | 2 +- > >> 1 file changed, 1 insertion(+), 1 deletion(-) > >> > >> diff --git a/devtools/test-null.sh b/devtools/test-null.sh > >> index f39af2c06..548de8113 100755 > >> --- a/devtools/test-null.sh > >> +++ b/devtools/test-null.sh > >> @@ -20,7 +20,7 @@ if [ ! -f "$testpmd" ] ; then > >> fi > >> > >> if ldd $testpmd | grep -q librte_ ; then > >> - export LD_LIBRARY_PATH=3D$build/lib:$LD_LIBRARY_PATH > >> + export LD_LIBRARY_PATH=3D$build/drivers:$build/lib:$LD_LIBRARY= _PATH > >> libs=3D'-d librte_mempool_ring.so -d librte_pmd_null.so' > >> else > >> libs=3D > >> -- > >> 2.17.1 > >> > > I'm surprised to see this. > > So far, the CI ran fine without it, so something is different in the > > environment. > > > > I can see that the RPATH entry disappeared from the testpmd binary. > > Xenial: > > # readelf -d build/app/dpdk-testpmd |grep RPATH > > 0x000000000000000f (RPATH) Library rpath: > > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX] > > > > (not sure what XXXX purpose is, but different topic) > > > > Bionic: > > # readelf -d build/app/dpdk-testpmd |grep RPATH > > It looks like RPATH just changed to be named RUNPATH in Bionic: > > # readelf -d build/app/dpdk-testpmd | grep R.*PATH > 0x000000000000001d (RUNPATH) Library runpath: > [$ORIGIN/../lib:$ORIGIN/../drivers:XXXXXXXXXXXXX] Did some experiment with some test program and .so of mine. TL;DR, if I understand correctly, RPATH on the binary applies to all lookups, even in a subsequent .so code. But RUNPATH only applies to the current ELF, meaning that the dlopen() in my intermediate .so does not get it. dlopen() is called from librte_eal.so, and RUNPATH on testpmd is not enough= . Details: [dmarchan@dmarchan plop]$ cat main.c extern void loader(void); int main(int argc, char *argv[]) { loader(); return 0; } [dmarchan@dmarchan plop]$ cat loader/loader.c #include #include void loader(void) { if (dlopen("lib.so", RTLD_NOW) =3D=3D NULL) fprintf(stderr, "%s\n", dlerror()); } [dmarchan@dmarchan plop]$ cat lib/lib.c #include __attribute__((constructor)) static void plop(void) { fprintf(stdout, "plop\n"); } # no rpath/runpath [dmarchan@dmarchan plop]$ gcc -o lib/lib.so -Wall -Werror -shared -fPIC lib/lib.c [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -shared -fPIC loader/loader.c -ldl [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c loader/loader.so [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH [dmarchan@dmarchan plop]$ ./main lib.so: cannot open shared object file: No such file or directory # using rpath on final binary [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c loader/loader.so -Wl,-rpath,loader:lib [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH 0x000000000000000f (RPATH) Library rpath: [loader:lib] [dmarchan@dmarchan plop]$ ./main plop # using runpath on final binary [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c loader/loader.so -Wl,-enable-new-dtag,-rpath,loader:lib [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH 0x000000000000001d (RUNPATH) Library runpath: [loader:lib] [dmarchan@dmarchan plop]$ ./main lib.so: cannot open shared object file: No such file or directory # using runpath on loader [dmarchan@dmarchan plop]$ gcc -o loader/loader.so -Wall -Werror -shared -fPIC loader/loader.c -ldl -Wl,-enable-new-dtag,-rpath,lib [dmarchan@dmarchan plop]$ readelf -d loader/loader.so |grep R.*PATH 0x000000000000001d (RUNPATH) Library runpath: [lib] [dmarchan@dmarchan plop]$ gcc -o main -Wall -Werror main.c loader/loader.so -Wl,-enable-new-dtag,-rpath,loader [dmarchan@dmarchan plop]$ readelf -d main |grep R.*PATH 0x000000000000001d (RUNPATH) Library runpath: [loader] [dmarchan@dmarchan plop]$ ./main plop > > Adding Bruce and Kevin, as I think this is the same issue than: > > http://mails.dpdk.org/archives/dev/2019-December/153627.html > > Could it be a change in meson? > > Yes, looks like the same error to me. > > I'm not sure this is solely a meson issue, I often need to pass -d > librte_mempool_ring.so when using make builds too. Maybe I'm missing > something here? In a make build directory, all libraries ends up in the same lib/ directory and the test-null.sh script work with the current LD_LIBRARY_PATH. Ruifeng patch seems valid to me, but I would love some explanations in the commitlog. -- David Marchand