From: Shahaf Shuler <shahafs@mellanox.com>
To: Adrien Mazarguil <adrien.mazarguil@6wind.com>,
Thomas Monjalon <thomas@monjalon.net>
Cc: "Nélio Laranjeiro" <nelio.laranjeiro@6wind.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Yongseok Koh" <yskoh@mellanox.com>
Subject: Re: [dpdk-dev] [PATCH] mk: fix application compilation with lmnl and mlx5
Date: Wed, 25 Jul 2018 06:39:32 +0000 [thread overview]
Message-ID: <DB7PR05MB4426F27993854AB44CFA49EDC3540@DB7PR05MB4426.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <20180724151349.GT5211@6wind.com>
Tuesday, July 24, 2018 6:14 PM, Adrien Mazarguil:
> Subject: Re: [PATCH] mk: fix application compilation with lmnl and mlx5
> On Tue, Jul 24, 2018 at 01:03:28PM +0000, Shahaf Shuler wrote:
> > Tuesday, July 24, 2018 3:56 PM, Adrien Mazarguil:
> > > Subject: Re: [PATCH] mk: fix application compilation with lmnl and
> > > mlx5
[...]
> > > > Can we consider different options:
> > > > 1. always statically link libmnl
> > > > 2. dlopen libmnl just like we do for verbs/mlx5
> > >
> > > Regarding 2, unlike rdma-core/MLNX_OFED, libmnl should be available
> > > pretty much everywhere iproute2 can be found. The minimal version
> > > supported
> > > (1.0.3) was released in 2012.
> > >
> > > Using the glue approach for such a small library seems overkill;
> > > should we choose this path, we must also consider to get rid of it
> > > entirely since doing so would require more glue code than what mlx5
> needs from this library.
> > >
> > > So with the current approach, either the application or the PMD
> > > inherits a dependency to libmnl, depending on whether
> > > CONFIG_RTE_BUILD_SHARED_LIB is respectively disabled or enabled.
> > >
> > > If disabled, applications that want static linkage can specify
> > > -static as part of their compilation flags to let the compiler
> > > automatically look for libmnl.a as needed.
> >
> > Can you confirm static compilation w/ libmnl is working w/o any issues?
>
> Challenge accepted.
😊
Using -static is not something DPDK applications usually
> do and needs a few tweaks to compile successfully though (unless you
> meant something else by "static compilation"?)
Yes this is what I meant. However I was under the impression we can statically link to libmnl while keeping the rdma-core w/ dynamic linkage.
>
> First you need to force rdma-core to generate static versions of
> libmlx4/libmlx5 and libiberbs as it's not the default:
>
> $ cmake -DENABLE_STATIC=1 .
>
> Next, patch DPDK to remove references to -lgcc_s, -fPIC and -export-
> dynamic:
>
> $ sed -i 's/[[:space:]]*-lgcc_s//' $(git grep -l -- -lgcc_s) $ sed -i 's/[[:space:]]*-
> fPIC//' $(git grep -l -- -fPIC) $ sed -i 's/[[:space:]]*-export-dynamic//' $(git
> grep -l -- -export-dynamic)
It seems the fPIC and export-dynamic are defined only when compiling the DPDK as shared library.
The gcc_s is from eal. Do you think it is correct to put it only when compiling as shared library? It seems this approach was taken in the rte.vars.mk files.
>
> Then append rdma-core dependencies to libmnl users (mlx5) in order to
> avoid undefined references to libnl/libm stuff normally pulled by libibverbs:
>
> $ sed -i 's/-lmnl/-lmnl -lnl-route-3 -lnl-3 -lm/' $(git grep -l -- -lmnl)
>
> Configure DPDK with mlx5 enabled:
>
> $ make config T=x86_64-native-linuxapp-gcc $ sed -i
> 's/^\(CONFIG_RTE_LIBRTE_MLX5_PMD\)=.*/\1=y/' build/.config
>
> Finally add -static to $CC (the easiest method I could find) while compiling
> DPDK:
>
> $ make CC='gcc -static'
>
> You can ignore warnings about statically linking "getaddrinfo" and friends.
> What matters is we now have a testpmd binary with no dependencies:
>
> $ readelf -d build/app/testpmd | grep NEEDED $
>
> $ ldd build/app/testpmd
> not a dynamic executable
> $
>
> Now start testpmd with a mlx5 adapter and a couple of representors for fun:
>
> # ./build/app/testpmd -w 06:00.0,representor=[0-1] -n 4 -c 0x80 -- -i --rxq=1
> --txq=1
> EAL: Detected 32 lcore(s)
> EAL: Detected 2 NUMA nodes
> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: PCI device 0000:06:00.0 on NUMA socket 0
> EAL: probe driver: 15b3:1017 net_mlx5
> [...]
> Configuring Port 0 (socket 0)
> Port 0: 24:8A:07:8D:AE:F6
> Configuring Port 1 (socket 0)
> Port 1: A6:41:D9:89:DB:12
> Configuring Port 2 (socket 0)
> Port 2: FE:A8:15:80:DE:C0
> Checking link statuses...
> Done
> testpmd>
>
> Phew!
Nicely done.
So if indeed it is possible w/ some fixes on the DPDK to fully support static linkage of both rdma-core and libmnl, maybe we should consider such compilation flag for mlx drivers. This can be very good alternative to the current DLOPEN approach.
>
> > To put this in perspective, this also applies to all other
> > dependencies
> > > it will collect while compiling DPDK (libz, libdl, libpcap, libnuma
> > > to name a few).
> > >
> > > In my opinion, the purpose of *_DLOPEN_DEPS is to deal with large,
> > > nonstandard libraries where versioning issues are commonplace. This
> > > doesn't apply to libmnl, which shouldn't be a maintenance nightmare
> > > to package maintainers. I suggest to leave things as is.
>
> --
> Adrien Mazarguil
> 6WIND
next prev parent reply other threads:[~2018-07-25 6:39 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-24 9:29 Nelio Laranjeiro
2018-07-24 9:32 ` Adrien Mazarguil
2018-07-26 12:05 ` Thomas Monjalon
2018-07-24 11:21 ` Shahaf Shuler
2018-07-24 11:44 ` Nélio Laranjeiro
2018-07-24 11:53 ` Shahaf Shuler
2018-07-24 12:55 ` Adrien Mazarguil
2018-07-24 13:03 ` Shahaf Shuler
2018-07-24 15:13 ` Adrien Mazarguil
2018-07-25 6:39 ` Shahaf Shuler [this message]
2018-07-25 9:07 ` Adrien Mazarguil
2018-07-25 13:23 ` Shahaf Shuler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DB7PR05MB4426F27993854AB44CFA49EDC3540@DB7PR05MB4426.eurprd05.prod.outlook.com \
--to=shahafs@mellanox.com \
--cc=adrien.mazarguil@6wind.com \
--cc=dev@dpdk.org \
--cc=nelio.laranjeiro@6wind.com \
--cc=thomas@monjalon.net \
--cc=yskoh@mellanox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).