Re: [dpdk-dev] [PATCH v2 2/4] buildtools: get static mlx dependencies for meson

[Bruce Richardson <bruce.richardson@intel.com>]

On Wed, Jan 29, 2020 at 03:37:51PM +0000, Bruce Richardson wrote:
> On Mon, Jan 27, 2020 at 04:44:00PM +0100, Thomas Monjalon wrote:
> > The shell script options-ibverbs-static.sh was used with make
> > in forcing static linkage of ibverbs libraries.
> > 
> > When choosing to link with a static dependency in meson,
> > the generated .pc file will not force such static linkage.
> > The solution will rely on using this script in meson.
> > 
> > If linking with libraries installed in a non-standard path,
> > an option -L is provided via EXTRA_LDFLAGS in case of using make.
> > With meson, tuning PKG_CONFIG_PATH for pkg-config should be enough.
> > When statically linking an application, the .pc file must save the
> > -L path so the application link will work without any extra option.
> > That's why --libs-only-l is replaced with --libs which includes -L.
> > Options which are neither -l or -L are filtered out because not needed
> > and can cause compilation issues with the legacy system using make.
> > 
> > The other change in this script is to move the main library file
> > (libiverbs.a) at the end of the list of dependencies.
> > It fixes some undefined references when linking a static application
> > using libdpdk.pc.
> > 
> > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >  buildtools/options-ibverbs-static.sh | 10 ++++++++--
> >  1 file changed, 8 insertions(+), 2 deletions(-)
> > 
> > diff --git a/buildtools/options-ibverbs-static.sh b/buildtools/options-ibverbs-static.sh
> > index 0f285a343b..43578a37f3 100755
> > --- a/buildtools/options-ibverbs-static.sh
> > +++ b/buildtools/options-ibverbs-static.sh
> > @@ -9,6 +9,12 @@
> >  #
> >  # PKG_CONFIG_PATH may be required to be set if libibverbs.pc is not installed.
> >  
> > -pkg-config --libs-only-l --static libibverbs |
> > +lib='libibverbs'
> > +deps='pthread|nl'
> > +
> > +pkg-config --libs --static $lib |
> >  	tr '[:space:]' '\n' |
> > -	sed -r '/^-l(pthread|nl)/! s,(^-l)(.*),\1:lib\2.a,'
> > +	sed -r "/^-l($deps)/! s,(^-l)(.*),\1:lib\2.a," |   # explicit .a
> > +	sed -n '/^-[Ll]/p' |   # extra link options may break with make
> > +	sed "/$lib/d"   # move main lib at the end
> > +echo -l:$lib.a
> 
> To improve my understanding of the desired end result with this patchset
> applied, I tried running different build configs with make with this patch
> applied. I get a build error on my system with the following settings
> adjusted from default (i.e. shared build with static link of ibverbs):
> 
> diff .config.orig .config
> 32c32
> < CONFIG_RTE_BUILD_SHARED_LIB=n
> ---
> > CONFIG_RTE_BUILD_SHARED_LIB=y
> 219c219
> < CONFIG_RTE_LIBRTE_MLX5_PMD=n
> ---
> > CONFIG_RTE_LIBRTE_MLX5_PMD=y
> 225c225
> < CONFIG_RTE_IBVERBS_LINK_STATIC=n
> ---
> > CONFIG_RTE_IBVERBS_LINK_STATIC=y
> 
> Error message I got is below. This is on Ubuntu 19.10 with gcc 9.2:
> 
>   LD librte_pmd_mlx5.so.20.0.1
> /usr/bin/ld: /lib/x86_64-linux-gnu/libmlx5.a(mlx5.c.o): relocation R_X86_64_PC32 against symbol `stderr@@GL
> /usr/bin/ld: final link failed: bad value
> collect2: error: ld returned 1 exit status
> make[5]: *** [/home/bruce/dpdk.org/mk/rte.lib.mk:100: librte_pmd_mlx5.so.20.0.1] Error 1
> make[4]: *** [/home/bruce/dpdk.org/mk/rte.subdir.mk:35: mlx5] Error 2
> make[3]: *** [/home/bruce/dpdk.org/mk/rte.subdir.mk:35: net] Error 2
> make[2]: *** [/home/bruce/dpdk.org/mk/rte.sdkbuild.mk:48: drivers] Error 2
> make[1]: *** [/home/bruce/dpdk.org/mk/rte.sdkroot.mk:99: all] Error 2
> make: *** [Makefile:12: all] Error 2
> 

After applying all 4 patches, and testing with meson, I see the same issue.
It appears that the mlx4/5.a files in Ubuntu are not relocatable for
including in a shared library.

/Bruce

[1/2] Linking target drivers/librte_pmd_mlx4.so.20.0.1.
FAILED: drivers/librte_pmd_mlx4.so.20.0.1
cc  -o drivers/librte_pmd_mlx4.so.20.0.1 'drivers/a715181@@rte_pmd_mlx4@sha/meson-generated_.._rte_pmd_mlx4.pmd.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_ethdev.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_flow.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_intr.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_mp.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_mr.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_rxq.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_rxtx.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_txq.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_utils.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx4@sta/net_mlx4_mlx4_glue.c.o' -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_pmd_mlx4.so.20.0 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_ethdev.so.20.0.1 lib/librte_eal.so.20.0.1 lib/librte_kvargs.so.20.0.1 lib/librte_net.so.20.0.1 lib/librte_mbuf.so.20.0.1 lib/librte_mempool.so.20.0.1 lib/librte_ring.so.20.0.1 lib/librte_meter.so.20.0.1 drivers/librte_bus_pci.so.20.0.1 lib/librte_pci.so.20.0.1 drivers/librte_bus_vdev.so.20.0.1 -Wl,--version-script=/home/bruce/dpdk.org/drivers/net/mlx4/rte_pmd_mlx4_version.map -lpthread -l:libbnxt_re-rdmav22.a -l:libcxgb3-rdmav22.a -l:libcxgb4-rdmav22.a -l:libefa.a -l:libhns-rdmav22.a -l:libi40iw-rdmav22.a -l:libmlx4.a -l:libmlx5.a -l:libmthca-rdmav22.a -l:libnes-rdmav22.a -l:libocrdma-rdmav22.a -l:libqedr-rdmav22.a -l:libvmw_pvrdma-rdmav22.a -l:libhfi1verbs-rdmav22.a -l:libipathverbs-rdmav22.a -l:librxe-rdmav22.a -lnl-route-3 -lnl-3 -l:libibverbs.a /usr/lib/x86_64-linux-gnu/libbsd.so -Wl,--end-group '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' -Wl,-rpath-link,/home/bruce/dpdk.org/build/lib -Wl,-rpath-link,/home/bruce/dpdk.org/build/drivers
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libmlx4.a(mlx4.c.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
[2/2] Linking target drivers/librte_pmd_mlx5.so.20.0.1.
FAILED: drivers/librte_pmd_mlx5.so.20.0.1
cc  -o drivers/librte_pmd_mlx5.so.20.0.1 'drivers/a715181@@rte_pmd_mlx5@sha/meson-generated_.._rte_pmd_mlx5.pmd.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_ethdev.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_flow.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_flow_meter.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_flow_dv.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_flow_verbs.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_mac.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_mr.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_nl.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_rss.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_rxmode.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_rxq.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_rxtx.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_mp.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_stats.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_trigger.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_txq.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_vlan.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_devx_cmds.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_utils.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_socket.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_rxtx_vec.c.o' 'drivers/a715181@@tmp_rte_pmd_mlx5@sta/net_mlx5_mlx5_glue.c.o' -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_pmd_mlx5.so.20.0 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_ethdev.so.20.0.1 lib/librte_eal.so.20.0.1 lib/librte_kvargs.so.20.0.1 lib/librte_net.so.20.0.1 lib/librte_mbuf.so.20.0.1 lib/librte_mempool.so.20.0.1 lib/librte_ring.so.20.0.1 lib/librte_meter.so.20.0.1 drivers/librte_bus_pci.so.20.0.1 lib/librte_pci.so.20.0.1 drivers/librte_bus_vdev.so.20.0.1 lib/librte_hash.so.20.0.1 -Wl,--version-script=/home/bruce/dpdk.org/drivers/net/mlx5/rte_pmd_mlx5_version.map -lpthread -l:libbnxt_re-rdmav22.a -l:libcxgb3-rdmav22.a -l:libcxgb4-rdmav22.a -l:libefa.a -l:libhns-rdmav22.a -l:libi40iw-rdmav22.a -l:libmlx4.a -l:libmlx5.a -l:libmthca-rdmav22.a -l:libnes-rdmav22.a -l:libocrdma-rdmav22.a -l:libqedr-rdmav22.a -l:libvmw_pvrdma-rdmav22.a -l:libhfi1verbs-rdmav22.a -l:libipathverbs-rdmav22.a -l:librxe-rdmav22.a -lnl-route-3 -lnl-3 -l:libibverbs.a /usr/lib/x86_64-linux-gnu/libbsd.so -Wl,--end-group '-Wl,-rpath,$ORIGIN/../lib:$ORIGIN/' -Wl,-rpath-link,/home/bruce/dpdk.org/build/lib -Wl,-rpath-link,/home/bruce/dpdk.org/build/drivers
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libmlx5.a(mlx5.c.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.