From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 72BFD1DA4 for ; Tue, 24 Jul 2018 17:14:07 +0200 (CEST) Received: by mail-wr1-f68.google.com with SMTP id h9-v6so4536416wro.3 for ; Tue, 24 Jul 2018 08:14:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=2U3sZdSnjbIc07zTcW8wWvsyOSsapJfucaxF5erRDrc=; b=j562enMp3+PiNMFn4//bg8uaNXIZzrU6H74GSE/NN7kqRcDYLFwEH8AqfthPfEHGg/ LdYlqGiVeGHMykfWSt8jDHCDeKPWkjd424ZNLZv/Irb+UVG/awoO65DrVwBG/lv1by8K pwUhOf3omnQGBQdSjbnVArNHQgoavVfkKnDfnkOlNbyxal/TWQc4oiGySIWDdHEqQiAP XDlVIZ/lR5dzI3Vrb2lh6rKZ/1RHqWl8t+ohaCN0fJfUQAf5+qs5ZlL3mOJ6DZgtyDe1 wnLkaqVKS6v9071FIEaQ068isu5/EznLqx4UiHKUJxGDKOjfDTeeA96JOA0xnUv2LNLt L5OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=2U3sZdSnjbIc07zTcW8wWvsyOSsapJfucaxF5erRDrc=; b=t+PHtkQL5EdMUFx0LrJsb9juBZICa8NxJLQstaSdXqZHWGE0YP2mcH2c8m/UZXpar2 /CZsRsWYUTY0xb9VMmH3sFhaJYsuxpGjsSU2k3CFylv9SXYYoE0uc3tzKahpWT101UNA DJ3cv9wimG2vANWCzWNDQeYwIuporVpugwAJIKNugVQPcfLMYq//c4+skvWPyJcNLG3X V7gWi6ChDof0uQPYVEtcLi5ZOuEsmjduIbB9JhjjIzoiKpCwH3OstbIYKZG+UggOdaNp FHph97mKmqI481x5Ro3JbUsDkBIGwkuQP0raonVNdjQMNb2E+IV/QNRAbOeLkgTnpSTS 4epw== X-Gm-Message-State: AOUpUlG+r0x1vpJqfW5Bw48ENaeFnFc5m5QElC7DKh7FI4suYVxZZtIG 7ZKOeJtY91yRa16ECftfm4VpVg== X-Google-Smtp-Source: AAOMgpf45CzowJ0iI01+o8UIsGxLMbolWAAJEZlj6dL56TjNlEycQIIvh/Iy0te/tMLZ0LcocLsFtw== X-Received: by 2002:adf:e491:: with SMTP id i17-v6mr12444876wrm.145.1532445247117; Tue, 24 Jul 2018 08:14:07 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id u4-v6sm21411556wrt.31.2018.07.24.08.14.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Jul 2018 08:14:06 -0700 (PDT) Date: Tue, 24 Jul 2018 17:13:50 +0200 From: Adrien Mazarguil To: Shahaf Shuler Cc: =?utf-8?B?TsOpbGlv?= Laranjeiro , "dev@dpdk.org" , Yongseok Koh Message-ID: <20180724151349.GT5211@6wind.com> References: <14690e825609ee181e3cd522302d4788ef436f35.1532424524.git.nelio.laranjeiro@6wind.com> <20180724125556.GS5211@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [dpdk-dev] [PATCH] mk: fix application compilation with lmnl and mlx5 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 15:14:07 -0000 On Tue, Jul 24, 2018 at 01:03:28PM +0000, Shahaf Shuler wrote: > Tuesday, July 24, 2018 3:56 PM, Adrien Mazarguil: > > Subject: Re: [PATCH] mk: fix application compilation with lmnl and mlx5 > > > > On Tue, Jul 24, 2018 at 11:21:52AM +0000, Shahaf Shuler wrote: > > > Tuesday, July 24, 2018 12:29 PM, Nelio Laranjeiro: > > > > Subject: [PATCH] mk: fix application compilation with lmnl and mlx5 > > > > > > > > When Mellanox MLX5 PMD is compiled with > > > > CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS=y, the external dependency > > on > > > > libmln is missing. > > > > > > > > Fixes: 4d5cce06231a ("net/mlx5: lay groundwork for switch offloads") > > > > Cc: adrien.mazarguil@6wind.com > > > > > > > > Signed-off-by: Nelio Laranjeiro > > > > --- > > > > mk/rte.app.mk | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk index > > > > f4d28c2da..ff39d37aa > > > > 100644 > > > > --- a/mk/rte.app.mk > > > > +++ b/mk/rte.app.mk > > > > @@ -149,7 +149,7 @@ else > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -lrte_pmd_mlx4 - > > > > libverbs -lmlx4 > > > > endif > > > > ifeq ($(CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS),y) > > > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 -ldl > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 -ldl > > - > > > > lmnl > > > > > > This issue raise some more basic question. > > > The DLOPEN mode was introduced to run in systems which don't have > > verbs/mlx5 libs installed, because those were the only dependencies for the > > PMD back then. > > > Now we have the libmnl, which is external dependency just like rdma-core, > > and following your fix, hard linked also in case of DLOPEN option. > > > It means the whole DPDK binary/lib will be depended on libmnl and this is > > not what we want with DLOPEN. > > > > > > Can we consider different options: > > > 1. always statically link libmnl > > > 2. dlopen libmnl just like we do for verbs/mlx5 > > > > Regarding 2, unlike rdma-core/MLNX_OFED, libmnl should be available pretty > > much everywhere iproute2 can be found. The minimal version supported > > (1.0.3) was released in 2012. > > > > Using the glue approach for such a small library seems overkill; should we > > choose this path, we must also consider to get rid of it entirely since doing so > > would require more glue code than what mlx5 needs from this library. > > > > So with the current approach, either the application or the PMD inherits a > > dependency to libmnl, depending on whether > > CONFIG_RTE_BUILD_SHARED_LIB is respectively disabled or enabled. > > > > If disabled, applications that want static linkage can specify -static as part of > > their compilation flags to let the compiler automatically look for libmnl.a as > > needed. > > Can you confirm static compilation w/ libmnl is working w/o any issues? Challenge accepted. Using -static is not something DPDK applications usually do and needs a few tweaks to compile successfully though (unless you meant something else by "static compilation"?) First you need to force rdma-core to generate static versions of libmlx4/libmlx5 and libiberbs as it's not the default: $ cmake -DENABLE_STATIC=1 . Next, patch DPDK to remove references to -lgcc_s, -fPIC and -export-dynamic: $ sed -i 's/[[:space:]]*-lgcc_s//' $(git grep -l -- -lgcc_s) $ sed -i 's/[[:space:]]*-fPIC//' $(git grep -l -- -fPIC) $ sed -i 's/[[:space:]]*-export-dynamic//' $(git grep -l -- -export-dynamic) Then append rdma-core dependencies to libmnl users (mlx5) in order to avoid undefined references to libnl/libm stuff normally pulled by libibverbs: $ sed -i 's/-lmnl/-lmnl -lnl-route-3 -lnl-3 -lm/' $(git grep -l -- -lmnl) Configure DPDK with mlx5 enabled: $ make config T=x86_64-native-linuxapp-gcc $ sed -i 's/^\(CONFIG_RTE_LIBRTE_MLX5_PMD\)=.*/\1=y/' build/.config Finally add -static to $CC (the easiest method I could find) while compiling DPDK: $ make CC='gcc -static' You can ignore warnings about statically linking "getaddrinfo" and friends. What matters is we now have a testpmd binary with no dependencies: $ readelf -d build/app/testpmd | grep NEEDED $ $ ldd build/app/testpmd not a dynamic executable $ Now start testpmd with a mlx5 adapter and a couple of representors for fun: # ./build/app/testpmd -w 06:00.0,representor=[0-1] -n 4 -c 0x80 -- -i --rxq=1 --txq=1 EAL: Detected 32 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:06:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1017 net_mlx5 [...] Configuring Port 0 (socket 0) Port 0: 24:8A:07:8D:AE:F6 Configuring Port 1 (socket 0) Port 1: A6:41:D9:89:DB:12 Configuring Port 2 (socket 0) Port 2: FE:A8:15:80:DE:C0 Checking link statuses... Done testpmd> Phew! > To put this in perspective, this also applies to all other dependencies > > it will collect while compiling DPDK (libz, libdl, libpcap, libnuma to name a > > few). > > > > In my opinion, the purpose of *_DLOPEN_DEPS is to deal with large, > > nonstandard libraries where versioning issues are commonplace. This doesn't > > apply to libmnl, which shouldn't be a maintenance nightmare to package > > maintainers. I suggest to leave things as is. -- Adrien Mazarguil 6WIND