From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr0-f193.google.com (mail-wr0-f193.google.com [209.85.128.193]) by dpdk.org (Postfix) with ESMTP id 3ACB11B00C for ; Thu, 25 Jan 2018 00:25:19 +0100 (CET) Received: by mail-wr0-f193.google.com with SMTP id 100so5761846wrb.7 for ; Wed, 24 Jan 2018 15:25:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=noIYQ3GA6bzC2Bm9ZU4nba8n4euhYqVigdjggUHOmT0=; b=bV53rfk+LiktsIIvlYplBear/8olCPKSMPle5QmGrhGed2H8OVH0DxmUj6Ko2oQQGX U0aJeJIXJRYbmAObgH37aNOA7vTdsZRpw4p/1//qVuLRbjl2QGNpEHZpjRjDzqk4vsR2 qCCZhw7X9BVuhgB0B2HoRCb0n5nKTjopfbPB05F6CWBKroFSFrPyn34SVb/MMSYfZA2E Ult0cAmafKEOLW6QUC4eMdmz/5XWtZZronwQuG98hQr+WavqM4zPIsLV0oqc5NdVm00u yyqbb3me08Nm4NTWUFTp26IFaIThklxzBSUkdcDoV2hRRzBTg9bpXFKOfi+0cShiH5pA Odeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=noIYQ3GA6bzC2Bm9ZU4nba8n4euhYqVigdjggUHOmT0=; b=RoBekO813iAp4UoZWuIxrVX8KdwvNjHoqgFlFWudf2jrzgJl5OZnoaNJfX+3uuaXjQ 81oUxVv+8Xk9F+7Beo3tf84woss92pYTJ+rB9qPF8BLi0To7lDr0R2adOvUrgPFLPO/y N4EYY0fmgac4bzEniZL4Gnkhx8BM2Uahtzcb/QFusjIfAzgceUGsNZSFIYAPoUBHdXzE 6HHE4I7D6ic65shp0cVv5CZpDdVI2keiipJAISHmW/nS8RQ/DMRk4Z8toS1am7zu0tRJ 1gSNzV8VwCoygqDhfJ4HpoM4iQ8GW81ow6VpfP3Wy8dUyXm+AvdGLI+YghGsSf1i6tEt OT/A== X-Gm-Message-State: AKwxytdxzfcs3vQSAQMqM5W1c1KaFtRrli0pwAspKokUiJ4MJgI7elrf 0hiKT+7YmHAotKC06B0RDijcj+zC X-Google-Smtp-Source: AH8x227YoxRkjuGNLFWROXiIlxSTTfEXhDwL8IuDuwLKX1B8eJXRIbtqnUdw1xYv1/XgIQvwd5EpbA== X-Received: by 10.223.152.83 with SMTP id v77mr734139wrb.233.1516836318938; Wed, 24 Jan 2018 15:25:18 -0800 (PST) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id o18sm3913477wrc.62.2018.01.24.15.25.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jan 2018 15:25:18 -0800 (PST) Date: Thu, 25 Jan 2018 00:25:06 +0100 From: Adrien Mazarguil To: Shahaf Shuler Cc: Nelio Laranjeiro , dev@dpdk.org, Marcelo Ricardo Leitner Message-ID: <20180124223625.1928-5-adrien.mazarguil@6wind.com> References: <20180124223625.1928-1-adrien.mazarguil@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180124223625.1928-1-adrien.mazarguil@6wind.com> X-Mailer: git-send-email 2.11.0 Subject: [dpdk-dev] [PATCH v1 4/4] net/mlx5: spawn rdma-core dependency plug-in X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 23:25:19 -0000 When mlx5 is not compiled directly as an independent shared object (e.g. CONFIG_RTE_BUILD_SHARED_LIB not enabled for performance reasons), DPDK applications inherit its dependencies on libibverbs and libmlx5 through rte.app.mk. This is an issue both when DPDK is delivered as a binary package (Linux distributions) and for end users because rdma-core then propagates as a mandatory dependency for everything. Application writers relying on binary DPDK packages are not necessarily aware of this fact and may end up delivering packages with broken dependencies. This patch therefore introduces an intermediate internal plug-in hard-linked with rdma-core (to preserve symbol versioning) loaded by the PMD through dlopen(), so that a missing rdma-core does not cause unresolved symbols, allowing applications to start normally. Signed-off-by: Adrien Mazarguil --- drivers/net/mlx5/Makefile | 40 +++++++++++++++++++++ drivers/net/mlx5/mlx5.c | 79 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+) diff --git a/drivers/net/mlx5/Makefile b/drivers/net/mlx5/Makefile index bdec30692..35525f6d0 100644 --- a/drivers/net/mlx5/Makefile +++ b/drivers/net/mlx5/Makefile @@ -53,7 +53,11 @@ SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_rss.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_mr.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_flow.c SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_socket.c +ifneq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) +SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_glue_lib.c +else SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5_glue.c +endif # Basic CFLAGS. CFLAGS += -O3 @@ -65,7 +69,12 @@ CFLAGS += -D_DEFAULT_SOURCE CFLAGS += -D_XOPEN_SOURCE=600 CFLAGS += $(WERROR_FLAGS) CFLAGS += -Wno-strict-prototypes +ifneq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) +CFLAGS_mlx5_glue.o += -fPIC +LDLIBS += -ldl +else LDLIBS += -libverbs -lmlx5 +endif LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs LDLIBS += -lrte_bus_pci @@ -158,7 +167,38 @@ mlx5_autoconf.h: mlx5_autoconf.h.new $(SRCS-$(CONFIG_RTE_LIBRTE_MLX5_PMD):.c=.o): mlx5_autoconf.h +# Generate dependency plug-in for rdma-core when the PMD cannot be linked +# directly, so that applications do not inherit this dependency. + +ifneq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) + +mlx5_glue_lib.c: mlx5_glue_lib.so + $Q printf '#include \n' > $@ + $Q printf '#include \n\n' >> $@ + $Q printf 'const uint8_t mlx5_glue_lib[][16] = {\n' >> $@ + $Q od -vt x1 $< | \ + sed -ne '/^[[:xdigit:]]\{1,\}/{' \ + -e 's///;' \ + -e '/^$$/d; ' \ + -e 's/[[:space:]]*$$//;' \ + -e 's/[[:space:]]\{1,\}/\\x/g;' \ + -e 's/^/ "/;' \ + -e 's/$$/",/;' \ + -e 'p;' \ + -e '}' >> $@ + $Q printf '};\n\n' >> $@ + $Q printf 'const size_t mlx5_glue_lib_size = %u;' $$(wc -c < $<) >> $@ + +mlx5_glue_lib.so: mlx5_glue.o + $Q $(LD) $(LDFLAGS) $(EXTRA_LDFLAGS) \ + -s -shared -o $@ $< -libverbs -lmlx5 + +mlx5_glue.o: mlx5_autoconf.h + +endif + clean_mlx5: FORCE $Q rm -f -- mlx5_autoconf.h mlx5_autoconf.h.new + $Q rm -f -- mlx5_glue.o mlx5_glue_lib.* clean: clean_mlx5 diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index b36cc611f..7ef724585 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -56,6 +57,7 @@ #include #include #include +#include #include #include "mlx5.h" @@ -966,6 +968,78 @@ static struct rte_pci_driver mlx5_driver = { .drv_flags = RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV, }; +#ifndef RTE_BUILD_SHARED_LIB + +extern const uint8_t mlx5_glue_lib[][16]; +extern const size_t mlx5_glue_lib_size; + +/** + * Initialization routine for run-time dependency on rdma-core. + */ +static int +mlx5_glue_init(void) +{ + char file[] = "/tmp/" MLX5_DRIVER_NAME "_XXXXXX"; + int fd = mkstemp(file); + size_t off = 0; + void *handle = NULL; + void **sym; + const char *dlmsg; + + if (fd == -1) { + rte_errno = errno; + goto glue_error; + } + while (off != mlx5_glue_lib_size) { + ssize_t ret; + + ret = write(fd, (const uint8_t *)mlx5_glue_lib + off, + mlx5_glue_lib_size - off); + if (ret == -1) { + if (errno != EINTR) { + rte_errno = errno; + goto glue_error; + } + ret = 0; + } + off += ret; + } + close(fd); + fd = -1; + handle = dlopen(file, RTLD_LAZY); + unlink(file); + if (!handle) { + rte_errno = EINVAL; + dlmsg = dlerror(); + if (dlmsg) + ERROR("cannot load glue library: %s", dlmsg); + goto glue_error; + } + sym = dlsym(handle, "mlx5_glue"); + if (!sym || !*sym) { + rte_errno = EINVAL; + dlmsg = dlerror(); + if (dlmsg) + ERROR("cannot resolve glue symbol: %s", dlmsg); + goto glue_error; + } + mlx5_glue = *sym; + return 0; +glue_error: + if (handle) + dlclose(handle); + if (fd != -1) { + close(fd); + unlink(file); + } + ERROR("cannot initialize PMD due to missing run-time" + " dependency on rdma-core libraries (libibverbs," + " libmlx5)"); + return -rte_errno; +} + +#endif + /** * Driver initialization routine. */ @@ -985,6 +1059,11 @@ rte_mlx5_pmd_init(void) /* Match the size of Rx completion entry to the size of a cacheline. */ if (RTE_CACHE_LINE_SIZE == 128) setenv("MLX5_CQE_SIZE", "128", 0); +#ifndef RTE_BUILD_SHARED_LIB + if (mlx5_glue_init()) + return; + assert(mlx5_glue); +#endif mlx5_glue->fork_init(); rte_pci_register(&mlx5_driver); } -- 2.11.0