From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 81F54A046B for ; Mon, 22 Jul 2019 18:43:28 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B7A0B1BF16; Mon, 22 Jul 2019 18:43:27 +0200 (CEST) Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by dpdk.org (Postfix) with ESMTP id D8B351BF16 for ; Mon, 22 Jul 2019 18:43:25 +0200 (CEST) Received: by mail-pl1-f194.google.com with SMTP id k8so19444893plt.3 for ; Mon, 22 Jul 2019 09:43:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ELsTZ6bB2bVyU8mMy8M9pSJ2dTN8ErKoolDd9JsCW7k=; b=stqfgjQwjzqIyUenZ6cwZkDwjBxRNnErvAanNTp93niwbgIwHobVIUzrU4FxavECoN WWpgWnaMeBJWszAG0tORORUnyBGJQls42G4tNjMRc9SLzf0csb3KAL4hX9dCA5s5EX0A refy/PVoDbAr92rJfFGlfSYDfN/Qzy6/P5wj55YzH3gsWKp5sfya4dd0AzanLAw/o8TD Fhf2yrd1kLq/BP5fki8DXuXkDp3GCQ0Mn8inJQyZxt4QFKqSOMTZK/QmHMSa9DyANQNU SXQkAYK44PRe85MSZh84lK7eGGxYSTmtFM3j6G2muK73zAaOG+fmtqFheknyu3N/4B8K fllA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ELsTZ6bB2bVyU8mMy8M9pSJ2dTN8ErKoolDd9JsCW7k=; b=ASyBNrj7fK8tY+bhuBq4HSQNRfh7kA3glh+55iLBlIVhWcqg80Quf8Yj+a0qmYqXAE ZHYhnYYPe4xhvBVJzpokq9Vc/AZtZNzH5js/xzag05lNrPdzz/WuZ3Lp6j1Hwm0r2mnk 7b8uQC9MTHKz1KCVn6apex/xrxKhawsnD1Md6yM0wQqRY43CkbHPS9knjnXva+7BzlOt rDlyzE5kgUgFW5/KiykZkPNDkMEKFBbA6aG+bQrwzldX7Toaf5F0y+fs5XPi5qv9eJaR lpktODdLJn8yqKZMikxqpgYtCsM7iwNW+tvAeGFFQ87OAASRMBLdQYeRisW8oZapTu5w UYzQ== X-Gm-Message-State: APjAAAVmpiVD8E5Ls9RZl1YAX3Aa7CV4XMGkmR7HtNCKN+vOn9+0Js5/ YXoO4qW9n58IoWy75dvI0ws= X-Google-Smtp-Source: APXvYqydmyr+bLRTmVDotFecO9GzOchJhkCf6uBTVEx+enPUmzrRjjmO1twqR1B1UROEB2A+V9AlKQ== X-Received: by 2002:a17:902:29a7:: with SMTP id h36mr78163667plb.158.1563813804850; Mon, 22 Jul 2019 09:43:24 -0700 (PDT) Received: from xps13 ([167.220.63.53]) by smtp.gmail.com with ESMTPSA id g1sm73136991pgg.27.2019.07.22.09.43.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 22 Jul 2019 09:43:24 -0700 (PDT) Date: Mon, 22 Jul 2019 09:43:23 -0700 From: Stephen Hemminger To: Bruce Richardson Cc: Thomas Monjalon , dev@dpdk.org, stable@dpdk.org Message-ID: <20190722094323.613cb090@xps13> In-Reply-To: <20190722090610.GA289@bricha3-MOBL.ger.corp.intel.com> References: <20190715234136.3526-1-stephen@networkplumber.org> <20190716084423.GB561@bricha3-MOBL.ger.corp.intel.com> <20190719135540.7d889722@hermes.lan> <4295239.Udij6XshuM@xps> <20190722090610.GA289@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] pci: fix missing pci bus with shared library build X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, 22 Jul 2019 10:06:11 +0100 Bruce Richardson wrote: > On Mon, Jul 22, 2019 at 09:38:27AM +0200, Thomas Monjalon wrote: > > 19/07/2019 22:55, Stephen Hemminger: > > > On Tue, 16 Jul 2019 09:46:04 +0100 > > > Bruce Richardson wrote: > > > > > > > On Mon, Jul 15, 2019 at 05:19:12PM -0700, Stephen Hemminger wrote: > > > > > On Mon, 15 Jul 2019 16:41:36 -0700 > > > > > Stephen Hemminger wrote: > > > > > > > > > > > If DPDK is built as a shared library, then any application linked > > > > > > with rte.app.mk will not find any PCI devices. When the application > > > > > > is started no ethernet devices are found. > > > > > > > > > > > > This is because the link order of libraries on the command line matters. > > > > > > And PCI is before EAL. That causes there to be no dependency on PCI > > > > > > so linker ignores linking the library. > > > > > > Swapping the order fixes this. > > > > > > > > > > > > Fixes: c752998b5e2e ("pci: introduce library and driver") > > > > > > Cc: stable@dpdk.org > > > > > > Signed-off-by: Stephen Hemminger > > > > > > --- > > > > > > mk/rte.app.mk | 2 +- > > > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > > > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk > > > > > > index a277c808ed8e..470b92e4d73e 100644 > > > > > > --- a/mk/rte.app.mk > > > > > > +++ b/mk/rte.app.mk > > > > > > @@ -90,8 +90,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_STACK) += -lrte_stack > > > > > > _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2 > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring > > > > > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal > > > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched > > > > > > > > > > It still happens with 19.08. Testpmd works but only because it is > > > > > linked with so many things. But l3fwd fails... > > > > > > > > > > # ./examples/l3fwd/build/l3fwd -n4 -l0-3 -w 02:00.0 > > > > > EAL: Detected 8 lcore(s) > > > > > EAL: Detected 1 NUMA nodes > > > > > EAL: failed to parse device "02:00.0" > > > > > EAL: Unable to parse device '02:00.0' > > > > > EAL: Error - exiting with code: 1 > > > > > Cause: Invalid EAL parameters > > > > > > > > I don't think the position of these is going to be the cause here, the more > > > > likely cause is that the pci bus driver - and all other drivers - are not > > > > linked into apps for shared library builds. You always need to pass "-d" > > > > parameter to load drivers at init time (or have them installed in the > > > > correct driver path). For example, for me with a shared library build the > > > > following gives a no ports error: > > > > > > > > sudo ./build/l2fwd -c F00000 -- -p 3 > > > > > > > > while this succeeds and runs fine > > > > > > > > sudo ./build/l2fwd -c F00000 -d $RTE_SDK/$RTE_TARGET/lib/librte_pmd_i40e.so -- -p 3 > > > > > > The root cause is that recent gcc won't run constructor on unused libraries. > > > Testing a patch to take --as-needed off of PCI library. > > > > > > See: https://stackoverflow.com/questions/11631161/force-to-link-against-unused-shared-library > > > > The constructor is run when calling dlopen, right? > > > > Note: dlopen with -d is a feature. > > The original idea was to be able to specify which driver we want to use. > > If we want an automatic dlopen, like modprobe, then we need more scripts. > > But I understand you are against the whole dlopen idea. > > > > This issue is more of a problem for development systems where we EAL path > is not really usable for finding the drivers. For a properly deployed > system where we use DPDK installed to /usr/local or /usr, the EAL PMD path > will be correctly configured and properly probe all drivers. The problem is that bus drivers register themselves in constructors and these construtors are not run with as-needed. One part of fixing this is: diff --git a/mk/rte.app.mk b/mk/rte.app.mk index df917f946497..46bdff8ec5e8 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -130,6 +130,9 @@ ifeq ($(CONFIG_RTE_LIBRTE_FSLMC_BUS),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_COMMON_FSLMC) += -lrte_common_fslmc endif +# Bus devices register in constructor so always link +_LDLIBS-y += --no-as-needed + _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += -lrte_bus_pci _LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += -lrte_bus_vdev _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA_BUS) += -lrte_bus_dpaa @@ -137,6 +140,8 @@ ifeq ($(CONFIG_RTE_EAL_VFIO),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += -lrte_bus_fslmc endif +_LDLIBS-y += --as-needed + ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n) # plugins (link only if static libraries)