From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 9D01CA046B
for <public@inbox.dpdk.org>; Mon, 22 Jul 2019 11:06:19 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 5ACDF1BD62;
Mon, 22 Jul 2019 11:06:19 +0200 (CEST)
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
by dpdk.org (Postfix) with ESMTP id DA6741B9C9;
Mon, 22 Jul 2019 11:06:16 +0200 (CEST)
X-Amp-Result: UNKNOWN
X-Amp-Original-Verdict: FILE UNKNOWN
X-Amp-File-Uploaded: False
Received: from orsmga006.jf.intel.com ([10.7.209.51])
by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
22 Jul 2019 02:06:15 -0700
X-IronPort-AV: E=Sophos;i="5.64,294,1559545200"; d="scan'208";a="174171169"
Received: from bricha3-mobl.ger.corp.intel.com ([10.237.221.51])
by orsmga006-auth.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
22 Jul 2019 02:06:13 -0700
Date: Mon, 22 Jul 2019 10:06:11 +0100
From: Bruce Richardson <bruce.richardson@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: Stephen Hemminger <stephen@networkplumber.org>, dev@dpdk.org,
stable@dpdk.org
Message-ID: <20190722090610.GA289@bricha3-MOBL.ger.corp.intel.com>
References: <20190715234136.3526-1-stephen@networkplumber.org>
<20190716084423.GB561@bricha3-MOBL.ger.corp.intel.com>
<20190719135540.7d889722@hermes.lan> <4295239.Udij6XshuM@xps>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4295239.Udij6XshuM@xps>
User-Agent: Mutt/1.11.4 (2019-03-13)
Subject: Re: [dpdk-dev] [PATCH] pci: fix missing pci bus with shared library
build
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On Mon, Jul 22, 2019 at 09:38:27AM +0200, Thomas Monjalon wrote:
> 19/07/2019 22:55, Stephen Hemminger:
> > On Tue, 16 Jul 2019 09:46:04 +0100
> > Bruce Richardson <bruce.richardson@intel.com> wrote:
> >
> > > On Mon, Jul 15, 2019 at 05:19:12PM -0700, Stephen Hemminger wrote:
> > > > On Mon, 15 Jul 2019 16:41:36 -0700
> > > > Stephen Hemminger <stephen@networkplumber.org> wrote:
> > > >
> > > > > If DPDK is built as a shared library, then any application linked
> > > > > with rte.app.mk will not find any PCI devices. When the application
> > > > > is started no ethernet devices are found.
> > > > >
> > > > > This is because the link order of libraries on the command line matters.
> > > > > And PCI is before EAL. That causes there to be no dependency on PCI
> > > > > so linker ignores linking the library.
> > > > > Swapping the order fixes this.
> > > > >
> > > > > Fixes: c752998b5e2e ("pci: introduce library and driver")
> > > > > Cc: stable@dpdk.org
> > > > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > > > > ---
> > > > > mk/rte.app.mk | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> > > > > index a277c808ed8e..470b92e4d73e 100644
> > > > > --- a/mk/rte.app.mk
> > > > > +++ b/mk/rte.app.mk
> > > > > @@ -90,8 +90,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_STACK) += -lrte_stack
> > > > > _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring
> > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2
> > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring
> > > > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci
> > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal
> > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci
> > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline
> > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder
> > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched
> > > >
> > > > It still happens with 19.08. Testpmd works but only because it is
> > > > linked with so many things. But l3fwd fails...
> > > >
> > > > # ./examples/l3fwd/build/l3fwd -n4 -l0-3 -w 02:00.0
> > > > EAL: Detected 8 lcore(s)
> > > > EAL: Detected 1 NUMA nodes
> > > > EAL: failed to parse device "02:00.0"
> > > > EAL: Unable to parse device '02:00.0'
> > > > EAL: Error - exiting with code: 1
> > > > Cause: Invalid EAL parameters
> > >
> > > I don't think the position of these is going to be the cause here, the more
> > > likely cause is that the pci bus driver - and all other drivers - are not
> > > linked into apps for shared library builds. You always need to pass "-d"
> > > parameter to load drivers at init time (or have them installed in the
> > > correct driver path). For example, for me with a shared library build the
> > > following gives a no ports error:
> > >
> > > sudo ./build/l2fwd -c F00000 -- -p 3
> > >
> > > while this succeeds and runs fine
> > >
> > > sudo ./build/l2fwd -c F00000 -d $RTE_SDK/$RTE_TARGET/lib/librte_pmd_i40e.so -- -p 3
> >
> > The root cause is that recent gcc won't run constructor on unused libraries.
> > Testing a patch to take --as-needed off of PCI library.
> >
> > See: https://stackoverflow.com/questions/11631161/force-to-link-against-unused-shared-library
>
> The constructor is run when calling dlopen, right?
>
> Note: dlopen with -d is a feature.
> The original idea was to be able to specify which driver we want to use.
> If we want an automatic dlopen, like modprobe, then we need more scripts.
> But I understand you are against the whole dlopen idea.
>
This issue is more of a problem for development systems where we EAL path
is not really usable for finding the drivers. For a properly deployed
system where we use DPDK installed to /usr/local or /usr, the EAL PMD path
will be correctly configured and properly probe all drivers.