DPDK patches and discussions
 help / color / mirror / Atom feed
From: Stephen Hemminger <stephen@networkplumber.org>
To: Bruce Richardson <bruce.richardson@intel.com>
Cc: Thomas Monjalon <thomas@monjalon.net>, dev@dpdk.org, stable@dpdk.org
Subject: Re: [dpdk-dev] [PATCH] pci: fix missing pci bus with shared library build
Date: Mon, 22 Jul 2019 09:43:23 -0700	[thread overview]
Message-ID: <20190722094323.613cb090@xps13> (raw)
In-Reply-To: <20190722090610.GA289@bricha3-MOBL.ger.corp.intel.com>

On Mon, 22 Jul 2019 10:06:11 +0100
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Mon, Jul 22, 2019 at 09:38:27AM +0200, Thomas Monjalon wrote:
> > 19/07/2019 22:55, Stephen Hemminger:  
> > > On Tue, 16 Jul 2019 09:46:04 +0100
> > > Bruce Richardson <bruce.richardson@intel.com> wrote:
> > >   
> > > > On Mon, Jul 15, 2019 at 05:19:12PM -0700, Stephen Hemminger wrote:  
> > > > > On Mon, 15 Jul 2019 16:41:36 -0700
> > > > > Stephen Hemminger <stephen@networkplumber.org> wrote:
> > > > >     
> > > > > > If DPDK is built as a shared library, then any application linked
> > > > > > with rte.app.mk will not find any PCI devices. When the application
> > > > > > is started no ethernet devices are found.
> > > > > > 
> > > > > > This is because the link order of libraries on the command line matters.
> > > > > > And PCI is before EAL. That causes there to be no dependency on PCI
> > > > > > so linker ignores linking the library. 
> > > > > > Swapping the order fixes this.
> > > > > > 
> > > > > > Fixes: c752998b5e2e ("pci: introduce library and driver")
> > > > > > Cc: stable@dpdk.org
> > > > > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> > > > > > ---
> > > > > >  mk/rte.app.mk | 2 +-
> > > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > 
> > > > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk
> > > > > > index a277c808ed8e..470b92e4d73e 100644
> > > > > > --- a/mk/rte.app.mk
> > > > > > +++ b/mk/rte.app.mk
> > > > > > @@ -90,8 +90,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_STACK)          += -lrte_stack
> > > > > >  _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING)   += -lrte_mempool_ring
> > > > > >  _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += -lrte_mempool_octeontx2
> > > > > >  _LDLIBS-$(CONFIG_RTE_LIBRTE_RING)           += -lrte_ring
> > > > > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI)            += -lrte_pci
> > > > > >  _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL)            += -lrte_eal
> > > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI)            += -lrte_pci
> > > > > >  _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE)        += -lrte_cmdline
> > > > > >  _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER)        += -lrte_reorder
> > > > > >  _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED)          += -lrte_sched    
> > > > > 
> > > > > It still happens with 19.08. Testpmd works but only because it is
> > > > > linked with so many things. But l3fwd fails...
> > > > > 
> > > > > # ./examples/l3fwd/build/l3fwd -n4 -l0-3 -w 02:00.0
> > > > > EAL: Detected 8 lcore(s)
> > > > > EAL: Detected 1 NUMA nodes
> > > > > EAL: failed to parse device "02:00.0"
> > > > > EAL: Unable to parse device '02:00.0'
> > > > > EAL: Error - exiting with code: 1
> > > > >   Cause: Invalid EAL parameters    
> > > > 
> > > > I don't think the position of these is going to be the cause here, the more
> > > > likely cause is that the pci bus driver - and all other drivers - are not
> > > > linked into apps for shared library builds. You always need to pass "-d"
> > > > parameter to load drivers at init time (or have them installed in the
> > > > correct driver path). For example, for me with a shared library build the
> > > > following gives a no ports error:
> > > > 
> > > > 	sudo ./build/l2fwd -c F00000 -- -p 3
> > > > 
> > > > while this succeeds and runs fine
> > > > 
> > > > 	sudo ./build/l2fwd -c F00000 -d $RTE_SDK/$RTE_TARGET/lib/librte_pmd_i40e.so -- -p 3  
> > > 
> > > The root cause is that recent gcc won't run constructor on unused libraries.
> > > Testing a patch to take --as-needed off of PCI library.
> > > 
> > > See: https://stackoverflow.com/questions/11631161/force-to-link-against-unused-shared-library  
> > 
> > The constructor is run when calling dlopen, right?
> > 
> > Note: dlopen with -d is a feature.
> > The original idea was to be able to specify which driver we want to use.
> > If we want an automatic dlopen, like modprobe, then we need more scripts.
> > But I understand you are against the whole dlopen idea.
> >   
> 
> This issue is more of a problem for development systems where we EAL path
> is not really usable for finding the drivers. For a properly deployed
> system where we use DPDK installed to /usr/local or /usr, the EAL PMD path
> will be correctly configured and properly probe all drivers.

The problem is that bus drivers register themselves in constructors and
these construtors are not run with as-needed.

One part of fixing this is:
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index df917f946497..46bdff8ec5e8 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -130,6 +130,9 @@ ifeq ($(CONFIG_RTE_LIBRTE_FSLMC_BUS),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_COMMON_FSLMC)   += -lrte_common_fslmc
 endif
 
+# Bus devices register in constructor so always link
+_LDLIBS-y      += --no-as-needed
+
 _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI_BUS)        += -lrte_bus_pci
 _LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_BUS)       += -lrte_bus_vdev
 _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA_BUS)       += -lrte_bus_dpaa
@@ -137,6 +140,8 @@ ifeq ($(CONFIG_RTE_EAL_VFIO),y)
 _LDLIBS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS)      += -lrte_bus_fslmc
 endif
 
+_LDLIBS-y      += --as-needed
+
 ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n)
 # plugins (link only if static libraries)
 




  reply	other threads:[~2019-07-22 16:43 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-15 23:41 Stephen Hemminger
2019-07-16  0:16 ` Stephen Hemminger
2019-07-16  0:19 ` Stephen Hemminger
2019-07-16  8:46   ` Bruce Richardson
2019-07-16 14:46     ` Stephen Hemminger
2019-07-19 18:11     ` Stephen Hemminger
2019-07-19 20:39     ` Stephen Hemminger
2019-07-19 20:55     ` Stephen Hemminger
2019-07-22  7:38       ` Thomas Monjalon
2019-07-22  9:06         ` Bruce Richardson
2019-07-22 16:43           ` Stephen Hemminger [this message]
2019-07-22 17:04             ` Thomas Monjalon
2019-07-22 17:13               ` Stephen Hemminger
2019-07-22 17:31                 ` Thomas Monjalon
2019-07-22 18:34                   ` Stephen Hemminger
2019-07-23  7:59                     ` Thomas Monjalon
2019-07-23 18:29                       ` Stephen Hemminger
2019-07-23 18:35                         ` Thomas Monjalon
2019-07-22 18:53                   ` Stephen Hemminger
2019-07-23 12:30                     ` Bruce Richardson
2019-07-23 18:11                       ` Stephen Hemminger
2019-07-24  8:56                         ` Bruce Richardson
2019-07-23 18:47                       ` Stephen Hemminger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190722094323.613cb090@xps13 \
    --to=stephen@networkplumber.org \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=stable@dpdk.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).